It's easier for me to write about the castles I don't like... Knights and Barbarians... it feels like in all subsequent versions, the developers were apologizing and rehabilitating these castles...
M - Knight castle
S - Barbarian castle
L - Sorceress castle
XL - Warlock castle
For example, I played the campaign as a Knight because it was very easy to gather an army in the first scenario and defeat the dragons with it.