It wasn't just the developers of Heroes 1 who decided to make it so that castles would match the type of map.
M - Knight's castle
S - Barbarian's castle
L - Sorceress's castle
XL - Warlock's castle
For example, I played through the campaign as a Knight because it was very easy to gather an army in the first scenario and defeat the dragons with it.
Well, I don't know... What about the Tower and the Necropolis?