Good and Evil.

We say Lenin, we mean the party...
We need to keep two factions:
1) Good - Castle, Rampart, Tower;
2) Evil - Inferno, Necropolis, Dungeon.
The rest - overboard.
IMHO.

That's too selfish. For example, I was planning to play as the neutral factions... if they remove them, I'll leave... and many people like to play as certain neutral castles, so if you don't like them, you shouldn't say they should be removed. IMHO.