Barbarians are ideal for attacking, and Dwarves are ideal for defending a castle. But that's just my opinion.

I wouldn't say that Barbarians are even good for attacking, let alone ideal. They have only one ranged unit (1/8 damage, cycles don't count), there's essentially nowhere to build up rage, and their only flying unit is the Wyvern, but rage doesn't affect it. And Barbarians have no magic. The only plus is the free Cyclops that break down walls, but this can be neutralized by, for example, the Dungeon faction.
I don't know about the Dwarves; they defend perfectly well even without a castle. Besides, there are several other factions with their own defensive advantages. Mages can easily destroy a catapult (three ranged units + a hero with Chaos magic), and Warlocks can force the enemy to retreat with powerful spells. There's no clear leader here.