I wouldn't say that Barbarians are even good for sieges, let alone ideal. They have only one ranged unit (1/8 damage, siege units don't count), there's essentially nowhere to gain morale, and the only flying unit is the wyvern, but morale doesn't affect them. And Barbarians have no magic. The only advantage is the free cyclops that break down walls, but this can be neutralized, for example, by the Dark faction.
I don't know about the Dwarves; they defend perfectly well even without a castle. Besides, there are several other factions with their own defensive advantages. Mages can easily destroy a catapult (three ranged units + a Chaos Knight hero), and Warlocks can force the enemy to retreat with powerful spells. There's no clear leader here.