You're saying that the Unicorn has much worse defense than the Djinn or the Nightmare?
And that the blue-clad, sword-wielding creatures also have poor defense? Or the Dragons? Do they have virtually no defense?
In general, for me, the only difficulty when playing as the light-aligned factions is the huge amount of wood required by their castle. That's it.