You can only build all of them if you have several castles. So, are you suggesting building one in one castle and another in a different one? If you don't have a clear advantage in everything else, that's a sure way to lose.
And you can compare the usefulness of units, for example, a skeleton and an imp:
Their strength increases approximately the same amount and is small, but the imp runs far and collects resources better, and even in small numbers, it can cause a lot of trouble in battle by sucking 1 mana from angels and disrupting their resurrection, and in large numbers and with tactics... And a skeleton is just a product of early necromancy and mediocre meat (although it's great for Sacrifice, but that's already an exotic use).
And there are also key units for development, like vampires and djinn.
Regarding your example:
That's exactly the case where imps are better for gathering resources, and skeletons are good in large numbers (thankfully, necromancy is designed for that).
So, the choice of units depends on what you need... at the moment.
After all, you won't have to gather resources for the entire game...