Nargott, I completely understand your position and agree with it. Each faction should have its own motivation to gather into an army. Culture, style, history, and worldview should be shared. If we look at the standard factions, we have: two elf factions (light and dark), three human factions (humans, mages, and undead), and two demon factions (Inferno and half-demons - orcs). But each faction has a reason to exist. In the mod for the third part, "Heroes of Might and Magic III", a great pirate city has been added. Although they are humans, they are significantly different from the Griffin Empire.
The swamp faction is significantly different from the Naga Sanctuary in terms of culture, worldview, combat style, and unit set (although, these are all lizard-like humans). If you try, you can come up with an interesting story for the Factory as well.
And now about the lore... Using simple logic, you can explain many units. Of course, you shouldn't add cities from Winx, but if something is appropriate, why not?