The castle of your dreams.

Yes, a "Bandit" faction wouldn't hurt. There's a "Loot" (supposedly good) faction, but no "Bandit" one. There's definitely room for various bandits, nomads, and so on.
The rest – leave them as they are. And there's no need to divide these elementals into different elements. Elementals are elementals because they are elemental spirits (of earth, fire, air, water, etc.). Therefore, they belong in one castle.