In my subjective opinion, it all depends on the map, but more often than not, it's more convenient to collect different resources in smaller quantities than to focus on collecting one specific resource in large quantities, as it simply increases the chances of finding different resources.
1. Castle, Tower - other stones
2. Rampart, Stronghold - crystals
3. Fortress, Dungeon - sulfur
4. Conflux, Necropolis, Inferno - mercury
So, each faction has one such resource!