Not all factions will be able to learn these types of spells quickly enough to destroy walls: Necropolis, Academy, and Dark Elves (if they learn Summon and find Earthquake), and also Dwarves. But most likely, Necropolis or Academy will use Earthquake because they have Summon in their castle, and there's a chance of getting a good one... As for the combat factions, even if they learn Earthquake, the damage will be terribly low. The exception is the High Elves; they have Summon, so all they need to do is find Earthquake and get Elder Druids, and the walls won't stand a chance...
However, Gremly do their job of causing trouble perfectly in all other cases.