It's one thing to fight a hundred zombies with your starting 85 Goblins, and another to fight 120 Trolls.
No one plays with the Tower against Dungeon anyway.
P.S. And, by the way, I don't remember the Tower taking more time in battles. In my opinion, it's the opposite; the tactics there are usually simple - you build yourself in a corner and shoot, more sophisticated battles are rare. With Dungeon, the number of battles where you need to think and calculate moves is higher. For example, battles against all creatures with a speed equal to that of Trolls are quite challenging.