This doesn't create an imbalance because the forces don't combine, and it won't be significantly harder for the attacker to take the town.
Alex's question, as I understood it, was different: what happens if a hero from a third faction approaches instead of your hero or the defender? Does he have to wait? Does he join the battle? And if so, how? That's where the difficulties began.