I agree. But it's also easier to take out archers from towers. Of course, you'll get a response, but it's still better than just standing there and waiting for them to attack...
And while the enemy is breaking down the gate, you can do a lot of damage.
Is it worth attacking the elf on the tower to deal 1/4 or 1/8 of his health, knowing that you'll take full damage in return?! But usually, there's no other choice, and you have to do it! :D :cool: