You're right, it's best to apply the Fire Sign by level 6, and even earlier if possible...
Regarding the combination of walls with bears and harpooners... No decent player will exit the wall in a way that allows bears or harpooners to push them back... And bears sometimes deal negative damage, you run to block archers, and they push them directly into the archers' line of fire.
Bow down and hear our cry,
Our earthly rulers falter,
Our people drift and die,
The walls of gold entomb us,
The swords of scorn divide,
Take not thy thunder from us,
Take away our pride.
G.K. Chesterton: English Hymnal