I didn't mean the Castle, I meant the dwellings; I misspoke... And sometimes even the Castle. You constantly need to manage the growth, hang around each dwelling, and return to it every growth turn. Eight heroes with 2500 each is noticeable, and instead of developing, you can undermine the economy.