Why not make any road blocks impossible?
Some guards on the road should be left in any case. But their size should be limited so that they do not create deadlocks, that is, they can be overcome by the hero and the starting army for any castle without critical losses.
Visually, the optimal limit seems to be 5K value (minor guardian, imp guardian).