The map is scarce in gold and resources (in my opinion). With 200% production on the first week, it's impossible to build cows in any of the three castles, unless you start on the second week. And even then: two Fortresses, three Castles... or not?
Guys, what do you think is the minimum army size/stats needed to defeat a garrison, without taking morale into account? You can lose the entire army.