Don't say that! Whoever stays with the main army in the castle will 100% win.
When I said that you can win with any castle, in some cases you'll have to keep the hero in the main castle, and what's the problem with that?
If you've developed better, it doesn't really matter where you fight, and if you're weaker, it's the same "no matter what." Exceptions happen when both players are almost equally developed, but that's rare. Even rarer than having "bad skills."