Good idea, I'll think about it. We could make it so that he appears less and less frequently each time, or so that he stops appearing altogether after the Inferno castle is captured. But the castle can be captured quickly.

The simplest idea is to introduce a quest. I captured the Inferno castle right away and without any particular problems. Perhaps the quest could be triggered when reputation reaches 50 with any of the cities. Or, the quest could involve placing a large army with the enemy, so that it is initially invincible. You know better)
And by the way, a separate thank you for the puzzles and interesting quests that required some thought. I happily put down my laptop and got out a pen and notebook to write down the details and think about the answer.