Here's how it actually happened:
I was playing on XL, and there were a lot of neutral creatures. I was fighting them (for experience and artifacts, because there weren't many castles). And so on (especially since there were islands). I arrive, and he's sitting in his castle. And for no reason, I defeated him.
Auto-translated
