OK, fixed

The correct solution is that you could only move around like that through the garrisons.

By the way, I finished the map. It seems interesting, but what's up with the water? I didn't swim, but even so, it's noticeable that the blue water transitions into swamp water in a way that doesn't make sense.

Also, I somehow thought the map was difficult, so I started playing as a "Warrior." It turned out that the AI is so dumb that it didn't even clear out the dungeon when I arrived on its territory (of course, after I opened up my entire half of the map and the dungeon). So, I advise everyone to play at least on "Veteran" (although maybe even "Hero" would be better).