The path to knowledge.

This is a problem for all Dungeons on this map. Just keep clicking the exit, even though the cursor isn't actually pointing at it. It seems this is how the restriction on visiting is implemented for those who are pursuing you.