There are signposts everywhere, you can follow them to get to the dungeon.
I have a question: should the red enemy caravan, which you need to intercept as part of the quest, be moving, or just standing in a corner of the map?
It should be moving... Perhaps you had Monster Week and they blocked the road, which is why it wasn't moving.