You get 3 days when an enemy hero is wandering around the map. Specifically, a hero, not a unit. ;)
I'll join Leshy. Let's consider another unit with a Tomb. And then - one without a Tomb, but that can visit a Tavern. And so on. Even an enemy caravan lasts for three days.
There's also a really bad scenario when your own ally holds it.
Either you're mistaken,
or it's an undocumented feature of Equilibr,
or I haven't read the documentation.