NHF – new heroes for Heroes V 3.1.

I repeat: the guard is virtual, the mine remains empty all the time until the hero decides to add troops from his own inventory. When a hero touches an object, the owners of both are checked, and if they don't match, a battle begins; otherwise, it's just a normal interaction.

This is very bad: attacking something unknown.

However, there is a way out: when a special unit settles in, remember the creatures. When your hero touches it, remove only those creatures from the garrison for selection, then add them back again (but you also need to figure out when to do this again...).
But this is definitely a hero's ability. You can add this feature as an ability in the skill tree, as there are free perks available.