Pandora!

As I understand it, if you capture a Pandora's Box near an enemy Castle, the units inside will be of the enemy's faction. That is, they won't be tied to the hero's or player's race who captured the Pandora's Box. They don't seem to be tied to the terrain type because one terrain type can be suitable for multiple races.

Yes, if you capture a Pandora's Box near an enemy Castle, the units (if any are inside) will belong to that Castle!
That is, even if you're playing on Jebus, where you get 2 additional villages besides the main city, the Pandora's Box will still contain troops from the main Castle. For example, you start as the Tower faction, approach a village (let's say a Fortress) that's on your location, and see a Pandora's Box nearby. In this case, the box will give you units from the Tower, not from the Fortress!
There was a case that confused me: when I entered the Treasure Room, I saw two cities with a Fortress built in each (these were a Castle and an Inferno). The terrain type was sand. So, when I captured a Pandora's Box right next to (3-5 tiles away) the human Castle, I got 15 Efreet. I couldn't understand how that was possible, but then I noticed that around the same human Castle, there were dwellings of Inferno monsters (kennels, fire pits, and demon gates), but not a single archer tower or stable. That's when I realized that the human Castle was a secondary city with a Fortress built in it, and the entire Treasure Room belonged to the Inferno. That's why the Pandora's Box near the human Castle gave me Efreet!