It all depends on the number of Djinns; even if an Ogre-Mage doesn't defeat a Djinn in a 1-on-1 fight,
The most effective way to use Djinns is with illusions, but Ogres have very good Bless, so illusions will be easily countered, and you won't be able to cast many.
Ogres have 50% magic resistance, which doesn't allow Djinns to run wild...