The magic system in Heroes V [Discussion]

I would be very grateful if someone could tell me what is needed to gain control of a town's portal. Even after maxing out all four magic skills, it still doesn't want to come to me:(
Honestly, I don't know what to do...
And in one campaign, there was a hero who didn't take any magic skills at all!!!:eek: