Antimagic will help you. His tactic is to bring you down with slowness, weakness, and other negative effects. First, I used water elementals to remove the spells, and then I used Berserk or Blessing to improve the situation. You need to take units that are 1-2 turns slower than the Crystal Dragon, but not vulnerable to magic, like a green dragon or someone else.
This is how I did it when I was playing on a medium difficulty level.
By the way, did you kill the Azure Dragons before entering the main Dungeon?