Not quite. If there are fewer ifrits, then they take less damage.

How is that possible?

You can inflict exactly as much damage to one ifrit as it has hit points, right? :D :D :D
Therefore, the damage from one ifrit will be quite weak, and it can be ignored. But if there are so many ifrits that one hit won't kill them, then you need to be careful. Especially vampires :)
So, it's important not only to reduce the number of units in the attacking ifrit army, but also the decreasing number of ifrits themselves :smile01:

You're getting something mixed up. Check it yourself - take a pack of, say, a hundred ifrits, and take one peasant. Disable retaliation on the ifrits, and attack with the peasant. According to your logic, the peasant should die. But I assure you, it will survive. ;)
Because how much damage will our peasant inflict on the ifrits? That's right, very little, I don't remember exactly, let's say one. In this case, the fire shield will reflect 1/3 of that damage back onto the peasant (plus some modifiers). How much health does the peasant have? That's right, more than the reflected damage.
That's how it works.