Without the skill, there will be no post-effects. No stuns, no freezes, and their damage will be mediocre. It wasn't great even before with senior druids, and now, without the skill and without specialists in chaos with a reduced set of spells... No, I don't think so.
However, okay, I've started a debate here. If people want it, let them do it. But I don't see the point, seriously.

The skill can remain with a high chance, like, for example, the Necromancer's chance for chaos. I was talking about a city guild, i.e., having a higher level of chaos in it. Having a ring or fireball in the guild at level 3 already justifies the chaotic downpour. It's just that the chance for this is twice as low.

Although in my mod, the elves won't even have a high chance for chaos, but "Secrets of Chaos" (bonus chaos spells) are added to the racial perks, and half of the spells in the school are not very demanding of the skill.