The most optimal option for the Library in the Tower has long been proposed:

Rebuilding does not change the spells that were added to the slots after the Library was built.

1) There is no need to ban the Library.
2) It provides its bonus + allows upgrading Mages.
3) Rebuilding works in the Tower the same way as in other towns (i.e., 5 level 1 slots are rebuilt, 4 level 2, 3 level 3, 2 level 4, and 1 level 5).

I hope this exact option will be implemented in the next HeroesWorld patch.