In the original, where you play starting with only one city, it might work.
Of course, if you use multi-racial armies and heroes of different class-races, then this (loss of mini-artifacts) will be a significant drawback.
Also, sworn enemies (in the sense, those already killed, who can be set as enemies) are not recognized, and they will also be lost when replacing heroes, which is also not good.
For this, it will be necessary to somehow extract the guild from the city and simulate the hero touching it, and upon exiting, return the hero to the old place on the map.
If this option is possible, then mini-artifacts for the Academy can also be changed in the field.
And a completely incredible option is to open the interface of the Rangers' guild with an additional button in the hero's fire (like the Skill Wheel, although I think this is definitely impossible).
Although, of course, the +10% effect is extremely weak and will have to be supplemented with some +1 to attack.
And isn't this effect customizable?
Giving double bonuses is still more confusing. If the bonus cannot be changed, it is probably better to remove the building with an additional 10% altogether.
But with 3 native cities and Forest Cunning, the chance of double damage to a Sworn Enemy is already 40+10+3*10 = 80%
P.S.
Then it would be logical not to remove perks, but to come up with other effects for the racial one.