Here's the situation: you have wings, you need to build level 5 (or whatever the rules say...), you fly, everything is fine; and then the enemy takes your Castle with level 5, and... I'm sitting here wondering if I can still fly or not. It's interesting. On one hand, you built it, so you should be able to fly, but on the other hand, you don't have a Castle anymore.
Personally, I think this is perfectly fair regarding the building requirement for scrolls. Today, I played a game and found an explosion scroll guarded by 20-49 Gorgons, who were also guarding a can of food, a hut, and resources. It's scary to think how many there would be if they were only guarding one scroll. I personally found a Berserk scroll guarded by 20-49 simple Lizards. But building a Guild will be more difficult.