Which castle allows for the fastest development?

I think you can start the fastest in the Stronghold.
Goblin Barracks - Wolf Den - Rocky Lair - Monster Cave.
With a barracks and den built early, you can simply build the rocky lair, and on the second day, you'll have Monsters (if the resources allow for the chosen difficulty level). Although, it's not that difficult to gather resources for all of these.