Civil war.

I also sided with the Empire; many of their goals are the same as those of Ulfric (except for the assassination of the king, for example, because in some cases, overthrowing the ruler can be beneficial).
Moreover, Ulfric's actions, despite his policy of "Death to the elven invaders!", will only weaken both Skyrim and the Empire against them. And he lacks many of the personal qualities that are useful for a ruler.