So why is Varys a Targaryan supporter? In Jaime's speech he stated how Varys told the mad king not to open the gates. If Varys is all about whats best for the realm, you'd have to think the mad king being taken out was a step in that direction. Especially with Raegar dead already.
Because the Mad King was the exception, not the rule. With the Targaryens in charge, the Realm experienced 300+ years of relative peace. Before their time, the 7 kings fought with each other and with lesser lords constantly. The common folk in that time were nothing but pawns in the chess game their lords played with other lords. After the Targaryens conquered the land, the life of the common man improved immeasurably. If Varys is truly for Targaryen rule, it has to do with this.