I went with my girlfriend last year. Once you're up at the castle, you don't have to go back down between tours. You just schedule what times you want to do the tours, which you can do online or that day at the ticket counter. I think it's best to do there, as the lady at the counter was helpful. After doing one tour, you can hang around and then return to the top of the stairs where you came up when getting off the bus upon arriving at the house. You'll want to do this about 20 minutes after the scheduled time (because it takes them that long to get up the hill). There is a guide who hovers there around the top of the stairs, and he will advise you and help you get into the right tour when it arrives.
I did the main tour and the upstairs tour. The main tour was cool but it's really just like 3 large rooms -- a living room, dining room and movie theater where you watch a lil screening of the celebrity guests back in the castle's heyday. You see a ton of cool furniture and artwork from different historical eras that Hearst was able to buy from Europe at discount prices after WWI. I actually found the upstairs tour more interesting, going through his libraries, bedroom, guest bedrooms, and the bedroom of his mistress. It felt like a more substantial tour. The guide provided more details, there were only about 10 people on the tour so it was more personal and he answered questions, and you had a little more freedom and space to walk around and look at things.
For your time up at the house while not on tour, I think an hour is sufficient. You just want to take in the main pool with its Roman architecture and walk through the gardens. On the way to the bus as you're leaving, you'll go by the covered pool which is also very cool. There's also a free movie on Hearst history down at the welcome center that I think is best to watch when you first arrive, before going up to the house.
