Ben and Eli have been better under pressure (i.e., in the playoffs) than Rivers has. Ben showed tremendous poise from day one. Eli, after showing questionable leadership earlier in his career, has become nails.
Rivers is more accurate than Ben or Eli, and has a quicker release. I think he's just as tough as Ben (which is saying a lot). He's not nearly as mobile as Ben, but he's probably as mobile as Eli. His arm isn't as strong as Ben's or Eli's, but it's strong enough to get the job done. I think Rivers makes better, quicker reads than Ben or Eli — more of a "coach on the field" type.
A GM wouldn't go wrong by picking any of the three. I do have concerns about Ben's character. Not just the rape thing, but his arrogance in general I could see rubbing his teammates the wrong way. I don't have any particular concerns about Eli's character, but Rivers is super elite upper tier in the character department.
I think Rivers has generally performed the best of the three so far. Look at stuff like YPA, TD-INT ratio, completion percentage, passer rating, etc. He lacks rings, and he hasn't been as good in the playoffs as he has during the regular seasons (although the playoffs have presented extenuating circumstances, like a torn ACL). So I can't say that it'd be wrong to take Ben or Eli over Rivers. But I'd personally take Rivers.
Not only has Rivers generally performed the best of the three so far (IMO), but I trust him the most to keep focused on improving each year. He regressed in 2011, which is evidence against my position; but in general, he strikes me as a person who lives and breathes football — who loves football, and would play for free if he had to. Maybe Ben and Eli are the same way and I just haven't noticed it because I'm not as familiar with them. But with Rivers, I get the sense that he'll always be completely dedicated to his craft the same way that Peyton Manning has been — studying tons of film, thinking about the game, constantly working to improve. I don't trust Ben the same way. I'm coming around on Eli, but if I had to pick right now, I'd go with Rivers.