Gonna disagree... first, the "losers" were happy to have something of a treat to eat at all, it really meant something to them. They said when the winners got back, "yeah, you had a sandwich, but we had fresh ripe papaya, so it wasn't all bad", and then Ricard is basically like "well now I've had it too, so you're not special", bad move. It was their consolation prize and he just stepped right in it.
Secondly, he even said "I'm shocked there's papaya here. I didn't even know we had papaya on this island", so it was a rare treat for them on that count as well.