"And I will give them to your neighbor."
This doesn't mean God physically picks them up and hands the wives off, it means they will leave and become the other men's wives.
That isn't the context of the events in 2 Samuel. Nathan is chastising King David for sleeping with Uriah's wife while Uriah was away fighting a war for David.. and for eventually setting Uriah up to die in battle so that he can marry his wife, who he had been sleeping with.
God said that he was going to take David's wives and give them to one of his friends (someone close to David, not a neighbor outside the city) and this friend will have sex with his wives in broad daylight. Verse 12 says "you did this in secret, but I will do this in broad daylight" or in public to add to the humiliation and expose King David for his acts of adultery.
Uriah trusted David, his commander and king. David betrayed him by having an affair with his wife in secret. God is angry and threatens to do the same to David (have his wives to have sex with someone David trusts, but not in secret.. in public to bring to light David's sins). The wives in question here are simply property to be given and taken to make a point.
Is it rape? maybe, maybe not. Do the wives have a say in the matter? Would it matter if they didn't consent to this punishment of David at God's hand?