You may be surprised, but I 100% agree with you. The game's designers paid so much attention to the minute detail and interactions and interesting things to find and see in this game that having a fast travel runs completely counter to this and subverts the intent -- and maybe the entire point of this kind of imnmersive game.
But I'm more of
@shadyridr's mindeset on this in that having it doesn't mean you can't or won't immerse yourself in the game, but that you can choose not to depending on the way you want to play the game, or situationally.
If there were more random encounters, or varying dialogue with strangers that you meet on the road, or the hideouts or people camping off the roads had varying degrees of set-ups, difficulties, interactions/outcomes where you learned something from overhearing dialogue or picked up varying degrees of better loot (instead of simply becoming cookie-cutter repeats of the exact same rob-the-guy or kill-all-the-gang-members for a tin of chewing tobacco and an open health tonic), then I could see justifying not leaving the choice in a player's hands even more.
Point being, I think that choice should always be in the player's controls. With fast travel, I would still likely not use it 90% of the time, and reserve it for some of the grindy things I like to do like hunt those ten boars and 9 beavers and 15 snakes the trapper needs in different points of the map, be able to get to the trapper, and back to camp without that being the ONLY thing I can do in the 1 hour time slot for gaming I have a few times a week.
Wait, what? Does it explain everything and tie things all together? Only on CH3 and understand there is a LOT of mystery surrounding Emerald Ranch - the girl in the window, the shot-up saloon with the grave outside, the general gloominess of the place and multiple people mentioning parts of its backstory (Hosea for one, a woman you save whose horse has died on her and asks to be taken there). Would be cool if there is one mystery in this game that becomes solved by a tangible thing you can find.