While discussing this issue I think it’s important to go back to some first principles. What are sports for? Why do we value them? What are the benefits?
People seem to have very different ideas about these questions and that may be driving some disagreement.
I agree. This is one of those issues that would probably generate a little more light and little less heat if we start from first principles, in part because most of us probably agree on most of them. Here's where I come down, and my guess is that you and I are probably on the same page at first:
1) Sports aren't that big a deal. We're trying to get through a global pandemic that has up-ended society for nearly a full calendar year at this point. We're struggling to deal with issues regarding systemic racism and police brutality. Economic inequality and the distribution of wealth has always been a massive issue and will continue to be so until the problem of scarcity disappears somehow. Outside of the political realm exists a whole host of issues involving religion, art, philosophy, etc. that help people make sense of their lives. Who gets to play what sport is really an insignificant issue in the broader picture.
2) Having acknowledged (1), we can also acknowledge that sports aren't totally unimportant either. I don't think of myself as an athlete, but I do enjoy running. When I was at my peak, I was a marginally-above-average marathoner. On a typical day against a typical field, I could expect to finish around the 70th percentile or so -- toward the top end of the median half, but still a mid-packer. I was never really competing for anything, but it was something fun to do. I still like running, and I just got back from a short run. Back in March, when we were talking about lockdowns and curfews, I was one of a handful of people who said that I would continue to run by myself even if it were made illegal to do so, so obviously it matters to me at least a little. I'm therefore safe in assuming that sports also matter to at least some other people too.
3) Sports are something enjoyable that we do for leisure. They have similar value to any other leisure activity. They don't determine our worth as human beings or anything like that, but they provide a nice, safe outlet for our competitive instincts. I see this point as following from (1) and (2), mostly.
4) Some people aren't good at sports for one reason or another, and some of those folks are going to be excluded from serious competition no matter how we structure athletics. This point falls squarely in the "life isn't fair" bucket, and it's not necessarily a problem in need of fixing.
I suspect we more or less agree so far.
5) There is no one obviously correct way to structure competitive sports. For example, we could decide as a society that we're just going to have open leagues where everybody competes against everybody else. I would advocate against that position, but there's nothing inherently wrong that arrangement (in contrast to system racism, for example).
6) That said, there are massive population-level differences between males and females when it comes to strength, speed, endurance, agility, etc. Some females can outperform most males in athletics, but no females can outperform the top males. These differences are mostly rooted in biology, which is why it's good to use the term "male" and "female" -- conflating sex and gender identity has the effect of downplaying this point.
7) Given (6), it's reasonable to let females have their own sports leagues or divisions where they can compete among themselves. Again, there's no moral imperative here, but it opens up a valuable leisure activity to females that wouldn't exist otherwise. This is the reason why we have "men's sports" and "women's sports" and it's sort of the reason why Title IX exists in the sporting world.
8) Once you accept that we're segregating by biological sex, not gender identity, then the issue of trans women in sports isn't difficult. They're males, and they can compete with other males if they want. Probably that means that most trans women won't be winning a lot of medals, but some people aren't good at sports for one reason or another, and that's not a problem in need of fixing.
I'm sure we diverge somewhere in those last several points.