This has been a fantastic example that goes way bigger than polling.
We have to think about the system as a whole. As the big picture.
I'm attracted to smart people. I love the smart things places like 538 does.
Clearly, they have super smart people working there and doing stuff.
But the lesson for us is, "What's the material they're working with?"
For lots of this, the material, the source product, is people answering questions.
We have to ask ourselves, "Why do we trust that?"
Because you can have the most amazing technology and people ever to take those answers and manipulate them, but if the source material is faulty, nothing else matters.
This to me was the big error in 2016 and 2020. And from what I understand, from the Brexit polling.
I have a dozen people I'd say I'm very close to who all voted Trump. They wouldn't remotely consider a Trump bumper sticker. They wouldn't consider truthfully telling a pollster they were Trump voters. Not even close.
We got bad source material for this process of analyzing polls. It sometimes happens. And I think it's a good lesson for much bigger things.