Q&A of the Day – Which Presidential Poll Is The Most Accurate?
Each day I feature a listener question sent by one of these methods.
Email: brianmudd@iheartmedia.com
Social: @brianmuddradio
iHeartRadio: Use the Talkback feature – the microphone button on our station’s page in the iHeart app.
Today’s Entry: This is why people don’t trust political polls (he included a link to the Fox News and Rasmussen polls). How can the polls come out within one day of each other, each say that they only have a 3% margin of error and yet one says Biden’s up 2 and the other says that Trump’s up 9?
Bottom Line: Today’s note takes us to the two most recent national polls that rolled in late last week and it reignites a regular conversation and hotly debated topic about how much trust any of us should ever put in political polls. If you read the summations from the two different polls it’d seem like you’re living in the Bizzaro World. The Fox News polling summary starts off by saying this: President Biden is the frontrunner in a hypothetical matchup against former President Trump for the first time since October. Meanwhile Rasmussen’s poll shows Trump with his largest lead of the cycle. In fact, the poll shows Trump performing better than any nationally accredited poll has shown Trump performing in any election cycle. That’s dichotomy for sure. What’s more, as was noted by the listener who submitted today’s question, both polls assigned a 3% margin of error to their samples that differ with one another by 11%. Woof. So, what’s real? Is Biden really back on top or is Trump really performing better than he ever has in any election?
First, there’s one really important distinction between these polls that leaves for the possibility that they’re both possibly accurate within their margin of error. While on the surface both polls appear to be sampling the same thing, and at about the same time, they didn’t. The Fox News poll was conducted over a three-day stretch (14th-17th). Rasmussen’s poll was conducted entirely on the 20th. We’re not talking about a huge difference in time, but we are still talking about samples that differ by three to six days so that could matter. But the much bigger difference is who was sampled.
Right now, most of the pollsters are still using “registered voter” samples in their polling. This is an easier and cheaper way to obtain samples for the polls compared to exclusively sampling “likely voters”. Likely voter samples are always the best survey samples for the obvious reason. It doesn’t really matter who someone suggests they would vote for if they don’t plan on actually voting for anyone. This dynamic I believe is an especially important one to watch in the presidential election this year. My hypothesis, dating back about two years ago now, was that a Trump vs. Biden rematch would likely result in lower voter turnout compared to the 2020 election due to multiple factors (COVID voting and ballot practices and negative favorability ratings for both Biden and Trump in particular) and that the beneficiary of that change would be Trump. While that remains to be seen here’s a look at the current difference in the national polling averages between registered voter and likely voter samples.
- Registered voters: Even
- Likely voters: Trump +5%
That’s obviously a big difference. This by the way is also how the Fox News poll and Rasmussen poll would have the potential to be accurate within their margins of error. If you applied a 3-point swing towards Trump in the Fox News poll and a 3-point swing towards Biden in the Rasmussen poll you end up with a difference of exactly 5-points – which just so happens to be the difference between the average of all accredited national pollsters using registered and likely voter samples (this doesn’t mean that both or even either of the polls is accurate – only that it’s possible for both to be). In terms of pollster accuracy between these two pollsters let’s go to the scorecard:
- Fox News: Final poll had Clinton up 4% nationally
- Rasmussen: Final poll had Clinton up 2% nationally
The actual result was Clinton by 2.1%. Fox News oversampled Clinton’s support by 2%. Rasmussen was one of only two pollsters in 2016 to get the national popular vote (which is what national polls represent) right.
- Fox News: Final poll had Biden up 8% nationally
- Rasmussen: Final poll had Biden up 1% nationally
The actual result was Biden by 4.5% meaning Fox News overstated Biden’s support by 3.5% while Rasmussen overstated Trump’s support by 3.5%. Going through this exercise we see that Rasmussen has been the more accurate pollster with an average outcome that’s been off by 1.75% over the two previous elections while Fox News has averaged being off by 3.25% over that same time. Also, notably, both of the misses with Fox News’ polls were the result of oversampling support for the Democrat candidate for president (while the one time that Rasmussen was off it was due to oversampling Trump’s support).
In context what this means is that Trump is likely performing better than Fox News suggests, and he’s likely not performing quite as well as Rasmussen suggests – though it's likely that Trump’s current support is closer to what Rasmussen’s polling reflects than not – which is potentially great news for him as the two get ready for Thursday night’s first showdown of the cycle.