Q&A of the Day – How Polls Have Changed from Previous Presidential Elections
Each day I feature a listener question sent by one of these methods.
Email: brianmudd@iheartmedia.com
Social: @brianmuddradio
iHeartRadio: Use the Talkback feature – the microphone button on our station’s page in the iHeart app.
Today’s Entry: Brian, thank you for helping to calm me down. I was really upset after the debate when I thought Trump blew it. I truly appreciate how you we able to informationally demonstrate that the debate didn’t have the same effect on undecided voters that I feared it might have. I have a related question for you. You’d mentioned that Trump is still running ahead of where he was in the polls in both of the prior presidential elections. Have pollsters changed the way they’re polling significantly since then? IE are the polls being weighted more favorably for Trump than they were before, since they under sampled his support previously?
Bottom Line: The answer is that the pollsters have made significant changes over the prior two presidential election cycles and while we obviously won’t know if they’re more accurate until after the votes have been counted, we can account for what’s changed right now.
Last year the Pew Research Center conducted a study of how political pollsters are conducting their polls. What they found is that the industry has rapidly been changing. The first big takeaway from their study is that over 60% of pollsters changed their sampling and their methodology between the 2016 and 2022 elections. The fastest growing change within the industry is the use of multiple methods to obtain samples for polls with significant growth in the use of digital samples. Specifically, with fewer people answering their phones for call out surveys, the industry has flipped from about two-thirds of surveys being conducted over the phone with live interviews to about two-thirds being done through digital surveys. Here’s the breakout of the percentage of pollsters using each of these methods:
- Online opt-in: 67%
- Probability-based panel (recurring panels of people who’ve agreed to be regularly polled): 33%
- Live phone interviews: 32%
- Texts: 13%
- Mailed surveys: 10%
- Interactive voice response (call-based voice automated polling system): 6%
Just over two weeks ago Pew published an update to their research that’s specific to what pollsters are doing this cycle. As they note: Confidence in U.S. public opinion polling was shaken by errors in 2016 and 2020. In both years’ general elections, many polls underestimated the strength of Republican candidates, including Donald Trump. These errors laid bare some real limitations of polling. In addition to the changes in how polls are conducted, Pew notes these changes as being significant in this cycle:
- More pollsters producing more samples (total samples have doubled over the prior 22 years)
- The way “prominent pollsters” are weighting samples
- Determining who are “likely voters”
- The timing of polling samples around major events
- Increased transparency by top pollsters
- Increased emphasis on swing state polls
With most of these changes it’s easy to see how the potential to produce more accurate results. For example, in the current RealClear Politics polling average of national polls, the range over the past two weeks in individual polls is Harris plus 5 to Trump plus 3. Similarly using only post-debate samples, the range in the polls is Harris plus 5 to Trump plus 3. The RCP polling average prior to the results of the Presidential debate showed Harris up nationally by 1.5%. The average as of today stands at 1.7%. With more data coming from more sources more quickly it’s easier to balance out the outliers. For example, it’s likely safe to say that Kamala Harris isn’t really up 5-points nationally – which would mean that she would win the national popular vote by greater than twice the margin that Hillary Clinton did and by more than Joe Biden did in 2020. In other words, what would be a rather easy and comfortable win in the Electoral College for her. At the same time, it’s safe to say that if the election were today, Donald Trump wouldn’t win the national popular vote by the largest amount since George H.W. Bush in 1988 – in what would amount to an Electoral College landslide. A race where Harris is up 1.7% nationally, a number that’s 0.4% lower than Hillary Clinton’s actual results in 2016, seems to be about right.
The real wild card with the polling changes for this cycle is on the weighting and how the changes made are affecting it. As Pew notes, about the way polls are now being weighted: Weighting a poll on just a few variables like age, race and gender is insufficient for getting accurate results. Some groups of people – such as older adults and college graduates – are more likely to take surveys, which can lead to errors that are too sizable for a simple three- or four-variable adjustment to work well. Adjusting on more variables produces more accurate results, according to Center studies. A number of pollsters have taken this lesson to heart. For example, recent high-quality polls by Gallup and The New York Times/Siena College adjusted on eight and 12 variables, respectively. Our own polls typically adjust on 12 variables. In a perfect world, it wouldn’t be necessary to have that much intervention by the pollster. But the real world of survey research is not perfect.
That’s well said. And for that reason, we won’t know if what they’ve done is better until we get there. In the meantime, what we do know is that as of this moment Trump is running much better in these polls than he did four years ago in his loss to Biden and ever-so-slightly better than he was when he beat Clinton in 2016.