Can The Polls Be Trusted?

There’s no question that Democrats’ chances in the midterms have improved dramatically over the course of the summer. From the point in May when Sam Alito’s draft of the Dobbs decision leaked to the press until now, the generic ballot average has shifted just over 3.5 points in Democrats’ direction. For the generic ballot average, that’s a big shift. But could the polls themselves be underweighting Republican strength? Like many others I’ve been figuring that Trump being on the ballot has been the key in recent elections. He and his party’s chances were underrated in 2016 and 2020. In 2018, they were pretty much on the mark. But Philip Bump of the Post and I were apparently both watching G. Elliott Morris’s Twitter feed yesterday when he noted that this wasn’t quite true. The national averages were on the mark in 2018. But at the state level key contests still underweighted Republican chances. Bump pulled the numbers himself and sure enough, that skew was there in Senate races in 2018 too.

So what does this mean? It is hard to say. Pollsters have been trying to adjust for this “silent Trump voter” issue for a few cycles now, a gap that seems driven by the increasing correlation between partisanship and education levels. They’re still working on it. But at least as of 2020 they haven’t clearly solved it. It’s important to note that these are very small sample sizes. So in any big sense you can’t say they’re conclusive. They also include some significant misses in the opposite direction. In 2018 and 2020 polls underestimated Democratic strength in Pennsylvania, Arizona and Georgia. But the upshot has to be to be prepared for polls to underestimate GOP strength by a few points. And obviously a few points can be a very big deal in a battle for the Senate.

Further adding to the uncertainty we’ve seen a number of actual partisan contests so far this cycle and those have generally underestimated Democratic strength, though those races are generally very, very lightly polled if at all and polling off cycle special elections is very difficult in any case. One might also figure in the Kansas abortion rights referendum.

Again, we’re dealing with small samples (in terms of numbers of cycles). There are a number of contending factors. But the unmistakable takeaway is that there’s been a pattern in recent cycles of a small but non-trivial skew underestimating GOP strength at least at the state level. That pattern has been strongest in the industrial midwest. So that clearly implicates races in Ohio, Pennsylvania and Wisconsin. My hunch is that John Fetterman is far enough ahead to put him reasonably outside the margin of all but deep systemic polling error. Ohio remains basically tied though and has been a charnel house of Democratic dreams for years now. The smattering of polls from Wisconsin since Mandela Barnes securing the nomination have shown him significantly ahead. But again, a tough state for Democrats in recent years.

It all comes down to the same thing. Recent polling skews produce real uncertainty about any close races. So no one should be complacent or take anything for granted.

Dear Reader,

When we asked recently what makes TPM different from other outlets, readers cited factors like honesty, curiosity, transparency, and our vibrant community. They also pointed to our ability to report on important stories and trends long before they are picked up by mainstream outlets; our ability to contextualize information within the arc of history; and our focus on the real-world consequences of the news.

Our unique approach to reporting and presenting the news, however, wouldn’t be possible without our readers’ support. That’s not just marketing speak, it’s true: our work would literally not be possible without readers deciding to become members. Not only does member support account for more than 80% of TPM’s revenue, our members have helped us build an engaged and informed community. Many of our best stories were born from reader tips and valuable member feedback.

We do what other news outlets can’t or won’t do because our members’ support gives us real independence.

If you enjoy reading TPM and value what we do, become a member today.

Latest Edblog
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Investigations Desk:
Reporters:
Newswriter:
Editor at Large:
General Counsel:
Publisher:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: