Can The Polls Be Trusted?

Start your day with TPM.
Sign up for the Morning Memo newsletter

There’s no question that Democrats’ chances in the midterms have improved dramatically over the course of the summer. From the point in May when Sam Alito’s draft of the Dobbs decision leaked to the press until now, the generic ballot average has shifted just over 3.5 points in Democrats’ direction. For the generic ballot average, that’s a big shift. But could the polls themselves be underweighting Republican strength? Like many others I’ve been figuring that Trump being on the ballot has been the key in recent elections. He and his party’s chances were underrated in 2016 and 2020. In 2018, they were pretty much on the mark. But Philip Bump of the Post and I were apparently both watching G. Elliott Morris’s Twitter feed yesterday when he noted that this wasn’t quite true. The national averages were on the mark in 2018. But at the state level key contests still underweighted Republican chances. Bump pulled the numbers himself and sure enough, that skew was there in Senate races in 2018 too.

So what does this mean? It is hard to say. Pollsters have been trying to adjust for this “silent Trump voter” issue for a few cycles now, a gap that seems driven by the increasing correlation between partisanship and education levels. They’re still working on it. But at least as of 2020 they haven’t clearly solved it. It’s important to note that these are very small sample sizes. So in any big sense you can’t say they’re conclusive. They also include some significant misses in the opposite direction. In 2018 and 2020 polls underestimated Democratic strength in Pennsylvania, Arizona and Georgia. But the upshot has to be to be prepared for polls to underestimate GOP strength by a few points. And obviously a few points can be a very big deal in a battle for the Senate.

Further adding to the uncertainty we’ve seen a number of actual partisan contests so far this cycle and those have generally underestimated Democratic strength, though those races are generally very, very lightly polled if at all and polling off cycle special elections is very difficult in any case. One might also figure in the Kansas abortion rights referendum.

Again, we’re dealing with small samples (in terms of numbers of cycles). There are a number of contending factors. But the unmistakable takeaway is that there’s been a pattern in recent cycles of a small but non-trivial skew underestimating GOP strength at least at the state level. That pattern has been strongest in the industrial midwest. So that clearly implicates races in Ohio, Pennsylvania and Wisconsin. My hunch is that John Fetterman is far enough ahead to put him reasonably outside the margin of all but deep systemic polling error. Ohio remains basically tied though and has been a charnel house of Democratic dreams for years now. The smattering of polls from Wisconsin since Mandela Barnes securing the nomination have shown him significantly ahead. But again, a tough state for Democrats in recent years.

It all comes down to the same thing. Recent polling skews produce real uncertainty about any close races. So no one should be complacent or take anything for granted.

Latest Editors' Blog
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Editor at Large:
General Counsel:
Publisher:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: