Josh Goes Back to the Big Senate Data Dive to Obsess About Polling Error

DETROIT, MICHIGAN - NOVEMBER 5: Detroit voters at the polls inside Central United Methodist Church on November 5, 2024 in downtown Detroit, Michigan. Americans cast their ballots today in the presidential race betwee... DETROIT, MICHIGAN - NOVEMBER 5: Detroit voters at the polls inside Central United Methodist Church on November 5, 2024 in downtown Detroit, Michigan. Americans cast their ballots today in the presidential race between Republican nominee former President Donald Trump and Democratic nominee Vice President Kamala Harris, as well as multiple state elections that will determine the balance of power in Congress. (Photo by Sarah Rice/Getty Images) MORE LESS

Back in August I did a write-up on a deep dive on polling error put together by Republican consultant/pollster Pat Ruffini. When it comes to Ruffini’s work, I have a funny mix of assumptions and impulses. Ruffini’s a data guy and not a bullshitter. His numbers and analyses are real and solid. He’s also a Republican partisan. So he’s not above looking for the most favorable possibilities in the numbers. And I think it’s fair to say the series he’s done on this is generally aimed at Republicans and trying to send the message, “It might be okay” or “It’s not as bad as it looks.” At the same time, I am eager for it to be as bad for Republicans or worse than it looks.

Take that for what you will. Just a little truth in advertising and transparency for all involved. Ruffini just did a follow-up on this deep dive where, amazingly, there’s a new reason to think it might not be as bad for Republicans as it looks! Again, though, Ruffini’s a smart guy with a lot of data up his sleeves and he might be right.

In any case, let’s review his earlier findings. Essentially, what he found is that in the last four election cycles, polls have consistently overestimated Democrats in Senate contests during the summer. In presidential years, the polls get more accurate toward Election Day but there’s still a substantial average bias in favor of Democrats. In midterms, though, that bias basically evaporates by Election Day. Individual polls might miss but, on average, polls are quite accurate. Back in August the argument was, sure it looks bad now but this is the summer and they’re about to start trending back to the Republicans!

But that didn’t happen. In fact the polls got worse for Republicans just as they are supposed to be trending toward more accurate reads of the final result. And that’s where we are now.

So in Ruffini’s new update we find out, there’s more! Sure, the the polling error on average basically evaporates. But the average conceals something which now looks pretty helpful for Republicans. Despite the averages, Republicans end up doing a bit better in red states and Democrats do a bit better in blue states. But even with that basic pattern, there’s still a bit more final advantage for Republicans. That’s welcome news for Republicans since basically the whole playing field now is in red states. (I’m being a bit unfair here to Ruffini. This fact is in the earlier deep dive. But it’s buried. It’s the centerpiece of the new update.)

He notes something else that fits with our general understanding of the Trump-era electorate but is worth pointing out explicitly. It’s not just red states. It’s especially red states with large number of white non-college voters. So if we’re going to see this break toward Republicans in the actual election results, we’d expect it to be more likely in, say, Iowa and Ohio than Texas.

The other issue is the number of undecided voters. What we’re really talking about with Democrats and Republicans doing better in their “own” states is a reversion to the mean, or muscle memory. If candidates are tied at 45% in a red state, you figure that partisan muscle memory is going to kick in with the undecided voters and they’ll disproportionately break to Republicans. Same goes for Democrats in a Democratic state.

There used to be a conventional wisdom — and it was before the era of data-diving so I’m not sure it was true — which held that undecideds tended to break against the incumbent. The thinking was that the voter knows the incumbent. If they’re still undecided by the end of the campaign, most of those undecideds actually won’t vote for the incumbent. They just haven’t made that decision conscious yet. Or they haven’t quite come around to the necessary implication of that inability, which is that they are going to vote for the challenger. If this was ever true it seems to have been rendered inoperative by our more intensively partisan era.

What’s the upshot of all this? I guess just the obvious one that some of these red state contests that are super close could end up being a couple more percentage points favorable to the GOP. And the Republican wins. That looks most obviously possible in Iowa and Kansas, which are very close to ties in the polls.

Another point I haven’t mentioned much is that Ashley Hinson, the Republican Senate nominee in Iowa, is a strong candidate. No generational talent but a strong candidate. She’s kept the race close to even amidst an incredibly unfriendly environment. Here you see the contrast with Roger Marshall in Kansas. And really with Ashley Moody (FL), John Husted (OH), Ken Paxton (TX), Michael Whatley (NC), Mike Collins (GA), Dan Sullivan (AK), Pete Ricketts (NE). These range from meh candidates to horrible candidates. Some are from states that don’t have many contested general elections; others were gifted appointments or nominations by Donald Trump or Ron DeSantis. And as I’ve noted earlier, in addition to the bad environment, this is playing an important role in this cycle. Hinson and Maine Sen. Susan Collins are the two candidates who have real skills, and they’re both in contention.

It’s important to remember that these are general patterns in the last four election cycles. That’s a very small data set. Things tend to be the same until they’re not. You can’t just assume recent patterns in future results. And there are factors in this election — sharp swings of Hispanic voters against Trump, rural discontent tied to Trump’s trade policies, just Trump’s extreme unpopularity. What still remains to be seen is what will happen with whites without college degrees, the demographic group Trump has a particular hold on and with which pollsters have the most challenges.