There’s Something Going Wrong With The Polls — At Least When Trump is On The Ballot

This year, none of the big prognosticators nailed that brag-worthy, March Madness-bracket-esque, rub-it-in-your-face perfect electoral map forecast.

Some got damn close: Sabato’s Crystal Ball team at the University of Virginia nailed 49 of the 50 states. Only pesky North Carolina tripped them up. Cook Political Report did similarly well, though its sole error was giving Georgia to President Donald Trump.

Georgia going blue but North Carolina staying red was the most consistent trip-up for election prognosticators. And that, forecasters told TPM, was because of some dynamics that weren’t picked up by the polls underlying their forecasts.

“One explanation for some of the disparity in polling error is that North Carolina has a higher share of non-college white voters than Georgia,” G. Elliot Morris, data journalist for The Economist, told TPM. “Since polls overestimated Biden’s support with non-college whites pretty much everywhere, he would have more room to fall in North Carolina than Georgia.”

Professor Sam Wang, who runs the Princeton Election Consortium, hypothesized that the disparity could be due to Georgia’s larger share of Black voters.

“I think the fact that Georgia flipped and Florida didn’t is historically kind of odd, but we did see Biden’s various demographic weaknesses there,” Kyle Kondik, Crystal Ball’s managing editor, told TPM. “The polls were also tricky to navigate — they really overstated, in many cases, Democratic strength in some places.”

That’ll be a familiar tune to many political observers. Polling error in 2016 left Democrats heartbroken, stunned and, as proven by many meltdowns in left-leaning homes on the night of Nov. 3, emotionally scarred.

To be clear, modeling error and polling error are not the same thing, though they are, of course, intrinsically linked.

Morris said that The Economist’s 2016 model — which was built before he joined the magazine — was relatively accurate because it was able to detect some underestimation of then-candidate Donald Trump’s popularity in the polls by, for example, paying attention to whether the pollster weighted by education. But while their model had Hillary Clinton ahead by less in the key midwestern states than other models did, she still lost — something the model did not predict.

“Our model did pretty good, but the polls were still wrong,” he said.

The same was true this year. While most models were wrong about a couple states — which is about what you can expect, prognosticators often warn — the polls again seemed to miss Trump voters in a significant way.

“If I had anything to say about it, there should be a harsh reckoning about why the polls underestimated candidates in the same direction and similar magnitude in the same states in the last three election cycles,” Morris said.

While the polling misses on the presidential level were perhaps the most heart-stopping errors this year, the wildest swings ended up being on the House level. The chamber was considered so safe that it barely registered, drowned out by speculation about the White House and the Senate. Now, Democrats have a slimmer majority in the face of significant Republican gains.

That’s not what it looked like would happen after studying a slew of public and partisan polling from both parties, said Nathan Gonzales, editor and publisher of Insider Elections.

“We know now that the polls underestimated President Trump whose performance was much closer to 2016 and meant that those Democratic members who represented districts Trump carried in 2016 were in an extra vulnerable position, and many ultimately lost or are on their way to losing,” he told TPM. “I’m frustrated.”

After the outpouring of rage in 2016 about the polling miss, a flurry of explanations was offered up to explain how polls failed to capture some voters’ enthusiasm for Trump. One theory was the “shy Trump voter.” It posits that people contacted by pollsters liked Trump and intended to vote for him, but were too embarrassed to say so — at least, to a stranger on the phone. There hasn’t ultimately been much evidence found to support the theory, though it still gets airtime.

Another is that pollsters failed to sufficiently weight by education, thereby overloading their samples with highly educated voters who generally don’t like Trump. Most polls controlled for that factor this time around though, and it didn’t seem to make much difference.

Wang, of the Princeton Election Consortium, thinks some of the issue could lie with undecided voters.

“From a cognitive perspective, voters don’t necessarily know how they will vote, any more than you or I know what we’re going to have for lunch on some particular date in December,” he told TPM. “That’s different from the ‘shy Trump’ concept.”

Morris too thinks it’s hard for polls to capture changing dynamics in the final weeks of a campaign, but is more focused on the specific group that keeps getting underrepresented in the data: Trump voters.

One of his theories centers on the idea that these people have “low social trust,” a concept introduced by Democratic data guru David Shor. Morris hypothesizes that those people, generally white with no college education, would be disinclined to pick up a pollster’s call, or to stay on the line when they discover who’s on the other end.

Shor too found that his theory applied to this cycle, telling New York Magazine that liberals were “really excited” to answer polls, and spoke to pollsters at higher rates than Trump voters.

“These voters don’t trust institutions or each other, and feel pressured by society to hold certain beliefs so they don’t speak openly about theirs,” Morris said. It’s different than the “shy Trump voter” theory, but would be no less difficult for pollsters to get around.

And while these kinds of people may have always distrusted institutions and declined to talk to pollsters, it’s become more of a problem now when they’re increasingly siloed in one party. Back in 2008 and 2012, Morris said, there were many white, non-college-educated voters in unions, a group that historically went Democrat and helped this dynamic slip under the radar. That’s not so much the case anymore.

“Even if they weren’t responding, there wasn’t such a severe issue in the partisan makeup of the polls,” he said. “Now, those people are voting by 60 percent for Republicans and Trump routinely, and that can skew the polls by a few points.”

For the polls, and the prognosticators that depend on them, it’s a point of reckoning.

“Now we know that in elections where Trump is on the ballot, polling gets a little wacky,” said Gonzales from Insider Elections. “But is public opinion polling irreparably broken, or is it just broken in elections where Trump is on the ballot?”