What Do the Polls Tell Us?

With 9 days to go before election day, I thought I’d take a moment to go over where the polls are as of today and what if anything they can tell us about what the headlines will be on November 7th.

If you’re a polling or data diehard a good bit of what follows may seem obvious. But not everyone’s a data nerd. So I want to go through a lot of data points and basic questions we get in one go. So here goes.NATIONAL VOTE

Clearly, we’re going to have a very tight outcome in terms of the national popular vote. With the exception of two extreme outliers — Gallup and Rand — all the national tracking polls show a race that is either tied or within 1-3 points. You can see the latest here.

So why is there such a variance? It’s worth noting that in statistical terms this isn’t that big a variance. There is an inevitable amount of noise in all polling data. But if you look at the tracking polls individually you’ll note that they’re actually not bouncing around that much at all. Among other things that points to the conclusion that more than noise the difference here is likely voting models — how many people and which people are actually going to show up. So for instance, you’ll note that Gallup currently has Mitt Romney up by 4 percentage points among likely voters but has Obama beating Romney by 1 point among registered voters. Pollsters are making different bets about the scope of voter turnout and just which groups are going to show up.

On balance, the national vote looks like a tie when you average all the numbers out, with a fractional advantage to either guy depending on which day you look. Nate Silver has some data which suggests, though doesn’t prove, that when the national popular vote and the state by state numbers go out of alignment that the state numbers are more reliable — not just because they determine the electoral college but also that they are surer guides to the national popular vote. Nate seems inherently skeptical (and I share this skepticism) that the popular vote and the electoral vote will give us a split decision, even if the polls suggest a decent likelihood of it happening.

So what about the states where the whole thing really gets decided?

Here the clearest reality is that Mitt Romney has to do something like running the table to get the 270 electoral votes necessary for victory. On the other hand, Barack Obama has to just hold on where he already has leads. Now, that’s basically good news for Obama. But it’s not necessarily quite as bad for Mitt Romney as it sounds. Because ‘running the table’ could be achieved by just outperforming the polls by a two or three percentage points in a handful of states.

So let’s look at the details. And I’m going to focus on Ohio and Virginia because even though other states like Iowa, Colorado, Florida and New Hampshire are still in contention, I think there’s little doubt that these two states will determine the election. Romney probably has to win both to win.


First, Ohio, where we’ve long known the election could either be won or lost. The current PollTracker Average gives Obama a 3.2 percentage point lead (when I finally published this post it was at 2.8 and will likely bounce more as new polls come in). That’s not a huge lead. But there’s probably never been a state polled this intensively. Since October 15th, no fewer than 18 polls have been released for Ohio, only one of them has shown Romney in the lead, by 1 point. And that was a low quality robopoll put out by Rasmussen subsidiary Pulse Opinion. Also notable: as Tom Kludt explained on Friday, there’s a significant disparity between polls that do and do not place calls to cell phones. Those which do show significantly larger leads for President Obama.

TPM Reader GD wrote in this afternoon asking how it is Obama supporters can be so sure Obama will take Ohio when John Kerry was also ahead in Ohio going into the 2004 election only to see George W. Bush win. My answer: I wouldn’t be sure of anything. And I really mean that. However, Obama’s lead is significantly different than Kerry’s because it’s bigger. Nate Silver put together a helpful chart yesterday looking at how 10 day out poll leads tend to pan out on election day. The upshot is that within 1 percentage point, they don’t seem to have much predictive value. (Kerry’s was .8.) But once you get over about 1.3 percentage points, they’re seldom wrong — when the average is based on decent number of polls. If history is any guide, a 3.3 advantage has a strong likelihood of holding up.

The PollTracker trend chart tells the story pretty clearly. President Obama opened up a substantial lead in September. It narrowed a lot after the first debate. But Romney never closed the gap. Indeed, even in individual polls Romney never managed more than a handful of polls out of a few dozen with even a 1 point lead. Since mid-October the race has basically been stable with Obama slightly expanding his lead.


Virginia at this point looks like a toss up with an advantage to President Obama.

The current PollTracker Average gives Obama a two point lead in the state. But this data is significantly impacted by partisan and/or low quality polls on both sides. The key data point is the Washington Post poll released two days ago which showed President Obama holding a 4 point lead in the state. Also worth noting is the Purple Strategies poll from the day before which showed a tie race at 47%. Purple is a quality poll whose results have leaned somewhat to the GOP through this cycle (note: this does not mean bias, it simply means they’ve been slightly more GOP-friendly in their results than the mean.)

Taking all the polls together, as you can see in the trend chart, Romney’s support grows significantly beginning just before the Denver debate but then crests and begins to fall in the second half of October. At the moment, this looks like at best a tie race for Romney and quite possibly a small but growing margin for Obama. Romney almost certainly has to win it. For Obama it’s a possible fall back route to victory if Ohio eludes his grasp.


So do the polls take early voting into account? Probably.

The answer is yes, pollsters do take early voting into account. If you’re in a state where there’s early voting most pollsters will have this as a question. Have you already voted and if so, for whom did you vote. (Obviously someone who’s already voted is a certain voter, a likely-plus, as it were.) So in theory and probably in practice the topline numbers in the polls should entirely account for early voting and you shouldn’t have to look at those breakdowns of early voters or get excited if your candidate is way, way ahead among early voters.

The problem is that it’s not clear to me that pollsters’ methodologies have fully wrapped their figurative heads around the phenomena of early voting. 2008 is really the only good data we have and early voting has grown substantially since then, despite Republican efforts to reduce or eliminate it wherever possible. So we don’t have a lot of good empirical data and the whole thing is a moving target.

My own suspicion or perhaps simply a question is whether the polling models and ‘screens’ are fully able to account for the fact that ‘already’ voters are more than ‘likely’ voters and particularly whether they’re able to model those early voters against relevant demographic groups since early voting seems to be disproportionately a minority and youth phenomenon. That’s merely the curiosity/suspicion of a non-statistician, non-pollster. So don’t read too much into it. I’ll be curious to hear what pollsters make of this question.


So, what does it all mean?

I’ll start by stipulating to the flurry of sometimes contradictory data and that fact that anything can happen. But certain things are considerably more likely to happen than others. The clearest way to view the race at present is that President Obama has measurable leads in just enough states to win the electoral college. Meanwhile, Mitt Romney has, among the remaining swing states, a measurable lead in North Carolina and possibly a very tiny one in Florida. That leaves Virginia and Colorado, where Obama has tiny leads — the same size of larger than Romney’s in Florida.

Romney has to win North Carolina, Florida, Virginia and Colorado and then pick off at least one state where President Obama has a lead to get to 270 electoral votes.

By those metrics, a lot has to fall into place to get Romney an electoral college victory. This reasoning is mirrored in this analysis piece out tonight from AP, Nate Silver’s statistical model and commentary, most other sober analyses I’ve seen and most public betting sites.

The one piece of discordant data is the national poll averages which seem to show it either a dead heat or a very small advantage to Romney.

There are three possibilities. 1) The state numbers are off. 2) The national numbers are off. 3) We’ll have different winners for the popular and electoral college votes. History tells me three is unlikely. It’s also notable that the national numbers are much closer than the electoral numbers. I’m basically persuaded by the argument that the state by state numbers are usually of a higher quality and have more data behind them. But at the end of the day this one is just a bit of a mystery and one I dont — just speaking for myself — find entirely resolvable.

On balance, barring major changes — and the polls seem pretty stable at the moment — President Obama has a much more plausible route (actually a handful of routes) to 270 than Gov. Romney.

Watch Virginia and Ohio. They’ll tell the story.