Opinions, Context & Ideas from the TPM Editors TPM Editor's Blog

Those Internal Polls

C2lxbpq1vawpggpqdshh
Newscom

During a campaign we tend to think of released internal polls as crap. And the real thing is the public polls. But of course, it's only the released ones that it's right to be skeptical of. The internal polling itself is probably quite good. Because no one has a bigger interest in accuracy than the campaign itself. And the campaign's pollsters get to work with data provided by the campaign. So they should know much more about the contours of the race than some national pollster that drops into a state every few weeks while they're conducting polls in all sorts of other places.

In any case, here's the story. Neither side's internal polls in Nevada were showing what the public polls were showing. And they weren't far off: Neck and neck but with at least some advantage to Reid. Both sides internal polling apparently showed results that were pretty consistent with the final result.

So why the difference? The difference seems to be much more aggressive modeling, call-back persistence, cell phone calling and a lot of other stuff that's really just old school, phone survey due diligence. Who does or doesn't answer the phone obviously throws a huge wild card into the data. And the only way to adjust for that -- or the key way -- is to have really good demographic models of the electorate. That allows you to 'control', at least to a degree, for variable response rates as well as just random noise.

During the campaign frequently you'll see people dig into the internals of a poll and come up with something like, "Wow, this Nevada senate race has Hispanics voters splitting 50/50 between Reid and Angle. No way that's true. There's something wrong with poll."

Often in cases like that you're talking about a relatively small poll -- say 500 voters. And the demographic group is a much smaller subgrouping. So you're getting down to a sample size where the margin of error is really, really big. You can't necessarily read that much into it. In other cases, we figure that with so much polling these variations sort of come out with the wash. And if all the polls are the same big picture you cannot ignore that.

But in the case of Nevada at least it looks like the public pollsters really were missing a number of younger voters, Hispanic voters and just voters in general who, for a number of reasons, were much harder to reach. And those voters heavily favored Harry Reid. In theory, a pollster can still overcome some of that sample bias by controlling and 'weighting' for the different underlying demographic groups. But it looks like a lot of the public pollsters just didn't do that as consistently or as aggressively as they should have.

I haven't had a chance yet to look over the race results against the polling data systematically across the country. There were at least a few cases of polls significantly underestimating Democratic strength. The Pennsylvania senate race is the one that comes to mind to me.

If the Reid result had been replicated across the country, it would be a devastating verdict on the state of public polling today. But it didn't. By and large, looking across the entire country, the verdict has to be that the public polls got it pretty much right. A bit underestimating Democratic strength on the Senate side and a bit overestimating it on the House side. Overall, what the polls told us to expect is what we got. But perhaps it points to growing problems cropping up on the margins with all the different forces conspiring against accurate public opinion data -- pervasive cell phone use, discount robopolling and all the rest of it.