Polling Note

Start your day with TPM.
Sign up for the Morning Memo newsletter

Emerson came out with a series of polls today in Ohio, Pennsylvania and Michigan – with Senate polls in addition to presidential polls in the first two states. All of the results show numbers substantially more friendly to Republicans, either the given Senate candidates or to Trump. Now, that could conceivably be the leading edge of a trend or simply an outlier. It’s always important not to start hyper-examining the details of poll that either surprises you or has results you don’t like. It’s very easy to end up fooling yourself. But here there’s a specific reason of caution.

Cell phone use has become increasingly ubiquitous over the years. 2012 was probably the last cycle when a pollster could get away with calling landlines only. As most of you likely realize, it’s not just that cell phone use is so common today. It’s that it skews heavily toward the young and to a lesser degree along racial lines – particularly those who only have a cell phone. If it were evenly distributed over the population it wouldn’t be as important. But it’s not.

If you don’t use cell phones, you’re likely missing a lot of people in demographics who heavily favor Democrats.

It turns out this Emerson poll is landline only. The exact language from their description is: “Data was collected using an Interactive Voice Response (IVR) system of landlines only.” IVR means a robopoll. That’s not at all the norm these days. I can’t say precisely how uncommon it is. But I think it’s fair to say that at least some cell phone calling is the norm, certainly for the more quality polls.

I should note that this doesn’t necessarily mean that this poll includes only older people. A pollster can weight results by demographic group to more closely approximate the results for demographic groups who tend to be excluded by not calling cell numbers. But there’s a reason pollsters now call cell phones. That’s just not the best way to do it. It’s a workaround at best.

Because PollTracker’s methodology has a strong bias toward inclusion, by design, we have included the Emerson poll in our averages. But for those keeping score at home I think there are pretty good reasons to think this poll is oversampling Republicans.

Late Update: A bit more poking around reveals this

Our first step in weighting is to survey more than enough people. This allows us to then be able to systematically reject individual surveys from demographics that are over represented. Next, survey data is weighted with a 3 point decrease in Conservative opinion and a 3 point increase in Liberal opinion to offset the bias in land line only telephone polls.

I don’t want to be mean. But this seems like a very clumsy way of weighting to avoid the skew created by landline only. We have decided to continue to include the poll because of the bias to inclusion. But it puts the bias toward inclusion to a real test.

Latest Editors' Blog
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Editor at Large:
General Counsel:
Publisher:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: