Let me start this post with a bundle of qualifiers. The NYT’s Upshot is a great new data/explainer journalism project. I’m jealous (but don’t count us out in this game yet.) I’m a big fan of Nate Cohn. And I think the point of his article here – that the efflorescence of partisan polls is creating a serious challenge for poll aggregation – is right on the mark. It’s actually been something we’ve been trying to think through in our own approach to poll aggregation and averaging at PollTracker. But let me show you this one collection of data about the Arkansas Senate race because I think something doesn’t quite add up.
Here’s the data set in the article illustrating the fact that who’s ahead depends a lot on whether you’re looking at partisan polls from each party of non-partisan polls. As Cohn describes it, “The possibility that partisan polls could skew the averages is perhaps best illustrated in Arkansas, where partisan polls are breaking along predictably partisan lines.”
But wait a second, let’s look more closely. The Democratic polls and the non-partisan polls are nearly identical. And actually, you only get that number by including two polls from Rasmussen. They may be a bit less problematic since Scott Rasmussen left the company. But by any reasonable measure they count as a partisan pollster. If you remove Rasmussen from the non-partisan set the non-partisan number balloons to Pryor +8.
So, without putting too find a point on it, the Dem-leaning ‘partisan’ polls seem right in line with the non-partisan polls. They actually seem to trail them in showing the extent of Pryor’s lead. It’s really Dem-leaning polls and non-partisan polls on one side and Republican polls on the other.
Not every race is going to be like this certainly. And just eye-balling it, it’s a little hard for me believe he’s quite as far ahead as NBC/Marist seems to show. But still. This data set doesn’t really capture what it purports to show and I think the pattern – that this is more an issue with Republican pollsters than Democratic ones is very real.
