Nate Silver Calls Competing Election Forecaster’s Model ‘Wrong’

This post has been updated.

Election forecaster Nate Silver did something unusual on Wednesday: He openly criticized another election forecaster’s modeling.

The target of his critique was Princeton University’s Sam Wang, whose 2014 forecasts have been published by The New Yorker. Silver took the unusual step, he wrote at FiveThirtyEight, because of the disparity between his model and Wang’s. Silver’s most recent forecast Tuesday gave the Republicans a 53 percent chance of taking over the Senate. Wang’s Wednesday forecast shows Democrats with a 70 percent chance of keeping it.

“I don’t like to call out other forecasters by name unless I have something positive to say about them — and we think most of the other models out there are pretty great,” Silver wrote. “But one is in so much perceived disagreement with FiveThirtyEight’s that it requires some attention. That’s the model put together by Sam Wang.”

“That model is wrong,” Silver said.

Silver took issue with the Princeton model’s methodology and weighing of polling averages.

“It substantially underestimates the uncertainty associated with polling averages and thereby overestimates the win probabilities for candidates with small leads in the polls,” he said. “This is because instead of estimating the uncertainty empirically — that is, by looking at how accurate polls or polling averages have been in the past — Wang makes several assumptions about how polls behave that don’t check out against the data.”

He cited specific examples like the 2010 Nevada Senate race. Wang’s model gave Senate Majority Leader Harry Reid (D-NV) only a 30,000-to-1 chance of keeping his seat against Republican candidate Sharron Angle, Silver said, while FiveThirtyEight by contrast afforded Reid 5-to-1 odds.

Both were wrong, he acknowledged, but “those are very different forecasts.”

Wang indirectly responded to Silver’s critique in a post of his own on Wednesday, writing that there was a difference between his model’s Senate poll snapshot (which gave Democrats an 80 percent on Wednesday) and its forecast which allowed for some uncertainty between now and November (and in turn put the Democratic odds at 70 percent).

In an email to TPM, Wang said: “I do not want to turn this into a shouting match — it’s really unnecessary.”

But he said that he would distinguish between models like his (which he said uses only polling data) and those like Silver’s (which he said also factor in other electoral conditions).

“I think the real point is that (the Princeton model) is doing a polls-only calculation, as are HuffPost, Electoral-Vote.com, DailyKos Poll Explorer, and TPM. If one adds assumptions about where the Senate race ought to be, then a somewhat different outcome results,” Wang said. “Basically, several major models (NYT, 538) have said that at the start of 2014, conditions favored the GOP. However, for most of the year, polls have shown that Republicans are slightly underperforming, relative to those expectations.”

“In my view, it is helpful for readers to be able to separate what the polls say, and where data pundits say the polls ought to be (“fundamentals”),” he said.