Nate Silver Calls Competing Election Forecaster’s Model ‘Wrong’

Nate Silver holds his phone as he sits on the stairs with his laptop computer at a hotel in Chicago on Friday, Nov. 9, 2012. The 34-year-old statistician, unabashed numbers geek, author and creator of the much-read F... Nate Silver holds his phone as he sits on the stairs with his laptop computer at a hotel in Chicago on Friday, Nov. 9, 2012. The 34-year-old statistician, unabashed numbers geek, author and creator of the much-read FiveThirtyEight blog at The New York Times, correctly predicted the presidential winner in all 50 states, and almost all the Senate races. (AP Photo/Nam Y. Huh) MORE LESS
Start your day with TPM.
Sign up for the Morning Memo newsletter

This post has been updated.

Election forecaster Nate Silver did something unusual on Wednesday: He openly criticized another election forecaster’s modeling.

The target of his critique was Princeton University’s Sam Wang, whose 2014 forecasts have been published by The New Yorker. Silver took the unusual step, he wrote at FiveThirtyEight, because of the disparity between his model and Wang’s. Silver’s most recent forecast Tuesday gave the Republicans a 53 percent chance of taking over the Senate. Wang’s Wednesday forecast shows Democrats with a 70 percent chance of keeping it.

“I don’t like to call out other forecasters by name unless I have something positive to say about them — and we think most of the other models out there are pretty great,” Silver wrote. “But one is in so much perceived disagreement with FiveThirtyEight’s that it requires some attention. That’s the model put together by Sam Wang.”

“That model is wrong,” Silver said.

Silver took issue with the Princeton model’s methodology and weighing of polling averages.

“It substantially underestimates the uncertainty associated with polling averages and thereby overestimates the win probabilities for candidates with small leads in the polls,” he said. “This is because instead of estimating the uncertainty empirically — that is, by looking at how accurate polls or polling averages have been in the past — Wang makes several assumptions about how polls behave that don’t check out against the data.”

He cited specific examples like the 2010 Nevada Senate race. Wang’s model gave Senate Majority Leader Harry Reid (D-NV) only a 30,000-to-1 chance of keeping his seat against Republican candidate Sharron Angle, Silver said, while FiveThirtyEight by contrast afforded Reid 5-to-1 odds.

Both were wrong, he acknowledged, but “those are very different forecasts.”

Wang indirectly responded to Silver’s critique in a post of his own on Wednesday, writing that there was a difference between his model’s Senate poll snapshot (which gave Democrats an 80 percent on Wednesday) and its forecast which allowed for some uncertainty between now and November (and in turn put the Democratic odds at 70 percent).

In an email to TPM, Wang said: “I do not want to turn this into a shouting match — it’s really unnecessary.”

But he said that he would distinguish between models like his (which he said uses only polling data) and those like Silver’s (which he said also factor in other electoral conditions).

“I think the real point is that (the Princeton model) is doing a polls-only calculation, as are HuffPost,, DailyKos Poll Explorer, and TPM. If one adds assumptions about where the Senate race ought to be, then a somewhat different outcome results,” Wang said. “Basically, several major models (NYT, 538) have said that at the start of 2014, conditions favored the GOP. However, for most of the year, polls have shown that Republicans are slightly underperforming, relative to those expectations.”

“In my view, it is helpful for readers to be able to separate what the polls say, and where data pundits say the polls ought to be (“fundamentals”),” he said.

Latest Livewire

Notable Replies

  1. I guess we’ll know who’s right and wrong in 6 weeks.

  2. Alt headline:
    Nate Silver Critiques Wang

  3. Avatar for mames5 mames5 says:

    Wasn’t Sam Wang more accurate on the Senate races last time around than Silver?

  4. Avatar for storm storm says:

    No. Can’t know right or wrong by looking at a model correlation to one election result.

  5. True. I thought about that right after I posted it.

Continue the discussion at

97 more replies


Avatar for bdtex Avatar for kenneth_thomas Avatar for slbinva Avatar for austin_dave Avatar for littlegirlblue Avatar for CuriousOne Avatar for ncsteve Avatar for matthew1961 Avatar for mames5 Avatar for commiedearest Avatar for xyxox Avatar for teenlaqueefa Avatar for musicman495 Avatar for krush Avatar for storm Avatar for daveyjones64 Avatar for ottnott Avatar for 538liberal Avatar for captaincommonsense Avatar for unclechris Avatar for jp1064 Avatar for vlharpley Avatar for occamsrazor2 Avatar for meta

Continue Discussion
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Editor at Large:
General Counsel:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: