The Washington Post’s Dana Milbank took aim at “Big Data” earlier this week in a column that seemed to rehash much of the same skepticism that was directed at the likes of Nate Silver back in 2012.
But he insisted in an email to TPM on Thursday that he’s not becoming a poll truther.
Milbank opened his column on Tuesday with some gentle mockery toward the idea that “[e]ven close races can now be predicted with mathematical precision.”
“Throw all of these into our election model, add eye of newt and toe of frog, stir counterclockwise and — voila! — we can project with 84 percent confidence that Republicans will control the Senate next year,” Milbank wrote.
He quoted “old-school forecaster” Charlie Cook, who said that the models go wrong when they assume “a precision and confidence that doesn’t exist in human behavior.”
And although he acknowledged that some of the forecasters “have good records, and the theory behind them is sound,” Milbank suggested that “there can be too much of a good thing — as when media outlets put too much faith in the numbers the models churn out.”
Modelers like to think Big Data can revolutionize election coverage the way “Moneyball” changed baseball recruitment. And some editors see it as an alternative to poll-driven coverage of politics. But models rely heavily (in some cases, exclusively) on polls and are subject to the garbage-in, garbage-out rule. Models also rely on historical patterns, and in politics, past performance is no guarantee of future results.
At first blush, Milbank’s piece had echoes of the doubts levied at Silver in the lead-up to the 2012 election, when pundits like David Brooks and Joe Scarborough were dismissive of the polling guru’s forecast that President Obama was poised for a second term.
“If you tell me you think you can quantify an event that is about to happen that you don`t expect, like the 47 percent comment or a debate performance, I think you think you are a wizard,” Brooks said during an October 2012 appearance on PBS. “That’s not possible. The pollsters tell us what`s happening now. When they start projecting, they`re getting into silly land.”
Brooks made the same point in his New York Times column the same month.
“Look, I know in the cool light of rationality how I should treat polling data. First, I should treat polls as a fuzzy snapshot of a moment in time. I should not read them, and think I understand the future,” he wrote. “If there’s one thing we know, it’s that even experts with fancy computer models are terrible at predicting human behavior. Financial firms with zillions of dollars have spent decades trying to create models that will help them pick stocks, and they have gloriously failed.”
Scarborough was even more vocal in his criticism.
“Anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes,” the MSNBC host and former Republican congressman said days before voters went to the polls.
Silver, of course, was vindicated by the results that year, forcing Scarborough to offer a half-hearted apology after the election.
Is Milbank setting himself for a similar misstep? In an email to TPM, the columnist made it clear that he’s not being dismissive of the forecasters.
“My objection isn’t to the models but to their decimal-point statistical certainty,” Milbank wrote in the email, echoing a sentiment expressed by other pundits.
Unlike the Scarborough’s skepticism from two years, Milbank said he believes what virtually every forecaster is pointing to this time around; he just doesn’t care for the decimal probability.
“In other words, I’m inclined to believe the models are right: Republicans will take the Senate, Hagan wins, McConnell wins, Landrieu loses,” Milbank wrote. “The polls, on which the models are based, suggest the same. But I find that putting statistical precision on it — whether 59% or 98% — is a gimmick.”
Silver was able to call the 2008 and 2012 presidential elections with almost perfect precision, because there was a lot of good polling data from the swing states.
That’s not going to work in a congressional election. There is not a lot of good polling in the Alaska senate race. I’m getting five calls a day from campaigns and “surveys.”
Most of the time, I just hang up on them.
To a certain extent, Milbank and other critics of Silver are correct. The data that Silver and other statisticians create is not absolute. This does not mean that they are necessarily wrong, but they create a narrative that can be wrong or moderately wrong.
I don’t think Milbank is totally wrong. The truth is that Silver and Wang’s feats in 2008 and 2012 were less impressive than their boosters and they themselves would have it. Sure, 49/50 states called correctly sounds fantastic, but at least 40 of those were not in doubt. In fact, in both cases, only a few states were really in doubt (NC, IN and MO in 2008; FL in 2012).
As for Moneyball, very nice, but the Oakland As won 6 pennants from 1972-1990 and 0 in the Billy Bean era.
Um, why should anyone care what Dana Millbank thinks? Isn’t he just a typical beltway media pundit? Someone help me out here.
Milbank better watch himself. His boss is Jeff Bezos, who doesn’t make a trip to the bathroom before first consulting a statistical model. The Washington Post is, and likely will remain, the home to all manner of right-wing foolish opinions. But with the new boss there the quickest way to get fired there will be to question math.
I’m not kidding about Milbank needing to watch himself. If Milbank keeps arguing that the future can’t be predicted with math I can confidently make another prediction: he won’t last the year at the Post. I worked for a few years at Amazon where I had many meetings with Bezos. Those who don’t know their businesses down to the 100th decimal point don’t last long with him.