Nate Silver — author of “The Signal and the Noise: Why So Many Predictions Fail–But Some Don’t” and polling guru for the New York Times’ FiveThirtyEight blog– recently spoke to TPM about the state of the election, what makes a good forecaster, and the experience of writing his first book.
Which pollsters do you think have most accurately captured the state of the race to this point?
I think pollsters can do more things that are wrong than right. I’ve become more of a traditionalist when it comes to polling, in part because we’ve seen more of a split between what some of the more traditionally thorough polls say and what some of the automated and robopolls say. If you get numbers published by NBC and the Wall Street Journal or The Washington Post and ABC News, those are pretty good. Even within the range of the high quality pollsters, you have a spread between basically a tie and a rather overwhelming, 2008-style Obama lead. That’s kind of normal. If you go back through history and see what the polling spread is like at any given time, you often will see a 5- or 7- or even 10-point gap between one poll and another.
Has Obama been the favorite for longer than journalists and pundits have been writing and saying he is?
For some reason, the conventional wisdom went from, “Oh, it’s a tossup” to “Oh, Obama’s way ahead or a clear favorite.” I tend to see it more in shades of distinction. When we launched our forecast in June, we had him as a 60-40 favorite. The race didn’t move all that far from that number. One thing people maybe don’t grasp as a very basic thing is the more time runs off the clock, the more holding a narrow advantage is likely to hold up. If you were making an assessment of what Obama’s chances were in the spring, then you have to worry about a blowup in Europe, you have to worry about something happening in the Middle East — maybe it still will in the Middle East. Maybe instead of having merely mediocre jobs numbers, actually going back into a recession. There are all these shoes that can drop on you and they haven’t.
You were seeing the same range of about a 2-, 2.5-, 3-point lead very, very consistently until the conventions, really, where the first really bad signal for Romney was that he got very little convention bounce. And historically, if you don’t pull ahead in the polls after your own convention, then you don’t win. It looks like he maybe pulled into a tie, but probably not much more than that. That was bad piece of news No. 1. And then the theory was, maybe conventions don’t produce a bounce anymore. And then Obama got a decent sized one — not great, it’s actually fairly small by historical standards. But the fact is, it’s almost pure gain for him because Romney didn’t really eat into Obama’s numbers at all. If you go from having a 2-point lead to a 4-point lead, that’s one thing, and that’s pretty meaningful. Also the fact that conventions start in late August to now, a month has gone by, you’ve eaten up almost a third of the time remaining until the election. All of a sudden it’s kind of like, well Romney has a shot here, but we’re looking for him to really win the debates or October surprises or this convention bounce peters out. We’ve had Obama’s odds going from 60 percent to 70 percent over the summer, and then from 70 percent to closer to 80 percent now. I’m not sure what the exact threshold is, when’s it a tossup and when’s it likely Obama?
So you’re not ready to make a prediction yet, Romney or Obama winning?
Well, we have Obama with a 77 percent chance on the site. My personal prediction would be pretty close to that. Those would be the odds I would be willing to establish at a betting line. In the book I say we should try to make probabilistic forecasts and almost look at ourselves as being sort of bookmakers or gamblers. That’s the closest compromise between the uncertainties in the real world and our ability to perceive them. That’s as well as you can do. At this point it does look like we have a clear favorite, but we could be surprised.
What should people be looking for in the several weeks before the election?
People should be looking to see if the consensus of data moves. If there’s a meaningful shift, then it’s not that hard to detect. It wasn’t that hard to detect when Obama got his convention bounce. We’ll be looking to see if Obama’s convention bounce is sticking. This is about the last week when Romney can say, “Oh, the momentum’s gonna fade out.” You have three debates, you have two jobs reports, you have various news in the Middle East. It’s kind of more a question of Obama clearing those hurdles. There’s always a chance there’s some organic shift back toward Romney or, maybe also equally likely, is things could just kind of totally fall apart for Romney, where the wheels comes off and people lose faith in him and all of a sudden Romney is losing by 7 or 8 points. For the most part, we should assume whatever numbers Obama has about a week from now represent some sort of equilibrium in the race.
How long did it take to write the book?
Oh God. I signed the contract in November 2008, probably didn’t do very much in the first six months. I was working on it for a good three years, I would say.
How was it to write while still having a day job?
It’s very challenging, because they require different parts of your brain. The blogging part of your brain has to be very caffeinated and reactive and news-driven and conscientious of needing to get information out reasonably quickly. Whereas a book, you really need some space where you shut out distractions and are able to do some reflection and deeper thinking. You need to be well-rested, which I’m terrible at most of the time. On the one hand you’re trying to take these different subjects, and you might call them verticals — politics and poker and global warming and everything else — on the other hand, you’re trying to weave themes throughout the book so it’s not just a series of essays or an anthology that drives to a coherent point. That requires a lot of energy. On top of that you have to use narrative and structure and dramatic interpretation of things, and you also have to be accurate to a series of facts you’re presenting, some of which are new to you. It’s a lot to do, I think, to write a quote-unquote “real book” when you’re making an effort to step back and make a real argument and have some nuance and complexity in it. This last crunch in particular, from about February or March onward, has been just the busiest period of my life, professionally. I’m looking forward to sitting on a beach somewhere in December.
You write in the book about forecasters’ biases and the hurdles to making objective predictions. How can forecasters overcome that bias?
In the long run, the best test is: How do your predictions do? People are sometimes too slow to change course when they’re clearly doing something wrong. Politics is one of the more obvious examples of this, I suppose, where you have pundits like, say, Dick Morris, where they’ve made a lot of predictions and their track record is really terrible. I see prediction as the means by which we test our theories against reality, test are we being objective, are we in touch with reality? I mean that fairly literally. If you keep making these statements that are totally and completely wrong, that Barack Obama is going to win Tennessee, or that Donald Trump’s going to be a major force in the Republican race or that Republicans will win 100 seats in Congress, then you have to examine your world view, and say, “Am I remotely in touch with what’s really happening?” People don’t like to change, they don’t like to admit faults, they like to couch their predictions in a way.
That’s why at FiveThirtyEight, we’re very explicit about what we do. It’s easy to say, for example, Obama’s the favorite, but Romney still has a chance. Everyone can agree with that premise at this point. But how much of a chance does Romney have? What are the odds? The more you commit to something, the more blame you should take if you’re wrong and the more credit you should get if you’re correct. But people like to make predictions and we don’t like so much to go back after the fact and evaluate them. There ultimately is a kind of zen-like state that you must acquire as well, not in a literal, spiritual sense. But you have to be aware of the fact that, in some fields — like in politics, where there are only presidential elections once every four years — there are not gonna be enough elections in my lifetime to prove one way or the other, is the FiveThirtyEight method better? What I recommend people do is look toward cases where you can test predictions frequently, so something like a weather forecast that’s made every day, or something like sports bettors who are betting on NBA games almost every day. And look at what makes those people successful, and you adopt the habits and attitudes that they have. That’s the best way to know if you’re on the right track in the absence of having hundreds of predictions to test, which you can’t in a lot of fields.
This interview has been edited for clarity and space.
