Gallup Admits Mistakes, Plans Changes After Dismal 2012

One of the best known polling firms in the nation plans next month to reveal big changes to the way it does business after falling flat during last year’s election.

Gallup’s editor-in-chief Frank Newport wrote in an email published Monday by Politico that “a blue ribbon group of outside experts” is conducting a review of the firm’s “methodological issues” during the 2012 election. The findings will be unveiled during an event on June 4 at Gallup’s Washington, D.C. office.

The move comes after an embarrassing stretch for Gallup in which the firm was widely panned for its 2012 election polls. Soon after the election, Fordham University put the firm near the bottom of its rankings for pollster accuracy. And more recently, MSNBC’s Joe Scarborough launched a blistering on-air attack last week against the firm.

Newport responded to the attack in the email to Politico.

“Our review includes a significant amount of new research, including fielding new research experiments focused on six specific areas of pre-election polling, along with simulations and re-examination of every component of the pre-election polling process,” Newport wrote. “All of this research, some of it focused on the future of election polling, is one reason why it is taking several months. A number of changes have already been implemented in our on-going tracking, and others will continue to be tested, including experimental tests involving voter sampling and likely voter models.”

Newport added in a phone call to Politico that Gallup’s changes will include a different method of asking respondents about their race and what he described as a different way of bringing “landline and cellphone users into the sample.”

About a month before last year’s election, Gallup adjusted the sample of its daily tracking poll to an even split between respondents reached by landline telephones and those reached by cell phones.

But that change did little to improve the accuracy of the organization’s results. After shifting to a narrower sample of likely voters, the Gallup tracking poll showed Mitt Romney with solid leads over President Barack Obama in the closing weeks of the campaign — although the final Gallup poll before Election Day showed the Republican nominee’s lead had fallen to only a point.

Weeks after the election, Newport wrote a post on the firm’s website in which he defended Gallup’s findings and appeared to take a thinly veiled shot at polling aggregators.

“Individual farmers can each make a perfectly rational decision to graze their cows on the town commons,” Gallup wrote. “But all of these rational decisions together mean that the commons becomes overgrazed and, in the end, there is no grass left for any cow to graze. Many individual rational decisions can end up in a collective mess.”

But its performance in the 2012 election left Gallup with no shortage of critics, including Obama’s 2012 campaign manager, Jim Messina. He said Gallup’s poor performance was nothing new.

“Gallup has been wrong repeatedly in presidential elections for a long time,” he said.

Dear Reader,

When we asked recently what makes TPM different from other outlets, readers cited factors like honesty, curiosity, transparency, and our vibrant community. They also pointed to our ability to report on important stories and trends long before they are picked up by mainstream outlets; our ability to contextualize information within the arc of history; and our focus on the real-world consequences of the news.

Our unique approach to reporting and presenting the news, however, wouldn’t be possible without our readers’ support. That’s not just marketing speak, it’s true: our work would literally not be possible without readers deciding to become members. Not only does member support account for more than 80% of TPM’s revenue, our members have helped us build an engaged and informed community. Many of our best stories were born from reader tips and valuable member feedback.

We do what other news outlets can’t or won’t do because our members’ support gives us real independence.

If you enjoy reading TPM and value what we do, become a member today.

Latest Dc
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Investigations Desk:
Director of Audience:
Editor at Large:
General Counsel:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: