Election Pollsters Face A Big Test In 2022 After Recent Polling Errors

ASBURY PARK, NJ - NOVEMBER 02: Supporters react as New Jersey Governor Phil Murphy speaks during an election night event at Grand Arcade at the Pavilion on November 2, 2021 in Asbury Park, New Jersey. New Jersey resi... ASBURY PARK, NJ - NOVEMBER 02: Supporters react as New Jersey Governor Phil Murphy speaks during an election night event at Grand Arcade at the Pavilion on November 2, 2021 in Asbury Park, New Jersey. New Jersey residents went to the polls Tuesday to vote in the gubernatorial race where Murphy faced off against Republican challenger Jack Ciattarelli. (Photo by Mark Makela/Getty Images) MORE LESS
Start your day with TPM.
Sign up for the Morning Memo newsletter

This article is part of TPM Cafe, TPM’s home for opinion and news analysis. It was originally published at The Conversation.

When it became clear his poll had erred in the 2021 New Jersey governor’s race, Patrick Murray, director of the Monmouth University Polling Institute, acknowledged:

“I blew it.”

The campaign’s final Monmouth poll estimated Gov. Phil Murphy’s lead over Republican foe Jack Ciattarelli at 11 percentage points — a margin that “did not provide an accurate picture of the state of the governor’s race,” Murray later said in a newspaper commentary. Murphy won by 3.2 points.

It was a refreshingly candid acknowledgment by an election pollster.

More broadly, the error was one of several in the recent past and looms among the disquieting omens confronting pollsters in the 2022 midterm elections. Will they be embarrassed again? Will their polls in high-profile U.S. Senate and gubernatorial races produce misleading indications of election outcomes?

Such questions are hardly far-fetched or irrelevant, given election polling’s tattered recent record. A few prominent survey organizations in recent years have given up on election polling, with no signs of returning.

Treat polls warily

It is important to keep in mind that polls are not always in error, a point noted in my 2020 book, “Lost in a Gallup: Polling Failure in U.S. Presidential Elections.” But polls have been wrong often enough over the years that they deserve to be treated warily and with skepticism.

For a reminder, one need look no further than New Jersey in 2021 or, more expansively, to the 2020 presidential election. The polls pointed to Democrat Joe Biden’s winning the presidency but underestimated popular support for President Donald Trump by nearly 4 percentage points overall.

That made for polling’s worst collective performance in a presidential campaign in 40 years, and post-election analyses were at a loss to explain the misfire. One theory was that Trump’s hostility to election surveys dissuaded supporters from answering pollsters’ questions.

In any case, polling troubles in 2020 were not confined to the presidential race: In several Senate and gubernatorial campaigns, polls also overstated support for Democratic candidates. Among the notable flubs was the U.S. Senate race in Maine, where polls signaled defeat for the Republican incumbent, Susan Collins. Not one survey in the weeks before the election placed Collins in the lead.

She won reelection by nearly 9 points.

Recalling the shock of 2016

The embarrassing outcomes of 2020 followed a stunning failure in 2016, when off-target polls in key Great Lakes states confounded expectations of Hillary Clinton’s election to the presidency. They largely failed to detect late-campaign shifts in support to Trump, who won a clear Electoral College victory despite losing the national popular vote.

Past performance is not always prologue in election surveys; polling failures are seldom alike. Even so, qualms about a misfire akin to those of the recent past have emerged during this campaign.

In September 2022, Nate Cohn, chief political analyst for The New York Times, cited the possibility of misleading polls in key races, writing that “the warning sign is flashing again: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.”

There has been some shifting in Senate polls since then, and surely there will be more before Nov. 8. In Wisconsin, for example, recent surveys suggest Republican incumbent Ron Johnson has opened a lead over Democratic challenger Mandela Barnes. Johnson’s advantage was estimated at 6 percentage points not long ago in a Marquette Law School Poll.

The spotlight on polling this election season is unsurprising, given that key Senate races — including those featuring flawed candidates in Pennsylvania and Georgia — will determine partisan control of the upper house of Congress.

Worth doing?

Polling is neither easy nor cheap if done well, and the field’s persistent troubles have even prompted the question whether election surveys are worth the bother.

Monmouth’s Murray spoke to that sentiment, stating: “If we cannot be certain that these polling misses are anomalies then we have a responsibility to consider whether releasing horse race numbers in close proximity to an election is making a positive or negative contribution to the political discourse.”

He noted that prominent survey organizations such as Pew Research and Gallup quit election polls several years ago to focus on issue-oriented survey research. “Perhaps,” Murray wrote, “that is a wise move.”

Questions about the value of election polling run through the history of survey research and never have been fully settled. Early pollsters such as George Gallup and Elmo Roper were at odds about such matters.

Gallup used to argue that election polls were acid tests, proxies for measuring the effectiveness of surveys of all types. Roper equated election polling to stunts like “tearing a telephone book in two” — impressive, but not all that consequential.

A screenshot of a story in the New York Times about polling mistakes in the US 2016 presidential election.
Pollsters in 2016 predicted Democratic presidential candidate Hillary Clinton would win some states that she actually lost. Screenshot, New York Times

Who is and isn’t responding

Experimentation, meanwhile, has swept the field, as contemporary pollsters seek new ways of reaching participants and gathering data.

Placing calls to landlines and cellphones — once polling’s gold standard methodology — is expensive and not always effective, as completion rates in such polls tend to hover in the low single digits. Many people ignore calls from numbers they do not recognize, or decline to participate when they do answer.

Some polling organizations have adopted a blend of survey techniques, an approach known as “methodological diversity.” CNN announced in 2021, for example, that it would include online interviews with phone-based samples in polls that it commissions. A blended approach, the cable network said, should allow “the researchers behind the CNN poll to have a better understanding of who is and who is not responding.”

During an online discussion last year, Scott Keeter of Pew Research said “methodological diversity is absolutely critical” for pollsters at a time when “cooperation is going down [and] distrust of institutions is going up. We need to figure out lots of ways to get at our subjects and to gather information from them.”

So what lies immediately ahead for election polling and the 2022 midterms?

Some polls of prominent races may well misfire. Such errors could even be eye-catching.

But will the news media continue to report frequently on polls in election cycles ahead?


After all, leading media outlets, both national and regional, have been survey contributors for years, conducting or commissioning — and publicizing — election polls of their own.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The Conversation
Latest Cafe

Notable Replies

  1. Avatar for bdtex bdtex says:

    I know I’m not paying much attention to polls lately. My guess is that they’re probably all over the map right now. I haven’t clicked on 538 or RCP for over a year now.

  2. Avatar for daled daled says:

    Polls have been wrong, but the key information for the article is missing: WHY are they wrong? There’s no discussion, no real analysis.

    So, too long and not much worth reading and could be just 1 paragraph:

    Polls have been wrong and nobody really knows why, so nobody has really figured out how to make them more accurate. Networks and Media will continue to publish “results,” but we recommend that if you care about the results of an election you ignore any stories based on polls and make sure you vote.

    Even shorter: Ignore polls and stories based on polls, and get your ballot in!

  3. The Kansas abortion polling showed my why they are no longer reliable.

    Before the election.

    Poll shows tight race on Kansas abortion amendment, with supporters narrowly ahead as Election Day nears

    Andrew Bahl | Topeka Capital-Journal | July 23rd 2022

    A new poll released Wednesday shows a close race in the passage of a proposed anti-abortion amendment to the Kansas Constitution…

    The actual results:

    Kansas voters defeat abortion amendment in unexpected landslide


    OVERLAND PARK — Kansas voters in a landslide Tuesday defeated a constitutional amendment that would have stripped residents of abortion rights, defying polling and political observers who expected a close result.

  4. Back in 1792, anyone who could vote pretty much did vote. Even if you were a farmer, a days ride away from the polling place, you could vote. [Apparently, Tuesday was chosen so you could go to church Sunday, and still make it to the polling place in time to vote]. And people probably weren’t inclined to lie about who they would vote for. If there had been any polling people in those days, they probably would have got things pretty accurate. Today, there are all sorts of obstacles in the way of voting, and people do lie. No surprise it is tough to figure out who will vote, and how.

  5. I imagine that the only folks still reliably participating in any kind of poll are the same guys who buttonhole the ladies who work the sample carts at CostCo and lecture them on why they shouldn’t have to wear a mask.

Continue the discussion at forums.talkingpointsmemo.com

14 more replies


Avatar for discobot Avatar for bdtex Avatar for daled Avatar for pluckyinky Avatar for jep07 Avatar for jkrogman Avatar for commanderogg Avatar for harrync Avatar for khyber900 Avatar for not_so_fluffy Avatar for grants Avatar for gatesfl Avatar for txlawyer Avatar for ladyfair Avatar for IBecameACitizenforthis Avatar for timbomov

Continue Discussion
Masthead Masthead
Founder & Editor-in-Chief:
Executive Editor:
Managing Editor:
Associate Editor:
Editor at Large:
General Counsel:
Head of Product:
Director of Technology:
Associate Publisher:
Front End Developer:
Senior Designer: