Ann Selzer always worries. She did so again on Monday.
The Iowa Poll that she conducts for the Des Moines Register came out last weekend showing Joni Ernst, the Republican nominee for Senate, leading her Democratic opponent by 7 percentage points.
The lead was significantly larger than those listed in other public polls measuring the Iowa Senate contest. Braley's campaign issued a memo calling the Register's poll an "outlier." Then, Quinnipiac University released a poll the day before the election that showed a dead heat. Who was right? Both the Iowa Poll and Quinnipiac have good reputations for being methodologically sound and correct.
Increasingly, it seems everybody's got a poll. Universities, media outlets, special interests, and consultants are all doing them and, in the Internet era, they make for easy coverage of a political race and data-point for a campaign trying to lure donors and supporters. All that polling traffic, though, has a downside: they're getting harder to do and, if wrong, can incorrectly sway the narrative of a race. That is one of the takeaways of the 2014 midterms.
"It happens more often than you might think that your poll doesn't agree with others," said Selzer, who also does polling work for Bloomberg Politics and Bloomberg News. "If you're in this business, there are times when you will be golden and there will be times when you are a goat."
As it turned out, Selzer was golden this time. Ernst beat Bruce Braley by 8.5 percentage points, one point higher that the Iowa Poll predicted.
Peter Brown, assistant director of the Quinnipiac poll, said talking about methodology was not his area of expertise. Douglas Schwartz, the poll's director, didn't respond to an email seeking comment.
In Iowa and other states, the run-up to Tuesday's midterm election wasn't the finest hour for public opinion polling. In short, many polls were wrong.
A CNN/ORC International poll conducted Oct. 27-30 on the North Carolina Senate race showed Democratic incumbent Kay Hagan leading Republican state House Speaker Thom Tillis by 2 percentage points. Tillis wound up winning by two points. In Virginia's Senate race, a Christopher Newport University poll taken Oct. 23-29 showed Democratic Senator Mark Warner beating Republican challenger Ed Gillespie, 51 percent to 44 percent. The actual election was much more of a nail-biter for both sides, with Warner ahead by about 1 percentage point as the vote counts are reviewed by election officials.
A NBC News/Marist poll in Kentucky taken Oct. 27-30 showed Republican Senator Mitch McConnell leading Democrat Alison Lundergan Grimes by 9 points. He won by 15.
In Georgia, polling in Georgia was closer to reality. A NBC News/Marist poll taken Oct. 27-30 showed Republican David Perdue getting 48 percent and Democrat Michelle Nunn receiving 44 percent. Perdue won with 53 percent of the vote.
Wisconsin became a bright spot for at least one polling outfit, which scored Governor Scott Walker's re-election bid against Democratic Mary Burke close to dead on. The final Marquette Law School Poll had Walker at 50 percent and Burke at 43 percent. Walker won with 52 percent of the vote.
Because some of the underlying polls were significantly off, it affected the statistical modeling and prediction that has become popular in recent years. The New York Times' Upshot, for example, gave Hagan a 71 percent chance of winning in its final update before the election.
Two reasons why things went so wrong are money and the competitive nature of the midterms. A map packed with seemingly tight Senate elections in small-population states meant that a narrow slice of the electorate was getting bombarded with political messages–and phone calls from pollsters.
That made it harder for survey researchers to corral enough people on the line for a representative sample.
“It's becoming a much more difficult, nerve-wracking business,” Geoff Garin, the president of Hart Research Associates and a leading Democratic pollster, told Bloomberg reporters and editors before the election. “The willingness of respondents to participate in polls has declined, the move to cell phones has had an impact.”
After the 2012 election, the Gallup Poll reviewed its methodology of selecting likely voters after it missed the presidential election, predicting in its final pre-election poll that Mitt Romney would beat President Barack Obama, 49 percent to 48 percent. Obama won the election by four percentage points. “When the next presidential election rolls around, we think we’ll certainly be in a position at the accurate end of the spectrum,” Frank Newport, Gallup’s editor-in-chief, said in 2013.
Differences among polls often have to do with how the surveys define likely voters, Selzer said. "In my world, the less I make a judgment about what the electorate will look like, the better we have done," she said. "We don't play that game."
With the exception of a few years during the mid-1990s, Selzer has worked with the Register since 1987. During that time, she said, accurate polling has gotten harder for a variety of reasons. "I think we are potentially on the verge of turning off anyone," she said. "You can see it on the horizon."