Iowa Results Underscore Polling Industry Vulnerabilityby
Cruz win in Republican nominating contest contrary to surveys
Pollsters battered by recent misreads in Israel, Greece, U.K.
A disappointing night in Iowa for Donald Trump was just the latest setback to pollsters around the world.
The results in the state’s presidential caucuses on Monday showed Texas Senator Ted Cruz beating Trump in the Republican contest by 28 percent to 24 percent -- an outcome that was contrary to poll after poll that showed the billionaire winning.
In the Democratic contest, the virtual tie between Hillary Clinton and Bernie Sanders was uncomfortably close for both the former secretary of state and pollsters who showed her with a narrow but steady lead heading into the caucuses.
The results underscored the vulnerability of a polling industry that has been battered in recent months by inaccurate readings of public sentiment in votes from Israel, to Greece to Spain and the U.K. -- often by underestimating conservative turnout.
On Monday night, Iowans defied expectations by attending caucuses in unprecedented numbers to hand Cruz a momentum-shifting victory. Those misfires have raised doubts about the validity of polling at time of technological and social shifts even as political and business professionals increasingly rely on surveys and the data they collect.
Most pre-caucus polls showed Cruz within striking distance and a tight Clinton-Sanders race. In the final Bloomberg Politics/Des Moines Register Iowa Poll before the caucuses, conducted Jan. 26 through 29, Clinton held a 3 percentage point advantage over Sanders, while Trump led Cruz by 5 points. Both results on Monday night were within the margin of error.
The Cruz victory is just the second time since 1988 that the Iowa Poll, conducted by Selzer & Co., has incorrectly picked either the Democratic or Republican caucus winner in Iowa. The other exception was Republican Rick Santorum’s 34-vote victory in 2012 over Mitt Romney. Selzer didn’t conduct the poll in 1992.
So what happened?
For one, Cruz’s strategy appears to have paid off perfectly. J. Ann Selzer, president of Selzer & Co., said that while her final poll projected a victory for Trump, all of the variables that favored Cruz came together.
“We knew to look at evangelicals, and we tested it at 60 percent” of the Republican electorate, Selzer said. In the end, 64 percent of caucus-goers were evangelical or born again, according to a CNN entrance poll, giving Cruz important momentum.
“We always thought, and we certainly heard about, the ground game,” Selzer said. “You can be for Trump, but if you don’t show up, it doesn’t matter.”
The Bloomberg Politics/Des Moines Register Iowa Poll was in line with the average of eight polls compiled by RealClearPolitics, which showed Trump with a 4.7 point advantage heading into caucus day, and Clinton with a 4 point lead. The polls included in the average, though, showed the volatility and uncertainty of measuring voter intentions. The results ranged from an 11-point lead for Clinton in a Gravis Marketing survey to a 3-point edge for Sanders in a Quinnipiac University poll.
In many ways, Iowa epitomizes the uphill battle pollsters face in accounting for rapid social and technological changes.
Those shifts -- from the abandonment of landline phones to changing demographics -- have battered the polling industry globally with missed trends and sentiments. The degree of difficulty only increased in Iowa, where convoluted caucus structures, a history of low turnout and this year’s unorthodox candidates combined to inject confusion.
In caucus rooms, voters heard from neighbors and friends, raising the possibility of last-minute switches in loyalty for a candidate.
“These are people you know from the neighborhood, people you might respect, and that may sway your vote,” said Timothy Hagle, a professor of political science at the University of Iowa. “Those last minute changes make a difference, particularly if you have a really close race.”
Further complicating matters, turnout in Iowa is low compared to a primary or general election, increasing the odds of an off-base poll result. In 2012, 121,503 Republicans -- 19.7 percent of the state’s 614,913 registered GOP voters -- cast a ballot. In 2008, a banner year for participation thanks to enthusiastic support for then-Senator Barack Obama, just 23.3 percent of registered Democrats participated. Iowa Republicans surprised everybody Monday by turning out in record numbers -- but not overwhelmingly for Trump, the outsider candidate.
“People say on the phone, ‘Yeah, I’m going to turn out the caucus,’ but will they?” Hagle said. “Who knows if you’re going to get the right ones?”
Nate Silver, the polling specialist behind FiveThirtyEight, a statistical analysis website, said the miss in the Bloomberg Politics/Des Moines Register Iowa Poll of Republicans mirrored the average error in a primary or caucus poll, which are “really tough” to survey.
“It looks like there was some late-breaking movement toward Rubio and Cruz that her poll wasn’t in the field late enough to pick up,” Silver wrote in a blog post. “It won’t be a poll she brags about, and perhaps it’s an argument for keeping a tracking poll in the field until the very last day of campaigning. But all of this is fairly par for the course.”
Efforts to properly screen potential voters are made more now that voters can no longer be counted on to reliably answer their landlines, if they even have one. Response rates on mobile phones are notoriously low, driving up the costs of proper polling methodology.
Those trends have forced polling operations to fundamentally alter how they gather data, said Mike Traugott, a political scientist at the University of Michigan who has advised Gallup on its polling methods.
The Pew Research Center, for example, got around a quarter of its responses from mobile phone users during the 2004 election. But in 2016, the firm expects to use mobile phone respondents for 75 percent of its surveys, despite it costing twice as much to survey those users.
Pollsters like Selzer cross-reference poll respondents against registered voter lists in an effort to improve the quality of their responses. But that’s time-intensive and costs more money, and not all media organizations are willing to shoulder that financial burden.
Better methods “would require more time, and more money,” said Traugott , who has served as president of the World Association for Public Opinion Research. “The news organizations are not inclined to pay for that.”
The result has been a series of high-profile misfires.
In Greece, voters re-elected Syriza leader Alexis Tsipras by a margin that was far greater than polls predicted. That followed a Greek referendum in July over whether to accept bailout conditions by the European Commission and International Monetary Fund. Polls suggested voters might barely reject it, but the referendum was ultimately defeated by 22 points.
Israeli Prime Minister Benjamin Netanyahu and British Prime Minister David Cameron each won re-election last year despite late polls suggesting they were in trouble. And in 2012, Republican presidential candidate Romney didn’t draft a concession speech in part because his internal polls showed him leading Obama.
Pollsters were nervous entering Monday’s contest in Iowa because the Republican field featured elements that have tripped up past surveys: a large group of candidates and an unorthodox frontrunner whose support was largely rooted in voters who hadn’t previously participated in the caucuses.
Cruz’s surprise victory was actually in line with some past polling misses. Polls conducted by the Register in 1988, 1996, and 2012 all significantly underestimated the candidate who would earn the backing of evangelical voters -- the very group that appeared to have pushed the Texas senator to victory.
Polls are also often late to recognize insurgent candidacies -- like Santorum’s come-from-behind win in the final days of the 2012 caucus. A week ago, Rubio’s RealClearPolitics average was just 12.2 percentage points, before spiking to 16.9 percent hours before the caucuses, suggesting the Florida senator was primed for a surprise. He finished Monday with 23 percent -- trailing Trump by 1 percentage point, even though earlier polls had shown him behind the real estate mogul by double digits.
The Democratic side featured a smaller field, but similarly difficult-to-quantify variables. Young voters drove Sanders support, but the 18-to-34 year-old demographic is historically fickle when it comes to showing up to vote.
Monday’s results mean pollsters are likely in for a fresh round of criticism over the reliability of polls that could further erode confidence in their survey methods.
“Pollsters need models, and sometimes the models they use don’t work any longer,” Traugott said.