Gallup Reviews Poll Methods After 2012 Prediction SkewedJonathan D. Salant
Gallup, whose final 2012 election poll showed Republican challenger Mitt Romney leading President Barack Obama, said today that its likely-voters screen and too few interviews on the East and West coasts contributed to its failure to forecast the final vote accurately.
The Gallup poll commissioned a study of its methods after its last pre-election survey gave Romney a one-point lead, 49 percent to 48 percent, over Obama, who won re-election by almost four percentage points.
“That’s not where we wanted to be,” Frank Newport, Gallup editor-in-chief, told reporters at a briefing today in Washington.
The firm said it was reviewing its models, had already changed some of the methods it follows in conducting surveys, and would make other adjustments as needed.
Newport said that Gallup’s techniques were sound and that other polling organizations’ likely-voter models also leaned more toward Romney than the actual electorate, although not to the extent that Gallup’s did. Gallup’s survey of registered voters gave Obama a three-point lead, 49 percent to 46 percent.
“The basic procedures are sound, but they are significant enough that they made a significant difference in our overall estimate of who was going to win the presidential election last fall,” Newport said.
Gallup’s failure in 2012 was magnified as other polling organizations called the election correctly. Nate Silver, the computer expert who gave Obama a 90 percent chance of winning re-election, successfully called each of the 50 states as well as Obama’s final percentage of 51 percent. The Pew Research Center for the People & the Press and ABC News/Washington Post surveys put Obama up by three points.
In a report prepared with the University of Michigan, Gallup said the way that it determined likely voters, such as screening for those who have voted in previous elections and their interest in this one, tilted its sample toward Romney.
“What we’re doing is evaluating the whole process of estimating likely voters,” Newport said.
With fewer adults willing to participate in telephone polls, Newport said that the respondents may be self-skewed toward those more likely to turn out on Election Day, requiring a change in how the sample is weighted.
“It is possible that with higher refusal rates, Gallup and others are doing some screening already,”he said.
The firm said its samples skewed more to Midwestern and Southern voters rather than those on the East and West coasts, where Obama was strongest.
Gallup discovered that its telephone samples appeared to be tilted toward Republicans and raised questions about the way it weighted for respondents who have only mobile phones.
Its questioning to determine race and ethnicity of respondents also produced flawed information, Gallup said.