All ‘Brexit’ Polls Are Wrong But Some Are More Wrong Than Others

Analog beats digital when it comes to Britain’s EU referendum polls, study finds.

How Is 'Brexit' Risk Being Priced Into the Market?

When it comes to predicting the result of Britain’s European Union membership referendum, the telephone surveys showing a lead for the “Remain” side are probably closer to the mark, despite their flaws, according to a study.

Polls conducted online that suggest the race is too close to call and those done by phone are all missing important data, but the Internet polls are out by a wider margin, according to Matt Singh, co-author of the report released Tuesday, who called last year’s general election correctly on his Number Cruncher Politics blog.

QuickTake Perils of Polling

According to Singh, the “blind spot” is a failure to weight the results according to the social attitudes of interviewees. At present, companies usually weight their samples for age, gender and how people voted in the past.

Matt Singh, a former trader who set himself up as a political blogger, warns of potential polling blind spots.
Matt Singh, a former trader who set himself up as a political blogger, warns of potential polling blind spots.
Photographer: Matthew Lloyd/Bloomberg

“The online samples have got too many socially conservative people, and the phone polls have got too many socially liberal people,” Singh said in an interview. “On the referendum, the true picture is closer to the phone polls, but they’re both wrong.”

The June 23 vote on a so-called Brexit is a worry for Britain’s polling companies, who were humiliated by their failure to predict the election result. With different polling methods yielding different results for the public attitude to the EU, they face the prospect of a second disaster.

In 2015, the different mistakes of different polling methods meant they were all wrong in the same direction. In the EU referendum, the effect of their errors is pushing them in different directions.

Skewed Samples

Ahead of the election, “the phone polls had too many Labour voters, but the Labour voters they had were socially liberal,” Singh said. This meant they missed the flow of Labour voters to the anti-immigration U.K. Independence Party. Online polls meanwhile missed out on socially liberal Conservative voters, so they over-estimated the flow of Tories to UKIP. Both ended up overstating Labour support and understating Conservative votes.

Working with polling by Populus Ltd., Singh and his co-author James Kanagasooriam, head of analytics for the company, estimate that online polls lean too far toward a “Leave” vote by about 3 percentage points because of the bias in their sampling. Phone polls overstate support for “Remain” by about 5 percentage points but a second factor then helps correct that error: phone polling discourages people from choosing “Don’t know.”

Source: Number Cruncher Politics

Online polls make it easier for people to say they haven’t decided how they’re going to vote than phone polling does. When these voters are pushed, they tend to break in favor of the status quo. This effect adds another 5 percentage points to the “Leave” vote in online polls that isn’t reflected in the electorate as a whole, according to the study.

The last three telephone polls have shown the “Remain” campaign with a lead of between six and 11 percentage points.

The official inquiry into what went wrong with last year’s polls will report on Thursday. Its interim results, published in January, found the companies had been talking to the wrong voters.

Before it's here, it's on the Bloomberg Terminal. LEARN MORE