Pollsters Are Worse Than Ever
If you weren’t following the British election returns last night, you missed an amazing show. Virtually everyone went into the race expecting that the ruling Conservative Party would lose seats, and the biggest question was whether the Scottish National Party would cost Labour enough seats to make it impossible for any one party to form a government.
Well, we can all relax about the constitutional crisis. Astoundingly, the Conservatives actually gained seats, and now have enough to form a government without their former coalition partner, the Liberal Democrats. (And lucky for them, because the Lib Dems were absolutely crushed, dropping from 57 seats to eight.) Labour’s shellacking in Scotland was utter, worse than even the direst predictions. The party, which has dominated Scotland for decades, held 41 of 56 seats going into the election. When the returns were in, they held … one, the same number as the Conservatives do.
I won't opine on What It All Means. But let's talk about the surprise factor: The polls were wrong. And as Nate Silver points out, this seems to be a troubling trend, not just in Britain, but around the world. The polls on the Scottish independence referendum were way off. So were the ones that missed the Republican sweep in the 2014 midterms. Israel’s pre-election and exit polls both missed Likud’s solid victory.
We’ve always known that polls had problems. You can get very different answers depending on how you ask the question, as “Yes, Prime Minister” effectively dramatized. Sampling problems arise when people who don’t get chosen for the poll, or refuse to respond, are systematically different from the rest of the population. (This is how the infamous “Dewey Defeats Truman” headline happened.) Even with problems, however, polls remain useful — as long as you keep those problems in mind.
I’ve seen a lot written today about how this shows the need to fix polling. I’ve seen few people asking what seems like the more pertinent question: What if polls can’t be fixed?
The second half of the 20th century was the golden age of survey data. But toward the end of the century, changing technology began to threaten the accuracy of polls. We now have caller ID, voicemail, and millennials who regard talking on the phone as a barbarism akin to the chamber pot. The modern American workday also compresses housekeeping and socializing into a few narrow hours, during which people are less likely to humor an unsolicited caller. In part because we've also seen the proliferation of robocalls in the survey industry and beyond.
Pollsters say that by carefully calibrating for the missing responders, they can still get accurate results. But what if it’s getting too hard?
In some sense, that doesn’t matter all that much for elections. For one thing, there are still betting markets, which, as my colleague Leonid Bershidsky points out, were doing an excellent job of predicting elections long before we had representative national surveys. For another, in the realm of politics we always get the answer we need eventually -- on election day.
But there are broader issues where we don’t know the answers, and would very much like to, about the lives of the people who live within our borders, and what they want from their governments, businesses and civic institutions. Without good survey data, all those institutions will be operating blindly, groping toward answers they used to be able to get just by picking up the phone.
This column does not necessarily reflect the opinion of Bloomberg View's editorial board or Bloomberg LP, its owners and investors.
To contact the author on this story:
Megan McArdle at email@example.com
To contact the editor on this story:
Philip Gray at firstname.lastname@example.org