The 'Wisdom of Crowds' Is Not That Wise
I had to chuckle when I read the following on Bloomberg yesterday:
Paddy Power Plc said it was left “red-faced” after paying out early to gamblers who incorrectly bet that Greek voters would back an austerity referendum. Ireland’s biggest bookmaker “paid out five figures” in winnings on July 1, days before Greece rejected further austerity, with a larger-than-forecast 61 percent of the vote for the “no” campaign. - Paddy Power Left ‘Red Faced’ After Early Payout on Greek Vote
The delicious irony of a prediction market making an early payment on an incorrect prediction of a political referendum was simply too wonderful to pass up without comment. So today's column is on why predictions markets fail, and that contrary to common beliefs, the crowd often has surprisingly little wisdom.
The “wisdom of crowds” is a colloquial way of describing the market as a complex system. The work on wisdom of crowds shows that when certain conditions are met — diversity, aggregation, and incentives — markets tend to be efficient. Conversely, when one or more of those conditions are violated, markets can and do become inefficient, i.e., price is no longer an unbiased reflection of value. - Michael Mauboussin (emphasis added)
Wisdom, "having the quality of having experience, knowledge, and good judgment," hardly describes the random, chaotic behavior of markets in general, but it especially does not describe prediction or betting markets.
Before we get into the details, a preface: Markets are by definition a crowd. They are the place where the collective actions of buyers and sellers set actual prices. But it is an unthinking, automatic mechanism of volume and sheer dollar brute force. We can call it "dumb" in the sense that it is not the result of any conscious, intelligent decision-making that determines price. Rather, collectively, all of the decisions made by all of the participants -- good, bad or worse; smart, not-so-smart and just plain foolish -- add up to the day-to-day trading prices of commodities, bonds, equities, etc.
The very specific flaws of prediction and futures markets have been revealed over the past decade or so. When compared with "real" markets (mentioned above) the differences are both qualitative and quantitative.
Real markets are deep and diverse, and they trade in dollar volumes measured in trillions. Prediction or betting markets do not. They are thinly traded, by number of shares and by dollar amounts at risk, and they rely on a relatively small number of traders, with limited diversity. These markets lack the necessary elements for success.
Consider some of the more spectacular examples where prediction markets got it wrong:
- Iowa Primary 2004 (Howard Dean versus everyone else)
- 2006 GOP Senate retention
- New Hampshire Primary 2008 (Obama versus Clinton)
- Michael Jackson trial
- Morgan Stanley CEO Purcell resignation
- Greek referendum
The closer the traders are as a group to the actual decision makers, the better their track record. What Paddy Power needed among its gamblers was more Greek voters. Of the conditions Mauboussin outlined for the wisdom of crowds, two were inadequate: not even diversity and not enough incentives.
The reason for these errors was that the crowd of bettors did not resemble the final arbiters of each of these events. As Dan Gross observed, "these are less futures markets than immediate-past markets." Bettors aggregate polling data that is already out -- not a wise way of forecasting the unknown future.
It is not as if Paddy Power or the Iowa electronic market bettors own any special insight. They don't "know" anything individually or collectively that the rest of the public doesn't. They act more like a focus group than a crowd with specialized knowledge. The essence of a focus group or polling sample, of course, is that it be representative of the decision-making population. (Nate Silver made a science of this in his political polling work at FiveThirtyEight.) So prediction markets are like a focus group … but not a very good one.
That is similar to why the prediction markets got the Michael Jackson trial or the Morgan Stanley CEO Purcell resignation wrong and the Greek referendum wrong; much smaller groups such as jurors or directors or Greek voters don't lend themselves very well to focus groups, polling or sampling. One imagines that Greek voters can be polled -- but given how quickly the referendum came together, apparently not on short notice, and not by Paddy Power bettors.
You can call market prices random, as Burton Malkiel did in "A Random Walk Down Wall Street"; you can say they are irrational, as Robert Shiller did in "Irrational Exuberance." You can even call them chaotic and turbulent, as Benoit Mandelbrot did in "Misbehavior of Markets: A Fractal View of Financial Turbulence."
I remain unconvinced you can call prices "wise," no matter what market sets them. Perhaps the most constructive comment one can make about the crowd in market prices is that there are no better alternatives yet invented for determining the price of any item to be bought or sold. "The best we've got" hardly rises to the level of "wisdom."
To contact the editor responsible for this story:
Philip Gray at firstname.lastname@example.org