There are a lot of things we're not good at predicting.

Photographer: Martin Bernetti/AFP/Getty Images)

Economists' Biggest Failure

Noah Smith is a Bloomberg View columnist. He was an assistant professor of finance at Stony Brook University, and he blogs at Noahpinion.
Read More.
a | A

One of the biggest things that economists get grief about is their failure to predict big events like recessions. Even the Queen of England, that most reserved of personages, got in on the game, back in 2008, according to the U.K. Telegraph:

During a briefing by academics at the London School of Economics on the turmoil on the international markets the Queen asked: "Why did nobody notice it?"...Professor Luis Garicano, director of research at the London School of Economics' management department, had explained the origins and effects of the credit crisis...Prof Garicano said: "She was asking me if these things were so large how come everyone missed it."

The queen is correct. Here, courtesy of Institute for Monetary and Financial Stability economist Volker Wieland and Goethe University economist Maik Wolters, is a picture of how badly economists’ models failed to predict the Great Recession:

As you can see, even in the third quarter of 2008, the best models we have missed the big recession entirely. 

Economists didn’t just fail to see that monster recession; they routinely fail to see economic events coming. The best models we have -- the ones central banks use, which take graduate-level training in order to handle -- have about as much forecasting power as simple, naïve mathematical techniques that any undergraduate statistics major could whip up in a few minutes. 

Pointing this out usually leads to the eternal (and eternally fun) debate over whether economics is a real science. The profession's detractors say that if you don’t make successful predictions, you aren’t a science. Economists will respond that seismologists can’t forecast earthquakes, and meteorologists can’t forecast hurricanes, and who cares what’s really a “science” anyway. 

The debate, however, misses the point. Forecasts aren’t the only kind of predictions a science can make. In fact, they’re not even the most important kind. 

Take physics for example. Sometimes physicists do make forecasts -- for example, eclipses. But those are the exception. Usually, when you make a new physics theory, you use it to predict some new phenomenon -- some kind of thing that no one has seen before, because they haven’t bothered to look. For example, quantum mechanics has gained a lot of support from predicting the strange new things like quantum tunneling or quantum teleportation.

Other times, a theory will predict things we have seen before, but will describe them in terms of other things that we thought were totally separate, unrelated phenomena. This is called unification, and it’s a key part of what philosophers think science does. For example, the theory of electromagnetism says that light, electric current, magnetism, radio waves are all really the same phenomenon. Pretty neat!

What’s important about these predictions is, first of all, that they’re testable -- the evidence isn’t going to give you an ambiguous answer. It’s also important that they’re novel -- each theory can predict more than just the phenomena that inspired the theory. As Richard Feynman, the great physicist and amateur philosopher of science, put it:

When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

So that’s physics. What about economics? Actually, econ has a number of these successes too. When Dan McFadden used his Random Utility Model to predict how many people would ride San Francisco's Bay Area Rapid Transit system, it was a totally new experiment. And he got it right. And he got many other things right with the same theory -- it wasn’t developed to explain only train ridership. 

Unfortunately, though, this kind of success isn't very highly regarded in the economics world -- at least, not that I’ve seen. If you manage this kind of success you can start a consultancy, but your fellow academics will think it merely a feather in your cap. The kind of theories that are held in the highest regard are usually not empirically successful ones, but new ones -- theories that use new kinds of math, for instance. These are prized because they give a lot of other economists work to do,specifically making variation upon variation of the cool new model. 

When economists do praise a model for its empirical success, it’s usually about how well the model fits the data on the phenomenon the model was created to describe. This, as Feynman pointed out, is a pretty low bar. If that’s the only hurdle models have to clear, you can make one new theory to describe each new phenomenon. If you have a million phenomena, you end up with a million models. The models probably contradict each other, but that doesn’t matter, since each model is only judged on how well it “explains” the thing it was created to describe. Which, at least in the macroeconomics literature, is pretty much what we see.

How did econ get this way? My guess is that it’s because economics didn’t evolve from science -- it evolved from literature. Back in the days of Adam Smith and David Ricardo, there was no such thing as economic data -- all you had were thought experiments and casual observations. So economics wasn’t able to hold itself to scientific standards of validation in the old days, and it’s been an uphill battle to graft those standards onto the discipline. Not that people haven’t tried -- if you read Milton Friedman’s “The Methodology of Positive Economics” from 1966, you will find him saying much the same things I’m saying.

Maybe now, with the ascendance of empirical economics and a decline in theory, we’ll see a focus on producing fewer but better theories, more unification, and more attempts to make novel predictions. Someday, maybe macroeconomists will even be able to make forecasts! But let’s not get our hopes up.

This column does not necessarily reflect the opinion of Bloomberg View's editorial board or Bloomberg LP, its owners and investors.

To contact the author on this story:
Noah Smith at nsmith150@bloomberg.net

To contact the editor on this story:
James Greiff at jgreiff@bloomberg.net