7 Data Disasters More Embarrassing Than Reinhart and Rogoff'sMatthew Zeitlin
Yesterday, a group of authors at the University of Massachussetts-Amherst released a paper calling into question one of the intellectual cornerstones of post-financial-crisis economic thought. The authors, Thomas Herndon, Michael Ash and Robert Pollin, said they had found three errors in research done by the economists Kenneth Rogoff and Carmen Reinhart, who had demonstrated that there seemed to be a tipping point where a country with public debt greater than 90 percent of its gross domestic product would face sharply slower growth.
The most glaring error: In an Excel data set of countries' annual GDP growth and their public debt, Rogoff and Reinhart apparently forgot to select an entire row when it came to averaging growth figures, leaving out Australia, Austria, Belgium, Canada and Denmark. Rogoff and Reinhart acknowledged the error, writing that "it leads to a notable change in the average growth rate for the over-90-per-cent debt group." This means that one of the most influential claims in public discussions and government policies related to austerity, debt and stimulus -- "that there is a sharp fall-off in growth when debt reaches 90 percent of GDP -- was partially due to a simple Excel error.
And it's not the only time that esteemed academics, engineers or financial professionals have gotten into serious trouble because of data mistakes. History is littered with coding errors -- failure to convert units, missing data, incorrectly entered figures -- that had global and even extraterrestrial consequences. Here's a few other times when data broke bad:
1. Mariner 1, July 1962
Less than 5 minutes after the unmanned Mariner 1 spacecraft launched on July 22, 1962, another rocket launched by NASA destroyed it. Mariner's navigation code malfunctioned, making it impossible to navigate the craft, so NASA destroyed it before it could crash-land on earth. The problem? A missing hyphen in Mariner's code responsible for transmitting navigation instructions.
Because of a malfunctioning beacon and the incorrectly written code, Mariner's navigation computer started guiding the ship into a series of unnecessary course corrections with erroneous steering commands which finally threw the spacecraft off course, according to a NASA history. The science fiction author Arthur C. Clarke called it "the most expensive hyphen in history."
2. Intel Corp., 1994
When Intel shipped two new micropressor chips -- the 486DX and Pentium -- in personal computers, there was a slight difference from its previous chips: They could do calculations using floating point numbers (non-integers, such as 4.8 or 2.3) on their own. This meant that the new chips could do certain complex calculations much faster. However, because of an error in the chips' instructions, certain floating point numbers were divided incorrectly. When a math professor found the problem, Intel publicly stated that nearly all users were very unlikely to have any problems running programs or doing calculations. Yet by the end of the year, after sustained public pressure, Intel announced it would replace the chips for anyone who asked.
3. USS Yorktown, 1997
The Yorktown, a Ticonderoga-class missile cruiser, had to be towed back to Norfolk, Virginia, after its propulsion system failed. The reason? Its database couldn't divide by zero. When a system administrator entered zero into the data used by one of the ship's central computer programs, the database overflowed and the computer systems failed. A civilian engineer investigating the failure wrote that "your $2.95 calculator, for example, gives you a zero when you try to divide a number by zero, and does not stop executing the next set of instructions. It seems that the computers on the Yorktown were not designed to tolerate such a simple failure."
4. Mars Climate Orbiter, 1999
It's become the favorite story for elementary school math teachers. In September, 1999, NASA launched a $125 million craft that was supposed to work as a communication relay for the Mars lander already on the surface and then spend one Martian year (687 days) collecting data on the Martian atmosphere and surface. Instead, it plunged into the Martian atmosphere and high speed and was disabled. What happened? Engineers working on the craft ten months before it was lost forgot to convert English units into metric. The problem was that contractors working on the orbiter used English units while the Jet Propulsion Laboratory, in charge of navigation, used metric. Thus those navigating the craft assumed that a set of thruster firings were being reported in newtons when instead they were being reported in pounds. Before anyone could figure out the data, it was too late and the orbiter was lost.
5. Barclays PLC and Lehman Brothers Holdings Inc., 2008
When Lehman Brothers was going through bankruptcy, Barclays scooped in to buy a huge chunk of the defunct investment bank. This ended up including 179 contracts that had been accidentally added to a spreadsheet prepared by a first year associate at the law firm Cleary Gottlieb Steen & Hamilton LLP. What went wrong? The associate was reformatting a spreadsheet of nearly 1,000 rows and more than 24,000 individual cells and, when converting the spreadsheet to a PDF, accidentally added in the new contracts
6. European flight ban, 2010
After Eyjafjallajokull, a volcano in Iceland, erupted in March, thousands of flights to, from and within Europe were cancelled because of fears that ash would damage jet engines. However, European officials said the models used to justify the shutdown were "based on incomplete science and limited data." A European Union transportation official told the Financial Times that "many of the assumptions in the computer models were not backed by scientific evidence." Just five days of the shutdown ended up costing airlines around $1 billion in lost revenue.
7. JP Morgan Chase & Co., 2012
The London Whale fiasco -- in which JPMorgan lost $6 billion on a credit-derivative bet gone wrong -- isn't just the story of poor supervision and greedy traders. It's also the story of bad spreadsheets leading to worse results. When it came time to implement a new model to calculate the risk of the trade at the heart of JPMorgan's historical loss, the man in charge of doing the calculations from spreadsheets with trade data made a mistake: Where he was supposed to divide by the average of the new and old "hazard rates," he divided by the sum, according to the U.S. Senate's report. The report claimed that this error understated the volatility in the new calculation by a factor of two and ended up lowering the trade's reported risk.
(Matthew Zeitlin is a contributor to the Ticker. Follow him on Twitter.)