A Highly Dubious Measure of Innovation

When you think “innovation,” do Switzerland, the United Kingdom, and Sweden come to mind? No? Well, those countries are at the top of the annual Global Innovation Index, released last Friday.

In case you hadn’t noticed, “innovation” has become quite the buzzword. It attracts attention, particularly in business circles. However, there’s no real metric to measure innovation. Researchers at Cornell University, INSEAD, and the World Intellectual Property Organization aspire to have the go-to benchmark, the way, say, S&P/Case-Shiller is for home prices.

For the 2014 edition of the index, the researchers ranked 143 nations based on 81 indicators ranging from the obvious, such as patent applications and high-tech exports, to the more questionable, such as YouTube uploads and press freedom.

I chose to drill down into 5 of the 81 indicators in the study and chart the top 10 countries in each category, along with the U.S. The exercise immediately raised red flags about the index’s validity.

Estonia: No. 1
U.S.: No. 37
As a share of the population, no other people are as devoted to creating or editing Wikipedia entries as Estonians are. That’s odd. What’s weirder, though, is that someone thought “Wikipedia edits” could be a proxy for innovation (though it could be argued that Wikipedia is a good platform for invention).

U.S.: No. 1
This chart is straightforward: These are the countries that produce the highest number of MBA aspirants. The metric, however, is not an entirely accurate proxy for how many people end up applying to business school. And, since the test is only offered in English, including it in the index privileges nations such as the U.S. Also, consider this: Consulting and finance are the top two industries employing MBA grads. Do you think of them as cradles of innovation?

Thailand: No. 1
U.S.: No. 84
Upon viewing this chart, you might think, “Golly, look how far the U.S. trails behind!” Don’t be fooled—the countries are ranked by the percent of “tertiary” (read “college”) education graduates with a science or engineering degree, and not the absolute number. The U.S. has more college graduates than countries like Oman, Myanmar, Tunisia, and, yes, Thailand, and it would be hard to imagine more than 50 percent choosing science or engineering as their fields of study. And what of Iran? Are its science and engineering graduates staying in the country? Or are they emigrating abroad, where they might encounter greater opportunity.

Hong Kong: No. 1
U.S.: unranked
How reliable is this data set if the researchers couldn’t even include a number for the U.S.? It merits noting that several countries in the above chart are tax havens. Many of the businesses that register in those locales have no physical presence there, outside of a post office box.

Lesotho: No. 1
U.S.: No. 45
This chart can be very misleading, and completely unjustified as a measure of innovation. Lesotho spends 13 percent of its gross domestic product on education, while the U.S. spends 5.4 percent. The GDP of the tiny African nation in 2010, the year the data correspond to, was $2.2 billion. For comparison, the New York City school district had an expenditure of $19 billion in 2010 (PDF), 8.6 times Lesotho’s GDP. And while Lesotho can be rightly proud of the fact that its adult literacy rate is among the highest in Africa, at 82 percent, it is still classified among the “low human development countries” by the United Nations.

There are 76 other factors considered, but on the basis of a closer examination of these five alone, it’s hard to take the index seriously. But don’t take our word for it. Explore the rest of the indicators for yourself here.

Since this article was published, a WIPO spokesperson has responded to its content:
“The Global Innovation Index is a robust study that offers a fresh look at how economies are performing around the world. Each year we update our methodology, always looking for the latest data from which we can glean important lessons about fostering innovative economic environments. When taken out of context, certain variables may seem arbitrary to some. But viewed in aggregate, the data sketch a much larger picture of human innovation that is useful to decision makers around the world.”

Before it's here, it's on the Bloomberg Terminal.