Metrics Madness

Quantifying innovation is key. Here's how to do it right and avoid the big mistakes managers often make

Slide Show >>

The buzz around innovation is expanding to monstrous proportions. Nowhere is this frenzy more evident than in the pell-mell rush to metrics. Where once managers obsessed about measuring quality and cost, they now focus on measuring the innovation process. A wise use of metrics can improve a company's innovation "hit rate," and help companies make the right choices, faster, with less risk of failure.

But metrics madness can lead to confusion, dysfunction, and less innovation, not more. The common mistakes are putting in too many metrics, measuring the wrong things, misaligning metrics within organizations, and counting what can be counted, not what counts. Looking solely at the number of ideas in a pipeline without measuring successful outcomes in terms of revenue and margins won't help.

Metrics can mislead. Whatever gets measured will change. Focus heavily on patents, and soon employee incentive programs will develop to reward patent generation. For some companies such as IBM (IBM ), patents do reflect success in innovation. But how many patents do Samsung or Apple Computer (AAPL ) hold relative to IBM? Does it really matter to their core strategies?

Which are the best metrics to use? Try an innovation dashboard or index, a portfolio of eight to ten significant measures. More than that can overwhelm; fewer don't provide enough information to be useful. And remember: Executives in the corner suite need a different set of measures than do the people executing design strategy.

Innovation consultants believe that companies should look at metrics holistically, including other measures that add balance for a fuller picture. IN talked with the experts at Booz Allen, Boston Consulting Group (BCG), Jump Associates, and Kaiser Associates to pull together the good, the bad, and the downright ugly on the nascent practice:


Before you install a massive system of metrics, take the time to study and understand your company's innovation DNA. What resources are available to foster innovation, in terms of money, people, and processes? Your metrics flow from your resources.


How much money is available for research and development, design and implementation? Is this amount sufficient? Too much?

Warning! It doesn't do much good to measure spending levels if you don't ultimately track them against final outcomes. It's the organic growth and profit margins that count in the end. Never lose sight of that.


The folks that really matter to a process must always be part of it. This is often a group so small that you can call the members out by name and count them on one hand.

Warning! Focus on building a team of people who can create -- and deliver -- the product/service. Titles and silos do not matter. Designers, ethnographers, and engineers are key, but people who understand the company culture and can keep new projects on course are needed, too.


A formal electronic system that manages new ideas provides a visible means of generating and measuring their volume. It can be a useful tool to get employees to contribute.

Warning! Ideas are cheap. What matters most is quality. Companies should look carefully at the screening mechanism they have for sifting the gold from the thousands of suggestions they may get. And be aware that ideas from senior execs often get the green light, not necessarily because they are more innovative.


Know what you can do and what you can't do. Understand how the systems in your culture work and what your resources are.

Warning! Companies don't do everything equally well. Know your limits and design those hindrances into your strategy. If going through the IT department always means a six-month delay costing $2 million, deal with it.


Innovation processes must be robust and repeatable. Metrics that measure them must keep your vision on course.


Speed is everything. In a recent BCG survey of 269 companies, time to market was the innovation that metric companies reported to be most valuable.

Warning! Managers often make the mistake of citing the average, not the median. It's the median, however, that allows them to course-correct. If all your products reach market, say, after 16 weeks, except one that takes an inordinately long time, it's critical to identify the problematic one and find out why. Similarly, find out why the fastest products are quick to reach shelves, and repeat that process.


Take an early look at whether or not your innovation projects are fully staffed. If they are 50% staffed, you may be slipping even if you are only two months into a 15-month effort.

Warning! The success of the later stages depends on the quality and timing of the work up front. This metric relies on a disciplined company culture. A catch-up-later mentality won't work. The measure alone will do no good if managers don't or can't quickly bring projects up to staff.


Break projects into discrete, measurable deliverables, and then make sure they are delivered.

Warning! To make it work, companies must hold employees accountable for meeting milestone targets. If your corporate culture allows people to let things slip, the process doesn't work. If a supervisor accepts half-finished projects, it also fails.


Companies' tolerance for risk varies, and understanding those limits is key to placing your innovation bets. To identify risk, some businesses identify 10 to 15 key attributes, such as the number of new technologies involved in a new product or the length of time it will take to bring the product to market.

Warning! Risk is culture-specific for companies. Without analyzing the corporate culture and knowing the tolerance and limits of risk, innovation teams waste their efforts and fail.


Strong metrics that measure outcomes can be best used to drive future performance. They can help convince senior managers to alot more resources to future innovation efforts.


TSR is the most used innovation metric. It represents the amount shareholders gain from a rise in stock prices plus dividends. Most innovation consultants argue that companies with a hearty innovation process should see it reflected in their TSR.

Warning! Research on TSR is mixed. Some research points to higher stock returns for companies that focus on consumer experiences and design. Others point out that there is no statistical relationship between how much you spend on R&D and total shareholder return.


Measuring the number of patents encourages companies to increase their development of intellectual property. It's one of the only metrics that is publicly available from all companies, so many use it to benchmark their progress against competitors.

Warning! Invention is not innovation, and patents do not automatically lead to innovation. A patent focus can become distracting. Many companies focus on the legal aspects of protection so much that they lose sight of the business.


This is one of the top three most widely used metrics, according to a recent BCG survey.

Warning! It's easy to game. Companies measure new products differently. If you change the color of a toothbrush, is it a new product? The percentage also goes up if overall sales go down, so a particularly bad year for sales may disguise itself as a great year for innovation.


Most companies flag this as important, and they're anxious to figure out how to develop it, but few currently employ this kind of ROI.

Warning! Return on investment is a very difficult measure to construct. No one has a commonly accepted ROI metric, and many are in use.


This is the number of ideas that you decide can't work. If your kill rate is too low, you are not loading enough new ideas into the front end of your innovation process. You are also not taking enough risk.

Warning! Companies kill most ideas at the front end. They should cull ideas at each stage of the process. Managers often fail to document why an idea didn't work, and the company is destined to repeat mistakes over again.

By Jessi Hempel

    Before it's here, it's on the Bloomberg Terminal.