Big Data Not Living Up To Its Promise? Change the Way You Work

Photograph by Colorblind/Corbis

From all the talk, you might think Big Data is powering multinational corporations and transforming their businesses. But of the countless companies trying to leverage vast amounts of data, only a few have been truly successful. To benefit, companies need to make a significant commitment to re-engineer their decision-making processes.

For most companies, Big Data is not fully integrated into their procedures. Workers are overwhelmed by the volume of data, and either they don’t know how to use it or they can’t use it in a timely way. Although many people talk about turning data into insights, the under-appreciated problem is turning those insights into decisions. This means getting data analytics to the right decision maker at the right time, which requires a new way of working.

Even when a company has great software and great analysts generating great insights, it can struggle to make those deductions useful. Leaders on the business side have to commit to using them.  Decision makers need to trust them. And information needs to flow properly, with a timeliness that’s aligned to decision-making cycles.

In managing a company’s conversion to Big Data, it’s best not to jump in all at once. Piloting the data and its analysis in small areas—taking a category or a set of specific products, for example—offers an affordable way to test the waters and take down barriers to change. A pilot program can show a company what works before it makes the massive investment to roll out a company-wide plan.

This way, a company can gain a better understanding of what capabilities and skills are needed. Think through the re-engineering process of getting all that expensive technology to do what it’s supposed to do. A lot of things can hold back change, but one of the biggies is that if you make it difficult and you put the onus on the individual, the change is less likely to happen.

Getting from the data to making day-to-day decisions involves a special combination of science and art. There’s science involved in assembling and comprehending the data—companies may need new job titles, such as data scientist—and there’s also an art to making data relevant and meaningful. Not only do companies need new roles, titles, and skills—they also need to mesh the new processes with existing employees and ways of working.

The scope of these issues can give some companies pause. Where do you start? How far do you go? If a customer-focused business suddenly has an abundance of customer data, it touches everyone. Managing that change can become overwhelming.

Other issues to overcome include so-called analysis paralysis—the common feeling among managers that ‘Now I know everything but I have no time to use it.’ If companies were to streamline the decision-making process, they wouldn’t have to make one-off decisions about how to use data analytics; it would simply become part of what the company does.

Next, the company must design new information flows, define roles and ownership, and get decision makers engaged. It all adds up to a more integrated way of working. This has to be an iterative process as data models evolve, which they are constantly doing—that’s part of the art. But in the best cases, this rigorous change-management approach sets a standard for the entire company to follow in learning to apply the Big Data vision.

Before it's here, it's on the Bloomberg Terminal.