The Need for Speed with Big Data

The Need for Speed with Big Data
Too many data mining projects produce 'stale' insights because managers forget the importance of 'time to insight" (Photograph by Brad Wilson/Getty Images)
Photograph by Brad Wilson/Getty Images

Companies around the world have jumped on the Big Data bandwagon, excited about finding new ways to exploit the volume and variety of data flooding in from social media, mobile devices, sensors, and applications. Unfortunately, too many companies forget to focus on the other “v”—velocity. It’s the pace at which data can be gathered, sorted, and analyzed to produce insights that business leaders can act on quickly to generate instant value.

The need for speed should be obvious, but a lot of Big Data projects produce stale insights because managers have forgotten the importance of “time to insight.” Imagine you’re an insurer trying to fight fraud. Finding out three months after paying out on a fraudulent claim is clearly less valuable than identifying the fraud while processing the claim.

There are technologies—and new skills—that can make you faster. For example, advances in computer memory and databases, such as in-memory computing, will help cut the time it takes to access and analyze information from days to seconds because the data is in the computer where it’s handy, not stored on disk drives that take time to search. Think of having the papers you need on your desk versus having to search through a filing cabinet to find the right material.

Technology is only part of the equation. Companies also need to court data scientists—statisticians who can construct speedy analytic models. In the last few years, some businesses have gone a step further, creating a senior management position, chief data officer, to accelerate their data strategy.

Increasing data velocity isn’t just an obscure objective; it’s a business necessity that gives companies a chance to open up a lead on competitors. After all, what good is Big Data if you move like a lumbering giant?

Before it's here, it's on the Bloomberg Terminal. LEARN MORE