A Shortcut to Cure Big Data Headaches

Demand for database outsourcing attracts nimble startups

A Shortcut to Cure Big Data Headaches
Demand for database outsourcing attracts nimble startups like Cloudant and Orchestrate (Illustration by Shawn Hasto)
Illustration by Shawn Hasto

Two years ago, Hothead Games was drowning. On March 1, 2012, Apple promoted the company’s mobile game Big Win Soccer on its App Store—great news, except that Hothead wasn’t prepared for the massive influx of visitors that followed. Within six hours, traffic to Hothead’s servers spiked a hundredfold, says co-founder and director of technology Joel DeYoung. The company’s software couldn’t manage all the data from the crush of users quickly enough. Players experienced sustained game delays. As Hothead’s user base ballooned from the tens of thousands into the millions, “it was like we were in the kiddie end of the pool, and we were thrown into the deep end,” DeYoung says. Later that month his company managed to stabilize its service by outsourcing its database creation and storage. It hired Cloudant, a startup founded in 2008, which is using cloud technology to take a bite out of the $36 billion database market by managing enormous amounts of user information on behalf of other businesses.

It’s not just overnight successes such as Hothead that need help. Companies of all kinds are turning to on-demand database services in lieu of paying for more staffers and equipment to deal with the data they collect. In addition to PCs, smartphones, and tablets, a host of other devices—smart cars, thermometers, even traffic lights—constantly scoops up data about the people who use them. As Cloudant Chief Executive Officer Derek Schoettle says, “The mantra now is collect it and save it, because you don’t know how you may want to use it.”

At the moment, Amazon.com has the most popular on-demand database services. Google is in the game, too. Cloudant (which IBM acquired in March), along with Rackspace and a slew of startups, is helping to put the “database as a service” industry on track to make more than $2.8 billion annually by 2016, according to an estimate by Matt Aslett, a research director at market analyst 451 Research. Last year the industry made $516 million. Cloud technology is key to the startups’ nimbleness. Even companies such as Microsoft, Oracle, and SAP that have long sold pricey conventional database software are ramping up external database services. “I firmly believe that database-as-a-service absolutely has a role to play,” says Steve Lucas, who heads that effort at SAP.

The traditional database software sold by Microsoft, Oracle, and SAP often requires companies to purchase dedicated hardware run by a full-time staff. So far, small companies that can’t and won’t put up with the expense remain the likeliest customers for Cloudant and its rivals. In Portland, Ore., each of the dozens of cars tracked by security-monitoring startup Cargo Labs generates several gigabytes of data every few hours. Cargo, which apart from its three co-founders has a grand total of two interns, couldn’t afford to hire full-timers to maintain its database, says co-founder Matt Wallington. Instead, it hired Orchestrate, also in Portland, to create a series of databases that let it keep a real-time eye on customers’ cars.

Some companies may be reluctant to switch. More established companies may want to squeeze as many years as possible out of the expensive hardware they’ve already bought. Many larger corporations also worry about the security and privacy trade-offs that come with storing their data on a third party’s servers in the cloud. In some cases, they’re phasing in the use of on-demand database services for tasks that don’t require the utmost security, such as bug-testing new apps. “We certainly see this as the direction it’s going, but some of these things just take time as customers get more and more comfortable,” says Eron Kelly, a general manager in Microsoft’s data platform division.

Younger companies aren’t as patient. Year-old Orchestrate has roughly doubled its client base to 15 in the past month, according to CEO Antony Falco. Its clients include Angry Birds maker Rovio, which outsourced the building of a database for its user names. Since being acquired by IBM, Cloudant has doubled its staff from 70 and expects to employ more than 200 people by yearend, says CEO Schoettle. Puneet Agarwal, a partner at Orchestrate investor True Ventures, predicts that on-demand database service will eventually displace company-specific hardware as the majority of the market. “This is how the next generation of developers is going to think about how they are using a database,” he says.

Before it's here, it's on the Bloomberg Terminal. LEARN MORE