Sizing Up Your Web Payoff

Forecasts of gains from e-business investments don't always pan out

Last year, when Jonathan Wharton, the e-business manager for Canadian energy producer Suncor Energy Inc., began sizing up the potential benefits of a new package of Internet software, he didn't take his supplier's claims at face value. The idea was to install Plumtree Software Inc.'s corporate Web portal to help Suncor's 4,000 employees get information and communicate better. Using a workbook Plumtree provided to help estimate the return on his roughly $1 million investment, he initially calculated a $13 million annual bounty. But Wharton tossed out a handful of items--including a $10 million revenue gain due to better dissemination of information. It could happen, but he wouldn't count on it. He came up with a more conservative number: $2 million. "We're a tough audience," says Wharton. "I needed to have something I could believe in and could convince my boss to believe in."

Ultimately, the deal was done. Wharton is confident he will pay for his initial investment within eight months after the software starts running next year. And while the process of sizing up the potential benefits was bumpy, he wouldn't dream of buying software without first going through the exercise of calculating his return on investment--or ROI.

Wharton is at the forefront of a phenomenon that's sweeping the corporate world: the ROI study. In the Internet's heyday, corporations didn't demand proof that e-business projects would save them money or boost revenues. Today, many corporations won't commit to new tech purchases unless they see the benefits spelled out in black and white. So tech companies are devising all sorts of studies to persuade customers to buy.

As vital as ROI analysis has become to corporations, not all studies are created equal. And some promise returns that will never materialize. Online calculators--where a company asks questions about a business and gets answers based on a formula--give only rough estimates of possible savings. Case studies of companies that have completed projects don't necessarily apply to companies in other industries. A detailed analysis of an individual company's prospects should yield more useful results, but even then, variables such as an economic slowdown are hard to anticipate.

Not surprisingly, ROI studies tend to be overly optimistic. Only 28% of major tech projects fully meet expectations, according to researcher The Standish Group. Yet a September survey of nearly 500 corporate information technology executives by researcher Jupiter Media Metrix Inc. showed that 59% of their do-it-yourself ROI studies forecast gains.

To get dependable assessments, corporations need to drill deep into the numbers. They have to identify their variable costs, then calculate the money they can save or the new sales they can log adding technology. This can be a major undertaking. Consider electronic controls company Cutler-Hammer, a division of Eaton Corp. (ETN ). A three-month study that led to buying Metreo Inc. software to automate the company's pricing system involved interviews with 200 staffers. Cutler-Hammer calculates it will take 18 months to pay back the $250,000 cost of the first phase of the project, started in May. Already, the time it takes to analyze a new piece of business and come up with a price has dropped by 40%, to four days.

By contrast, ROI calculators can give rough estimates in as little as 10 minutes. The calculators, often on a supplier's Web site, walk a company through a series of questions, and then spit out a forecast based on the answers. These estimates are the least reliable of all the ROI approaches. When Polly Foote, a human resources business analyst at plumbing distributor Ferguson Enterprises Inc., used the basic version of PeopleSoft Inc.'s (PSFT ) ROI calculator last June, she came up with an estimated 400% return on her investment in human resources software over five years. "That was unbelievable," she says. So she spent two weeks coming up with her own calculations--yielding a 77% internal rate of return, which takes into account the cost of money and other factors. Ferguson bought the software.

Even though tech suppliers have an obvious bias, they also have expertise that can help customers size up their products. Intraspect Software Inc., a Brisbane (Calif.) maker of knowledge management software, two years ago forecast a $1.5 million return in 12 months on an initial $280,000 investment by customer Hill & Knowlton, a public relations firm. Ultimately, the return was even better, $2 million, according to Edward W. Graham, Hill & Knowlton's director of knowledge management. Intraspect led Hill & Knowlton people through a series of questions about each of their business activities--such as prospecting for new clients--and helped them estimate exactly how much time they could save on each piece of the process by using its software.

It's prudent to get a second opinion, though. Consultants such as Gartner Inc. (IT ) and Meta Group (METG ) are the least likely to forecast a false-positive ROI since they have no vested interest in the outcome. "An external consultant can be your best bet. They've got a proven methodology and can benchmark you against others in your industry," says David Taylor, research director at Jupiter Media Metrix.

Doing follow-up studies after a new technology is running can confirm whether it delivers as promised. Sometimes they even reveal pleasant surprises. Computer Services Solutions, a $185 million Dutch computer systems integrator, bought SAP supply-chain software to create online links with its suppliers of technology gear. Rather than storing computers in its warehouses, it arranged with suppliers to have them shipped directly to its customers. That allowed it to close its warehouses, eliminating $13 million in inventories. Then came the bonus. Through an audit of results by Andersen, CSS figured out it could use the same software to provide online purchasing for corporations. The result: a spin-off, Atcostplus, which links companies with their suppliers via Atcostplus' data center.

ROI studies aren't an exact science. But in times such as these, measuring results is a vital discipline. It could mean the difference between running efficiently and being an also-ran.

By Steve Hamm

Before it's here, it's on the Bloomberg Terminal.