Web Numbers: What's Real?

Competing methods of measuring traffic online leave advertisers, investors, and even Net companies almost flying blind

Life is good for 27-year-old Seth J. Sternberg. A year ago, he dropped out of Stanford Business School to work full-time on Meebo Inc., an easy-to-use service that has solved one of instant messaging's nagging problems: the inability to communicate with people who use an IM service other than yours. Today Meebo is going gangbusters. It has raised $3.5 million from the Silicon Valley crème-de-la-crème, including Marc Andreessen of Netscape fame and venture capital heavyweight Sequoia Capital. More impressively, the service attracts almost a million people every day, who swap more than 60 million messages.

There's just one hitch: Sternberg and his co-founders have a hard time proving the site is as popular as they say it is. Look up Meebo's Web traffic using the comScore Networks Inc. service, and you'll find that a European competitor eBuddy.com is four times as big. Alexa, a competing Web measurement service owned by Amazon.com Inc. (AMZN ), shows Meebo is bigger. Which is true? Probably neither. Sternberg's best guess is that the two rivals are about the same size. Yet even he doesn't know for sure.

The dirty little secret of Silicon Valley is that no one knows exactly who is going where on the Web. That flies in the face of the impression that online advertising is the most dependably trackable ad medium of all time, a big reason spending on Web ads is expected to grow 33% this year, to $16 billion. But confusion over traffic measurement could cast a chill over the Web 2.0 craze. Valuations for startups such as Facebook Inc. and YouTube Inc. appear to be doubling every few months, but those numbers are based on traffic figures that could be misleading.

From the start, measuring online traffic was a juggling act. Rather than simply relying on a Web site's traffic reports, advertisers traditionally compared that data with information from Nielsen//NetRatings Inc. (NTRT ) and comScore, independent services that recruit Web surfers to record their mouse clicks. Those outfits argue that there are many reasons not to just count the clicks off a Web site's server logs. For instance, comScore points out that servers would count pop-up ads as a page view if the tracking service didn't filter them out.

Independent traffic analysis becomes more important as bigger chunks of advertising flow online and the threat of "click fraud," which inflates ad bills, grows bigger. No wonder that a host of newer services, such as Alexa and Hitwise, are highlighting the weaknesses of the older traffic-measuring companies and are muscling onto the scene with alternatives. By providing some free traffic data via their Web sites, these outfits make it easier for anyone to publish an estimate. But they also have their own blind spots and are making side-by-side comparisons vastly more confusing.

To see why there's an opening for new ways of measuring traffic, look at what has happened to the old standard for gauging online growth, the page view. As the Internet evolved during the 1990s, advertisers came to rely on the number of pages a site served up each month as their most reliable metric. With the rise of new programming and distribution technologies, however, page views suddenly look less relevant. For instance, the beauty of a site such as Meebo is that it is built with software tools called Ajax, which speed up Web surfing. When you log onto Meebo, instead of loading a new page for every mouse click, only the log-in section is loaded. But no matter how long people stay on Meebo, they're technically viewing only one page.

Meebo isn't selling advertising yet. When it does, the company's executives will be challenged to persuade ad buyers to consider measurements other than page views. In June, Sternberg invited comScore's researchers to his Palo Alto (Calif.) office to look at his company's server logs. "Here is our data, and here is your data. Something is wrong!" he told them, to no avail. Sternberg and his co-founders have thrown up their hands and have decided to publicly disclose all their internal numbers. One thing they're counting on is that people will take into account the amount of time members spend on the service.

Ajax isn't the only technology that's upending traditional Web measurement. Real Simple Syndication, or RSS, lets people sign up to have news articles, blog posts, or audio interviews from their favorite sites sent directly to their computers. But since they aren't surfing around, none of the sites gets credit for the page view.

Then there are widgets, interactive icons that can be installed on a wide array of sites. Install the Meebo Me widget on a MySpace page, and people can just click on that to IM you when they visit the site. More than 40,000 Meebo Me widgets have been installed. Because the widgets don't direct you back to Meebo's site, they don't show up in the Meebo traffic figures.

For advertisers, the problem is that while any one method of measurement may capture certain Web technologies or demographics, it misses others. An alternative to counting page views, for instance, is to measure reach, which is a calculation of each site's usage as a percentage of all Net traffic. Alexa, which tracks Web surfers through a downloadable search toolbar, combines that approach with counting page views. But Alexa is limited because it compares reach only among the sites its users visit. Results also appear to skew more toward sites favored by techies rather than the wider Web, says Ed Sim, managing director of VC firm Dawntreader Ventures. Niall O'Driscoll, Alexa's vice-president of engineering, says the company doesn't believe that's the case.

Web outfits seem to agree that Alexa is flawed, but they continue to rely on it because the data are so addictive. Since Alexa's numbers are free and available online, they can easily be plugged into a PowerPoint presentation or onto a blog, providing a quick-and-dirty way to get a competitive snapshot. Blogs cite Alexa as gospel, and its graphs are part of nearly every startup's pitch to investors. "It's a giant pain," says George Zachary of Charles River Ventures. "If someone came up with something better, I'd fund them."


If Alexa favors early adopters and techies, the opposite complaint is made about comScore and Nielsen//NetRatings. Some VCs, startups, and established companies don't think these ratings methods adequately capture the growth of new services. Yahoo! Inc. (YHOO ) saw this firsthand with del.icio.us, a Web bookmarking site the portal bought last year. ComScore showed declining traffic at del.icio.us over the summer, but Yahoo says it had at least 10% monthly growth.

The best hope for clarity is for Web ad giants and small fry alike to work with the established measurement services to improve their tracking ability. ComScore and Nielsen//NetRatings are beginning to respond. ComScore, for instance, now reports the number of requested videos at online video sites, rather than page views or unique visitors.

Some observers expect that in time a variety of metrics, such as time spent online, will be applied to different services until one measurement that combines a set of factors can emerge. Until that happens, though, the Internet will have to deal with the discrepancies. And Web metrics, like company valuations, will remain a crapshoot.

BY Sarah Lacy

Before it's here, it's on the Bloomberg Terminal. LEARN MORE