A Search Engine For Every Subject

Google and Yahoo rule, but a flock of upstarts is offering new ways to find info

Ron Conway hit paydirt as one of Google's earliest bankrollers. But don't try to convince him the search giant has a hammerlock on the sector. Sure, Google, by some estimates, handles nearly two-thirds of the world's Internet queries. It also shells out half a billion bucks each year on research and development and plucks the top engineers from powerhouse universities. Nevertheless, Conway believes that some of the most interesting search innovations will soon emanate from companies not named Google (GOOG ) or, for that matter, Yahoo! (YHOO ) or Microsoft (MSFT ). Over the past two years, he has bet heavily on this hunch, putting his money in nearly two dozen startups. "Search is in its infancy today," says the seasoned Silicon Valley investor. "It's at 10% of its potential, maybe."

Conway is by no means alone. A flock of startups has descended on the lucrative search industry. Last year, $263 million in U.S. venture capital poured into 47 search outfits, says researcher Dow Jones VentureOne (DJ ). The figures have risen steadily since 2002, when $58 million went into nine startups. "It's shocking to go from drought to flood as quickly as we have," says Bradley J. Horowitz, Yahoo Inc.'s director of technology development.

The boom in startups suggests a technology renaissance ahead, and this could mean big changes for Internet users. While Google and other giants seem wedded to the current paradigm -- type words into a rectangular box, and get a list of 10 blue links -- startups are throwing caution to the wind. Without existing users to alienate, they can afford to rethink everything from the search interface to the formulas used to deliver info.

The result is likely to be a move away from a single ubiquitous search box, away from the reflexive hop to Google to find info on the Net. Instead, people may use several different search engines, each tailored to a specific task. One might specialize in blog postings, another in video clips, and a third in general information. The shift may look like the evolution of TV, from a world dominated by the Big Three networks to one in which hundreds of cable channels specialize in topics from cooking to history. "People are looking for targeted, specific information that search engines can't provide," says Michael Yang, CEO of Become.com, a search engine focused on Internet shopping.

Few fledgling firms are tackling the search leaders head on. Companies such as Google and Yahoo have spent years and billions of dollars designing powerhouse engines that include mathematical algorithms that scour, sift, and rank billions of Internet pages at the click of a mouse. It's an engineering feat that even deep-pocketed Microsoft Corp. can't easily replicate. But the upstarts are finding plenty of opportunities between the cracks of the bigger search players.

Many of their innovations could well end up in the pockets of Google or Yahoo, which haven't shied away from acquiring promising startups. In the past two months alone, Microsoft's MSN, Yahoo, and America Online (TWX ) have all snapped up search-related startups to bolster their technology offerings. Although prices haven't been disclosed, many have been selling in the range of $10 million to $50 million, say investors. Conway thinks that "75% of the companies in search will either be acquired or go out of business."

The dangers of falling into that latter category are omnipresent, particularly as the flood of startups builds. On one hand, promising outfits want to be known by would-be acquirers. Yet a Google or Yahoo could simply shift around an R&D project or two and wipe out the prospects of several entrepreneurs.


That's not the only challenge the biggies impose on startups. Google's voracious appetite for engineers has cleared out upper echelons of many a graduating class of computer science majors, according to professors from the University of Washington to Stanford University. When Google can't make the hire, Yahoo and Microsoft often pick up the slack. The ensuing talent shortfall has crimped many grand plans. For instance, Conway hoped to launch a business incubator, Evergreen Innovations, for startups. He had planned to build three teams, each with five engineers, to hatch fledgling companies. But after struggling to hire the techies, he scrapped the operation to focus on his roster of investments.

Despite perils, opportunity abounds. For instance, many startups are convinced the industry's methods for selecting the most relevant sites are set for an upgrade. Google provided a big jolt seven years ago, by analyzing the links that connect sites. Among other factors, Google decided that the more sites that link to your site, the more important your site.

Today a number of startups are attempting to infuse human preferences into computer algorithms to pinpoint relevant information better. For instance, somebody searching for a nearby bicycle store may benefit from knowing which shop's site has been most frequently bookmarked by people in the neighborhood, or described in the most favorable terms. "It gets interesting when human knowledge begins to intermingle with algorithmic search," says David Hayden, CEO of startup Jeteye Technologies Inc.

This "social search" is the source of a recent acquisition binge at Yahoo. In the past year the company has snapped up at least five startups focused on building online community content. "Our belief is: better search through people," says Yahoo's Horowitz.

It's not the only way search startups hope to improve discovery of sites. Many believe the answer will come in the form of niche sites. While Google and Yahoo have proved adept at filtering through billions of documents to come up with a list of pertinent results, this approach hasn't thrived in narrow industries, such as shopping, real estate, or health.

Why? The engines can't really know users' intent when they plug in one or two words and hit "enter." If people come to Google and type in "Xbox," they could be looking for the site Xbox Addict, a list of games, or possibly trying to buy a new video-game console. But a person who plugs in a similar query at a shopping search engine has already made a statement about intent.

The search behemoths are developing their own niche engines, but their tech prowess may not carry over. Often, relevant information such as a job posting or a real estate listing exists only fleetingly and is barely noticed elsewhere on the Web. Because other sites typically won't link to, say, a short-lived job opening, it means the technical advantages of major engines won't be a crucial factor. In addition, startups may get a more receptive greeting in industries such as classified ads, where data are like gold and Google is viewed as a threat. "The information that is important might be pictures, inventory information, freshness of data, or accurate product info," says Theresia Gouw Ranzetta, managing partner at venture firm Accel Partners and a backer of real estate site Trulia.com. "It doesn't lend itself to link analysis."

All these opportunities don't portend Google's demise by any means. Indeed, look for Google and its competitors to snap up many promising startups. But with less to lose and a clean technical slate, these newcomers will probably force sizable change in the industry -- and in the ways we all search for information on the Net.

By Ben Elgin

    Before it's here, it's on the Bloomberg Terminal.