Microsoft's Plan B for Search

Moving past the Yahoo saga, the software giant is buying semantic search engine Powerset as a way of closing the gap with Google

Still smarting from a failed attempt to buy Yahoo! (YHOO), Microsoft (MSFT) is trying another tactic to gain on Google (GOOG) in Web search.

Microsoft is buying Powerset, developer of what it hopes is a smarter way to search the Web. Powerset uses so-called "semantic Web" technology that brings up results based on an understanding of a word's meaning and the context of its use. That's in contrast to the method used by the major search engines, which work primarily by matching words in queries to those on Web pages. Microsoft announced the acquisition July 1 on a blog, saying it shares Powerset's vision "to take search to the next level by adding understanding on the intent and meaning behind the words in searches and webpages." News of Microsoft's interest in Powerset was reported June 26 by industry blog VentureBeat. According to the article, Microsoft has offered more than $100 million to acquire the company. The purchase price was not disclosed.

The purchase could give Microsoft a big leg up in efforts to catch Google. Powerset and other semantic search engines outperform Google in some cases (, 9/17/07). They respond particularly well when users want detailed answers to questions in specific subject categories for which there are a lot of Web pages with similar keywords, such as health or law. "Semantic search takes it to the third level," says Eric Tilenius, an early investor in Powerset and Kango, which applies semantic search technology to travel.

What's more, semantic search wouldn't be easy for Google to replicate. Large search engines, such as Google and Microsoft, have already scanned and indexed many of the pages on the Web. So their machines can concentrate efforts on analyzing the several million new Web sites created every year and adding them to their records. Adopting semantic search technology would require the big guys, in essence, to start from the beginning—rescanning every Web page according to the technology's fundamentally different method of analyzing and classifying Web pages. "You cannot do a patch job," says Riza Berkan, chief executive of Hakia, a semantic search engine working on scanning the entire Web. "We are building everything from scratch and this is what it takes to make it."

"Best Shot"

The technology itself is difficult to develop. Though Google has hired some semantic search experts, the technology behind semantic search engines has been in development for the better part of a decade. "Microsoft's acquisition of Powerset makes perfect sense and is probably the best shot at a disruptive technology that might allow it to leapfrog Google," says Andrei Hagiu, assistant professor of strategy, focusing on technology, at Harvard Business School.

Of course, Microsoft would have to rescan all its pages, too. But with just a portion of the $46 billion it was willing to pay for Yahoo, Microsoft could invest in the necessary equipment, such as the servers needed to scan and "read" all those pages. Plus, it has the underdog's willingness to take on risk and expense in hopes of finally generating search results that can rival Google's. (Google grabbed nearly 70% of searches in May, according to research firm Hitwise; Microsoft's share slipped to just under 6%.)

Microsoft and Powerset are complementary in other ways (, 6/17/08). Rather than focus on creating one consumer-facing site capable of answering any query, like Google has, Microsoft has split its search engine into specific categories—a comparison-shopping engine, Microsoft Live Cashback; a travel search engine, Farecast; and a health-specific search engine, Today, semantic search engines do best with such category-specific searches, which help them to scan a smaller set of pages in detail. Scanning the entire Web in that much detail is difficult to do quickly.

Powerset's Potential Problems

Yet, Powerset is no panacea. The processing power and time required to perform semantic searches are the main drawbacks of the technology. It can take a microprocessor as long as 20 seconds to analyze a page. As a result, many of the sites today scan just a subset of the pages on the Web. Currently, Powerset's site, for example, only scans Wikipedia articles. Cognition, another semantic search engine, scans pages in the health and legal fields. It will probably take years for the semantic search engines to be able to scan the whole Web in a timely manner.

A second potential problem is that there are a couple of semantic engines out there, each gunning for Google. Hakia, which has raised $21 million to develop a search engine capable of indexing the entire Web, hopes to someday be a Google competitor. "The promise is that we will be a competitor to the big search engines," says Berkan. "Definitely within five years we will see big changes in the search market."

Indeed, some of these engines are willing to be acquired by a Google. Brooke Aker is CEO of Italy-based Expert System, the maker of semantic search engine Cogito Focus. He sees a union with the major search engines as the fastest way to spread semantic technology across the Web. "I do see it merging," says Aker. "We don't have the big server farms like a Google does."

Microsoft's Challenges

And even if Microsoft can put in place the infrastructure semantic search needs, it will still face two challenges it has been trying to surmount for years. One concerns multimedia. Semantic search does not offer a great solution for recognizing videos and images. It can scan the words people use to describe a video or an image. But it can't yet analyze the soundtrack or images in a video to determine whether those descriptions are accurate. Google is working on better technology. Last September, it hired experts who could help it decipher images, such as scanned pages in books.

The other big hurdle is the power of Google's brand. The company's name has become synonymous with search and its users are loyal. To take share, semantic search will need to develop a reputation for providing better, more relevant results than Google.

Microsoft is determined, if nothing else. Company CEO Steve Ballmer has said repeatedly that the company is in the search game for the long haul. Having stumbled in its most audacious attempt at grabbing share, it's time for Plan B.

Before it's here, it's on the Bloomberg Terminal.