Online Extra: Speedier, Smarter, More Selective Searches

Improved technology promises to help e-consumers find what they're looking for -- even lead them to stuff they didn't know about

To succeed at online shopping, consumers have to be good at solving crossword puzzles. Try typing "car rental" into Yahoo!'s search window: You'll get a list of companies -- but no Hertz Corp. Enter "flowers in a vase" into the search window at and, alas, you'll face a blank page. But type in "vase," and you can view 46 bouquets in vases.

The irony is Yahoo! and have two of the best Web search engines among shopping sites, according to industry experts. Most other sites' searches "stink," says Harley Manning, analyst at technology consultancy Forrester Research. According to Forrester's June, 2000, report, 93% of the 30 e-commerce sites surveyed didn't offer keyword searches or failed at the most basic searches. "If someone can't find a product on your site, they can't buy it," says Manning.

COOL! But help is on the way. Over the next year, sellers and search-engine companies looking to better connect with shoppers will roll out dramatically improved search capabilities. From software robots that understand the contextual meaning of "Cool!" to artificial intelligence programs that will learn what users like over time, these new systems should provide far more accurate replies to queries, says Detlev Johnson, search expert with Position Technologies, a company that helps sites to place well with search engines. And that could prove a boon to e-commerce, both for consumers and business-to-business customers who have been stymied when looking for things on the Net.

Search problems haven't crippled Web commerce. According to e-business consultancy ActivMedia, online shopping should double from $56 billion last year to $112 billion this year. But impressive as those numbers are, they don't reflect the millions of unsuccessful online shopping expeditions.

In fact, most e-commerce companies have failed to do basic user testing of their search functions, says Hal Varian, dean of the School of Information Management & Systems at the University of California in Berkeley. Even comprehensive search engines and directories are hit-and-miss when it comes to helping shoppers find what they want.

SUBJECTIVE. To a certain degree, search-engine results are clouded by the recent rise of pay-for-placement, where commerce sites pay to get their name at the top of the return list when certain keywords are typed in as part of a query. But problems also come from the inherent reliance on humans at popular Web directories such as Yahoo! and Human editors look through each site to determine which category its content fits into and to make sure pages of the same site aren't listed as duplicate category entries. Unfortunately, this subjective process relies on human-formed associations that might or might not match up with those of Web surfers.

The directory process needs to be taken to the next level, with editors rating each site on design, prices, and organization, says Ted Zajac, vice-president for placement technology at Search Engine Ranking Magic, which helps companies get better search-engine placement. Directory, which will go live in the next few months, uses these broader ranking measures in a system that Zajac thinks will be a dramatic improvement over Yahoo!.

Ultimately, however, directories, big search engines, and shopping sites alike will need a greater level of personalization if search results will ever noticeably improve. For example, visitors who today chose "girls" the first three times they enter the Gap Kids site would be taken directly to the girls page the next time they typed in, explains Ramana Venkata, chief technology officer of search software company PurpleYogi.

In the future, the site might also remember that this visitor has a three-year-old daughter who wears size five shoes and likes the colors red and pink. And eventually, by comparing past purchases to the purchasing habits of other customers with similar taste, the site could make solid recommendations on what outfits to buy.

MAKE OR BREAK. Thus far, is one of the few sites to take advantage of this so-called "collaborative filtering" technology, albeit at a rudimentary level. After registering with the e-tailer, customers who disclose their favorite author or music receive fairly accurate personalized product recommendations. For instance, a fan of Ernest Hemingway would get a list of all of the author's books in stock. Such services will be a make-or-break e-commerce search technology: "Either you use it or you die," says Venkata.

Once these systems are adopted by consumers, they could easily be broadened to create proactive shopping services that collect a folder on a user's interests and tastes and alert that person when shoes he or she likes are on sale at a specific Web site. That could be a tough sell to people concerned about protecting their online privacy. But crude forms of these shopping systems already exist. Called shopping bots, they're personal search agents that a surfer can activate to scour the Web in search of a few specific items.

For now, this technology remains problematic. Web shoppers who unleashed shopping bots last winter in the quest for Sony Playstation 2 consoles caused traffic jams at many e-commerce sites as the queries from bots overburdened Web servers. And shopping bots lack the advanced collaborative filtering technology needed to understand a user's personal tastes and extrapolate further suggestions.

But search engines with interpretive capabilities that could enable better shopping bots are already being deployed. These engines can understand intelligent language by looking at concepts instead of fetching results tied to individual words. Copenhagen-based Ankiro has built a search engine equipped with intelligence that allows it to interpret exactly what a user wants, even if the desired result matches none of the words typed into the query, says Mikkel deMib Svendsen, Ankiro's vice-president for product development.

TALKING BOTS. For example, a question about marinara sauce might elicit an answer that also informed the reader about gourmet Parmesan cheese. Portal Terra Lycos already uses Ankiro's engine.

However, these highly intelligent search engines seem positively tame compared to the next wave. Ankiro is about to deploy dialog robots that would do customers' searches in a chat-like format. Users going to a search window would be greeted by name and asked how they can be helped today. Through a series of these questions, the robot would guide searchers to the content or destination they seek, explains Svendsen.

Unlike traditional search engines, Ankiro's bots will speak the vernacular. If you tell the robot "Cool!" it will interpret it as a positive response, says Svendsen. That implies future robots will accommodate not only different languages but also understand regional dialects and generational language differences.

Ankiro is about to unveil this technology at Denmark's libraries. The software, now available only in Danish, will be out in English and German versions by September. Most analysts say Ankiro's technology "should increase the [shopping] experience and improve sales," says Johnson of Position Technologies.

That would be a welcome change for frustrated Web shoppers tired of playing word games.

By Olga Kharif in New York

Edited by Alex Salkever

    Before it's here, it's on the Bloomberg Terminal.