Can the AP Out-Google Google?

To compete with what it deems Google's "misappropriation" of its news, the Associated Press wants to fight back by building its own news aggregator

The Associated Press is hopping mad over what it calls "misappropriation" of its content online. But the news service isn't just complaining about it or threatening legal action against Web sites that it says unlawfully reproduce its news stories.

In a largely overlooked aspect of its battle with Google (GOOG) and other aggregators of news content, the AP plans to build an online destination where it hopes Web users can easily find and read its news stories and those of other content creators. When it comes to compiling online news, the AP wants to out-Google Google. The Web search giant "has a wacky algorithm" for collecting news stories, AP Chief Executive Tom Curley says in an interview. "It does not lead people to authoritative sources."

As an alternative, Curley plans to create "landing pages" that would host articles from any news sources that allow their headlines on the site. Participating outlets would share revenue generated by ads placed on those pages, "monetizing content in an ecosystem that would be different from the Google ecosystem," Curley says. The sites would include both national and local media outlets. "There are some pretty exciting ways to showcase some of the core features of local newspapers," Curley adds.

Online Ad Slump Presses Publishers

AP's approach would challenge the status quo whereby aggregators such as Google collect and organize information from across the Web, often showcasing a small snippet of a news story. By clicking on the associated link, a user calls up the full story, which resides on a newspaper's own site or with some third-party host—in some cases, Google itself—that shares ad revenue with the content's creator. "What the AP is doing is trying to create a mechanism to reestablish the balance of power between search engine companies and content creators," says Art Howe, a former newspaper publisher and Pulitzer Prize-winning journalist from The Philadelphia Inquirer. "They are taking the first steps to creating a new model that benefits the publishers—the people who are paying writers and editors and photographers."

The old system worked well for years as advertisers dedicated ever-larger portions of their budgets to online advertising. It has come under increasing fire recently, however, as companies slash ad spending amid recession, says Alan Mutter, a media industry veteran who now operates the blog Reflections of a Newsosaur. "The entire philosophy of almost all news companies has been to build page views and to sell as many ads on those page views as you can," Mutter says. "It worked pretty well until the year 2008." These days, more Web publishers are chasing fewer online-marketing dollars.

Some newspapers have already experimented with their own aggregation tools. In December, The New York Times (NYT) introduced a trial service called "Extra," a version of its home page with links below each major story to similar stories from competing news sources.

Coding to Lock Out Google's Robots

To succeed, the AP and others will need to lure readers who have grown accustomed to searching for news through Google and other aggregators. Currently, 19% of all traffic to news and media sites comes from Google's search, news aggregation, and other pages; another 13% comes from Yahoo! (YHOO) pages, according to numbers obtained from Hitwise, which compiles data on Web traffic.

Sites that really want to keep their content separate from Google's pages could use code known as "robots exclusion protocol" that would block Google News and others from indexing the content in the first place. "It is the equivalent of Google and other search engines giving a key to publishers that empowers them to lock down their content to ensure that it is not indexed," says Google spokesman Gabriel Stricker.

That approach is likely to alienate readers who are used to browsing the Web however they choose, says Jeff Jarvis, director of the interactive journalism program at the City University of New York's Graduate School of Journalism and author of the book What Would Google Do? Jarvis says this is akin to asking: "How can we keep people from doing what they want?"

Schmidt: Google's Technology Works

Newspaper publishers complain that aggregators have created a system that lets Web surfers skim headlines and brief excerpts without clicking through to get the whole story. They also allege that aggregators promote stories from less credible sources.

Speaking to the Newspaper Assn. of America's annual conference on Apr. 7, Google CEO Eric Schmidt addressed those concerns, explaining that the company uses complex mathematical formulas to "determine rank and relevance." He added, "There's no question in my experience that the top brands represented in this room would, in fact, float to the top in our search ranking."

Jarvis argues that rather than trying to compete with Google, newspapers should band together to "try to make Google News better." He suggests they could do that by coming up with a system of inserting code in stories that could, for example, alert the search engine to the ones that were first with a particular piece of news, or those that have a particularly rare source.

Before it's here, it's on the Bloomberg Terminal.