Online Extra: How digg Uncovers the News

The founders of the wildly popular tech-news site explain how their mass of users sift and rank stories. Now they've got expansion plans

Overwhelmed by the fast-multiplying amount of content on the Web? Breathe easy: could provide some guidance, and some relief. The site, which mostly features news about the tech industry, filters stories from all over the Internet and presents them in a linear Google News-type interface. All digg's content comes from its users, who scour news sites, blogs, and other online sources for interesting tidbits.

The items users submit -- usually in the form of a short writeup and a link -- go into a queue, where members vote on their favorites. The 15 stories that attract the most votes -- or "diggs" -- are featured on the site's front page, which is updated several times an hour to keep the news fresh. It's this innovative "by the people, for the people" writing and editing system that sets digg apart from other Web sites. And it has some people calling digg the future of news.

Never heard of digg? You may soon. The 11-month-old site already has 80,000 registered members and 500,000 daily visitors, with 100,000 visitors being added every month. Fans include the influential media critic/blogger Jeff Jarvis and his 13-year-old son, Jake -- a loyal digg member who says he likes the power the site gives its members.

Now digg is about to get bigger. A coterie of Silicon Valley venture capitalists, including eBay (EBAY ) founder Pierre Omidyar's Omidyar Network, Netscape founder Marc Andreessen, and Greylock Partners recently invested $2.8 million in the startup. BusinessWeek Staff Editor Elizabeth Woyke spoke to digg founder and CEO Jay Adelson and co-founder and "chief architect" Kevin Rose at their base in San Francisco about how the site began. Adelson and Rose also discussed the epiphany they had after Paris Hilton's Sidekick was hacked and their aims to spread the digg way of life to the rest of the Net. Edited excerpts follow:

Where did you get the idea for digg?

Rose: [Last year, when I was working for Tech TV], I interviewed Rob Malda (aka Cmdr Taco), the founder of Slashdot, one of the largest tech-news sites. Slashdot is much like digg in that its users submit stories, but Slashdot has a handful of editors who promote the stories to the home page. [Malda] receives 400 to 500 submissions per day.

It seemed to me there might be some content I would find of interest that he might not be into. So I asked him, would you ever consider turning that content live to the users so they could browse it and find what they like? He wasn't really into adding that kind of functionality, but I talked with Jay later that week, and we started putting together digg.

Adelson: We started talking about this notion of how to leverage the collective mass of the Internet in various ways: applying it to content, using it to rank content, using it to make content more palatable to the masses. There are just so many different ways you can apply that.

What was the first big moment you had with digg, when you saw its potential?

Rose: Early on, in February, Paris Hilton's [Sidekick] cell phone got hacked. Images from it were posted on the Internet, along with some celebrity phone numbers. Someone within a few degrees of the hacker posted [the story] on digg. It was late at night, and of course it was dug to the home page in a matter of minutes. We woke up the next morning, couldn't pull up the site, and found out we were getting hit by an insane amount of traffic.

I can't even tell you how many users were coming to the site because the servers weren't up to capture it.… When we first put this together, it was very much an experiment run out of my house. It was during that point we saw, wow, this is really blowing up. We could see the power of breaking stories before anyone else.

Adelson: It attracted the attention of the news media immediately -- the fact that we had this incredible speed. Automated systems take time to crawl the net. Editorial systems have the human factor. They may decide they're not interested that day, or they'll do it tomorrow. In our case, there's no barrier, so the second a story would be interesting to this mass public, we can break it.

You let your users submit content and choose what's featured on the front page of the site. How do you make sure the stories are useful and interesting rather than just weird or sensational?

Adelson: The larger the critical mass of users and the collective wisdom applied to digg, the better and more relevant the stories get. The number of diggs needed to promote a story to the front page gets higher as the number of users increase, so you get a better editorial [product] than if you had a small group of users. Also, if users digg something and later change their mind, they can take the digg away. And they can only digg a story one time.

Rose: The users are very much a self-policing community. They have the ability to report a story if it's not appropriate for a particular category. {And] digg looks at those reports, and our computers will automatically remove stories once they hit a certain threshold of reports.

Adelson: We also have a system we call karma that helps prevent abuse of digging. The system knows the difference between users who log in and digg a story one time, or users that are created just to add a digg to a story, and someone who's on the site a lot, digging a lot of stories. We can rank those capabilities.

The power behind digg is collective wisdom. There are those who would argue it's more powerful than a search engine because a human's ability to determine whether a story is appropriate is much better than a computer's ability to do so. We're able to get a much better filter than if we relied on some kind of artificial intelligence.

digg has social-networking features -- like the ability to add friends to a profile -- and a pool of active, loyal members. Do you consider it a social-networking site?

Adelson: Instead of creating a social-networking platform and adding an application to it, we started with an application, and we're using components of social networking to expand the value of the site. There's no question that the power of the collective mass is what's interesting to us. In the case of social networking, it serves one very distinct purpose -- introduction.

Rose: One of the most popular features of the site is the "friends" section, where users can combine multiple feeds into one unified feed. I have about 15 different friends I've added. What digg does is take all the stories, comments, and other activity my friends are doing on the site and unifies it into a single RSS feed that's updated in real time. I can subscribe to that one feed and see what all my friends are digging and commenting on.

What do you think users find helpful about digg?

Rose: The biggest benefit users see is they're able to come to digg, read stories, and know they're reading them before they get picked up on the major news sites.… Users are really starting to use digg as a tool more than anything else.

We also added RSS to our search results. It sounds boring until you figure out the power of how it actually works. You can enter a keyword into digg, do a search, and bookmark that page. digg will work nonstop for you looking for stories with that keyword that have been dug to the homepage or have a certain number of diggs. You're getting instant, relevant content directly to your browser.

Adelson: [We've heard] that on digg content comes faster, it's more relevant, and it saves them time. There's this notion that once you're using digg, it really makes other ways of sifting through data obsolete. We saw testimonial after testimonial about this.

What's the story behind your popular podcast/videocast, diggnation?

Rose: That's something I came up with about four months ago. We had this great pool of stories, and a lot of them were very unique because other sites weren't picking them up. I come from a tech TV background and have friends with similar backgrounds, so I thought, let's do a weekly podcast and highlight what users are submitting and just have fun.

Apple's (AAPL ) launch of a podcasting section within iTunes has really helped its popularity. We stay in the top 20 ranking on their site, and that brings us a few thousand new listeners every day.

What do you plan to do with your new venture-capital funding?

Adelson: There's some capital up-front to expand the servers and bandwidth, but the real money is going toward the people it takes to develop the site, create features, and maintain operations. We're just beginning to create an office in San Francisco where we have developers who are going to be helping us with the next generation of digg. Right now, we have about six employees.

Can you talk about your plans to expand and improve the site?

Adelson: We created digg intending to focus on technology information. What we've seen is that the concept bleeds over and has a strong pull toward other areas of news. I think that's maybe what the future of digg will be -- to move into different areas of content besides tech.

Rose: Users are begging for other versions. They want politics and business and to really blow out the different science categories. So it's just a matter of time.

Adelson: One of the things we're already developing is making digg as customizable to the user as possible. You may want to create your own version based on certain interests or create category views that allow you to see those interests. There are lots of different ways we plan on presenting the data.

Rose: We're also working on freeing up a lot of the data we have. We're going to be offering an API [application program interface] in the next few months that will allow users to tie in to the data and manipulate it in any way they see fit. They can pretty much create any type of application they want around the Web site.

We're also learning about the content users are digging, so in the future digg will be a little bit smarter. So if you've dug stories around Linux and oolong tea, digg will know that and make recommendations.

What about taking your methodology and applying it to the rest of the Web?

Adelson: digg is all about leveraging collaborative wisdom to make the information on the Internet more convenient. There are a lot of ways you can apply this concept. We've been contacted by researchers and scientists about how digg's model could be applied to things like education, publishing papers, and collaborative science. The peer-review systems that exist in the world today could be easily translated from tens of peers to millions of peers. The world is open. The question is what are we going to choose to do first?

    Before it's here, it's on the Bloomberg Terminal.