Interview with Digg founder Kevin Rose, Part 1

Yesterday I interviewed digg founder Kevin Rose to discuss digg's popularity, its battle with spammers, the recent issues with GroupThink and digg's upcoming personalization features. This is the first of a two-part article presenting that interview. NB: Part 2 available now.

As you know, digg.com is a technology news site that over the past 6 months has begun to rival Slashdot in popularity among IT geeks. But whereas Slashdot content is controlled by editors, digg prides itself on being run by its community. Which articles make it to the coveted digg homepage is determined by how many "diggs" they get from the readers. digg calls this "non-hierarchical editorial control", in a none-to-subtle dig at Slashdot. 

We plan to add the ability to mark stories as false or inaccurate However in recent weeks there have been signs of possible cracks in this user-driven system, with some people wondering whether the 'wisdom of the masses' is more like a mob mentality at times. I myself have published a couple of posts critical of the digg system, firstly about their (what I at the time perceived to be) overzealous response to spam and then there was the highly controversial group tarring and feathering of an innocent O'Reilly writer. Both of those incidents made me question the system, although I continue to be impressed by digg's potential. 

I thought what better way to get to the bottom of these issues than to interview the people behind digg - and so I'm grateful that Kevin Rose agreed to talk to me. The interview will run over two days and here now is Part 1.

The Tipping Point

Richard: How and when did digg get started?

Kevin: We started working on developing the site back in October 2004. We started toying around with the idea a couple of months prior to that, but it was early October when we actually started creating what would become the beta version of digg. The site launched to the world on December 5th 2004.

Richard: How long did it take to gain momentum and a large user base? It's extremely popular now, but how long did it take to get to that tipping point?

Kevin: We never had any idea it would become this popular. It was just a side project of mine and something that we wanted to try out by giving power and control back to the masses. Typically with technology news sites a handful of editors choose which stories are relevant and which they believe the audience would like to read about. This was the first time that anyone experimented with allowing the general mass audience to decide what they believed to be the most important topic of the day. 

So initially there were a few months where everyone had to wrap their head around the idea of how the entire digg concept worked. Around February or March 2005 we started to see the site take off Because when you first come to the site you see the number of diggs, but initially you don't understand the benefit of digging something and how that's also bookmarked into your own personal profile. Every activity you do on the site is saved as an RSS feed and can be shared with friends. The site increases in value with each new user added to the critical mass already on digg. 

It was around February or March of 2005 that we really started to see the site take off. One of the things that pushed us into the mainstream was the Paris Hilton cellphone hack, in which her cellphone images and data were stolen and posted online. Someone posted the story on digg late one night and because of the popularity of the story it was quickly dugg onto the homepage. We were then indexed by Google and Yahoo that night under the keywords "paris hilton cellphone hack" and when we woke up the next morning, we saw our site was crawling along... because there were hundreds of thousands of people that were typing those keywords in - and the first site that would come up was digg. So it was a really strange incident at the time, but it showed us the power of using the masses to break news stories. 

It works by users finding relevant information and sending it to the digg community, which then takes that story and diggs it - and once it gains enough momentum and interest, it's promoted to the homepage for all to see.

Reputation Systems - does digg have one?

Richard: I'd like to talk now about reputation systems. I wrote a post recently about reddit's karma system, which is similar to Slashdot's reputation system. I wondered whether the same thing will be built into digg, or whether you think it's necessary even?

Kevin: Well there is a karma system that exists behind the scenes of digg.  It is basically used for spam and fraud detection. So it's an automated system where we can detect where there are people trying to abuse the system. It identifies those users and notifies an administrator of activities that violate our terms of service.  The administrator then takes appropriate action based on the nature of the violation. 

In the future when we release the next version of our comments system For the most part it's self-policing by the users we're going to allow people to gain a reputation by the ability to digg or bury comments that they like or dislike. But for the most part it's the self-policing by the users of the community that helps keep things in check. Users have the ability to report a story that they believe is irrelevant or doesn't pertain to tech or is a duplicate entry. It's the users of the community that are performing all the duties that would typically be performed by a standard admin on the site. So, for example, users can "bury" a story and when enough users do that it's removed from the queue, but not deleted from the site.

Richard: So is it fair to say then that rather than a reputation system for people, it's more of a system for the story itself? That is, you're not voting on whether the person commenting on the story is good or bad, or whatever, but you're voting on whether the story itself is good or bad. Is that a good way to look at it?

Kevin: That's a good way to look at it. Rating individual users really doesn’t scale well The problem with rating individual users is that it really doesn't scale well. We have 160,000 registered users and it would be impossible for everyone to go through and directly rate everyone else. What we have created, which is quickly becoming a popular way to discover new stories, is our friends system. This enables you to create a group of trusted users who read each others articles and trust each others content. You can combine articles and friends that you're tracking into something almost like a watchlist. You're into what they're digging, you're into what they're submitting and commenting on. And digg takes all that information under your own personal profile on digg and combines it all together in a single feed for you to pick through. 

The GroupThink Issue

Richard: That leads to the question of groupthink. There's been a bit of discussion recently about whether digg users actually add their own personal opinions on stories, or whether they just jump on board whatever train of thought the community has at the time. Do you think that's a concern for you, are you trying to address that issue? I'm thinking specifically of the incident with the O'Reilly writer - are you doing anything to improve the system in that respect?

Kevin: Since news breaks on digg very quickly, we face the same issues as newspapers which print a retraction for a story that was misreported. The difference with digg is that equal play can be given to both sides of a story, whereas with a newspaper, a retraction or correction is usually buried.

In the case that you mentioned with the O'Reilly writer, it was an inaccurate story that was promoted to the homepage, and then quickly thereafter, a correct story was submitted and made the digg homepage as well. 

We plan to add the ability to mark stories as false or inaccurate, so that other users can see they've been marked inaccurate. There's such a mass of stories coming in at a very fast rate -- it's nice to know that as quickly as digg can break stories, they can also break follow-up stories that might contain additional information or a follow-up to the original story. 

Richard: Staying on the issue with the O'Reilly writer for a minute, he posted a follow-up post on his blog saying that he was concerned about the original digg article being on the first results page of Google when people do a search on his name. Is there any way for digg, or Google even, to make amends on that - because obviously that's a bit harsh on the poor O'Reilly writer...

Kevin: Within the next week, we're turning a new system live that will immediately alert readers if a story has been marked as inaccurate - the story will display that right off the bat, right at the top of the story itself. So the users will know as soon as they see the story... much like a Wikipedia entry that contains information that is potentially inaccurate, it's marked in the same way.

In Part 2 of our interview, Kevin talks about digg's fight against spam and the upcoming release of new personalization features.