Techmeme's 6 editors signals potential trouble with Google PageRank

Gabe Rivera, founder of Techmeme, a popular news aggregator, yesterday announced he had doubled the number of editors to 6 people.

This is a very significant announcement because it shows that there is a problem in using search algorithms to discover new content. The problem is that there isn't enough linking happening between web sites (except by spammers).

Techmeme has been a staple of the tech news scene for many years. Gabe Rivera ran it by himself, relying on his search algorithm that monitored a fixed number of influential blogs and news sites.

He told me that to get the best results within a sector, you needed a large enough pool of blog/news sites otherwise the results would be unreliable.

The Techmeme algorithm counts the links to other blog posts coming from this core pool of influential sites. And thus Techmeme could run fairly well all by itself with occasional tidying up by Gabe.

I noticed about a year ago that blogs were linking less and less to each other, they were becoming a lot like old media, which hated to link to other sites. I asked Gabe about this.

I ran into Gabe last week at an event at the St. Regis in San Francisco and asked him about the state of the blogosphere. I pointed out that there seem to be few "real" bloggers left. Original bloggers such as GigaOm, ReadWriteWeb, TechCrunch, etc now all seem to be just online news sites and they read like an "old media" news site.

Gabe agreed, he said:"Techcrunch and the others used to link to each other and now they don't--they only link if they have to."

...And with fewer links, that means he has had to continually tweak his algorithm.

"I get around the problem by looking in many places for links or references to news stories, in places you might not normally look."

It looks as if Gabe ran out of places for his algorithm to look, because he has had to double his human editors to help find interesting new content.

Microcosm of search...

Techmeme is very much a microcosm of the search world. If Techmeme has had to go from 1 human editor (Gabe) and now to 6 in about a year, then this signals that all other search engine algorithms are in trouble, especially the biggest one of all: Google and its groundbreaking PageRank. PageRank lies at the heart of the secret Google algorithm. It is named after Larry Page, co-founder, and it also describes how it works.

PageRank counts the links to a web page and computes its relevance by assessing the importance of the linking site. If a page gets links from a high-ranked site that already has lots of links to it from reputable sites then that assigns a high rank to that page in Google results.

It works great. But only as long as people are linking to other relevant pages. It breaks down if people link less or not at all.

And that's what has been happening. I don't have any research to prove it but I have noticed over the past few years a large drop in the number of links on a web page to outside sites. I think it is a function of not having enough time to create the links; and also it isn't as fashionable as it once was, to pepper your content with dozens of links.

So, this means that Google's foundation for its search algorithm, PageRank is far less effective today than at any other time. It must roam further and look for signals of importance, and look for human help such as Twitter. But with Twitter, Google can't easily assess the importance of the "link-giver" because it is an individual, not a web site with history.

Google cannot do what Techmeme has done and hire editors to help surface the best search results. It would be impossible to cover the range of queries that Google handles every day.

This has to be GOOG's Achilles Heel. It will ultimately lose out to a horde of smaller search engines, such as Techmeme, using human editors to sort through the cache of links collected by algorithms.

But the algorithms, by themselves, won't be able to provide the same quality of results that they once did.

Techmeme doubling its editors to six is a watershed moment in search.