Scoble's summer movie doesn't kick Google's butt

Robert Scoble livened up the August vacation time, when companies prefer to hold their announcements until after Labor Day (in the U.S. at least), with 35 minutes of video proclaiming that "SEO-resistant technologies like Mahalo, Techmeme, and Facebook are about to upend the search industry," meaning Google's dominant position in search is in trouble. (I would prefer a transcript I could read and cut and paste from in 10 minutes to the 35 minutes of video.)

Robert is a human information processor, reading thousands of items a day from 800 feeds, which roll around in his brain and are then transformed into blogs posts, podcasts, videos, food for thought and Scoble-filtered links. He is not one to spend days in quiet solitude contemplating the inner workings of the Internet and it future destinations. He is more of real-time processor, getting conversations started, digesting vast amounts of data each day and throwing it into his blender and seeing what bubbles to the surface.

In this case Robert takes off on Jason Calacanis' rant from Gnomedex 2007 that search engine optimization is a kind of spam and that Mahalo, his new human-powered search engine is the answer.

It's not a zero-sum game. Mahalo and human-power search, what we used to call a directory in the early days of Yahoo, has a place, but it's not the ultimate search solution. Read what Krish has to say:

Human powered search engine in the Mahalo mould cannot scale like an algorithm based search like Google. Even if we assume that Mahalo grows like how Robert expects in the next four years, with 100K “volunteers”, it still cannot cover all topics in the world, it still cannot cover all languages in the world, it still cannot covers all the specifics in a particular topic, it cannot cover all the variations of a particular query, etc.. It might cover a search query “HDTV” much better than Google. But I searched Mahalo with queries “Dent on HDTV” and “Scratch on HDTV” an hour back. It returned back saying there are no matching results. Even though it offers better results for HDTV, it fails miserably when you get into specifics on HDTV. Imagine the variations in queries on HDTV from all over the world, “HDTV with a spot”, “HDTV with a broken edge”, “HDTV and Bestbuy problem”, “HDTV with loose switch”, “HDTV’s remote not working”, “I just peed on HDTV”, etc. There is no way Mahalo can scale to take care of such specifics on even a single topic, along with everything else. Comparatively, HDTV is an easy problem to solve. There are much more difficult problems in this world which Mahalo cannot solve or may not bother to solve. Let us say I make a web page about a topic HDTV that contains a sentence like “not gone triple ribbon belt mother cook with mould acumen”. If someone searches for HDTV, my page may show up after the first 50 pages of Google results and not at all on Mahalo. But if someone searches for the sentence I mentioned above with double quotes, it will show up in the Google results and there is almost zero chance that it will show up in Mahalo (unless if my content is a killer content except for that particular sentence I quoted above). Now imagine endless new queries. Ask.com CEO claimed that 60% of their searches are unique. Even Google seems to have claimed that 20-25% of their total queries were not made before. Mahalo cannot even attempt to solve the search engine needs of all the internet users in this world, even if it scales like what Scoble hopes. It is like trying to build a 200 storey building without involving any machines and just with one brick at a time. It is just not humanly possible. Scaling problems will ensure that any attempts to create a completely human powered search engine is a failure.

I am a big fan of Techmeme and would like to see Gabe Rivera's technology applied to the larger Web, but that doesn't scale either. Techmeme's ranking is based on links out to popular stories and links in from a relative handful of authoritative sources. Not enough links, not enough good information to create a useful content clusters.

Facebook is a closed environment that can eventually rival Google in scale, but can each user be his or her own Mahalo, which in turn creates a massive social search fabric that factors in x-degrees of separation on multiple vectors, as Robert suggests in his whiteboarding video? Again, a Facebook search service could reach massive scale, but it would be complementary to other kinds of searches, such as the statistical, algorithmic Web crawler search that Google and others have developed. In addition, the likelihood is that Facebook will make its wall semi-permeable, and Google will come up with its own PageRank social graph search algorithms.

The key is not to expect Google, Yahoo, Microsoft or some as yet unknown company to have their blinders on and cease to improve their search engines. Powerset, for example, is applying crawling, linguistic analysis with natural language processing, semantic search and human engineered wisdom of the crowds to build a search engine that can deliver more accurate results. Hakia is also taking a semantic approach to searching the Web.

Ethan Stock has a down-to-earth analysis of what's going on with each of the services and Robert:

Here are the relevance mechanisms:

Mahalo - perceived prestige and accuracy of sites -- gathered by many social mechanisms Techmeme - blog link graph and textual correlation Facebook - 'social graph' of friends' interests, links, activity Google - web graph of hyperlinks, and increasingly textual correlation -- links made by people! Scoble - "ear to the ground" in Silicon valley -- via phone, email, twitter, the center of a social graph all his own

And here are the SEO mechanisms:

Mahalo - Persuade an editor that your site is important - PR or payola Techmeme - Persuade three bloggers that your site is important - again, PR Facebook - Persuade a cluster of friends to use your site and drive newsfeed items - marketing Google - Persuade (or fake) a bunch of hyperlinks from important sites to yours - SEO Scoble - Persuade him that what you're doing matters - PR

For an eviscerating critique of Robert's thesis, Danny Sullivan, the dean of search engine news and analysis, goes in-depth on each of the points.

The important thing to remember about Robert's videos is that he spawned a lot of interesting conversation, even if you don't agree with him. More on Techmeme, the clubhouse for the tech bloggers.

In addition, the bigger picture is that over time computers will get much smarter than humans, which will make the argument of human researchers editing directory pages versus machine-powered search moot. Listen to my conversation with Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence.