Alexa turning its Internet index inside out
Via John Battelle's Searchblog, Amazon's Alexa Web search engine is offering its index and a set of Web services--such as search, category browsing and metadata crawling--for building custom search solutions. According to Alexa's site, the index has 4.5 billion pages from over 16 million Web sites.
A correction: The new service is the Alexa Web Search platform, not the Web Information Service, which is focused on Alexa's preprocessed data. From Alexa's Web Search Platform beta site:
The Alexa Web Search Platform provides public access to the vast web crawl collected by Alexa Internet. Users can search and process billions of documents -- even create their own search engines -- using Alexa's search and publication tools. Alexa provides compute and storage resources that allow users to quickly process and store large amounts of web data. Users can view the results of their processes interactively, transfer the results to their home machine, or publish them as a new web service.
Here's the FAQ.
From Alexa's site on the Web Information Service:
The Alexa Web Information Service offers a platform for creating innovative web solutions and services based on Alexa's vast repository of information about the web. Developers, researchers, web site owners, and merchants can get information about Web sites, such as traffic data, contact info and related links, as well as an xml-based search engine and browse service, and incorporate them directly into their own Web sites or services.
The first 10,000 requests per month are free. Thereafter, requests are charged at a rate of $.00015 each (just 15 cents per thousand requests.). For example, if you make 100,000 requests to the Alexa Web Information Service during a given month, you will be charged $13.50. Your first 10,000 requests are free, while your remaining 90,000 requests are charged at a rate of $.00015 each: .00015 * 90000 = $13.50
From Battelle's post on the Web Search Platform:
Anyone can also use Alexa's servers and processing power to mine its index to discover things - perhaps, to outsource the crawl needed to create a vertical search engine, for example. Or maybe to build new kinds of search engines entirely, or ...well, whatever creative folks can dream up. And then, anyone can run that new service on Alexa's (er...Amazon's) platform, should they wish.
It's all done via web services. It's all integrated with Amazon's fabled web services platform. And there's no licensing fees. Just "consumption fees" which, at my first glance, seem pretty reasonable. ("Consumption" meaning consuming processor cycles, or storage, or bandwidth).
The fees? One dollar per CPU hour consumed. $1 per gig of storage used. $1 per 50 gigs of data processed. $1 per gig of data uploaded (if you are putting your new service up on their platform).
In other words, Alexa and Amazon are turning the index inside out, and offering it as a web service that anyone can mashup to their hearts content. Entrepreneurs can use Alexa's crawl, Alexa's processors, Alexa's server farm....the whole nine yards.
More disruption in the search engine world. Now let's see how Google, Yahoo and MSN respond...
Update: More on the topic from Om Malik, Richard MacManus, Phil Wainewright, Michael Parekh for starters...