Creators of Alltheweb say that within a year the engine will be able to catalogue more than one billion pages of content, encompassing the entire Web.
Currently the Internet consists of around 600 million documents. Altavista and its rival Northern Light, search around 150 million pages at a time others, such as Excite refer to around 70 million documents. Alltheweb currently contains 200 million searchable documents and is designed to increase that figure daily to catch-up with the growth of the Internet by next summer.
Technical director of Fast, John Lervik claims it is not just the size of Alltheweb that makes it revolutionary. "Even though we are much bigger than other search engines, we are also faster," he says.
Lervik says Alltheweb is capable of searching all its 200 million documents within a second as well as building a document database in just 12 hours. Existing engines take days or even weeks to update their databases. Lervik puts this difference down to super-efficient algorithms that have been in development since 1990.
Lervik also revealed that Fast is in the process of developing a number of features designed to further improve the efficiency of Alltheweb. "We are not yet as efficient as some engines, but we are working on three or four features to improve the relevancy of searches and make us as good as the best within a few months."
Additional search facilities, including a separate Boolean search, will also be added to Alltheweb in the coming months.
By a startling coincidence, Excite today also revealed plans to develop a potentially all-encompassing search engine by mid-August. However, it has not yet been disclosed precisely how many documents this engine will search when it is launched.