The googlebot can find pages not directly linked

Summary:Google has the ability to find the location of pages/files that have no direct links to them. If it has a web address Google can find it.

Dell's confidential specs for future Dell notebooks were discovered and distributed by Google. Elinor Mills from here writes a web primer on how to avoid the problem by using a robot.txt file and by not linking to sensitive materials.

However, did you know that the Googlebot can find pages that have no direct link from the home page of a web site? That's what a Google engineer said at a search conference a couple of years ago.

If I find the reference I will post it, but the gist of it was that Google has the technology so that it can find and catalog information on a server without having to follow links.

And you would expect Google to have such technology since its mission is to index and copy all of the world's information. Not "all linked information" but "all information."

- - -

Here is an interesting discussion on this topic.

Topics: Google


In May 2004, Tom Foremski became the first journalist to leave a major newspaper, the Financial Times, to become a full-time journalist blogger. He writes the popular news blog Silicon Valley Watcher--reporting on the business of Silicon Valley. Tom arrived in San Francisco in 1984, and has covered US technology markets for leading comput... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.