Google blames 'human error' for search 'malware' hiccup

Google is blaming "human error" for today's search results screw-up that flagged the entire Web as serving up malicious code.

But the company's explanation, detailed in a blog post by VP of search products and user experience Marissa Mayer incorrectly linked StopBadware.org to the glitch, causing major problems for the non-profit group.

StopBadware, which is run by Harvard Law School, Oxford University and Consumer Reports WebWatch, was already under a denial-of-service attack because of the Google hiccup and had to scramble to get Google to correct the misinformation.

[ SEE: Google flags entire Web as 'malware' ]

Here's the gist of Google's explanation:

• What happened? Very simply, human error. Google flags search results with the message "This site may harm your computer" if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers. We maintain a list of such sites through both manual and automated methods. We work with a non-profit called StopBadware.org to come up with criteria for maintaining this list, and to provide simple processes for webmasters to remove their site from the list.
• We periodically update that list and released one such update to the site this morning. Unfortunately (and here's the human error), the URL of '/' was mistakenly checked in as a value to the file and '/' expands to all URLs. Fortunately, our on-call site reliability team found the problem quickly and reverted the file. Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes.

Google promised to "carefully investigate this incident" and put more robust file checks in place to prevent it from happening again.

In addition to labeling all sites as being harmful, the hiccup caused Firefox to put up an erroneous "This is an attack site!" warning when users attempted to get to certain Web sites.  Here's what I found this morning when visiting BitDefender.com, the home page of a company that provides anti-malware software:

Obviously, this is a big problem when a Google glitch affects the reputation of a security vendor and it provides a harsh lesson on our total dependence on a single vendor/search provider.   Like Dan Geer and others warned more than five years ago, the monoculture presents a major risk.

This is very much applicable today.