madison

Google fallout: One in seven users hit by outage

Rupert Goodwins ZDNet.co.uk | May 15, 2009 4:56 AM PDT

Summary

Google's network updates cause worldwide problems - routing traffic to the wrong locations. But the company promises it won't happen again.
About one in seven Google users around the world lost the use of some or all of the company's services on Thursday as a result of a major network outage.

Blamed by the company on a systems error that caused network traffic misrouting through Asia, the failure occurred at approximately 7:15 a.m. PST and lasted for two hours. The outage affected Googlemail, search and other services. Many UK users were affected, according to online comments on Twitter and other forums, although people reported rapid changes in connectivity over time and dependent on which ISP they were using.

Google issued a statement on its blog after the event. "An error in one of our systems caused us to direct some of our web traffic through Asia, which created a traffic jam. As a result, about 14 percent of our users experienced slow services or even interruptions. [...] We're very sorry that it happened, and you can be sure that we'll be working even harder to make sure that a similar problem won't happen again."

In October, Google extended its service guarantee for paying Gmail customers to other Google Apps, including Google Docs and Google Calendar. The agreement guarantees that service will be available 99.9 percent of the time to premium-service customers, or Google will pay a refund. In its announcement, Google pledged to improve communication with customers if its services went down.

"We've made a series of commitments to improve our communications with customers during any outages, and we have an unwavering commitment to make all issues visible and transparent through our open user groups," Matthew Glotzbach, director of Google Enterprise product management, wrote in a blog posting in October.

Although the exact reasons of Thursday's failure are not known, internet service provider operators and monitoring companies have detailed some of the effects.

Craig Labovitz of Arbor Networks, an ISP services company, reported that traffic across ten major North American ISPs collapsed from 15Gbps to around 1Gbps.

"Outage began roughly at 10:15am and lasted through 12:15pm EDT. Looking at the data, most large transit providers appear to have been impacted (e.g., Level3, AT&T, etc.). Other providers (e.g. large consumer DSL / Cable) showed no drop in traffic from/to Google", Labovitz wrote in a company blog.

This article was originally posted on ZDNet UK.

Talkback Most Recent of 10 Talkback(s)

  • Google News is broke today....
    8 am CDT and I can't find my news anywhere.
    ZDNet Gravatar
    nucrash
    15th May 2009
  • Look out of the window
    Look out of the window... that is a place called "the real world" - that's where the news is.
    ZDNet Gravatar
    Chalfont
    15th May 2009
  • Window doesn't show much
    There's an alley with some broken construction equipment - the trains are going by - can't tell if they're on time happy

    No - looking out my window doesn't tell me much about what's going on in the "real world"
    ZDNet Gravatar
    dougs.zdnet@...
    15th May 2009
  • WHERE IS THE "REAL" WORLD?
    I've never been to the "REAL" World...But my friend Brenda--one night-- many years ago in Silicon Valley looked UP into the night sky and said: "Do YOU have ANY IDEA How MANY billions of dollars are being 'transfered electronically' through the air up there?!"
    Anyone wanna help me write a wireless Program...?! Cause "THAT IS REAL!" Is that what you're talking about?!

    CaptainOfTheSees
    ZDNet Gravatar
    indexster@...
    18th May 2009
  • It is not Google's fault
    there just happens to be nothing news worthy going on today! happy
    ZDNet Gravatar
    GuidingLight
    15th May 2009
  • RE: Google fallout: One in seven users hit by outage
    Remember that Google uses only linux in their infrastructure and that is what caused this issue. So we can conclude that linux cannot scale well by any means and if you are thinking of implementing it in your enterprise you should plan to have global downtime while you try to get it working. I won't place the blame squarely on linux although that is where most of the blame lies, but some of the blame is on Google's employees as well. They have consistently neglected Google's services and would rather sit around playing with office toys all day rather than trying to build a stable service.
    ZDNet Gravatar
    Loverock Davidson
    15th May 2009
  • Lol....
    Heh, gave me a good laugh there. Care for an encore?
    ZDNet Gravatar
    Sephoroth
    15th May 2009
  • Thats a little too much information....
    For someone that has no clue of the true problem. Google will not release that and unless you work there we can't make any determination at this point. I think way too many people make assumptions in this world. Get the facts then lets talk about it.
    ZDNet Gravatar
    OhTheHumanity
    15th May 2009
  • ZDNet Gravatar
    VoiceOfLogic
    15th May 2009
  • routing
    It was a routing problem. Doesn't google use cisco hardware and IOS? The article should have looked a bit deeper and reported a bit more detail on what "system" caused this and what exactly was the "problem."

    Was an update rolled out? Hardware failure? Human error??
    ZDNet Gravatar
    pgit
    15th May 2009

Talkback - Tell Us What You Think

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity