Gmail's disappearing act: Blame the storage software update

Summary: Google said Gmail will return for all users in the next few hours and that lost email archives will return.

By Larry Dignan for Between the Lines | February 28, 2011 -- 19:30 GMT (11:30 PST)

Follow @ldignan

Google said late Monday that Gmail will return for all users in the next few hours and that lost email archives will return.

Gmail went dark for a small subset of users over the last few hours. In a nutshell, some folks found their Gmail account empty and archives were nuked.

Needless to say, this situation had a few folks reconsidering their cloud computing thoughts. At the very least, backup plans were being concocted. On the Enterprise Irregular list, it was evident that backup plans---and specifically options for Gmail---were sorely lacking.

Fortunately, Google didn't lose your data. Here's what Google outlined in a blog post.

0.02 percent of Gmail users were affected on Sunday.
Google is "very sorry."
The data wasn't lost and most of those affected has service restored.
Multiple copies of the data was stored, but a software bug deleted some of the mail.
Now Google is restoring the data from tape backups.
A storage software update introduced the bug and Google stopped the deployment.

What should you do in the future? Your options are to backup your Gmail in an email client, autocopy to another Google account via POP and autocopy to a non-Google account.

Topics: Cloud, Browser, Collaboration, Data Management, Google, Storage

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

20 comments

Log in or register to join the discussion

RE: Gmail's disappearing act: Blame the storage software update

When you outsource / move to the cloud / services you get what you pay for. To be fair Google has done great so far, but they're big enough to start having issues now. I don't trust them enough to dismiss the possibility that this wasn't malicious either.

ITSamurai
28 February, 2011 19:54

Reply Vote
- RE: Gmail's disappearing act: Blame the storage software update
  
  @ITSamurai So they'd try to lose all of those emails? Yeah... That sounds like a great business decision. Imagine how that hypothetical meeting would have gone, "Let's make ourselves look like an awesome email solution by losing tons of email!"
  
  snoop0x7b
  28 February, 2011 20:21
  
  Reply Vote
- RE: Gmail's disappearing act: Blame the storage software update
  
  @ITSamurai What exactly was so terrible as opposed to an Exchange blackout in a large company. Suppose this happened in a company with 100000 users effecting 200 users (0.02%). The service was restored, backup procedures worked...
  
  prof123
  1 March, 2011 07:36
  
  Reply Vote
  - RE: Gmail's disappearing act: Blame the storage software update
    
    @prof123 : In other words, GMail is as fault-prone as a non-cloud solution. Hence, we can erase any supposed reliability advantage GMail may have claimed.
    
    nomorebs
    1 March, 2011 16:40
    
    Reply Vote
RE: Gmail's disappearing act: Blame the storage software update

Let that be a lesson to Google... Don't cheap out on storage, buy EMC!

snoop0x7b
28 February, 2011 20:20

Reply Vote
- RE: Gmail's disappearing act: Blame the storage software update
  
  @snoop0x7b
  Oh brother.
  
  rossdav
  1 March, 2011 12:30
  
  Reply Vote
RE: Gmail's disappearing act: Blame the storage software update

Buy EMC? You mean the company that was behind the Virginia DMV's 3-day outage? ( Read about it here: http://www.zdnet.com/blog/btl/virginias-it-outage-doesnt-pass-management-sniff-test/38609 ) Every product has issues. EVERY ONE. That's why you wrap them in process and procedure -- and you make backups.

wcpreston
28 February, 2011 23:44

Reply Vote
RE: Gmail's disappearing act: Blame the storage software update

BTW, the point of my previous comment was not that EMC is bad. My point is that any good piece of storage can be ruined by faulty humans or lack of process. (The going story is that in the VA incident, someone replaced the GOOD piece of hardware in the fault tolerant system instead of replacing the BAD one. Ouch.)

wcpreston
28 February, 2011 23:47

Reply Vote
RE: Gmail's disappearing act: Blame the storage software update

One last thing. There are services that handle backup of gmail and other online accounts. Check out backupify.com and backupmyemail.com. (I do not sell either of them, BTW.)

wcpreston
28 February, 2011 23:52

Reply Vote
Google created panic for the rest of 99.9999% of users. PERIOD

Oh, pls. Google is one such company run most of thier apps over web in Beta for quite long time, and Gmail was such an app, was in Beta for years. Why in this world such a co. like Google kept it in Beta for so long, made it to general public after several years, went ahead against MS and filed a lawsuit against US State Govt. for they choosing MS Exchange as their email solution, oops...! I'm tired.

Now, their site is down, Oops, Gmail ? Then, next week, say comeback and say 0.00001 % accounts are RESET (oops !), then saying it's only 500,000, next day no no, it's only 40,000.

Give me a break ! They created a panic for rest of the 99.999999 % and awakened that Google is yet another Yahoo/Hotmail/Facebook nice to have app's over web, but, not reliable. Better keep it backed up in your local PC's, take 2-3 backup of it in external drives, have it backed up in some cloud servers like Amazon S3/Microsoft Azure thru backupify.com (I searched for one online backup solution yesterday and configured it) and have one of your external safest external drive in a firesafe box, etc.

Pls. dont' come and support Google.

jinishans
1 March, 2011 04:54

Reply Vote
- RE: Gmail's disappearing act: Blame the storage software update
  
  @jinishans
  Umm. Exaggeration much?
  You also have an unhealthy attachment to your email.
  
  rossdav
  1 March, 2011 12:31
  
  Reply Vote
I wonder what that Donnieboy's take on all this will be

since he made it a big deal when it happened to MS, saying use gmail because it won't happen.

Will he have learned a lesson about the cloud overall?

Will Farrell
1 March, 2011 06:02

Reply Vote
And then there was...

backupify

Dietrich T. Schmitz, ~ Your Linux Advocate
1 March, 2011 06:36

Reply Vote
- RE: Gmail's disappearing act: Blame the storage software update
  
  @Dietrich T. Schmitz, Your Linux Advocate
  Backupify has had issues.
  
  daikon
  8 March, 2011 16:32
  
  Reply Vote
Transparency is a beautiful, beautiful thing!

.02% of users are affected and it creates a national news event. This is a glorious time in technology for consumers, individuals and businesses. I believe this new level of transparency is one of the biggest advantages of the multi-tenant cloud computing environment.

For the first time the user community has a voice that is stronger than the vendor's and they know it. Previously the customer would be scrambling - "is it our hardware, Exchange Server config, the new SAN maybe, where's the DR plan?" Let's figure it out. Call the vendors, call the consultants, and work the issues. I always thought support payments for on-premise products were ironic since it is actually the customer that supports everything. Are other customers impacted by the same config/hardware/software issues? Is it our fault or the vendors? Who knows. "It's complicated."

Multitenancy raises the bar for the vendor. .02% is not OK. Who's problem is it? Google's problem. Who's fault? Google's. Who pays the bills to get it fixed? Google. Who goes through the after event review to learn from the mistake? Google. Who benefits from the lessons learned? Everyone.

Technology services are becoming a utility - always on, no excuses. This is extremely exciting.

@...
1 March, 2011 08:01

Reply Vote
It's not Linux's fault

that Google sucks so much. I personally blame the LSE.

iPad-awan
1 March, 2011 08:39

Reply Vote
Lack of SDLC Process @ Google?

Perhaps someone @ Google should explain why this 'storage software update' wasn't thoroughly tested before it was rolled into production. Here's a link for Google to reference in the event they are oblivious to the concept of SDLC:<br><a href="http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle" target="_blank" rel="nofollow"><a href="http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle" target="_blank" rel="nofollow">http://en.wikipedia.org/wiki/Systems_Development_Life_Cycle</a></a>

cybersec_guru
1 March, 2011 09:23

Reply Vote
Just rambling on to myself...

Back up stored email to local computer? Okay: got it! Back up Contacts to local computer? Okay: got it (including continuous synchronization)! Then, the cloud server goes down. If the cloud server goes down and an email comes and I don't see it, how do I know? Kind of like an etree falling in the eforest... I have an email server on my own domain, forwarding a copy to Gmail for spam filtering, then downloading to me. Double backup. But still, not foolproof. I'm a fool, and I'm danged good at it.

fjpoblam
1 March, 2011 09:55

Reply Vote
attent

<a href="http://www.yozgatsohbet.com/" title="yozgat sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbethatay.com/" title="hatay chat, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.sohbetadana.net/" title="adana sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.adanasohbet.org/" title="adana sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.antalyasohbet.gen.tr/" title="antalya sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.mersinsohbet.org/" title="mersin sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetmersin.org/" title="mersin sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.bursasohbet.tk/" title="bursa sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.trabzonsohbet.tk/" title="trabzon chat, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.denizlisohbet.tk/" title="denizli, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.mardinchat.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbethatay.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.ankarachat.tk/" title="ankara sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.izmirchat.tk/" title="izmir sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.gaysohbet.info/" title="gay sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.cinselchatsohbet.com/" title="cinsel sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.erzurumsohbet.com/" title="erzurum sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.evlisohbet.org/" title="evli sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.islamichatsohbet.com/" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.istanbulsohbet.name.tr/" title="istanbul sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.kalpgulu.org/" title="cet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.karadenizsohbet.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.lezsohbeti.tk/" title="lez sohbet, sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sicaksohbet.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.zalimcafe.com/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.zurna.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetodalari.ws/" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.sohbetodalari.biz/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetodalari.biz.tr/" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.sohbetodalari.bbs.tr/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetodalari.be/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetodalari.info/" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.sohbetodalari.name.tr/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetodalari.com.tr/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.avrupasohbet.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sexsohbetchat.com/" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.muglasohbet.tk/" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.ankarasohbet.gen.tr/" title="ankara sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.izmirsohbet.gen.tr" title="izmir sohbet, sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.sakaryasohbet.gen.tr" title="sakarya sohbet, sohbet odalari, sakarya chat" target="_blank">sohbet odalari</a>
<a title="sohbet kanallari" href="http://www.sohbetkanallari.gen.tr">sohbet kanallari</a>?<a title="radyo dinle" href="http://www.radyolardinle.net">radyo dinle</a>?<a title="radyo dinle" href="http://www.radyodinle.name.tr">radyo dinle</a>?
<a title="mirc, mirc indir" href="http://www.mirclerindir.com">mirc</a>?<a title="cet" href="http://www.cet.name.tr">cet</a>?<a title="dul sohbet" href="http://www.dulsohbet.gen.tr">dul sohbet</a>? <a title="chat odalari" href="http://www.sohbetodalari.com.tr">chat odalari</a>?<a title="chat odalari" href="http://www.konyachat.tk">chat odalari</a>?
<a title="sohbet odalari" href="http://istanbul.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://izmir.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://malatya.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://ankara.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://hatay.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari, kayseri sohbet" href="http://kayseri.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://ordu.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://elazig.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://igdir.sohbetodalari.com.tr">sohbet odalari</a>?<a title="sohbet odalari" href="http://agri.sohbetodalari.com.tr">sohbet odalari</a>?<a title="cet" href="http://www.eceerken.net">cet</a>?<a title="cet" href="http://www.sohbetkizi.org">cet</a>
<a href="http://www.girhadi.com" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.muhabbetodalari.net" title="sohbet odalari" target="_blank">sohbet odalari</a><a href="http://www.sohbetturkchat.com/" title="turk chat" target="_blank">turk chat</a>- <a href="http://www.turkcechat.tk" title="turkce chat, turk chat" target="_blank">turk chat</a> - <a href="http://www.kalpgulu.org/" title="cet" target="_blank">cet</a>
<a href="http://www.odalari.gen.tr" title="odalari" target="_blank">odalari</a><a href="http://www.islamichat.net" title="islami sohbet, islami chat" target="_blank">islami sohbet</a>
<a href="http://www.dinisohbet.gen.tr" title="dini sohbet" target="_blank">dini sohbet</a>
<a href="http://www.sohbetbu.net" title="sohbet odalari" target="_blank">sohbet odalari</a>
<a href="http://www.idealsohbet.org" title="sohbet odalari" target="_blank">sohbet odalari</a>
Beni takip edenin gotune koyim =)565

Firat31
15 August, 2011 13:11

Reply Vote
RE: Gmail's disappearing act: Blame the storage software update

I think you're too hard on by Google. In such a large company may have problems because of the staff. Who prevents you to store e-mail archive on my computer?
http://bank-of-ideas.ru

http://bank-of-ideas.ru/
30 August, 2011 11:26

Reply Vote