7 important survival tips Amazon's orphaned 0.07 percent can teach us

7 important survival tips Amazon's orphaned 0.07 percent can teach us

Summary: Service Level Agreements won't save your butt when the technology's out to get you.

SHARE:
TOPICS: Storage, Amazon, CXO, Hardware
6

Cloud technology is hard to build. Even if you're a company with all the cloud experience of Amazon.com (probably one of the two biggest brain trusts on the planet for cloud implementation), making it all work can be a challenge.

Yesterday, the Register reported that about 0.07 per cent of Amazon's lost EBS storage volumes in the East Region of its infrastructure cloud are not fully recoverable.

In other words, some data is permanently lost.

This, of course, is in reference to Amazon's devastating cloud failure last week.

See also: Seven lessons to learn from Amazon's outage

See also: Amazon Web Services outage: 'Detailed post mortem' coming

If there's any one company you would think you could trust with your data, it'd be Amazon. Even more than Google -- who's likely to hold your data, and then subject it to some sort of inhuman scrutiny, trying to extract the unified field theory, the meaning of life, and what your favorite breakfast cereal is from the data stored in your online documents -- Amazon has always seemed to be the most trusted keeper of the sacred cloud.

But those who fly too high are apparently destined to fall to Earth, and Amazon's cloud hid within it not just a silver lining, but bolts of lightning.

[It's at this point that your esteemed author must apologize for the metaphors he's mauling. As you all know by now, he can't help himself. Now back to our story.]

While Amazon's customers are undoubtedly disturbed by the existence (actually, lack thereof) of unrecoverable data, there are important lessons to be learned here, both for private sector customers and -- especially -- government customers moving increasingly to the cloud.

See also: Clouds, consolidation, culture: making federal IT less 'horrible'

See also: Why killing Data.gov is wrong-headed and stupid

See also: Senate committee stifles White House cloud computing caper

Lesson 1: Don't trust any single vendor

It has always astonished me that people trust their data to the cloud. As a guy who's operated my own mail server since before there were combustion engines, it's never seemed quite right that consumers and businesses would put all their critical information into someone else's hands.

I know it's a pain to manage it all yourself, and I know free seems like such a good price, but if you've been around the block more than just a few times, you'll realize that even some of the most promising companies can fail (or change their behavior).

If you care about your data (and you should, you really, really should), then make sure that you don't rely on just one vendor. That vendor could go out of business, be acquired by a competitor, or just turn into a complete and total a-hole.

Lesson 2: Always back-up your data

I know this should be obvious, but it's apparently not. I can't tell you how many whining babies I've encountered over the years who were too stupid or too lazy to back their stuff up.

Which brings me to...

Next: Five more lessons »

« Previous: Amazon's missing 0.07 percent

Lesson 3: If all your data's in the cloud, back up to a local environment

I don't believe that just because your data is on a cloud vendor's system and they supposedly back stuff up, that it'll really be backed up.

You should always keep a local copy (on your own computers -- big drives are cheap these days) of all your data.

If you don't know how to back up those live databases, learn. If you don't know how to extract data from your running systems, figure it out.

And if you think you have too much data or your data set is too large to download each day, consider using rsync, which will do excellent incremental backups and was designed for just this sort of thing.

And yes, this is my second mention of rsync this month.

See also: So, isn't it time, really, for iPhones and iPads to sync wirelessly with iTunes?

Lesson 4: Then back that up

So, now that you've downloaded your cloud data to your local machines, back that stuff up. Feel free to back it up to other machines, to something as archaic as tape, or even back to the cloud (but another vendor's cloud, please).

Lesson 5: Always plan for some sort of roll-over to another vendor

Don't keep all your eggs in one basket. More importantly, if you need to be running 24/7, make sure you have alternate servers you can jump to if your primary servers (or infrastructure) goes down.

On the other hand...

Lesson 6: Your alternate vendor might be using the same infrastructure

Of course, it could really suck if you sign up with a completely different vendor to provide alternate, fail-over services and then find out that their infrastructure provider is the same one as your primary provider.

So, make sure you know where your alternate, fail-over vendors get their resources from, and make sure you're using completely separate datacenter resources.

And, finally...

Lesson 7: Service Level Agreements won't save your ass when the technology's out to get you

Many customers rely on SLAs as the promise that they'll get the service they expect. That's all well and good, except sometimes technology fails. Sometimes technology fails even if your vendor is the best there is.

When that happens, don't just rely on a promise in a purchase document. Make sure your fail-over logistics are in place.

Otherwise, when your business or organization goes down hard, you might as well print your resume on the back of that celebrated SLA.

Topics: Storage, Amazon, CXO, Hardware

About

David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

6 comments
Log in or register to join the discussion
  • RE: 7 important survival tips Amazon's orphaned 0.07 percent can teach us

    Trust is not something you should bestow upon any Corporation. Corporations exist to make money. Selling your trust is probably not profitable for them, in the long run, but were that to change they would sell you out without a second thought.
    Tsingi
  • RE: 7 important survival tips Amazon's orphaned 0.07 percent can teach us

    If this data is lost in the cloud, and it's scheduled to rain here today, does that mean that people could be hit by falling data?
    jgm@...
    • You're right!

      @jgm@... It's been raining here a lot lately, and the gutters are clogging with gray stuff. I ran outside and scooped up handfuls of tiny 1s and 0s.
      There might be some valuable info here if I can organize them.
      kidtree
  • RE: 7 important survival tips Amazon's orphaned 0.07 percent can teach us

    Didn't you write a book about how one particular administration lost 100% of its data for some period of months? Also, didn't you blog about how China hacked the entire fed and downloaded terabytes of information during that same administration? What is losing .07% of one part of one region that hadn't opted for distributed backups compared to that?
    tkejlboom
  • RE: 7 important survival tips Amazon's orphaned 0.07 percent can teach us

    0. Trust but Verify!!!

    1. When you think you have enough copies of the backups - you're one short.

    2. If you can lose the computer and all the backups in one event you have wasted a lot of time doing backups because you really don't have any.

    3. You are putting much too much faith in technology if
    You have never verified the contents of the backup,
    You have never restored from backup,
    You have never verified another machine can read the backup media as created in the production machine,
    You believe a single set of 20 cent pieces of plastic archiving vital data for your entire operation is adequate. protection,
    You have put your entire future in virtual hands running a virtual place in a virtual universe.

    4. You can't trust the C-K interface to (a) perform all backup assignments true to the checklist, (b) actually read and react to failure messages or (c) cycle through any reusable media in the proper sequence.
    EJC05262
  • RE: 7 important survival tips Amazon's orphaned 0.07 percent can teach us

    I wouldn't trust corporations, or lawyers for this matter http://canlii.ca/en/on/onlshp/doc/2011/2011onlshp14/2011onlshp14.html
    will789