An entire folder of document scans -- over 600 individual documents -- was now empty. It hadn't been empty last week, but now, for reasons still to be definitively discovered, it was empty. Here's how we recovered it.
UPDATE: Clarification on the Drobo's role in the article. UPDATE #2:Mozy on Mozy (at end of article)
In the interests of disclosure, I should admit that I haven't had bacon for something like a whole month. So sad. But had I had bacon, real bacon, crispy -- but not too crispy ... mmmm! -- CrashPlan would have saved it.
Instead, CrashPlan, which is a cloud-based backup service, helped my wife and me recover some very important family documents we lost to the weird whims of Windows.
Yesterday, while I was quietly drinking my second cup of coffee and internally whining about the complete lack of bacon within my reach, I suddenly heard what Spider Robinson calls a "scroan" -- half screech and half groan -- coming from my wife.
It turns out that an entire folder of document scans -- over 600 individual documents -- was now empty. We've been serious about becoming a paperless home office, and my wife scans almost every important piece of paper that comes across our desks before shredding and recycling. The folder of scanned documents hadn't been empty last week, but for reasons still to be definitively discovered, it was empty yesterday morning.
The folder was a network share, so if the files had been accidentally deleted, they wouldn't be in the trash (this is a terrible failing of Windows, frankly). She was also missing a pile of desktop icons (no, "Hide desktop icons" wasn't turned on). Weirdly enough, the folder containing the missing files was the only desktop icon that hadn't vanished.
Detailed investigation didn't turn up even a single sign of virus, and many other probable scenarios that would explain the cause were also eventually ruled out. The closest theory we have now is that she'd been trying out a bunch of different, low-budget Bluetooth headphones (and their associated questionable drivers), and something conflicted with something else and ... boom! I know this makes no sense, but neither does any other theory we've come up with.
Anyway, although the cause is important (and I hope we figure it out before it drives us crazy), this is a parable about the importance of backups. I generally have a two stage backup for live data. First, every morning, I have a script that automatically syncs the contents of all our critical folders using SyncBack Pro to the Drobo.
The thing is, to avoid exceeding the Drobo's limited capacity, I have SyncBack set up to remove deleted files from the Drobo if they're not on the original machine and haven't been modified within a week.
So, when I went to the Drobo to recover the missing document scans, they weren't there. The files vanished slightly over a week ago, and we just discovered they were gone yesterday.
UPDATE: Just to be clear, the Drobo did exactly what it was supposed to do and didn't fail us at all. It had backups of the data I put on it. The gotcha was in order to manage space allocation, files deleted from the source drive eventually get deleted off the Drobo as well. Because a week had passed, the procedure I use had deleted those files. I'm actually quite happy with my Drobo. I've pulled data off of it a few times, and it's behaved exactly as it's supposed to do. But there's a difference between a local backup device with four 1-terabyte drives and a cloud backup service with theoretically unlimited storage.
This is why I employ multiple backup methodologies. I also subscribe to the CrashPlan family plan. For about a hundred bucks a year, I can backup all my computers on the network to CrashPlan's servers, with updated backups every fifteen minutes or so.
Now, you might think that if the files were deleted from our network share a week ago, they'd also be missing off the CrashPlan servers. But you'd be wrong, and this is how CrashPlan saved our bacon.
CrashPlan keeps snapshots, for which you can set the granularity. I found that it had kept daily snapshots going back 90 days, weekly snapshots going back a year, and monthly snapshots going back about the 30 months or so since I started using the service.
So I logged into the CrashPlan application, set the recovery date to August 23, looked for the folder, and voila! All 657 files were there, safe, sound, and snug. Five minutes later, I had them downloaded to my machine and another five minutes after that, they were on my network share, back where they belonged.
I want to compare that with my last experience of Mozy. To be fair to Mozy, I haven't used their service since I started with CrashPlan, and they may have improved things. You folks reading can let me know. (UPDATE: See Mozy's invited response, at the end of this article)
But there were two key reasons I dumped Mozy and moved to CrashPlan. First, it would take literally hours to generate a list of what was stored on Mozy's servers, before even attempting to do a restore. Then, once I did a restore for an earlier crash, and I found that Mozy didn't properly back everything up. Worse, there were gaps in the data I'd tried to recover.
By contrast, everything came back in perfect, original condition using CrashPlan.
About a month ago, I'd paid for another year of CrashPlan's service and this experience definitely showed how worthwhile that expense was. We now store a few terabytes of data on CrashPlan (probably a lot more since they have all those historical snapshots), and the recovery was essentially painless.
Even though this experience worked out well, one of my two backup tiers failed. I don't like that. I'm going to add another tier, which is an unused older server I have parked in a back closet. I'm going to power that thing up once a month, do a full backup, and power down. That way, I'll have another snapshot older than the one on the Drobo, just in case.
The moral of this story is that failures do happen and you should backup. But even if you do backup, your backup may fail. Good practice is to layer your backups so that if one fails (as mine did), a second or third tier will be around to save your bacon.
More disclosure:An older version of SyncBack Pro was provided to me a few years ago for my article in Computing Unplugged. I honestly can't recall if Mozy was a provided review product or not. I pay for CrashPlan personally, and I personally purchased the Drobo and its drives.
Mozy on Mozy: If I discuss a vendor's product, I always encourage them to respond if they feel so moved. The following is Mozy's response to my discussion about my experiences with their service a few years ago:
Dave, Since you last used Mozy in 2009 we have made many significant updates. We have completely overhauled Web Access with several major updates. Our North American customer support is 100% U.S.-based. We introduced mobile apps for Android and iOS. And Mozy also has a file synchronization feature called Stash. We'd encourage you to take another look at Mozy. We're much more than just an online backup company now, and we get high marks on these changes from our millions of customers. -- Ted Haeger, Product Manager, Mozy
Awesome! I love it when vendors continue to improve their products and encourage you to give Mozy a try. If any readers have used the service more recently, please share your experiences below.