Preparing for a Flickr Apocalypse (Updated, with Yahoo response)

What would happen to my food blog and precious memories if suddenly all 12,000 of my photos were suddenly to disappear off Flickr?

Tuesday's posts by ZDNet's Janice Chen and SAAS Blogger Phil Wainewright scared the living crap out of me. I had no idea that Digital Raiload, a high end photo hosting service for professional photographers had just shut down with little warning, and sinking with all its hosted data on it, leaving its users with no recourse other than their own backups.

Digital Railroad's demise is a tragedy, but compared to what would happen if a much larger photo service were to go under it's a sinking dinghy rather than the Titanic. Imagine, if you will, if suddenly Flickr shut down, with little or no warning.

Now, the odds of this happening to Flickr is small, but the idea is still disturbing. Flickr's owner, Yahoo!, is going through a lot of thrashing about these days and its stock is getting hammered, and Microsoft is now in the enviable position of saying it's no longer interested in entering a search deal or buying the company outright.

I have to believe that Steve Ballmer is doing this to drive Yahoo!'s stock price down so they can swoosh in like a giant vulture and grab the company for a mere fraction of their original proposed offer, hoping that Yahoo!'s board will boot Jerry Yang out the door before the marriage is consummated.

The worst case scenario is that Yahoo! goes Chapter 11 or Chapter 7 and that its assets would be sold off piecemeal or in chunks. Flickr is a particularly juicy chunk as it is the most popular photo hosting service on the Internet -- Google may very well want it, as might a few other companies. Still, there is the chance that it might not get saved, and all of everyone's photos might go down with it.

This is particularly scary because Flickr is used as a image hosting service by a lot of bloggers who own free or inexpensive blog hosting accounts and use Flickr's bandwidth and high speed pipes to store images. It's a lot cheaper to use Flickr to hold gigabytes of photos than an ISP would, and the end-user response time for image download is generally faster than say, WordPress.com's, where I host my own food blog, Off The Broiler.

In my particular case, I have 12,000 images hosted on Flickr, many of which are IMG linked in my blog posts, of which I have over 600, starting since 2006. If Flickr were to suddenly die, with no warning, they would all be gone. I've gone through too many computers and haven't backed up the originals -- I simply assumed that Flickr would be a financially stable entity. With today's economy, however, I'm not so sure.

Flickredit allows you to do batch downloads of Flickr photos.

So the first thing I did yesterday was look into how I could download my images from Flickr. How much storage would I need? I downloaded a cool utility program called Flickredit, which is Java-based and runs on Windows, Mac and Linux. Among other things, Flickredit allows you to batch download pages of Flickr photos, as well as your entire photostream.

However, I soon found out that 12,000 hi-res photos equals roughly 11.5 Gigabytes of data. Assuming sustained 6Mbps transfer speeds, that could take a while. Still, I have more than one PC, so I started the job on my Vista box and continued my daily work on my laptop. An hour later, Flickredit barfed on the transfer, it only pulled down 700 photos. So I would need a better solution, at least for the initial full backup -- I could use Flickredit for the incremental backups.

I saw an interesting post about flickrfs, a Linux-mountable virtual filesystem. Apparently, it's relatively easy to set up on Ubuntu. Essentially, what it does is mount your entire Flickr stream as a virtual disk, and can mirror the entire content of your account on your local hard drive -- this is a similar technology to rsync or IBM's TDMF software for enterprises, which is essentially host-based replication. I haven't tried this yet but I plan to mess around with it this weekend.

What I ended up doing to get the bulk of my data was use a pay-for-service called QOOP.  QOOP does a bunch of other stuff including printing services, but in particular they will backup all your photos and send them to you for $20 a DVD plus shipping. So for nearly 12,000 photos and roughly 11.5 GB of data, it took 3 DVDs, and with shipping was $70. That doesn't suck. Now I can just dump the contents of all my Flickr stuff onto a RAID disk or external hard drive, and use Flickredit to download weekly backups.

Of course, this doesn't solve the problem of what will happen if Flickr truly dies and suddenly I need to re-load this data into a new hosting service, and then go through all my 600 posts on my food blog and re-hotlink the images because the IMG links would now be broken.

Presumably, some enterprising person or company might figure out a way to automate this -- such as a Google or a Microsoft, given the huge impact of this type of outage on a very large amount of blogs, but I wouldn't necessarily count on it. Any way you look at it, a lot of blogs and websites would be affected and it would be catastrophic.

Do you host a lot of photos on Flickr or some other image hosting service? Talk Back and Let Me Know.

Yahoo's Response

“Preparing for a Flickr apocalypse”? Are you one of those guys with a fully stocked subterranean bunker at home, or are you simply ignoring the facts to make a juicier post? I’d like to know.

Brad Williams | VP, Corporate Communications | YAHOO! |