Anatomy of a storage overhaul: Facebook's photo system

Facebook's photo storage system holds 850 million photos and costs a lot of dough. Niall Kennedy has a nice overview of what Facebook is doing to minimize its storage costs. 

Facebook's system, dubbed Haystack, is custom built but relies on content delivery networks and NetApp. Facebook is trying to minimize the custom stuff and use commodity hardware. 

Kennedy does a nice job of synthesizing Facebook's storage system. In a nutshell:

• Facebook's previous system relied heavily on Akamai and Limelight to improve latency.
• That Akamai and Limelight use costs money.
• Facebook has invested in its own "blob" storage system designed to cut the total cost per photo on the social networks systems. 
• The company hired a former NetApp engineer to redesign the storage system. 

A lot of this storage architecture is complicated---and frankly over my head---but for the engineers in the house here's a presentation on Haystack from last year.