Facebook sees about two billion photos shared across its platform on a daily basis. With that astounding challenge in mind, Facebook's infrastructure team started to face the head-on reality that new systems were needed to handle increasing and changing demands.
The Menlo Park, Calif.-headquartered company most recently bolstered its data center in Altoona, Iowa last November with a data center fabric described as a next-generation architecture developed for speeding up machine-to-machine traffic at scale.
Essentially, the solution to the puzzle became cold storage facilities.
Facebook engineers Krish Bandaru and Kestutis Patiejunas explained further in a blog post on Thursday how the development process started with building servers that power on as needed, then managed by intelligent software constantly verifying and balancing data to optimize durability.
According to the engineering team, this full-stack approach toward establishing efficiency while reducing operating power is based on building racks with double petabyte (or two million gigabyte) storage systems that operate at a quarter of the power typically used by conventional storage servers.
"The goal was to make sure your #tbt photos from years past were just as accessible as the latest popular cat meme but took up less storage space and used less power," quipped Patiejunas and Bandaru. "The older, and thus less popular, photos could be stored with a lower replication factor but only if we were able keep an additional, highly durable copy somewhere else."
The engineering team acknowledged that they are "lucky" to have both the financial and technological support (not to mention encouragement) to build such hardware and software from scratch and experiment with them.
"Too often, we've all seen systems begin to get bogged down as they grow and become more utilized,"Patiejunas and Bandaru wrote. "So, right from the beginning, we vowed to make a system that not only didn't degrade with size but would get better as it grew."
Facebook has already established two cold storage facilities in the last year at its data centers in Prineville, Oregon and Forest City, North Carolina, together hosting "hundreds of petabytes" of data -- an amount Patiejunas and Bandaru admitted is ballooning daily.