Airbnb open sourcing its SmartStack automated cloud framework tools

Airbnb is taking its infrastructure tools public, making them open source on Github with the potential to both help and influence other startups.


Airbnb has grown to host more than 500,000 listings across 192 countries since launching 2008. Obviously, scaling both manpower and the back-end infrastructure must have been growing pains for the San Francisco startup.

But perhaps it wasn't as painful as one might expect given the rapid rise of the online accommodations service -- notably because Airbnb built its existing cloud infrastructure tools for scaling and operations from within.

Now Airbnb is taking those tools public, making them open source on Github with the potential to both help and influence other startups as they build their own infrastructures.

Dubbed Smart Stack, the automated set of tools are designed to facilitate a "cruise-control" system for Airbnb's distributed cloud computing infrastructure.

Airbnb engineers Igor Serebryany and Martin Rhoads explained in a blog post on Wednesday that Smart Stack has been undergoing internal tests for more than a year now, positing that the Airbnb approach for service discovery is more transparent and "simpler conceptually."

Companies like Airbnb often start out as monolithic applications -- a kind of swiss army knife which performs all of the functions of the organization. But as traffic (and the number of engineers working on the product) grows, this approach doesn't scale. The code base becomes too complicated, concerns are not cleanly separated, changes from many engineers touching many different parts of the codebase go out together, and performance is determined by the worst-performing sections in the application.

The solution to this problem is services: individual, smaller code bases, running on separate machines with separate deloyment cycles, that more cleanly address more targeted problem domains. This is called a services-oriented architecture: SOA.

Serebryany and Rhoads published detailed explanations about how to build upon Smart Stack as well as about alternative methods they rejected.

Some of the touted benefits to using Smart Stack as opposed to DNS or central load balancing, according to the Airbnb engineers, include more intelligent queueing and faster actions taken after health checks.

The Airbnb team asserted that this infrastructure and its concepts could be applied to a business of any size.