Mesosphere + DataStax + Confluent + Lightbend = Container 2.0... But is it complicated?
Mesosphere has a thing for names. Calling your company Mesosphere helps if you're into middleware, calling your product Data Center Operating System (DC/OS) helps put what it does across, and Container 2.0 is a sure hit way to state Mesosphere sees itself as leading the way to the next generation in Containers.
The Mesosphere Stack. (Image: Mesosphere/ Datastax)
But what exactly is Container 2.0, how do the strategic partnerships fit in the picture, how does this work, and what does it all mean for clients and vendors? In an effort to go beyond press releases and technical article jargon, we had a chat with Tobi Knaup, CTO at Mesosphere, and Patrick McFadin, Cassandra Evangelist at DataStax.
Container 2.0 = Container + State
For the last couple of years, there has been a lot of discussion on the merits of Containers versus Virtual Machines (VMs). Even though containers are generally seen as more agile, better suited to cloud architectures such as microservices, and more performant than VMs, one of their major limitations has been the lack of support for stateful applications. So, simplistically put, Container 2.0 = Container + State.
For DC/OS, the support for state in container apps is manifested via the DataStax - Confluent - Lightbend stack. Datastax is the enterprise version of Apache Cassandra, while Confluent is the enterprise version of Apache Kafka, and LightBend is an open-source application development environment for DC/OS.
But why Mesos for Cassandra, and why Cassandra for Mesos? It could have been any other Container orchestrator (e.g. Kubernetes/Docker Datacenter) for Cassandra, and any other NoSQL/SQL database (e.g. MongoDB/Oracle) for Mesos. Let's not forget, it's only been a few days since Kubernetes 1.3 general availability, introducing the ability to run stateful workloads within the same cluster with PetSets.
Even though both Knaup and McFadin claimed the decision was based solely on technical merits, and that does indeed make sense, one can't help but wonder what goes on behind the scenes and whether the fact that all vendors that comprise this stack are acting as the commercial branch for Apache projects had something to do with the forming of this alliance.
Can other frameworks be integrated in the Mesosphere stack, and will it be painless?
Either way, this array of partnerships has resulted in an pre-integrated stack that is functional and makes for a compelling offering. But there's a fine line between pre-integrated and rigid, and if the Mesosphere stack does catch on, both clients and vendors will be lining up to integrate their frameworks. Can this be done, and will it be painless? The answer is yes, and maybe.
DC/OS features its own app store of sorts, called the "Universe". The idea is that through this app store Mesosphere clients should be able to easily deploy their framework of choice in a container managed by DC/OS and subsequently invoke its services from their own apps running in other containers.
The secret sauce that enables integration of frameworks into DC/OS is its 2-level scheduler architecture. As having one scheduler to rule them all is cumbersome, DC/OS enables frameworks to run their own scheduler. DC/OS does the overall scheduling among frameworks, and each framework takes care of its own internal scheduling.
Even though a simple scheduler can be developed in no time, a production-strength scheduler is a different beast altogether. So, that's one thing. What about integrating schedulers?
Integration of framework schedulers in the Mesosphere stack can be done via an SDK. This SDK may be public, but according to Knaup, it's not really advertised or easy to access for the time being. As for the complexity of such an undertaking and the degree of collaboration it took to be accomplished, things are not exactly clear either. Yes, it did take getting some engineers from both vendors in the same room, but how many or for how long is not known.
Of course, you have to start somewhere, and Mesosphere seems to have made perfectly valid choices for their stack. But even though the brave new Container 2.0 world as supported by DC/OS makes for a solid offering, that offering is more attractive for clients with greenfield environments or clients already utilizing -- or willing to switch to -- the Mesosphere stack. For anything that deviates from this, it may have to take a bit of a wait-and-see before jumping on the bandwagon.