New corporate group wants to make building stable stateful Kubernetes databases easier
We use Kubernetes to manage our containers on clouds all the time. That's another way of saying we use Kubernetes for just about everything we can in IT these days. Unfortunately, Kubernetes doesn't do well at all at managing stateful applications. Now, the newly formed Data on Kubernetes Community (DoKC) consortium wants to change that.
We use stateful applications, such as e-mail and online banking, every day. They retain the data and context of previous transactions and your current transaction. Like web searches or print servers, Stateless processes have no stored data or references to past transactions.
In Kubernetes' early days, we usually used it to run web-based stateless services. If you needed stateful services, such as a database, you had to run them in virtual machines (VM) or as cloud-based services. Now, users want to deploy stateful apps on top of Kubernetes orchestrated containers, and that's not easy.
The problem is, while Kubernetes supports stateful applications with StatefulSet, it doesn't work that well. Kubernetes pods are by their very nature stateless and can vanish and be recreated in seconds. That's no problem for a stateless application, but a stateful program needs its data to be persistent; if its pod disappears, well, that's a problem. It can be done with StatefulSet using a Permanent Volume for data storage, but there are a lot of ways this can go wrong.
That hasn't stopped people from doing it anyway. Indeed, according to a forthcoming Data on Kubernetes Community survey, 70% of organizations are already running stateful workloads on Kubernetes. But, it's not pretty. As DoKC puts it, "interoperability and quality of Kubernetes operators remain top challenges."
You think?
So it is that DoKC, led by founding Platinum sponsors DataStax, EDB, MayaData, and Portworx, is working on best practices for using Kubernetes, stateful apps, and persistent data together.
DataStax VP of developer relations Patrick McFadin said, "Including data as a part of deployment completes your cloud-native application. DataStax believes in the power of open communities and is excited for a future of data on Kubernetes. It's time to work together and build the next generation of infrastructure."
"Organizations are standardizing on Kubernetes for container orchestration and are now looking at the potential for making data declarative, just like Kubernetes," added Melissa Logan, new DoKC director. "Achieving this will require a big tent that encompasses the world of data technologies, data infrastructure, and data governance. We are expanding the work of the DoK Community to help usher in a new era of declarative data."
Today, the DoKC hosts more than 4,000 individual members across its Slack and LinkedIn channels and has held close to 100 meetups ranging from how Adobe operates stateful applications on Kubernetes across multiple data centers and regions to practical questions such as how to deploy production-ready Postgres to Kubernetes. It recently announced the schedule for the second DoK Day at KubeCon North America 2021.
Related Stories: