Open source database ScyllaDB 4.0 promises Apache Cassandra, Amazon DynamoDB drop-in replacement
We first covered ScyllaDB on ZDNet back in 2017. Its story is one of deep tech, open source, and pivots. Started by Hypervisor and Linux Red Hat veterans Dor Laor and Avi Kivity, the database that positions itself as a faster Apache Cassandra did not set out as a database at all.
Having embarked on that course, however, it remains set. Today ScyllaDB announced version 4.0. of what it dubs a high-performance NoSQL database for real-time big data workloads. This release marks a significant milestone, as ScyllaDB moves beyond feature parity with Apache Cassandra, now also serving as an open source drop-in alternative to Amazon DynamoDB.
ZDNet connected with ScyllaDB co-founder and CEO Dor Laor to discuss the details of the new release, as well developments in the database world.
Turning the tables on Amazon
Improved performance is always good to have. ScyllaDB is built on that premise, and has always been big on benchmarks. This release is no different, with ScyllaDB's own performance comparisons showing it to deliver more than 5X higher throughput and nearly 10X lower P99 latencies than the upcoming Apache Cassandra 4.0 release.
There are also some new and noteworthy features, such as lightweight transactions, change data capture and a Kubernetes operator - the latter two in beta. What is probably most noteworthy about ScyllaDB 4.0, however, is Alternator, an Amazon DynamoDB compatible API.
DynamoDB, Cassandra and ScyllaDB are all branches of the same root, so to speak, namely Bigtable. DynamoDB and Cassandra have evolved separately, even though recently Amazon unveiled Keyspaces, a Cassandra DBaaS (Database-as-a-Service) offering. ZDNet's Tony Baer recently compared Keyspaces with Apache Cassandra and DynamoDB.
In a way, ScyllaDB 4.0 brings these branches together, by providing an offering which can interoperate with both Cassandra and DynamoDB. Scylla 4.0 provides a production-ready Amazon DynamoDB compatible API that lets DynamoDB users switch to ScyllaDB without changing a single line of application code, is the promise. Laor also observed that ScyllaDB's strategy vis-a-vis Amazon Web Services (AWS) is like a wrestling reversal:
"Where AWS monetizes by taking from open source (Keyspaces, DocumentDB), ScyllaDB is bringing closed-source capabilities to open source, along with the freedom to deploy on-premise or on other cloud platforms. And of course the distinction between ScyllaDB and DataStax is also instructive. Where DataStax is moving toward services under new leadership, ScyllaDB is doubling down on technology".
The database market is changing, and new products like ScyllaDB are aiming to get a bigger share from incumbents and cloud vendor offerings
Laor referred to a ScyllaDB client who set out as a DynamoDB user, but their requirements dictated that certain workloads had to be on-premise. Even though they were not a ScyllaDB client before, the ability to do this made them give Alternator a go, and today they are using both DynamoDB on AWS cloud and ScyllaDB on-premise seamlessly, said Laor.
AWS is also known to target on-premise deployments, with AWS Outposts, so we wondered whether that could be an option for clients looking to run DynamoDB on-premise. Laor could only offer an educated guess on this one, mentioning that it's probably not as simple as installing DynamoDB on AWS Outposts, since DynamoDB relies on AWS cloud infrastructure to work.
What Laor was clear on, however, was that this is an onboarding strategy for ScyllaDB. He believes that ScyllaDB's superior performance and lower total cost of ownership will eventually bring more users to ScyllaDB:
"Developers are no longer locked in to a platform and have new open source options available to them. They can run on-premises, on their preferred cloud platforms or on Scylla's fully managed database as a service, Scylla Cloud. They are free to access their data as they like, without pay-per-operation fees, and with more deployment options, including open source Docker and Kubernetes".
World database domination
ScyllaDB also brings some noteworthy features from a DevOps perspective. Change Data Capture (CDC) allows users to track changes in their data, recording both the original data values and the new values to records. Changes are streamed to a standard CQL table that can be indexed or filtered to find critical changes to data.
Scylla Operator is a Kubernetes extension for Scylla cluster management. It currently supports deploying multi-zone clusters, scaling up or adding new racks, scaling down and monitoring Scylla clusters with Prometheus and Grafana. Both CDC and Scylla Operator are currently in Beta, expected to be fully rolled out soon, as per ScyllaDB's development model.
Indeed, having watched ScyllaDB grow from relatively early in its lifecycle, we will have to ascertain the fact that it's catching up and adding new features at a rather fast pace. Laor mentioned they have a slew of more features in the works.
When discussing what it is that enables ScyllaDB to make such rapid progress, Laor said that the company now employs about 100 people, and business has been growing well, too. It counts the likes of Comcast, Starbucks, Samsung and IBM as clients, and has been growing its clientele rapidly and steadily over the last couple of years, Laor said.
ScyllaDB has always been big on benchmarks. Now it's set its sights on Amazon DynamoDB
Our conversation with Laor took place on the day CockroachDB announced its funding round, and CEO Spencer Kimball made it clear that they have big plans to displace the incumbents. ScyllaDB and Laor are no short of ambition either, so we asked Laor about his take on the database market at this time.
CockroachDB and ScyllaDB have a few things in common: they are both open source, multi cloud and hybrid cloud ready globally distributed databases. We see them as representing a rising class of products that aim to offer an alternative to both legacy proprietary databases and cloud vendor offerings.
They also have differences, however, and Laor was quick to point them out. Funding strategy is one, as ScyllaDB is more modest in that respect. Laor said that they prefer not to raise too much, and focus on execution. ScyllaDB's last funding round was a Series C in 2019, having raised a total of $60M in funding over 5 rounds.
Perhaps more to the point, however, CockroachDB is a SQL database, while ScyllaDB is built on Apache Cassandra's protocol and query language. Laor said that they do encourage people to use SQL products if their data needs are modest, but real-time big data use cases are a different ball game.
Perhaps ScyllaDB's next benchmark will also include some comparison to that class of products, hard as that may be. Until then, the comparison is up to potential users. Regardless, ScyllaDB is an interesting option, which just became even more interesting.