PlanetScale scales out MySQL
With necessity being the mother of invention, PlanetScale co-founder Jiten Vaidya needed a way to scale the MySQL databases that were running many of YouTube's operational systems. They developed a piece of middleware that would sit between the database client and logically shard the database in the underlying storage engine. That open source project became Vitess, and it was the technology around which PlanetScale, a company with $25 million in venture backing, was founded.
The company, which was founded in 2018, began with offering commercial support for Vitess, and over the past six months, has introduced its cloud transaction database-as-a-service, PlanetScaleDB.
The core technology, Vitess, is intended to enable SQL databases to scale out in a manner similar to NoSQL platforms like MongoDB, relying heavily on sharding. Acting as classic black box middleware, it logically shards the database on the presumption that most queries are going to be around data associated with specific records. So, it groups all data with common record IDs on the same shard. It also provides connection pooling to overcome the high memory overhead that is common with MySQL implementations so that the platform can handle high concurrency. And to further protect the database from getting overloaded, it has query limiters that throttle so-called "queries from hell."
Vitess has been initially designed to support MySQL and related platforms like MariaDB, providing a middleware tier that allows you to implement the regular edition of the database, with the middleware handling all the scale-out. According to Vaidya, the technology could also theoretically support PostgreSQL as well, but as a start-up, they are focusing on their existing sweet spot.
If this all sounds like a familiar story, it is. Vitess in many ways resembles a latter-day reinvention of the classic transaction processing (TP) monitors of the 1990s. At the time, distributed transaction databases were not feasible, and so the only alternative was to run transactions on a single node that would scale up. And given that most database platforms at the time were licensed based on processing power, TP monitors helped reduce server requirements by offloading transaction processing, and handling all the connection pooling. It was a highly contentious issue for database providers at the time – many would invalidate licenses if they caught their customers in the act. But when the Internet started delivering scales of transactions that overwhelmed even the most properly-licensed servers, TP monitors got reinvented as appservers, with many of those same database companies now biting the bullet.
Fast forward to the present, and there's growing demand for global applications that are managed as single logical instances. One answer has been the multi-master database, which in the NoSQL world can be found in DynamoDB and Cassandra, and in the relational world with Google Cloud Spanner and CockroachDB. Microsoft has adapted technology that came with the Citus Data acquisition for the Hyperscale editions of Azure SQL and Azure PostgreSQL databases. Oracle has also offered its RAC clustered database edition to run its eponymous database in shards. Probably the closest competitor in similarity is Yugabyte.
By contrast, PlanetScale's technology allows you to take a vanilla edition of MySQL or MariaDB and, in its middleware black box, scale it out. After starting business supporting the Vitess middleware, it is now packaging it with its own implementation of MySQL as a managed cloud service that is now supported on all three major public clouds. The company positions it as a multi-cloud and multi-region database, where you might run most of your shards on one cloud and reserve the other for disaster recovery.
On the horizon, PlanetScale intends to take this customers setting up their own Kubernetes clusters who want to mount their own private clouds.