The state of open source databases in 2019: Multiple Databases, Clouds, and Licenses
Seventy percent of a $46 billion market is a lot. This is the share of the overall database market open source databases are projected to hold, according to Gartner. An extensive survey done by Percona shows the attitude of the market towards open source databases.
The Open Source Data Management Software Survey was undertaken by Percona, a company offering services for open source databases, to capture usage patterns and opinions of the people who use open source databases. The survey, unveiled today at Percona's Open Source database conference in Amsterdam, included 836 of them from 85 countries, which means it's a good way to get insights.
Multiple databases and NoSQL
The first key finding seems to confirm the intuition and observations of many people working in the field. Practically everyone is running multiple databases, and multiple open source databases at that. Where things get more interesting, is when it comes to polyglot persistence, aka using different data storage technologies to handle different data storage needs.
Seventy-three percent of respondents are running both relational and other databases, while 54% are running a non-relational database. A bit of clarification is needed here though. In the graphic, those databases are denoted as NoSQL. While that's understandable, in terms of term brevity and familiarity, it may actually be misleading in terms of functionality.
Today, many non-relational databases (or data management platforms, if we want to include solutions like Hadoop, for example) actually support SQL. Although the NoSQL moniker has been redefined to "not only SQL", it's good to be aware of the fact that SQL does not just apply to relational databases. That's something to keep in mind when interpreting answers related to polyglot persistence.
Most organizations use multiple databases. Source: Percona
While it seems like practically everyone is using an operational relational database, a SQL interface may well be available for many other solutions too, such as analytical, streaming, multi-model, or time-series. We would even argue that solutions with the ability to offer a familiar SQL interface have an edge in terms of lowering the entry barrier to adoption.
Percona's survey goes to considerable lengths analyzing the use of MySQL and PostgreSQL and their variants. That's understandable, as these 2 appear as the dominant relational open source database platforms, and Percona's business revolves around them to some extent.
MySQL Community edition is used by 59% of respondents, who also use MariaDB Community edition and Percona Server (36 and 34% , respectively), while other options trail. In PostgreSQL, the picture is different: 46% use PostgreSQL, the next option is Amazon RDS for PostgreSQL, at only 9% , and other options show little adoption.
Percona, on its part, noted there are more details and a deeper dive into some database options, not because they are Percona products, but because they dominate the market. Percona also noted there were several questions where the options were wider, and respondents could add details. In any case, worth keeping the usual caveat about surveys in mind: results may not be 100% representative of the real world.
Open source databases in the cloud
We think this may apply, in particular, to input related to polyglot persistence. The survey's results show about 50% of respondents use Redis and Elastic, with Microsoft SQL Server, MongoDB, Oracle and SQLLite between 38 and 28%, and all others trailing far behind.
All others, in this case, includes solutions like Kafka and Cassandra. Not to mention more "exotic" solutions such as graph databases, which only score a negligible 3%. This does not match our own experience very well, so it's something to be aware of. Insights related to cloud adoption, on the other hand, are much more in line with what we see.
As the survey notes, most survey respondents are well-informed about using open source technology in the
cloud, and many of them do. Interestingly, however, these passionate open source evangelists championing cost-effectiveness, flexibility, and freedom from vendor lock-in often find themselves tied to cloud vendors with a single solution and large monthly costs.
In Percona's survey, relational databases still dominate the scene, although they are no longer the only game in town. Source: Percona
This, the survey goes on to add, might explain why over half of respondents have so far avoided the public cloud and opted for alternatives that allow for easy deployment and scaling but in a more "hands-on" environment. As company size grows, so does the likelihood of hosting databases in multiple clouds. The larger the organization, the more complex the hosting environment.
AWS continues to dominate the public cloud provider market, with over 50% of respondents using their cloud platform. Google Cloud and Microsoft Azure show similar adoption (around 20%). About a third of respondents use multi-cloud. As one respondent put it:
Also: What makes Google Cloud Platform unique compared to Azure and Amazon
"I think it's interesting that companies view cloud DBaaS (DB as a Service) as open source and not as enterprise software. We seem to throw caution to the wind there and look at it as a panacea rather than equivalent to running commercial software."
Support and licensing
This brings us to an interesting point - drivers and inhibitors for open source database adoption. Unsurprisingly for this cohort, practically everyone is retaining or increasing their use of open source database solutions. Some (10-20% ) also contribute. And about 70% cite cost savings as a reason to adopt an open source database. Avoiding lock-in, community, and ease of use also score high.
Lack of support and security concerns, on the other hand, seem to be the biggest inhibitors for adoption. What do people do to cover their support needs? About a third of all respondents rely on self-support, with small and medium companies surpassing this, and two thirds use some sort of support, be it vendor or 3rd party.
As the survey notes, too, there is a contradiction here: users want to benefit from the free to use software, but they seem hesitant to pay for support. Over 80% of respondents use community editions of open source databases, with only under 20% using enterprise editions. And yet, about 60% note support as a good reason to adopt commercial software.
And what about licensing? Despite recent controversy on new licensing models, whether they are "real" open source licenses or not and so on, only 15% of respondents seem to be concerned about it. Of course, there are different ways to read those results. Survey authors note that 68% of respondents are very likely to adopt totally open licenses like BSD, MIT, or Apache.
Maybe open source licensing is not such a big deal for most people after all: only 15% are concerned. Image: Percona
We would also note, however, that 65% are likely to adopt partially open licenses, and 79% are likely to adopt open with restrictions licenses. Our interpretation is that, when given the option, people would rather go for a "totally open" license. In practice, however, we believe reason will prevail: as much as we may like the "totally open" / free model, we doubt it is viable in the long term.
In a recent discussion with Redis, one of the "perpetrators" of new license schemes for open source databases, their spokespeople stated they believe that the OSI will revise their views about what constitutes an open source license. All things new are met with some resistance, which is particularly true for long-established institutions.
Add to this the somewhat ideological approach to open source by many people involved, and the different, often conflicting, business models that underlie different licenses, and the reasons for the flat-out rejection of new licensing schemes become clearer to understand.
Redis people expressed their willingness to engage in conversation, and said they have listened to input and made changes to their approach. They also expressed their conviction that the new licenses will eventually come to be accepted. While we have no vested interest in any vendor, we think this is a conversation that needs to be had, and hope to see the open source community moving forward.