Open Source growing pains: Is Open Core the answer?
In one of our first ZDnet posts a couple years back, we proffered the question of whether open source was becoming the new default business model for enterprise software. It's only been a couple years, but looking back, the question now seems a bit quaint given the recent actions of Redis, MongoDB, Confluent, and now, Elastic based on concerns on whether cloud giants like Amazon are, the words of MariaDB CEO Michael Howard, "strip mining" open source.
The latest round of volleys happened just last week. In competing wars of words, Elastic and Amazon each tried staking out the high ground as Steven J. Vaughan-Nichols reported.
Where there's smoke, there's fire.
There's little question of how open source has transformed the software landscape. Here in the New York area, where we're based, we've seen how open source has literally opened up the technology scene. Thanks to Wall Street, New York was always a tech town, but before open source, much of the technology was locked away as each firm had to invent the basic pillars of IT software infrastructure from high-performance messaging to compute grids and databases because off-the-shelf software couldn't hack it. Nobody talked to anybody else.
Wall Street firms are increasingly looking to open source first, as they don't want to reinvent the wheel, especially for core infrastructure software or machine learning algorithms; their unique IP is much higher up the value chain. And they don't want to lose good people who prefer open source on their resumes because they expect that their skills will become more portable. And Wall Street firms don't mind if their people go out and talk about their insights publicly or even start their own open source projects; rarely a night goes by without some meetup happening in town where practitioners are sharing their innovations.
The growing pains that open source companies are feeling today are directly related to the level of popularity of their open source projects. MongoDB doesn't want to see others profiting from the $300 million investment that it has made. As we stated last week, imitation might be the sincerest form of flattery, but for open source companies it's also an existential threat to future earnings. Open source players want to avoid becoming victims of their own success.
That's why companies like MongoDB and Redis, whose projects have drawn large communities of followers, feel threatened, while those like Cloudera, whose projects have narrower appeal do not. And that's why Cloudera views AWS and Azure as coopetition, not vampires.
Matt Asay has regularly commented on the unfolding saga. In his most recent post, he contends that most developers who download or use open source are not going to spend their time messing around with the code or contributing to it as they already have day jobs. So, the changes in licenses – especially those proscribing third parties from running SaaS services – won't matter to them. There's more than a grain of truth to that.
In the current spat, Amazon has opened up a new front by spearheading a new Open Distro for Elasticsearch project with customers such as Netflix and Expedia that will contribute all the bits back to the open source project. It's doing so based on its contention that Elastic is muddying the waters by comingling open source and proprietary code. Elastic is maintaining that its disagreement is with Amazon, not all cloud providers.
From the get-go, Elastics's business model has been based on a combination of open source and proprietary software. The core Elastic stack, which includes Elasticsearch, Kibana, Logstash, and Beats are all Apache 2.0 open source-licensed. The contention is over the Elastic Stack Features (extensions or plug-ins), formerly called the X-Pack, which handled security, alerting, monitoring, reporting, graph analytics, and machine learning. They were formerly treated as closed proprietary software, originally making the assemblage a classic open core offering, with some features available for free, while others were available only via paid subscription. As we'll state below, excluding the opening up of code for view, we believe that's where they should have stayed. And, as those features were proprietary, not surprisingly it spawned a third party ecosystem of alternatives for them.
Elastic made changes last year because it found the open core model divisive. In a blog post Doubling Down on Open dating back a year ago, Elastic CEO Shay Banon stated that creating a pay vs. free divide would fracture the community. He cited disconnects, not only in feature breadth, but also discontinuities in testing. He has a point – if the features are so intertwined, then divided code is just as unthinkable as, say, divided cities.
The challenge is that the road to confusion is paved with good, or more aptly, highly idealistic intentions. In an admirable move, Elastic subsequently made the source code for Elastic Features public and downloadable, with limits preventing third parties from offering them as commercial SaaS services. But in so doing, as AWS contended, the files in the downloadable folders ended up comprising a mix of Apache 2.0 and Elastic-licensed code, making things confusing. Elastic states that in the repository, the Elastic license and Apache license code are in different folders and maintains that the differences are quite plain to see. To our eyes, the differences are pretty subtle. Nonetheless, even if Elastic had built a bigger Chinese wall between the Apache and Elastic-licensed code, there's no guarantee that Amazon would have stood still.
In the interim, MongoDB and Confluent are doubling down on their plans. MongoDB is forging ahead with SSPL after withdrawing it from Open Source Initiative consideration because the two sides couldn't agree; it covers all aspects of the 4.0 platform. Confluent is pressing ahead with the Confluent Community License, which covers components that touch (but do not include) Apache Kafka: KSQL, Confluent Connectors, REST Proxy, Control Center and several other pieces.
Meanwhile, Redis, which announced the Commons Clause last summer for Redis Modules (not the database itself), has since loosened it up a bit under a new clause called the Redis Source Available License (RSAL). The good news here is that Redis Labs is calling a spade a spade: it makes no pretense that RSAL is an open source license.
The challenge that each of these players face is real. Open source opens up your addressable market and compared to traditional proprietary software, provides the best chance to build a critical mass community and, hopefully with it, installed base. But the double-edged sword is when your project, literally, becomes too popular for your own good. The classic challenge for open source providers is how to keep their competitive edge. It's one thing if you have a faster pipeline to new versions, but it's another once your rivals catch up. For instance, AWS used to lag with EMR in supporting the latest Hadoop versions, but has since gotten its act together to go live within 30 days of the latest stable Apache releases. At that point, time is no longer on your side.
The other risk is forking the technology, and with it, dividing the community that's given your project legs. That's a risk especially for open source projects that are vendor, not community-controlled, with MongoDB being the prime example. MongoDB will build new features off the latest version while third parties like Amazon and Percona develop off earlier versions not subject to the SSPL. The code will get forked and the question is whether that will end up fracturing the community. Not surprisingly, dividing the community is exactly what Elastic's Banon was concerned about by walling those using the free open source versions from the paid enterprise versions.
Open source companies need defensible IP. To date, Red Hat (soon to become part of IBM) remains the exception to the rule that pure open source companies whose products are based on community-led projects can be successful. Meanwhile, Cloudera would like break through and become the next outlier. For the rest of the ecosystem, are new open source variant licenses the answer? The danger is getting pushback from the legal and contract compliance folks on the customer side who might be getting license fatigue.
Maybe it's time to travel full circle. Don't do anything to plant doubts with your loyal base and stick to a well-known open source license. Around it, develop unique content that adds value to the open source project. Keep the value-added components sufficiently abstracted, but also unique and optimized for the open source content. Go ahead and publish the proprietary code with restrictions, but don't pretend that it is open source. Nobody said it will be easy. The open core model is so old that maybe it'll become new again.