Has the Hadoop market turned a corner?

With Cloudera's long-awaited IPO and Hortonworks' reporting of Q1 results, we now have real numbers from two out of the three Hadoop pure plays to paint a picture of the market. The good news is that, for both companies, the numbers have taken a positive turn with the sea of red ink starting the downward slope, and with product revenue growth outstripping professional services.

Drilling down, Cloudera's business is clearly more mature than Hortonworks; according to the S1, Cloudera has been growing at relatively constant 50+ percent rates over the past three years while Hortonworks settled at this rate only during the past year after the initial spurt of triple digit growth. On the other hand, from a revenue standpoint, Hortonworks benefited from a more developed market, getting to the $100-million mark a lot quicker.

And Cloudera is further along in becoming a product-centric business, with services accounting for 23 percent of revenues over the past year, compared to 31 percent for Hortonworks for the past year -- but 25 percent for the most recent quarter. Trends in the sea of red ink are more illustrative; Cloudera's losses started hitting the downward curve last year, while Hortonworks just got to this point in the last couple quarters.

Of course, all this comes amidst the angst that is predictable for any emerging technology - phenomena for which other analyst firms have coined trademarked names for such cycles. In the past few weeks alone, we've seen what seems like schadenfreude from posts with titles like Hadoop is falling, Is Hadoop Failing, and The Trouble with Hadoop, plus counterpoints like Hadoop is Not Failing, it is the Future of Data. We weighed with our own two cents last fall.

In one respect, what's happening with Hadoop is similar to that of any new technology. It takes time for practitioners to learn and for organizations to understand how they can benefit from it. For vendors, there's the need to educate the market, and when you're selling to early adopters, it's probably a high-touch, high cost, highly consultative sale. Not surprisingly Hadoop is a land and expand business, where the profits happen when existing customers renew and expand their subscriptions. With 90+ percent renewal rates, Cloudera and Hortonworks are on a path to profitability.

With these results, the good news for Cloudera and Hortonworks is that there is a real market there for them. But the question to investors is how big the addressable market will be and how long will be the path to profitability.

Cloudera estimates that it has penetrated barely 5 percent of its addressable market of the Global 8000 (companies with at least $1 billion revenues). OK, the potential market is big in terms of numbers of customers.

But how much will those customers pay? With open source technology running on commodity infrastructure, the prevailing expectation can be summarized by the old Meineke Muffler slogan, "We're not going to pay a lot for this muffler!" So, it's unlikely that Hadoop alone, which we estimate is a $600 million market today, is likely to hit the $10+ billion levels of the enterprise database market anytime soon.

The operable notion there is Hadoop alone -- because the addressable market that Hadoop serves is actually about Big and Fast Data. Cloudera, Hortonworks, and MapR are part of an ecosystem that also includes cloud based services for Hadoop, not to mention a la carte offerings for running Spark, machine learning, data pipelines, and streaming. Or to be snarky, it's the Big Data market, not the Hadoop market, dummy.

The cloud is responsible for expanding this ecosystem. By making all these alternative big and fast data services available the cloud is Hadoop's frenemy. But just as the cloud opens the gates for a la carte services, it also provides the way for Hadoop to surmount it's biggest challenge: that the platform is still too darn hard for most enterprises to implement. The cloud will be key to making Big Data - and Hadoop - accessible to the next wave of adopters who won't have the technical savvy or resources of the early adopters. We expect that by year end 2018, that most new Hadoop implementations will be in the cloud.

Managed cloud services can not only bury all those ugly configuration and patching tasks, but also provide fit-for-purpose platforms for specific use cases, like BI, data warehouse optimization/ETL, or Spark. A good example is the Hortonworks Data Cloud (HDCloud) that provides services optimized specifically or Hive and Spark workloads. Managed services are the key for Hadoop providers to fulfill their promise in a much broader ecosystem.