Pivotal pivots to open source and Hortonworks
A few days ago Pivotal made three major announcements: the creation of a Big Data Product Suite, a partnership with Hortonworks and the launch of an 'Open Data Platform'.
See: Pivotal unveils Hadoop Open Data Platform, deepens Hortonworks links and open-sources code
All three announcements tie together, but the Big Data Product Suite is the most important. Let's dissect the news.
San Francisco, February 17, 2015 - Pivotal, an open source pioneer, today announced a new release of Pivotal Big Data Suite, the next generation big data solution built to help customers accelerate the value they get from their big data. In an industry first, Pivotal will open source the core components of Pivotal Big Data Suite.
MyPOV - Pivotal has done a lot with open source - but they are not necessarily pioneers. This announcement is all about making the previous EMC acquisition of Greenplum and the previous VMware Spring Source acquisition of GemStone coupled with the HAWQ Hadoop SQL front end available on an open source core.
With over 25 years invested in development of Pivotal's big data product portfolio and global customers across every industry, Pivotal brings forward extensive experience to now offer customers an open source choice to accelerate their data-driven applications. Core components of the Pivotal Big Data Suite - including world-leading, analytical MPP data warehouse Pivotal Greenplum Database, the world's most advanced enterprise SQL on Hadoop analytic engine, Pivotal HAWQ; and the leading premium NoSQL in-memory database, Pivotal GemFire - will be, for the first time, based on an open source core. The contributions will be an open, fully functioning core that will provide mission critical resiliency, advanced client support, performance optimizations for demanding enterprise workloads and advanced operational tools.
MyPOV - The announcement effectively ends the rumors about what Pivotal will do with its BigData products, which never did as well as the sister PaaS product, CloudFoundry. The future will tell if Pivotal was slowed down with the late entry into the Hadoop Distribution market with Pivotal HD, or if Pivotal has realized it cannot compete with open source, despite all the remarkable development process (e.g. pair programming, agile etc.) that Pivotal has put in place. For original Greenplum and Gemfire investors and employees it may be as well a remarkable day, as their work now becomes part of open source. EMC and VMware probably paid close to a half Billion for these assets (prices were never disclosed) back in 2010.
"Pivotal Big Data Suite is a major milestone in the path to making big data truly accessible to the enterprise," said Sundeep Madra, vice president, Data Product Group, Pivotal. "By sharing Pivotal HD, HAWQ, Greenplum Database and GemFire capabilities with the open source community, we are contributing to the market as a whole the necessary components to build solutions that make up a next generation data infrastructure. Releasing these technologies as open source projects will only help accelerate adoption and innovation for our customers."
MyPOV - Well said, note the acceleration that is expected by making the open source contribution, effectively saying that Pivotal itself was not working as fast, and maybe not fast enough. It takes less resources when the majority of code is in open source and a vendor only needs to truly work at the revenue generating add-ons."Pivotal Big Data Suite is a major milestone in the path to making big data truly accessible to the enterprise," said Sundeep Madra, vice president, Data Product Group, Pivotal. "By sharing Pivotal HD, HAWQ, Greenplum Database and GemFire capabilities with the open source community, we are contributing to the market as a whole the necessary components to build solutions that make up a next generation data infrastructure. Releasing these technologies as open source projects will only help accelerate adoption and innovation for our customers."
Delivered On an Open Cloud PlatformCustomer focus has moved from storing the data to unlocking its potential--demanding a more agile way to wield their data. Pivotal Big Data Suite also offers customers an agile data solution based on open source software that can be flexibly deployed using cloud technologies.
MyPOV - Marketing lingo...
Pivotal Big Data Suite will provide support for bare metal commodity hardware, appliance-based delivery, virtualized instances, and now public, private, and hybrid cloud support. In addition, Pivotal Cloud Foundry Operations Manager will be included to provide Pivotal Big Data Suite capabilities as services inside and with Pivotal Cloud Foundry applications.
MyPOV - Good move to couple with the more successful Pivotal product, Cloud Foundry. We see the majority of next generation application projects being based on or needing some BigData capabilities, so combining Cloud Foundry with this is a smart move.
[...]Choice and Flexibility with a Single SubscriptionPivotal Big Data Suite will be the industry's first and only suite of enterprise class big data products based on open source at their core. Barriers to big data deployments will be removed through a single flexible suite, for both application developers and data practitioners. Enterprises now have access to the flexible deployment of data lakes, powerful tools for advanced analytics and data science, as well as a portfolio of building blocks for supporting custom data-centric scale-out applications in hybrid cloud environments.
MyPOV - One of the smart moves Pivotal made was to allow customers to trade data product licenses as they saw a need and their BigData projects evolved. Keeping the same in place now makes only sense. We wonder though how a recent paying customer for e.g. Greenplum may feel after this announcement.
Pivotal Big Data Suite to Include New Application ServicesPivotal Big Data Suite includes several new data services:(New) Pivotal Big Data Suite on Pivotal Cloud Foundry -- Leverage advanced data services using applications running in the leading open cloud platform as a service.(New) Spring XD -- Highest scaling open source distributed framework for data ingestion, batch processing and analytic pipeline management.(New) Redis -- Leading scalable open source key-value store and data structure server.(New) RabbitMQ -- Leading scalable open source reliable message queue for applications.
MyPOV - Good to see Pivotal bundle more assets into the offering, searching synergistic benefits. All increase the value and attractiveness of the new product suites.
Unified and Open Approach to Big DataResponding to customer and market needs for unification in the Big Data space, Pivotal is joining forces with other data industry leaders to provide customers a stable, secure and interoperable foundation to build on.In related announcements made today, Pivotal will be participating in the Open Data Platform to drive further collaboration and the adoption of modern, scalable data architectures by enterprises.
MyPOV - This is a smart move, combining all the forces of the 'beyond #2' runs in the market - the market leaders being Cloudera and AWS for BigData. However, like any other large group of otherwise competing vendors, it remains to be seen how long and how well these vendors can keep a working and the open platform going.
In addition, Pivotal announced a strategic alliance with Hortonworks to simplify the adoption of Apache™ Hadoop® by enterprises, which includes support for its advanced services like Pivotal HAWQ running on the Hortonworks Data Platform.
MyPOV - This is the third announcement (next to this press release and the Open Data Platform being joined by Pivotal). By making assets like HAWQ available on the Hortonworks' Data Platform, Pivotal goes with the #2 Hadoop distribution, which makes sense for the vendor.
Overall POV
Competing with open source is very hard -- too hard for most vendors. When IBM started to use Pivotal's CloudFoundry product for the new IBM PaaS BlueMix, it was clear that when super large vendors like IBM need to use open source, smaller ones will have to use it, too. Especially when the product is on a long downward trot as the former EMC and VMware BigData products (unfortunately) are. Putting them into Pivotal, telling a new story that worked out very well for the PaaS offering - CloudFoundry - did not change their overall market success.
Original investors and founders of Greenplum and others may shed a silent tear, but that's how the BigData market works these days. The interesting question will be if Pivotal will support the Cloudera ecosystem at some point. This support is unlikely to happen soon. In the meantime, good luck to Pivotal with the new approach; it needs less developers to only focus on the value adding ones that create revenue in an open source play.