Data 2021 Outlook Part II: Hedging the cloud
On Twitter, one of our most commented-on posts last year was when we when we reviewed the hype around multi-cloud. With a few exceptions (e.g., disaster recovery, public regulation), we remain cynical about the viability of running a single database logical instance across multiple clouds; we believe that for most organizations, a "multi-cloud" platform will in essence mean "freedom of cloud." Run the database or cloud control plane of choice in the public or private cloud of choice. We haven't exactly been a lone voice in the wilderness about the operational complexity of running a single instance of a database or application across multiple clouds; Matt Asay, Corey Quinn, and Gartner have shared similar concerns.
But we expect that freedom of cloud will have a louder ring in the coming year. It's not exactly like organizations will take their first halting steps into some alien cloud. We've noted that variety – intentional or otherwise – has long been the spice of life (and often, administrative headache) for most organizations. Chances are, the larger and more diverse the enterprise, the more likely they are to have one of everything, and why should that be any different with cloud? But as we note below, the choices over freedom of cloud won't necessarily be completely clear-cut.
Why the noise?
Managed services are pushing this dialog to the forefront, and it's why we think it will start proving an evergreen issue in the coming year. It's a tactical decision to put your DevTest or database on EC2 at AWS, whereas it's a strategic decision should your organization choose to run managed services such as Amazon Aurora PostgreSQL database or SageMaker, because at that point, you are not choosing a cloud to run some random compute cycles on, but a platform. You're making the Microsoft vs. Oracle decision all over again, but with a new and wider cast of the usual suspects and emerging players (Snowflake, Confluent, Databricks, anyone?).
In the 2020s, your choice of cloud platforms will drive your systems, tools, databases, and application choices. But just as it looked like the battle lines are hardening – e.g., choose Google Cloud SQL or AI Platform, and you can only run them on GCP – hybrid is changing all that. Hold that thought.
Conventional wisdom is that any form of lock-in is bad, but for most organizations, at some layer of the stack, it is inevitable. It's hardly a new issue, and, unless your organization homebrews all of its applications and databases and relies on itself for support, it will have to make some strategic technology choice somewhere. Your organization is choosing a platform that will drive its decisions around tools, databases, and applications. The technology provider for that strategic layer becomes, in effect, first-among-equals and a form of gateway: all other software choices must come either through the partner ecosystem or from suppliers who support and/or integrate to that platform.
During the mainframe era, that decision was driven by the system, but the emergence of distributed systems shifted that choice, first to the OS, and then to applications and/or databases. In the run-up to Y2K when modern enterprise applications and databases were implemented, that set the stage for the rivalry of Oracle and SAP: Which provider drove your choice of third party solutions?
Is cloud lock-in a black and white issue?
As enterprises are accelerating their embrace of the cloud, the choice becomes, appropriately enough, clouded. As we noted above, choosing from the cloud provider's portfolio of managed services likely ties you to that platform, because they are only run on that provider's cloud. Yes, we've italicized that term for a reason, and we'll get back to that in a moment.
Azure and Oracle Cloud excepted, most of the cloud providers have focused their managed services at the Platform-as-a-Service (PaaS) level, for databases, application development environments, integration, internet of things, analytics, and so on; they have not pursued enterprise SaaS applications. So, with PaaS, or in this case, Database-as-a-Service (DBaaS), enterprises have the choice: select a MongoDB-compatible database from AWS or Azure, or go to MongoDB Atlas for a MongoDB database that should promise an identical experience regardless of cloud. Until now that choice has been pretty black and white: stick with the cloud provider's service, which sticks you to that cloud, or go third party and feel free to choose the cloud.
That's where we rewind the tape and bring you back to that term "likely," regarding cloud vendor lock-in. In the coming year, we expect the lines to blur a bit as cloud providers raise the volume on their multi-cloud messages, while some third parties designate specific cloud providers as preferred partners that may involve joint go-to-market, support, and in some cases, joint product development.
And we could imagine a scenario where, for specific services they want to promote, some cloud providers may lower hurdles, such as selectively doing away with egress charges. For instance, we could well imagine Google Cloud getting creative here in promoting its AI services to customers running applications on AWS or Azure.
Hybrid starts to breach boundaries
For starters, let's look at hybrid cloud platforms, an area where we surveyed the market landscape last year. Google Cloud and Microsoft Azure have made noises about their software-defined hybrid platforms being able to run in alien territory, with Google Cloud Anthos now formally supported for running in AWS. Kubernetes (K8s) is supposed to be the great leveler that allows all this to happen. Keep in mind that each cloud provider's managed K8s service will be implemented slightly differently; for instance, GCP Anthos and IBM Red Hat OpenShift implement administrative functions for the Istio service mesh in their K8s offerings a bit differently. Nonetheless, with some tweaks, all those microservices should be portable.
But comparing K8s platforms is simply about the cloud control plane. What about the portfolios of higher-value managed PaaS services? At this point, the pickings are pretty slim. On Azure Arc, Microsoft currently only offers a couple of Azure data services (Azure SQL Managed Instance and PostgreSQL Hyperscale), while Google has just introduced BigQuery Omni (the extension of BigQuery that runs in Anthos on foreign soil).
The story is similar for AWS Outposts, although it is a bit of a different case. No, it's not a K8s platform, but instead is a turnkey system running in the customer's data center, not a foreign cloud environment. For now, Outposts runs a curated selection of AWS services including containers (Amazon ECS or EKS); RDS for MySQL or PostgreSQL database; and EMR for analytics.
Big Data
It's going to be a long while, if ever, when a critical mass of all those analytics, AI, database, integration tools, IoT, and other services any of the cloud providers makes it onto hybrid cloud systems, and potentially, cross-cloud software-defined offerings. But in the long run, we expect the 80/20 rule to apply with the most popular data, analytics, and machine learning services likely to make it to software-based hybrid clouds that venture into rival territory.
Third parties are hedging their bets – and so should you
At the other end of the spectrum, will you necessarily get the same third party experience across every cloud? What's emerging in open source databases, and we expect, AI and analytics tools, is that a growing array of third parties will anoint specific cloud providers as first among equals. Here are a few examples:
- Microsoft Azure has been especially proactive with its Embrace partnership with SAP for its S/4HANA NextGen ERP cloud service; SAS with its NextGen Viya cloud service; Redis, as a full stack alternative to Azure's cache-only offering; and Azure Databricks, a third-party service with native integration to Azure Machine Learning and other services.
- Google Cloud has also been prominent with its extensive roster of open source database partnerships that, for now, involve joint sales and support, but in the future could also involve some tighter integrations with other GCP services.
- AWS has cracked open the door with its new Managed Grafana service based on a joint go to market and product development partnership that we covered a couple weeks back.
In each of these cases, the third party services in question are, or will be offered in other clouds. But for these partnerships, some cloud providers will jump to the head of the line. In the coming year, we expect that the dividing line for cloud vendor lock-in will start to blur. Yes, for 95% of all managed services in the cloud provider's portfolio, your choice will continue to shape which public cloud (and in some cases where availability is limited, which cloud region) you run in. And for most third-party tools, applications, and databases, those with plans to support all major cloud providers will commit to delivering common experiences.
But in limited cases, cloud providers will penetrate enemy territory and third parties will feature preferred relationships with specific cloud providers, and this is what we expect to see more of this year. The issue of cloud lock-in will gradually take on shades of gray.
Note: Our Data Outlook for 2021 is in two parts. Click here for Part I, which addressed what's ahead for data and AI.
Disclosure: AWS, Microsoft, Oracle, SAS, and SAP are dbInsight clients.