Cloud computing and the return of the platform wars

Sun’s announcement last week that its new Cloud Compute Service would be API compatible at a storage level with Amazon’s popular S3 service is probably the first real evidence of the coming platform war in the cloud computing space. It’s a war that’s likely to be significant and protracted given the number of players that are lining up for a shot at what’s sizing up to be the next big development in the evolution of computing.
Written by Dion Hinchcliffe, Contributor

Sun's announcement last week that its new Cloud Compute Service would be API compatible at a storage level with Amazon's popular S3 service is probably the first real evidence of the coming platform war in the cloud computing space. It's a war that's likely to be significant and protracted given the number of players that are lining up for a shot at what's sizing up to be the next big development in the evolution of computing.

It must be easy to move existing applications and data into the cloud.The final outcome of this struggle, as it's been in many earlier platform battles over personal computer hardware, operating systems, databases, and even the Web itself, will be the result of a fairly predictable and oft-repeated cycle of events (see diagram below) for which a small number of large winners are likely to emerge victorious.

When we look back many years from now, it's probable that cloud computing will be regarded as both a momentous and major change of course in the history of software; many future computing platforms will be created and operated by what seemingly amount to utility companies. While this might seem like a boring future for computing, it's a necessarily pragmatic evolution as the very size and scope of modern software requires new economic models in order to remain cost effective. Virtually any online application these days has to scale to a few million users as quickly and inexpensively as possible.

However, cost is just one of the interesting aspects of cloud computing and the stakes are huge: The Wall Street Journal reported today that the cloud computing industry is estimated to reach $42 billion by 2012, or nearly half the entire software business.

Computing Platform Lifecycle
The world of software has recently, at least up until now, been moving slowly and steadily towards an increasingly commoditized, virtualized, and open sourced future. Cloud computing in its present form does appear to herald a return to the classical days of big vendor computing -- and all the baggage (good and bad) that it implies -- along with some unique twists of its own.

This means a lot of the old issues are back: Proprietary, commercial systems running our applications, very real risks of vendor lock-in, the requirements of adapting our businesses to difficult-to-customize one-size-fits-all computing models, and many others. While some companies are still dealing with these issues from the last round of computing platforms, a growing percentage of them have opted recently for more open and collaborative offerings such as open source, LAMP, and lightweight applications stacks from non-commercial vendors. Non-trivially, cloud computing also adds a number of all new concerns to the mix as well. Governance issues such as risk and trust are prominent as well as run-time concerns around the latency and performance of cloud-based applications.

The Growth of Cloud Computing: Open APIs, Storage, Computing, Infrastructure

The modern network era, however, has ushered in SaaS and Web 2.0 services which have been chipping away with a growing degree of effectiveness at the do-it-all-yourself view of IT that we've classically held for so many years. New computing models that take advantage of the inherent strength of networks to harness resources, distribute costs, and accumulate shared value have become compelling precisely because they've now become fully realized as products over the last year. They are now also (mostly) ready for prime-time for businesses to use and rely upon. This is true of both the open API model, which is a more application specific form of cloud computing as well as the more horizontal type such as storage, processing, and infrastructure, which is dominating the cloud discussion at the moment.

Cloud computing: A Faustian bargain?

Of course, many organizations would not consider dealing with these issues if it wasn't for the significant advantages that cloud computing offers in return for accepting some baggage. These advantages are:

  • Economies of scale. This is something that both commercial and open source software, at least in their traditional models where the customer runs everything on premises, has been poor at providing to customers. Network-based software in the form of shared resource cloud computing have been found to be very effective at building value in scale and passing it along to customers. While cloud computing can indeed be done without building economies of scale (for example, when using the development technologies in-house before actual deployment), one of the prime advantages of the model is distributed costs across as many other customers as possible. While cost is only one of the advantages of cloud computing, it is also one of its most compelling aspects and reports of 20% to 10x and more in savings are common in current cloud computing case studies. What's intriguing is how effective economies of scale build up across the board with cloud computing, from the cost of hardware and software to management, power, facilities, maintenance, backup, administration, etc. A good percentage of these costs are unburdened from the customer when they move to the cloud, are integrated, and then spread out to the size of the cloud provider's customer base.
  • Business and IT agility. Cloud computing can enable companies to access and activate significant resources on-demand that would be impossible, or at least financially untenable, as an individual organization. Entire enterprise-class data centers can literally be provisioned and deployed in minutes with almost no effort. They can then be resized, downsized, or decommissioned just as easily. The elasticity of most cloud computing offerings can be quite impressive when compared to traditional build-out efforts or even outsourced hosting.
    Granularity is a related topic and here there is a significant difference in the granularity of today's cloud computing offerings, particularly on the computing power side of things. Granularity of service typically ranges from the request level (Google App Engine) up to the virtual server instance level (EC2 or Sun's Compute Cloud), with substantially higher latency with the latter model than the former. Granularity of service tends to become important depending on the architecture of the applications you have. Which cloud you choose can also impose considerable architectural requirements on your applications depending on the one you select. In any case, organizations can use this agility to take advantage of new opportunities and business conditions (both good and bad) on much shorter timescales than with almost any previous computing model. It's also important to note that provisioning is only part of the cloud computing agility story; many cloud computing platforms (Force.com and Google App Engine especially) are highly productivity-oriented and provide an environment where the entire development lifecycle has been facilitated for the rapid creation of cloud-based solutions that natively take advantage of cloud capabilities.
  • Centralization of best practices and competency. Cloud computing vendors can use their economies of scale to make larger cumulative investments in improving their offerings and the technologies underlying them. They can also attract and leverage leading talent to making sure their designs, methods, practices, and infrastructure are the latest and most competitive. While very large enterprises can approach this level of best practice adoption, they are paying a significant premium for this and making investments in areas that are almost always outside of their core competency. This is one area where there will likely be considerable variety from vendors on this point (usually in direct correlation with cost of service) and discriminating cloud computing customers will periodically perform due diligence in this area.

Achieving success in the cloud

So given what we know from previous platform wars, what will the leading cloud computing offerings have to address? Here are the likely outlines:

  • Easy on-ramp and adoption. It can't just be easy to create new cloud computing solutions, it must also be easy to move existing applications and data into the cloud. Migration tools that can move entire Java and .NET applications and their databases will probably be popular. Even more so will be cloud platforms that play well with existing management and operations applications such as Tivoli and Patrol. In today's environment, there will be less tolerance for large migration and conversion efforts to the cloud, and more interest in effective tools for self on-boarding.
  • Support for cloud standards. While the strongest cloud computing offerings will actually set the standard (such as Amazon's widely used S3 service which has already been adopted by Eucalyptus and Sun's cloud and which it's been reported recently that the Cloud Computing Interoperability Forum is seriously considering adopting) the best way to get ahead in the platform wars is to take the most popular standards and provide the best implementations. The Gang of Nine famously did this with the ISA personal computer bus early on the PC revolution by taking an IBM platform standard and providing competing implementations behind the interface. We're now seeing something similar with the cloud industry and the S3 interface. Though open standards can emerge first and implementations later as it did with Web services, the cloud computing space is moving so quickly that the opposite is appearing to take place at the moment. Amazon has an early lead here but there is also a lot of standardization work left though it will continue to be very challenging for standards to keep up with the cloud computing industry. The bottom line: Today's customers understand that standards give them choice and flexibility (providing them with the ability to move between clouds that support the same standards, for example) and as I described above there is already evidence of initial coalescing around some implementations as de facto standards. Update: Read about the brouhaha this week about the Open Clouds Manifesto for the very latest perspective on cloud computing standards.
  • Robustness of platform. The bigger a cloud gets, the more value it can provide to everyone that is using it. However, that's only one axis of value: cost. The capabilities of a cloud will also be key to determining if it's suitable for a customer's use by meeting their existing needs or solving problems in new and compelling ways. Storage, computing, queuing, content distribution, management tools, application specific APIs, are all capabilities that various clouds currently offer in very different ways. Those with the ability to provide what customers need and in a form that they want, or at least a form they find highly usable, will discriminate the winners from the also-ran cloud contenders. For example, non-relational databases like Force.com's object store and Amazon's SimpleDB are some of the more commonly available ways to use cloud storage today. This is in stark contrast with how business applications generally work with data today which is primarily with relational databases based on SQL. Even though this has begun to change and block storage is now available in some cloud platforms, cloud providers are also clearly imposing certain architectural requirements in order to meet their SLAs and keep performance levels where their customers will expect it. This means that the providers that do a good job striking the balance between robustness and pragmatism will be rewarded. So too will cloud platforms that are robust with enterprise-class features. My recent exploration of cloud computing and IT noted that many cloud platforms often don't address enterprise requirements at all. However, the enterprise market still makes up the majority of strategic IT dollars spent and will probably be the bigger part of the cloud market overall, though decidedly a customer of a tougher nature.
  • Cloud development tools built by developers, for developers. Particularly with software platforms, developer adoption plays an absolutely crucial role for success. For years a significant portion of Microsoft's huge momentum with the Windows platform was due to its outstanding Microsoft Developer's Network and the flow of powerful tools, support, and information it provided. This made developers feel they were the center of the company's attention and gave them what they needed to succeed in using the platform. It's no accident that the leading cloud provider today, Amazon, is also very developer-centric with the primary evangelism and outreach being made to developers first and only to a lesser extent directly to businesses. Smart computing platform companies understand that they will live and die by their developer communities since this is the constituency that will actually be responsible for connecting organizations with their clouds. A heavy developer emphasis is also evident in Force.com and Sun's Compute Cloud, the latter which has a new REST Cloud API that will almost certainly set the standard for design excellence and best practices, which will help attract the best developers to build upon it. Lesson: Build your cloud for developers and the enterprises will come.
  • A good reputation and a proven operational track record. While courting developers is essential, businesses typically gauge their business partners via other yardsticks. This is where a cloud computing platform's reputation for safety, stability, reliability, and security will be crucial for strategic adoption by businesses. It's already become common, for example, to publish uptime information, real-time status, and recent reliability data. Amazon's Service Health Dashboard and Google's App Status Dashboard so that everyone can verify for themselves the quality of individual compute clouds. One area where there is little information yet is in latency and performance data; most companies don't know how well a cloud will actually perform for them until they try to use it. Other reputation issues, such as potential lock-in horror stories like impeding a customer's move to other clouds could hurt vendors if they happen but have largely been non-events up to this point (though this might be due to the industry's newness). Ultimately, a cloud computing provider's reputation and perceived ability to execute will be extremely important to customers who will, because of the fundamental infrastructure that cloud computing provides, will make this factor a key discriminator more than just cost alone.

Like many newer open business models, the current cloud computing landscape leaves a lot of unanswered questions for businesses today. Much of these are currently centered around risk, discomfort about losing control, and not-invented here syndrome.

Clouds: An agile springboard for 21st century enterprises?

Few major enterprises are prepared to make investments in cloud computing right now when it's unclear that their investment isn't going to be stranded in either an successful cloud or in security and governance issues can't be adequately addressed. This will likely result in several things happening:

First, it will encourage an industry-wide move towards cloud uniformity so that interoperability and data portability between clouds is one of their selling points, greatly lowering risk concerns. Interestingly, we've perhaps seen the glimmering of this with the industry's early ad hoc adoption of Amazon's S3 API. It will get more interesting and compelling when data transfer from other clouds is built into existing products so that customers can move their data around at the push of a button.

Second, the best clouds will begin adding features and certifications that "prove" they are safe, frequently audited by a reliable third party, and straightforward for businesses to regain full control if needed. This might look like an "ejection button" feature that begins an automatic migration back into a pre-configured data center or a hot-sync feature that keeps a live copy of all the data in the data center. These might be "training wheel" options for enterprises that are worried about the move to the cloud, though they are not sustainable for long-term operation.

However, the clouds are not waiting and it's been the small and medium sized businesses that have had no resources to build their own world-class data centers that have been blazing the trail so far. Since software powers so much of the world today, one interesting notion is whether cloud computing will be the springboard that allows them to create agile 21st century businesses that eclipse older, traditional firms that can't adapt. Combined with 2.0 business models, this is quite possible.

What are the reasons holding you back from using the cloud today? Please share your concerns and ideas in TalkBack below.
Editorial standards