Clearing the fog from cloud

People are making the cloud computing and storage much more complicated than it has to be. There are 2 key elements in cloud infrastructure: scalability and manageability. Those are the hard problems. If you don't have those you CAN'T have a cloud. The rest is pool.

Tempest in a cloud My ZDnet colleague, Phil Wainewright, recently opined

Yet in my view, the most important attribute of the cloud — too readily overlooked by many commentators — is that it lives in the Internet. The Internet dimension is crucial because it brings with it an obligation and a necessity to remain open to connections. It means that a cloud has to have:

• Open APIs
• Unlimited bandwidth
• Collective scrutiny and innovation

These are all Good Things, but only if they rest on scale and management. Otherwise, what's the point?

Public vs private James Hamilton, Amazon architect and a very smart guy, recently blogged about private clouds. In Private Clouds Are Not The Future he argues that economies of scale make public clouds much more efficient than private clouds.

I agree that several effects make web scale public clouds more efficient:

• Quality. Large clouds can economically employ experts to design and optimize their services and infrastructure. Security and server/storage design are two areas where deep expertise can provide more reliable and efficient service.
• Utilization. Power systems and power cost are optimized when data centers are run at 100% utilization. As utilization rises across the board so does the capital efficiency, i.e. work per invested dollar.
• Cost. Large-scale investments create their own lower-cost dynamic. Public cloud providers save money on infrastructure acquisition through volume buys. Volume lets them get high-efficiency power supplies or custom cost-reduced motherboards, that offer little economic advantage to small buyers.

With all these advantages it is obvious that private clouds are not the future. Or are they?

It isn't all about the Benjamins Economics is not the driver many assume. But some cloud capabilities will filter down to smaller-scale private clouds.

Enterprise data center costs are often 50%-70% labor. Internet data centers are as low as 3%. If cloud techniques allow enterprises to reduce their labor to 20% of cost, then private clouds become competitive with public clouds for critical apps.

Why? Often perceived benefits cannot be measured in dollars. Convenience, availability, consistency and control often relate to emotional needs and wants that are rarely quantified or questioned.

But we don't have to invoke those to understand why private clouds will be part of the computing landscape. Just a quick look at one of the large Internet data centers will tell us what we need to know.

Show me the power All the advantages of public clouds have analogs in the world of power generation and distribution. Power generation is cheapest when centralized and large-scale distribution systems move power at the lowest cost per watt.

Electrical power generation and distribution is over 125 years old. The technology is well understood, the industry is mature, and has a massive infrastructure for production and distribution.

And yet Google's dual 85,000 sq. ft. Dalles, Oregon data centers, built next to a substation a few miles from the nation's largest hydropower system - one of the world's most reliable power sources - flanks each data center with generators. I expect Amazon does the same.

Why do Google, Amazon and every other cloud service provider invest millions in private power production? They don't trust public power.

Same with the Internet.

Access Clearly, access to data is at least as important as access to power or why would data centers spend the money on uninterruptible power supplies?

Is the Internet that different?

We cannot rely 100% on Internet access to our data. Given the outages we've seen to date, even 99% is a stretch.

If the application is important enough, as judged by often subjective human criteria, we will keep our data as close as Google keeps its generators.

Even if it isn't the most economic choice.

The Storage Bits take I've grappled with the question of private clouds for the last couple of years. The advantages of web scale systems are obvious, but the need for reliable data access and control has not receded.

Public and private will not displace each other: they will coexist just as public and private power sources coexist today. No doubt public clouds will claim the majority of the market whether measured in dollars or exabytes, but private clouds will remain significant contributors to our data infrastructure for decades, if not centuries, to come.

Web-scale techniques will filter down to the enterprise and as the decades pass more and more legacy infrastructure will be migrated to them. Just as payroll is now outsourced - plenty of sensitive data there! - some apps will find their way to public clouds. But mission critical data storage will remain inside because Internet latency and bandwidth will always be slower and smaller than private local clouds.

Comments welcome, of course.