Between the Lines

Larry Dignan, Andrew Nusca and Rachel King

Why Microsoft, Google, and Amazon are racing to run your data center

By | June 15, 2009, 7:32am PDT

Summary: Several big technology vendors are racing to build a fleet of big data centers that will enable them to offer more Internet-based services to consumers and enterprises in the next 5-10 years. See why they think they will be able to talk you out of running your own data center.

The race for your data center has already begun. Google, Microsoft, and Amazon are the leading players in a global data center build-out that has not been slowed by the current economic recession and that over next decade will change the face of both consumer computing and IT departments.

The reason why these three companies are building out data center capacity around the world at a breakneck pace is that they want to be ready with enough capacity to handle the two big developments that are preparing to transform the technology world:

  1. Cloud computing: Applications and services delivered over the Internet
  2. Utility computing: On-demand server capacity powered by virtualization and delivered over the Internet

With both of these trends, the biggest target is private data centers. Cloud computing wants to run the big commoditized applications (mail, groupware, CRM, etc.) so that an IT department doesn’t have to run them from a private data center.

Utility computing wants to simply take over server capacity for private services and applications, using virtualization to seamlessly scale up and scale down those services so that an organization only has to pay for the bandwidth and server capacity that it uses. What most IT departments do now is pay for maximum capacity at all times, with very low utilization, and also risk downtime at peak times if their systems get overloaded because they haven’t planned for enough capacity at the high end.

This is an aerial view of Microsoft’s data center in San Antonio, Texas. Microsoft has been adding roughly 10,000 new servers per month as part of the its ambitious data center build-out. Photo credit: Microsoft

It makes perfect sense that Microsoft and Google would want to go after this market. Microsoft already runs a lot of the server software that powers the back office, while Google already has lots of expertise in running data centers to power the Internet. Amazon’s place here may appear odd to some, since the company started as an online book seller and has evolved into the Web’s biggest mega-retailer.

However, Amazon has arguably become the current market leader in utility computing by using the knowledge it gained in building the infrastructure for its e-commerce business and turning it into Amazon Web Services in which it rents server capacity to other companies. In 2008, CEO Jeff Bezos even revealed that Amazon Web Services now uses more bandwidth than Amazon.com (see chart below).

I also expect IBM and Hewlett-Packard to join the party. While neither of those two are being noticed for building new data centers the way Google, Microsoft, and Amazon are, both of them are in the midst of massive, multi-year data center consolidations, and it’s certainly possible that they are quietly building lots of extra capacity as part of those projects. HP has been a long-time proponent of utility computing, and IBM recently gave a public endorsement of cloud computing.

What all of these vendors will argue is that they can save organizations from overprovisioning and overspending on server capacity while also adding 24/7/365 monitoring, scalable load management, and a high level of IT service management (ITSM). Of course, the trade-off is that IT departments give up some control, and usually some staff positions as well.

This is essentially an outsourcing arrangement in which IT turns over a chunk of its operations to a third party. Many companies will be fine with that since IT is probably not one of their core competencies. They will welcome the expertise from a third party and will be happy to find a new way to control IT costs.

However, other IT departments and organizations are going to be far more reluctant to turn over their services, applications, and company data to a vendor. Just last week when Google announced that Google Apps can now use Microsoft Outlook as a client, I asked TechRepublic members, “Would you trust Google with your company’s Exchange Server data?

“I’m not sure it wouldn’t land you in prison in many countries for violation of various laws on privacy and data security,” wrote Deadly Earnest, an IT consultant in Australia.

“In the UK the Data Protection Act states that you must be able to disclose the location of your data, i.e. at any given moment you at least know what country is,” wrote Tom-Tech, a UK software developer. “I’m pretty sure Google doesn’t do this as they aren’t exactly forthcoming with the locations of their data centers and the data belonging to a specific company probably shifts between a few of them anyway.”

Zeplenith, an IT manager in Virginia, asked, “Where is the data stored? Are they SAS-70 certified?”

As such, security, privacy, and compliance are major hurdles that cloud computing and utility computing still must overcome. Nevertheless, we should expect that vendors are well aware of these hurdles and will be working with governments, regulators, and standards agencies to develop services that are fully compliant.

We can also expect that vendors will trip over each other trying to prove which one has the stronger security and privacy policies, because they know those factors are game-breakers. It’s not going to happen overnight, but these obstacles will very likely be overcome. The companies involves have invested too much in the future — and IT has too much to gain from a cost and management perspective — for these issues to not be resolved.

For more insights on cloud computing and other tech topics, follow my Twitter stream at twitter.com/jasonhiner

Bottom line

For governments, large financial institutions, and other high-security environments, outsourcing the data center will probably never make sense. For virtually everyone else, it’s going to become a very attractive option in the next 3-5 years. I suspect that a decade from now running your own data center will be the exception and not the rule, and IT departments will need a strong business case to justify the existence of a private data center.

The massive vendor build-outs currently underway are evidence that day is coming sooner than you might think, and IT leaders should prepare by experimenting with low-priority apps and workloads now, and thinking ahead about which parts of your current infrastructure will make the most sense in utility computing and which parts will present the biggest challenges.

Further reading:

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Jason Hiner is the Editor in Chief of TechRepublic. He writes about the products, people, and ideas that are revolutionizing business with technology.

Disclosure

Jason Hiner

Jason Hiner has nothing to disclose. He doesn't hold investments in the technology companies he covers.

Biography

Jason Hiner

Jason Hiner is the Editor in Chief of TechRepublic, an online trade publication and peer-to-peer community for IT leaders. He is an award-winning journalist who examines the latest trends and asks the big questions about the technology industry. He previously worked as an IT manager in the health care industry.

You can also find him on Twitter, , Facebook, and at JasonHiner.com.

Related Discussions on TechRepublic

Did you know you can take part in these discussions with your ZDNet membership?
30
Comments

Join the conversation!

Just In

Stick with your 8 tracks, CDs, and DVDs
taintpretty 17th Jun 2009
Every day inept sys admins discover their backups are incomplete and data is lost. Users lose licensing keys with such regularity that it provides countless jobs.

We have plenty of people that cannot imagine not having their data centers within reach located in a pretty raised floor DC in the corporate office proudly displayed for all to see. The price of electricity in a hydroelectric area as compared to a fossil fuel area?

These times they are a changing so to think this is just a fad shows a lack of the overall big picture. There are plenty of customers that will not take this route but to see how well these huge datacenters are managed, the security, redundancy, and horsepower is very impressive.

I just returned for Microsoft's new Chicago after installing the first cargo container of systems. This ain't no fad. The San Antonio DC has MAYBE tem Msft employees there and more systems than you can imagine. This is a degree of efficiency I could not have imagined when I was a sys admin at a company with a mere 1000 servers.

The best analogy was mentioned above considering money. You cherish your data for the money that it brinks into your company but you turn right around give that money to someone else to manage. We all know the banks have done a wonderful job of managing money and we continue to deposit our checks with them daily.
0 Votes
+ -
Old news repackaged
Chalkboy 15th Jun 2009
Cloud and utility computing (which might be one and the same) are a
multibillion dollar business in 2009. Are you guys for real about
reporting this like news? We have been running our business at
http://www.digitalchalk.com on the Amazon utility cloud for almost
three years.

0 Votes
+ -
Staff
Not news, but opinion
Jason Hiner 15th Jun 2009
This isn't a news story. It's an opinion piece.
0 Votes
+ -
importance and significance of it, and why the cloud
is so important to the big players like MS, Google,
and Amazon.
0 Votes
+ -
Been in IT for 10 years, unix/linux/win... give me a ring!

I think its great that these companies are moving toward cloud computing services. Services are what make money.. look at a godaddy and google.

These are sustainable and in demand. I will not say our company needs it or would move to it.. but maybe in the next 10 years. I imagine this will be common practice by then.

How long until everything runs from the internet as a service through a webpage?
Microsoft does not even have to make money on the
cloud, if it can reinforce the lock on the desktop
and office suite.
I'm surprised by the lack of Amazon Web Services and the Google App Engine/Google Docs teams' lack of response to Microsoft's announcement of SAS 70 attestation and ISO/IEC 27001:2005 certification.

Repeated requests to AWS and GAE for a statement re their certification intentions have been met with stoney silence.

--rj
http://oakleafblog.blogspot.com
0 Votes
+ -
I just don't get it.....
EdumacationIT 15th Jun 2009
Why would ANY company subject their IP to a 3rd party? Email and data should be protected and guarded... Not by a 3rd party company that is completely out of your control, but by trustworthy employees, bound by legal obligations, contracts, etc.

Letting a 3rd party manage your email and data is just begging for failure.

So seriously... Why? Why would you let strangers read your company confidential email?Why would you let strangers go through your data, Tax Records, Account numbers, HR confidential documents, etc. ???

Anyone???
0 Votes
+ -
I'm with you....
JoeMama_z 15th Jun 2009
The "cloud" has some very compelling numbers behind it, but you lose all real control. In effect you're being asked by a 800lb gorilla to "Trust me". Frankly that is THEE hurdle that the cloud will have to overcome.
0 Votes
+ -
Staff
Noted
Jason Hiner 15th Jun 2009
Remember that at the end of the article I noted
that high-security environments (that includes
companies with very valuable and secretive IP)
will likely avoid the cloud.

However, I think we'll also see stronger
security controls in place in utility and cloud
computing that restrict access based on two-
factor authentication that will keep prying
eyes away from the data on the vendor side.
0 Votes
+ -
Every company has IP they don't want others to see at some time or another.

Perhaps an employee is working on landing a client... Recipes.. Business practices... a CEO cheating on his wife.. A new product promotion... etc.

I don't know of any companies that don't have competition. And those companies don't want the competition to know what they are up to. So they all have valuable and secretive IP. If a company doesn't have valuable and secretive IP, then they are not competing and won't be in business long.

So again... I just don't see why any company would let strangers see, handle, touch and browse their information. The moment a company goes cloud, they might as well get rid of all document shreaders. Just not smart.
you are saying. Does that help you to realize how
stupid your post was????

Like employees are legally bound and trusted, but
companies are not????
0 Votes
+ -
Stupid?
Jeff_D_Programmer 16th Jun 2009
Banks pay you for the opportunity to handle your money. Plus, any money deposited is insured and replaced (albeit, limited amounts) if the bank screws up. If one of these IT firms screws up and all your data (or your clients data) end up public and spewed all over the internet, who's liable? You or the mega-million dollar corporation that can get protection by donating to a politician's re-election fund? Not to mention the difference between money (1 time use by the crooks) to data (knowledge is reusable).

Does that help you realize how stupid YOUR post was?
0 Votes
+ -
Get a clue DonnieBoy... sheesh...
EdumacationIT Updated - 16th Jun 2009
You need to go watch some schoolhouse rock dude?. They got it right? Knowledge is power. You can't substitute money for IP... It's IP dude... Get a friggin clue... Glancing at money versus a secret recipe, formula, or top-secret plans for a working cold fusion machine are nowhere near the same... Do you even comprehend what you type?

It doesn't matter what the business it is... It could be a secret recipe or a customer a business is trying to land. IP is stuff you don't want any outside eyes to see. It's not dollar bills or how much money you have in the bank. It's the little threads of information/knowledge that give one company an edge over the competition. And the competition would be willing to pay for that information if some corrupt bozo working at a cloud data center had the ambition to sell a few secrets on the side to supplement his income.

How would you tie that bozo into the fact that the competition now has your secret or could circumvent your plans to buy something or sneak in and land your client by underbidding you? There is no proof as to how the competition gained knowledge of your IP. That is why companies work so hard to make sure IP stays inside the company.

The two biggest commodities the world has ever seen are information and sex. And those will always be the two biggest so long as human nature holds out. You don?t think DOW Chemical wouldn?t pay for a secret formula from a competitor? Information can be more valuable than anything else in the world. Get a clue... Really??? A Bank and money? Sheesh....

Does that help you to realize how stupid your reply was????
0 Votes
+ -
I agree w DonnieBoy
skipbhe Updated - 16th Jun 2009
For too long, corporate IT departments have been expensive
roadblocks to technology progress and employee empowerment. Many
spend an inordinate amount of time scaring technophobic CEOs into
spending mega bucks on tecno-crazy security measures while
dumbing employee IT services to the lowest common denominator.
This makes their life really easy - watch the automated back-up
routines and rebuff any request for cutting edge empowering
technologies.

Commoditized data storage makes a lot of sense. No one really has
any idea what products will be offered. Client side encryption?
Redundant storage in multiple locations? Hard copy media backups
delivered by Brinks truck to your Swiss safety deposit box? Who the
hell really knows?

To say it can't be done securely is like saying man will never fly...
oops!

IT departments had better get use to the fact that most of what they
do has been commoditized. Just a thought but maybe they could re-
focus their efforts on serving, training, and empowering their
customers (fellow employees).
0 Votes
+ -
Another Fad
pizzaman7 16th Jun 2009
Fads come and go. Some companies will go for this in an attempt to save money but when they realize how little control they truly will have they will not be happy.

You call Google and they can't find your data and they'll get back to you tomorrow as they are swamped and have thousands of customers in line ahead of you.

An OS running on hundreds or thousands of computers has been know to go down for days at a time ("Clouds").

No matter what they do security-wise someone needs admin rights and someone will be able to see your data and that someone is not part of your organization.

I prefer to do my own stuff in-house. I try to minimize being at someone else's mercy. You get what you pay for in life !
0 Votes
+ -
Check out Google Earth--Zanzibar
BALTHOR 15th Jun 2009
Looks alien.
0 Votes
+ -
Organizations like AICPA...
bjbrock 15th Jun 2009
have warned their members that any data compromised through using services such as these opens them up for malpractice.

These services will be handy for small businesses looking to shrink their investment in infrastructure and in highly skilled personnel. But I don't see companies with mission critical services using this wholesale cloud.
try to do things themselves that should be done by professionals will open them up to a ton of
liability. If a large service provider screws up,
they can recover the damages by suing the service
provider. If the 22 year old programmer you hired
after he graduated from the community college
screws up - it is YOUR problem and YOU pay all of
the damages.
Stupid idea. Why would anyone trust another for their data? Besides, if these companies (Microsoft, Google, Amazon, etc.) systems crash or are hacked, it will compromise those who use the service. Dumb, dumb, dumb.
0 Votes
+ -
its all about people, people
muzza2005 16th Jun 2009
...as the lack of them. This has been coming now for years. IT was regularly tarred with "that bloody computer took my job". Now it has another coloured tar: beige - "that new f**kg computer architecture and cloud thingy took away my IT job".
Get used to it. Sooner or later the remote meta-data angle will be covered and then you are all screwed. Which then means the BRIC countries will simply reduce the running bill to almost zero and you have no industry left at all.
And don't even think of mentioning "infrastructure is toast, but software development is still in our realm". Not for long buddy... the writing is on the wall for developers - once the drag and drop across the screen re-does all the backend application dance and dbase re-shuffle - what do they need you for?
0 Votes
+ -
Dev
Jeff_D_Programmer 16th Jun 2009
Somebody's got to write that drag and drop application. There will always be a job for intelligent, knowledgable people. Of course, what the other 99.99% of the world's population is going to do is beyond me...

(relax, it's a joke. sort of...)
0 Votes
+ -
More bandwidth used
magallanes Updated - 16th Jun 2009
"In 2008, CEO Jeff Bezos even revealed that Amazon Web Services now uses more bandwidth than Amazon.com (see chart below)."

But you can't directly associate the bandwidth used with incomes. Instead, is more easy to associate bandwidth with expenses.
0 Votes
+ -
There is a logical explanation
EdumacationIT Updated - 16th Jun 2009
"In 2008, CEO Jeff Bezos even revealed that Amazon Web Services now uses more bandwidth than Amazon.com (see chart below)."


Sounds like somone was downloading porn on the company dime.... LOL


0 Votes
+ -
Speaking of bandwidth...
Jeff_D_Programmer Updated - 16th Jun 2009
Here's a factor that hasn't been adressed yet: Pushing your datacenter to the "cloud" may sound really great, but most companies - especially smbs - don't have dueling T1s for a connection.

If you've ever tried having 6 PCs all running SaaS at once over a 128k DSL, you'll see this idea crashes almost as fast as the machines themselves. Until mega-bandwidth is universally available and cost effective (basically - free), Cloud computing cannot take hold with most smbs. That means Amazon, Google, Microsoft, etc. are all waiting on AT&T - and AT&T can't even properly support an iPhone with their current system. How long will it be until they can support broadband cloud access for everybody?
Some people have commented that this is old, repackaged news but, it's not. It's foreshadowing the wave of the future for IT.

In my past corporate life, I personally watched one of the biggest pharma companies in the world scramble to cut their IT cost year after year. I don't mean by a couple hundred servers either, I mean dramatic cuts in order to save money.

The new Cloud and Utility computing models will ONLY increase that trend and will help it pick up even more steam.

The issues raised about security and compliance will be addressed and, in a lot of cases, already have been.

We work primarily with AWS and 3Tera to provide public and private Cloud Computing offerings to our clients and the combination is well received in our sphere of influence.

Paul Laskin, our CIO, has been preaching that the demise of the SMB data center will be within the next 5 years and this article only bolsters that.

We're on the brink of some very exciting transformations. I'm glad we're at the forefront of it and get to experience the outcome.

Mike
www.cirrhus9.com
0 Votes
+ -
People, IT is already 85% outsourced
vmirchan 16th Jun 2009
Some of the comments behave as if current IT is completely, pristinely untouched by outsiders.

If you add up the average CIOs budget only 15% of all IT and telecom spend comes from his/her staff - the rest is spent with sw companies, outsourcers, telecom companies. Plenty of outside fingers - many incompetent and over priced - already touch your process, infrastructure, data.

Hopefully Clouds will replace some of that spend and exposure - and in fact, shrink it
0 Votes
+ -
Have you ever spent 1 day without internet ?
stakabo@... Updated - 16th Jun 2009
This hole cloud computing is a great idea, but what happens
in case you have no internet. there are multiple reason internet why
could fail: major storm, war, no electricity ...

what happens then ...
Jeff_D_Programmer has an excellent point. Affordable bandwidth is a very critical point in the move to cloud computing. And reliable connectivity is also a huge issue the vendors gloss over. SMBs will need higher bandwidth and redundancy in their Internet connections to make this work.
Agree conpletey.Sooner or later, outsourcing of data centers will be the order of the day due to the extent of risk transfer and cost savings. However, it appears a new set of controls and regulations would come up in place of the existing ones to take care of data security and privacy aspects.
regards
Monalisa
0 Votes
+ -
Stick with your 8 tracks, CDs, and DVDs
taintpretty 17th Jun 2009
Every day inept sys admins discover their backups are incomplete and data is lost. Users lose licensing keys with such regularity that it provides countless jobs.

We have plenty of people that cannot imagine not having their data centers within reach located in a pretty raised floor DC in the corporate office proudly displayed for all to see. The price of electricity in a hydroelectric area as compared to a fossil fuel area?

These times they are a changing so to think this is just a fad shows a lack of the overall big picture. There are plenty of customers that will not take this route but to see how well these huge datacenters are managed, the security, redundancy, and horsepower is very impressive.

I just returned for Microsoft's new Chicago after installing the first cargo container of systems. This ain't no fad. The San Antonio DC has MAYBE tem Msft employees there and more systems than you can imagine. This is a degree of efficiency I could not have imagined when I was a sys admin at a company with a mere 1000 servers.

The best analogy was mentioned above considering money. You cherish your data for the money that it brinks into your company but you turn right around give that money to someone else to manage. We all know the banks have done a wonderful job of managing money and we continue to deposit our checks with them daily.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix