Why Microsoft, Google, and Amazon are racing to run your data center

Why Microsoft, Google, and Amazon are racing to run your data center

Summary: Several big technology vendors are racing to build a fleet of big data centers that will enable them to offer more Internet-based services to consumers and enterprises in the next 5-10 years. See why they think they will be able to talk you out of running your own data center.

SHARE:

The race for your data center has already begun. Google, Microsoft, and Amazon are the leading players in a global data center build-out that has not been slowed by the current economic recession and that over next decade will change the face of both consumer computing and IT departments.

The reason why these three companies are building out data center capacity around the world at a breakneck pace is that they want to be ready with enough capacity to handle the two big developments that are preparing to transform the technology world:

  1. Cloud computing: Applications and services delivered over the Internet
  2. Utility computing: On-demand server capacity powered by virtualization and delivered over the Internet

With both of these trends, the biggest target is private data centers. Cloud computing wants to run the big commoditized applications (mail, groupware, CRM, etc.) so that an IT department doesn't have to run them from a private data center.

Utility computing wants to simply take over server capacity for private services and applications, using virtualization to seamlessly scale up and scale down those services so that an organization only has to pay for the bandwidth and server capacity that it uses. What most IT departments do now is pay for maximum capacity at all times, with very low utilization, and also risk downtime at peak times if their systems get overloaded because they haven't planned for enough capacity at the high end.

This is an aerial view of Microsoft's data center in San Antonio, Texas. Microsoft has been adding roughly 10,000 new servers per month as part of the its ambitious data center build-out. Photo credit: Microsoft

It makes perfect sense that Microsoft and Google would want to go after this market. Microsoft already runs a lot of the server software that powers the back office, while Google already has lots of expertise in running data centers to power the Internet. Amazon's place here may appear odd to some, since the company started as an online book seller and has evolved into the Web's biggest mega-retailer.

However, Amazon has arguably become the current market leader in utility computing by using the knowledge it gained in building the infrastructure for its e-commerce business and turning it into Amazon Web Services in which it rents server capacity to other companies. In 2008, CEO Jeff Bezos even revealed that Amazon Web Services now uses more bandwidth than Amazon.com (see chart below).

I also expect IBM and Hewlett-Packard to join the party. While neither of those two are being noticed for building new data centers the way Google, Microsoft, and Amazon are, both of them are in the midst of massive, multi-year data center consolidations, and it's certainly possible that they are quietly building lots of extra capacity as part of those projects. HP has been a long-time proponent of utility computing, and IBM recently gave a public endorsement of cloud computing.

What all of these vendors will argue is that they can save organizations from overprovisioning and overspending on server capacity while also adding 24/7/365 monitoring, scalable load management, and a high level of IT service management (ITSM). Of course, the trade-off is that IT departments give up some control, and usually some staff positions as well.

This is essentially an outsourcing arrangement in which IT turns over a chunk of its operations to a third party. Many companies will be fine with that since IT is probably not one of their core competencies. They will welcome the expertise from a third party and will be happy to find a new way to control IT costs.

However, other IT departments and organizations are going to be far more reluctant to turn over their services, applications, and company data to a vendor. Just last week when Google announced that Google Apps can now use Microsoft Outlook as a client, I asked TechRepublic members, "Would you trust Google with your company's Exchange Server data?"

"I'm not sure it wouldn't land you in prison in many countries for violation of various laws on privacy and data security," wrote Deadly Earnest, an IT consultant in Australia.

"In the UK the Data Protection Act states that you must be able to disclose the location of your data, i.e. at any given moment you at least know what country is," wrote Tom-Tech, a UK software developer. "I'm pretty sure Google doesn't do this as they aren't exactly forthcoming with the locations of their data centers and the data belonging to a specific company probably shifts between a few of them anyway."

Zeplenith, an IT manager in Virginia, asked, "Where is the data stored? Are they SAS-70 certified?"

As such, security, privacy, and compliance are major hurdles that cloud computing and utility computing still must overcome. Nevertheless, we should expect that vendors are well aware of these hurdles and will be working with governments, regulators, and standards agencies to develop services that are fully compliant.

We can also expect that vendors will trip over each other trying to prove which one has the stronger security and privacy policies, because they know those factors are game-breakers. It's not going to happen overnight, but these obstacles will very likely be overcome. The companies involves have invested too much in the future -- and IT has too much to gain from a cost and management perspective -- for these issues to not be resolved.

For more insights on cloud computing and other tech topics, follow my Twitter stream at twitter.com/jasonhiner

Bottom line

For governments, large financial institutions, and other high-security environments, outsourcing the data center will probably never make sense. For virtually everyone else, it's going to become a very attractive option in the next 3-5 years. I suspect that a decade from now running your own data center will be the exception and not the rule, and IT departments will need a strong business case to justify the existence of a private data center.

The massive vendor build-outs currently underway are evidence that day is coming sooner than you might think, and IT leaders should prepare by experimenting with low-priority apps and workloads now, and thinking ahead about which parts of your current infrastructure will make the most sense in utility computing and which parts will present the biggest challenges.

Further reading:

Topics: Amazon, CXO, Cloud, Data Centers, Google, Hardware, Microsoft, Servers, Storage

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

30 comments
Log in or register to join the discussion
  • Old news repackaged

    Cloud and utility computing (which might be one and the same) are a
    multibillion dollar business in 2009. Are you guys for real about
    reporting this like news? We have been running our business at
    http://www.digitalchalk.com on the Amazon utility cloud for almost
    three years.

    Chalkboy
    • Not news, but opinion

      This isn't a news story. It's an opinion piece.
      jasonhiner
    • Sure, he did a good job of re-capping old news and discussing the

      importance and significance of it, and why the cloud
      is so important to the big players like MS, Google,
      and Amazon.
      DonnieBoy
  • Man i would love to run one of these sites or be number 2

    Been in IT for 10 years, unix/linux/win... give me a ring!

    I think its great that these companies are moving toward cloud computing services. Services are what make money.. look at a godaddy and google.

    These are sustainable and in demand. I will not say our company needs it or would move to it.. but maybe in the next 10 years. I imagine this will be common practice by then.

    How long until everything runs from the internet as a service through a webpage?
    Been_Done_Before
  • Yes, MS MUST maintain dependencies on Windows and Office.

    Microsoft does not even have to make money on the
    cloud, if it can reinforce the lock on the desktop
    and office suite.
    DonnieBoy
  • RE: Why Microsoft, Google, and Amazon are racing to run your data center

    I'm surprised by the lack of Amazon Web Services and the Google App Engine/Google Docs teams' lack of response to Microsoft's announcement of SAS 70 attestation and ISO/IEC 27001:2005 certification.

    Repeated requests to AWS and GAE for a statement re their certification intentions have been met with stoney silence.

    --rj
    http://oakleafblog.blogspot.com
    Roger_Jennings
  • I just don't get it.....

    Why would ANY company subject their IP to a 3rd party? Email and data should be protected and guarded... Not by a 3rd party company that is completely out of your control, but by trustworthy employees, bound by legal obligations, contracts, etc.

    Letting a 3rd party manage your email and data is just begging for failure.

    So seriously... Why? Why would you let strangers read your company confidential email?Why would you let strangers go through your data, Tax Records, Account numbers, HR confidential documents, etc. ???

    Anyone???
    EdumacationIT
    • I'm with you....

      The "cloud" has some very compelling numbers behind it, but you lose all real control. In effect you're being asked by a 800lb gorilla to "Trust me". Frankly that is THEE hurdle that the cloud will have to overcome.
      JoeMama_z
    • Noted

      Remember that at the end of the article I noted
      that high-security environments (that includes
      companies with very valuable and secretive IP)
      will likely avoid the cloud.

      However, I think we'll also see stronger
      security controls in place in utility and cloud
      computing that restrict access based on two-
      factor authentication that will keep prying
      eyes away from the data on the vendor side.
      jasonhiner
      • Every company has IP they don't want others to see.

        Every company has IP they don't want others to see at some time or another.

        Perhaps an employee is working on landing a client... Recipes.. Business practices... a CEO cheating on his wife.. A new product promotion... etc.

        I don't know of any companies that don't have competition. And those companies don't want the competition to know what they are up to. So they all have valuable and secretive IP. If a company doesn't have valuable and secretive IP, then they are not competing and won't be in business long.

        So again... I just don't see why any company would let strangers see, handle, touch and browse their information. The moment a company goes cloud, they might as well get rid of all document shreaders. Just not smart.
        EdumacationIT
    • We could easily substitute "Bank" for Third party, and Money for IP in what

      you are saying. Does that help you to realize how
      stupid your post was????

      Like employees are legally bound and trusted, but
      companies are not????
      DonnieBoy
      • Stupid?

        Banks <b>pay you</b> for the opportunity to handle your money. Plus, any money deposited is insured and replaced (albeit, limited amounts) if the bank screws up. If one of these IT firms screws up and all your data (or your clients data) end up public and spewed all over the internet, who's liable? You or the mega-million dollar corporation that can get protection by donating to a politician's re-election fund? Not to mention the difference between money (1 time use by the crooks) to data (knowledge is reusable).

        Does that help you realize how stupid <b>YOUR</b> post was?
        Jeff_D_Programmer
      • Get a clue DonnieBoy... sheesh...

        You need to go watch some schoolhouse rock dude?. They got it right? Knowledge is power. You can't substitute money for IP... It's IP dude... Get a friggin clue... Glancing at money versus a secret recipe, formula, or top-secret plans for a working cold fusion machine are nowhere near the same... Do you even comprehend what you type?

        It doesn't matter what the business it is... It could be a secret recipe or a customer a business is trying to land. IP is stuff you don't want any outside eyes to see. It's not dollar bills or how much money you have in the bank. It's the little threads of information/knowledge that give one company an edge over the competition. And the competition would be willing to pay for that information if some corrupt bozo working at a cloud data center had the ambition to sell a few secrets on the side to supplement his income.

        How would you tie that bozo into the fact that the competition now has your secret or could circumvent your plans to buy something or sneak in and land your client by underbidding you? There is no proof as to how the competition gained knowledge of your IP. That is why companies work so hard to make sure IP stays inside the company.

        The two biggest commodities the world has ever seen are information and sex. And those will always be the two biggest so long as human nature holds out. You don?t think DOW Chemical wouldn?t pay for a secret formula from a competitor? Information can be more valuable than anything else in the world. Get a clue... Really??? A Bank and money? Sheesh....

        Does that help you to realize how stupid your reply was????
        EdumacationIT
      • I agree w DonnieBoy

        For too long, corporate IT departments have been expensive
        roadblocks to technology progress and employee empowerment. Many
        spend an inordinate amount of time scaring technophobic CEOs into
        spending mega bucks on tecno-crazy security measures while
        dumbing employee IT services to the lowest common denominator.
        This makes their life really easy - watch the automated back-up
        routines and rebuff any request for cutting edge empowering
        technologies.

        Commoditized data storage makes a lot of sense. No one really has
        any idea what products will be offered. Client side encryption?
        Redundant storage in multiple locations? Hard copy media backups
        delivered by Brinks truck to your Swiss safety deposit box? Who the
        hell really knows?

        To say it can't be done securely is like saying man will never fly...
        oops!

        IT departments had better get use to the fact that most of what they
        do has been commoditized. Just a thought but maybe they could re-
        focus their efforts on serving, training, and empowering their
        customers (fellow employees).
        skipbhe
    • Another Fad

      Fads come and go. Some companies will go for this in an attempt to save money but when they realize how little control they truly will have they will not be happy.

      You call Google and they can't find your data and they'll get back to you tomorrow as they are swamped and have thousands of customers in line ahead of you.

      An OS running on hundreds or thousands of computers has been know to go down for days at a time ("Clouds").

      No matter what they do security-wise someone needs admin rights and someone will be able to see your data and that someone is not part of your organization.

      I prefer to do my own stuff in-house. I try to minimize being at someone else's mercy. You get what you pay for in life !
      pizzaman7
  • Check out Google Earth--Zanzibar

    Looks alien.
    BALTHOR
  • Organizations like AICPA...

    have warned their members that any data compromised through using services such as these opens them up for malpractice.

    These services will be handy for small businesses looking to shrink their investment in infrastructure and in highly skilled personnel. But I don't see companies with mission critical services using this wholesale cloud.
    bjbrock
    • They should be warning the businesses about data comprimised because they

      try to do things themselves that should be done by professionals will open them up to a ton of
      liability. If a large service provider screws up,
      they can recover the damages by suing the service
      provider. If the 22 year old programmer you hired
      after he graduated from the community college
      screws up - it is YOUR problem and YOU pay all of
      the damages.
      DonnieBoy
  • RE: Why Microsoft, Google, and Amazon are racing to run your data center

    Stupid idea. Why would anyone trust another for their data? Besides, if these companies (Microsoft, Google, Amazon, etc.) systems crash or are hacked, it will compromise those who use the service. Dumb, dumb, dumb.
    radesousa1
  • its all about people, people

    ...as the lack of them. This has been coming now for years. IT was regularly tarred with "that bloody computer took my job". Now it has another coloured tar: beige - "that new f**kg computer architecture and cloud thingy took away my IT job".
    Get used to it. Sooner or later the remote meta-data angle will be covered and then you are all screwed. Which then means the BRIC countries will simply reduce the running bill to almost zero and you have no industry left at all.
    And don't even think of mentioning "infrastructure is toast, but software development is still in our realm". Not for long buddy... the writing is on the wall for developers - once the drag and drop across the screen re-does all the backend application dance and dbase re-shuffle - what do they need you for?
    muzza2005