Inside a Microsoft data center

Inside a Microsoft data center

Summary: It takes mucho computes and terabytes to create the 3D models used in Microsoft's Virtual Earth online mapping service. So how do they cram 5,000 cores and 10,000 terabytes - 10 petabytes - of storage into 3 40 ft.

SHARE:

It takes mucho computes and terabytes to create the 3D models used in Microsoft's Virtual Earth online mapping service. So how do they cram 5,000 cores and 10,000 terabytes - 10 petabytes - of storage into 3 40 ft. shipping containers?

Containerized data centers Sun pioneered containerized data centers with their 2006 water-cooled Project Blackbox. But the Microsoft model is air-cooled and in production today in Boulder, CO.

Here's the exterior: Photo courtesy of the Microsoft Virtual Earth Gov Blog

It's what's inside that computes Putting ~3400 disk drives and 1700 cores into 320 sq. ft. isn't easy, which is what makes this picture so interesting:

Photo courtesy of the Microsoft Virtual Earth Gov Blog

Unlike Google's barebones racks, these racks are labeled Verari. Turns out that Verari Systems builds containerized data centers that look familiar:

Photo courtesy of Verari Systems

They also build high density storage and servers with very efficient cooling. They get 24 3.5" disks and up to 4 quad-core processors into 3 rack units - where most rackmount systems require 4U with half the cores.

Here's their high-density storage blade: Photo courtesy of Verari Systems

The Storage Bits take While I was skeptical at first, I'm now convinced that containerized data centers are the most economical and energy efficient alternative. Microsoft even bought wind-generated power for these units to underscore the point.

With the rapid growth of computing and data for science, commerce and entertainment we need to do it as efficiently as possible.

Comments welcome, of course.

Topics: Hardware, Data Centers, Microsoft, Storage

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

28 comments
Log in or register to join the discussion
  • Wow, on topic AND not brimming with anti-MS FUD!

    Will wonders ever cease? Good job Robin.
    NonZealot
    • Don't blame me. . . .

      My brand new Microsoft Natural Ergonomic Keyboard 4000 v.1 wouldn't
      let me!

      Other than that, love the keyboard.

      Cheers,

      Robin
      Robin Harris
      • Is that all it takes?

        Are you saying that for a $60 investment, we could buy Paul Murphy a keyboard and have him write something interesting for once? Sounds like a fantastic deal! :)

        The ironic thing is that you are known as the Mac $hill yet you use an MS keyboard and I'm known as the M$ $hill yet all of my peripherals are made by Logitech. :)
        NonZealot
        • Actually it was $20 from Dell

          I wouldn't have paid $60 for it - my old M$ Natural Keyboard is still like
          new. But I saw that price and had to have it. And its really good!

          Love my Logitech trackball too - though I broke 2 wireless ones in a
          month - so I stay wired now. Messy but it works.

          Robin
          Robin Harris
  • I reread this a few times...

    But I still can't find what would be confusing and/or shocking. It seems fairly logical to me.
    Spiritusindomit@...
  • God, but that's ugly!

    And I mean that professionally speaking. Certifiably, unspeakably, horrifically UGLY
    dilgreen
    • Uh.. It's a data center...

      ...Not something you put in your front lobby. It doesn't have to be pretty. It just has to get the job done.

      Wolfie2K3
  • RE: Inside a Microsoft data center

    anyone know how portable these units are? Can they be
    put into low earth orbit?
    green_collar
  • RE: Inside a Microsoft data center

    Hmmm looks a LOT like an IBM "mainframe" room
    with the "extras" i.e. Windows can be hacked
    and it crashes... totally pointless.
    devbravo
  • It's walking past that makes you vomit!

    They all appear to be outside, are ugly and come with dead trees!

    They are a techies drool and a designers nightmare..

    Someone throw a tarpaulin over them.
    graham.lv
    • Dead trees?

      Or maybe they're just taking a break for winter, as deciduous trees are known to do. The trees might appreciate Microsoft's use of wind energy as it means less acid rain producing coal power is used. If trees could appreciate things
      hungryjoe
  • Purpose of the stilts?

    I can imagine that they wouldn't want the containers sitting on the ground, but do they really need to be 6 feet off the groud (or whatever it is)?

    I think that's were the ugliness factor slips in. Those thick, spiraling stilts.

    At least on the vendor's home page, their container has trees painted on the side. Maybe if they'd lower it a bit it wouldn't be such an eyesore.
    Speednet
    • Electricians aren't mechanics

      Who wants to roll around on mechanic's creepers, trying to run power and cabling underneath the thing? Then again, even most busy mechanics would rather have the cars lifted six feet in the air, too.
      Worth2Cents
      • Not ceiling?

        I got the sense from the interior photo that the cabling was done through the ceiling, since it is a drop ceiling. I also did not notice any cables being routed underneath (from the exterior photo).

        Also, I can't imagine they would make holes in the bottom of the containers for cabling or anything else. That would not be very secure, and I think it would be easier routing them inside the drop ceiling.
        Speednet
        • Stilts keep containers well above any water

          Don't want equipment like THAT to get flooded.
          de-void-21165590650301806002836337787023
          • Datacenter Container on concrete stilts

            Good Guess! you're correct, we had to put them up as they are in a flood plain of a nearby creek.
            since this is in boulder, they have a local issue with not paving over the prarie dog habitat.
            Mark
            mark@...
        • Power from Ceiling, data from under the floor

          hi,
          the 2 480V 3phase whips per rack come in down from the false ceiling. its also the return path for the hot air.

          the network switches are in the back of each rack at the bottom in the 6u of utility space.
          each has mulitple 10G fiber links to the main datacenter
          mark
          mark@...
          • One of the efficiencies in Verari's design

            Verari takes the hot aisle/cool aisle design a step further by enclosing
            the racks so the cold air is sucked up through the racks and exhausted
            out the top. Seems they figured out that hot air rises.

            To paraphrase what firemen do, they pull the cold stuff past the hot
            stuff.

            Seems obvious, but it really makes for more efficient cooling.

            Robin
            Robin Harris
    • http://911review.org/search.html

      for flooding - a few places in New Orleans were sure glad they put their servers 6ft in the air

      http://911review.org/search.html
      911review
      • Ah, OK

        Thanks.
        Speednet