Why CERN chose OpenStack: A conversation with Rackspace's John Engates

Whenever I get the chance to chat with John Engates, Rackspace CTO, I jump at the opportunity. While I don't always agree with his point of view, I always learn something new.

Rackspace and CERN openlab

This time, John and I were discussing the fact that CERN openlab, the European Organization for Nuclear Research, has announced that it is collaborating with Rackspace to a deploy hybrid cloud computing environment. Openlab is an established framework that CERN uses to bring in external industry partners so that they can work together on technologies that may be used in the future. The goal is discovering the origins of the universe.

While CERN's computing environment is not representative of many commercial companies, the fact that this group of highly technical, highly expert people would select this environment is interesting.

It demonstrates that computationally intensive "technical computing" tasks can be happily running locally, out in the cloud, or some reasonable mix of the two.

I didn't speak with a CERN representative. Here are John's thoughts about their selection of Rackspace:

Large Hadron Collider is offline for refitting

During the time that the The Large Hadron Collider is offline for refitting, CERN is not sitting still. They are working to build an IT infrastructure to support the collider when it has twice the current capacity. This means building an infrastructure of 15,000 servers.

During the time that the collider is offline, CERN wanted to be able to use their server infrastructure for other purposes. They are envisioning being able to use both general-purpose and specific-purpose servers as part of an in-house cloud computing environment.

Why OpenStack?

Quite a bit of CERN's applications are written to run in the Phython development and run-time environment. Much of OpenStack is built using the same language. This meant that OpenStack would fit right in and be of immediate use.

At this point, CERN has four IT staff that have contributed code to OpenStack.

This also means that CERN can get "under the hood" of the cloud infrastructure and make it meet their needs. If what CERN needs is useful elsewhere, technology they develop can be made available to others. If what others develop is useful to CERN, they'll be able to easily take advantage of that technology. John described this as a "virtuous cycle."

Benefits CERN hopes to get

John's understanding is that they have a need for a tremendous amount of computing power. They have also invested time and effort in OpenStack and hope to leverage OpenStack to enhance what their limited staff can do.

They like the fact that one day, systems resources can be used to support an experiment running on the Collider, and the next day, those resources could be available to process and analyze the huge mass of data (they're collecting petabytes of data per minute during experiments).

Why RackSpace?

John believes that they chose to work with Rackspace because it is currently running one of the largest cloud resources in the world and that resource is running OpenStack. He thinks that they want to take advantage of OpenStack to create a private cloud today and, possibly, take advantage of public cloud resources at some point in the future.