Mirantis and IBM team up on OpenStack benchmarking tool

Benchmarks can either be useful tools for IT decision makers or, unfortunately, just another way for suppliers to offer misinformation to cloud an already complex product selection.
Written by Dan Kusnetzky, Contributor

Mirantis, in partnership with IBM's SoftLayer, just announced the availability of a benchmark test designed to help IT decision makers better understand key performance, scalability and managability criteria of the OpenStack environment and that environment executing in an IBM SoftLayer cloud computing data center.

Here are Mirantis' findings

Mirantis and IBM SoftLayer reported the following description of their test and the findings:

The benchmark used the latest Mirantis OpenStack 4.0 (Havana) distribution running on CentOS Ver 6.4, deployed in a high-availability configuration across distributed data centers -- from start to finish in about 8 hours.

Using 350 physical servers on an IBM SoftLayer bare-metal multi-datacenter cloud, Mirantis OpenStack stood up 75,000 virtual servers, provisioned in parallel streams levels ranging from 100 at a time to 500 at a time. The benchmark was designed to show how quickly and reliably OpenStack could respond to on-demand, real-world workloads requirements for provisioning cloud resources, achieving a sustained a rate of 9000 new virtual servers per hour for over 8 hours.

Key takeaways:

  • Enterprises considering where to run large-scale workloads can rely on OpenStack for a private cloud with up to 75,000 virtual servers.
  • Running Mirantis OpenStack on IBM SoftLayer provides highly responsive on-demand cloud resources, provisioned at over 9,000 virtual servers per hour.
  • Mirantis OpenStack and IBM SoftLayer enable resilient multi data center cloud deployments.

Snapshot analysis

Depending upon the way benchmarks are used, they can either be useful tools allowing IT decision makers to gauge how their workloads would fare when executing on a given system configuration or are just another way for suppliers to misinform or mislead potential clients. The findings reported by Mirantis, in partnership with IBM, appear to be in the first category.

It is impossible for IT decision makers to acquire and test every single possible system configuration to get a good idea of how their organization's workloads would run. Furthermore, it isn't always clear how a given configuration will handle growth or decline in workload resource requirements.

Benchmarks in the highest sense are designed to allow a fair comparison of different execution environments each performing the same set of tasks. Unfortunately, many players in the industry learned that they could game the system either by playing games with the standard benchmarks or by creating their own benchmarks that show their product in the best possible light, but can't be reproduced by anyone. These same suppliers often include prohibitions against running standard benchmarks and reporting the results.

Miraintis appears to have directly addressed many of the questions my clients have asked about using an OpenStack-based cloud service, including: How high and low can I scale a cloud computing environment; how long does it take for a new virtual machine to start up or shut down; or how can I create multi data center workload configurations to assure high levels of performance and reliability.

If your organization is considering OpenStack or cloud computing in general it would be worth taking time to learn more about Mirantis' benchmarking tool.

A detailed technical description of the benchmark configuration, tools and results is available at www.mirantis.com/openstackbenchmark.

Editorial standards