First ARM benchmarks: The race is on

First ARM benchmarks: The race is on

Summary: This week, sandwiched between the annual Structure Big Data conference and the International Supercomputing show in Hamburg, Germany, ARM startup and HP partner Calxeda also found time to release the first well-documented x86 versus ARM benchmarks.

SHARE:

This week, sandwiched between the annual Structure Big Data conference and the International Supercomputing show in Hamburg, Germany, ARM startup and HP partner Calxeda also found time to release the first well-documented x86 versus ARM benchmarks. The results, shown below, are very positive — while there are some caveats that we need to note, the first generation ARM SOCs seem to deliver on their basic promise of much better performance per Watt.

The benchmark, which compares anew ARM SOC from Calxeda to a Sandy Bridge (not Ivy Bridge) low-end Xeon server with the same number of cores, shows that the Xeon CPU, while delivering more performance, has a very large deficit in workload per Watt, which is one of the key value propositions of the ARM community. Benchmark details:

Click for more details

Click for more details

Interpreting The Benchmark

First of all, this is a single benchmark, and its relevance is limited to its domain — lightweight web serving on a small web server with 1 Gb network. We cannot interpolate results based on a faster network configuration (although my guess is that this configuration is bottlenecked by the network, and a faster Xeon would not make much difference), nor can we extend the interpretation to other workloads. But within the benchmark domain, this early comparison tells us some important things:

  • Even with the current V7 32-bit architecture, the ARM CPU does indeed deliver impressive power efficiency.
  • Absolute performance, especially considering the huge difference in clock speed, is higher than most of us expected.
  • As a basic proof point, this benchmark succeeds as a proof of concept — AMR servers are indeed in the ballpark versus their initial promises.

Even if we start to do some “Kentucky windage” on the benchmark results to try and guess the impact of substituting a newer, more power-efficient Ivy Bridge CPU such as the Intel Xeon E3-1200 v2 and assume a 50% improvement in power efficiency, the ARM CPU workload/Watt advantage might drop to the neighborhood of 10X. Further deflation for a modest improvement in Ivy Bridge performance (assuming that network bandwidth might not be a bottleneck), would bring it down a bit further, possibly to the region of 8/1. Still very compelling from an overall efficiency perspective.

But Intel is not laying down on the job, and the announcement this week of the “Gemini” server from HP as an extension of their Project Moonshot scalable fabric-based server family powered by the latest generation of Intel’s Atom CPU will probably move x86 performance per Watt closer to ARM when it is available later this year. We look forward with anticipation to the continued competitive playoff in server technology and what it means for users.

Topics: Hardware, CXO, Processors, Servers

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

6 comments
Log in or register to join the discussion
  • Those are good numbers re:power.

    Like to see how it stacks up in real life with VM use.
    droidfromsd
    • yghmm. What VM on non x86 system...

      And 4GB RAM limit due to 32-bit limitation. This is useless
      przemyslaw.cias
      • Plenty of use cases

        The vast majority of all cloud use cases fit well within 4GB - in fact a typical small web site can run inside 1GB.

        I should think that HP's strategy would have small footprint use cases run on Arm32, or Atom (for x86 code), large memory jobs run on Arm64, high performance jobs run on Xeon or AMD (with OpenCL). Something for everyone - and minimize data centre power consumption and cooling requirements. I suspect that anyone who chooses not to see this as a viable strategy has a vested interest in one architecture or another.
        dimonic
  • Intel's Handicap

    Intel's instruction set is saddled with legacy baggage for running old Windows and DOS apps that are completely irrelevant to these sorts of workloads.

    Oh, and you don't need to capitalize "watt".
    ldo17
  • poor benchmark

    first of all: the xeon system wouldn't comsume the max TPD of 80watt. The CPU Load is around 15%. with 5 more networkadapters the xeon cpu load will be around 90% - and 6 times more request could be handled.
    so the advantage would drop to 2.5 - and than think about a modern Xeon E3-1230V2 ...

    so overall, still good performance but not so impressiv...
    trax75
    • The paradigm has shifted

      Very few use cases want or need vast CPU power - and when they do, HP has servers (intel based) that can deliver. They are after providing a solution for the other 80%+ of their use cases, where the low power Arm can still saturate the network and provides a 15x advantage in day to day running costs.
      dimonic