Intel's new Xeon triples I/O performance; brings 10 GbE to the motherboard
With Intel's official announcement today of the general availability of new Xeon E5 family of processors, which the media has already been covering via leaked information since the late 2011 Intel announcement about this next-generation Xeon family, there will be a lot of discussion about how the move to Sandy Bridge microarchitecture and the increase in performance that these new CPUs bring to the server and workstation market. But the improved CPU performance is only a small part of this story, and not the one that is of greatest import to the datacenter market.
The real story is the huge improvements in I/O performance that the E5 Xeon family will bring. While earlier announcements spoke about this being the first generation of Xeon processors to support PCI Express 3.0 this is only part of the I/O improvement the new CPU brings. Intel is keenly aware that the key to success in the datacenter is the ability to scale; to support virtualization that means the needs of the customer and allows the customer to grow their enterprise without forcing them to immediately upgrade their systems to meet increased business demand.
The additional improvements in I/O come via two additional technologies; Intel Data Direct I/O and Intel Integrated I/O. Coupled with PCIE 3.0 Intel has seen triple the I/O performance when comparing the new E5-2680, which hit 66 GB/s in tested I/O bandwidth, compared to the previous Xeon X5670 which achieved only 18 BB/s in the same internal tests. The new X5's Integrated I/O reduces latency by up to 30% by moving the I/O management from an external chip to within the CPU, which would improve overall throughput on its own, but when coupled with the other technology announced today, the Intel X540 10GBASE-T Ethernet controller even greater performance is possible.
This is because the new Ethernet controller can take advantage of Intel Data Direct I/O. The controller writes directly into the CPU L3 cache, reducing the amount of work that needs to be done to move data from the network into the processor. In Intel's testing they have been able to support more than 50 active VMs on a single X540 adapter. And the technology scales, with tests showing over 250 GB of throughput through 16 adapters, connected via PCIE 3.0. As an added benefit, Intel has also seen a reduction in power consumption using this technology, with the avoidance of hitting memory resulting in a 60W drop in power use compared to previous generations of Xeon without Data Direct.
The X540 10GbE controller, while it can be integrated into an OEM NIC, is designed to bring 10 GbE to the motherboard, with a small package, low power consumption interface that is backwards compatible with 1GbE switches, allowing new servers with this technology to be rolled out without any networking infrastructure changes. With Intel supporting 10GBASE-T on CAT 6 or better wiring, their studies have shown that 96% of datacenters are already in position to take advantage of this technology. And when compared to existing 10GbE top-of-rack solutions, the X540 Lan-on-motherboard (LOM) comes in at only 25% of the cost of an SFP+ fiber per port deployment.
10GbE as a standard LOM connection, with the ability to run over existing copper networking will do a lot to drive the adoption of 10GbE as a standard beyond back-end interconnects in the datacenter and will likely accelerate the growth of 10 GbE in the datacenter, especially when compared to the fairly slow adoption rate that 1 GbE saw early in its deployment.