CSIRO upgrades supercomputing cluster
CSIRO is in the process of upgrading its GPU cluster, packing more hardware into the supercomputing facility to further increase its performance.
Supercomputers like the IBM Blue Gene/Q are designed for speed.
(Credit: IBM)
Although it is called a GPU cluster, it is actually a combination of Intel CPUs and Nvidia GPUs. It also has the ability to use Linux or Windows applications as required by scientists.
Melbourne-based Xenon Systems, which has been supporting CSIRO and the cluster with its past upgrades, is performing the upgrade, and will result in the cluster being powered by 268 Intel Xeon E5-2650 processors (totalling 2144 CPU cores) and 390 Nvidia Tesla M2050 GPU cards (totalling 174,720 GPU cores).
CSIRO said that one of the reasons for the upgrade is to free-up rack space for newer and future GPU cards, and to move the cluster to Intel's new platform architecture.
The new CPUs use Intel's Sandy Bridge-EP 8 core processors, and are expected to result in lower power consumption while almost doubling existing performance. Changes in GPU density will also mean that the CSIRO will be able to fit four Nvidia GPU cards per 1U of rack space, allowing for better use of physical space and better future planning.
The GPUs also make use of PCI-e 3.0, allowing I/O bandwidth to be increased up to 8 gigatransfers per second (1GTps) per PCI-e lane. Additionally, the increase in I/O bandwidth from upgrading the CPUs will result in a performance boost of about 20 per cent in the cluster's GPUs. CSIRO has also upgraded the InfiniBand interconnect, which allows data transfer between the cards. Its previously used DDR (double data rate) InfiniBand link will be upgraded to a non-standard FDR-10 (fourteen data rate) link, providing a 2.6x increase in maximum data rate.
The GPU cluster has been used in the past to crunch imaging data from the Australian Synchrotron, providing real-time, computer-generated models of objects smaller than the width of a human hair. when it would previously have taken several hours to compute. But the GPU cluster isn't limited to just imaging.
"We're in the business of turning information into knowledge, and quantifying uncertainty to help people make better decisions, whether that's in bio-chemical marine models, climate modelling or for medical imaging," said CSIRO head of Computational and Simulation Sciences Dr John Taylor.
"The faster we can process data, the faster we can get solutions to market — and that's in Australia's best interests."
Prior to its upgrade, the cluster ranked 212th on the Top500 list of the most powerful supercomputers, and 42nd on the Green500 list of energy-efficient supercomputers.
Taylor told ZDNet Australia that with the upgrade, the processing power of the cluster should jump from about 75 teraflops to around 120 to 130 teraflops. Of these, at least 10 teraflops can be attributed to the doubling of CPU speed and 15 teraflops to the 20 per cent increase in GPU performance directly from I/O bandwidth improvements.
While it is difficult to know where the cluster will rank, given that other supercomputing facilities are also always improving, Taylor is hopeful of seeing the cluster place within the top 100.