IBM storage management software will be the workhorse behind a 40-petabyte robotic tape library for archiving climate simulation data generated by supercomputers at the German Climate Computing Center (DKRZ).
DKRZ has launched its new storage management system at its facility in Hamburg, which will serve as the main interface of the climate research group's huge tape library archive.
The new system is based on High Performance Storage Systems (HPSS ) architecture, a collaborative effort to develop petabyte-scale hierarchical storage systems (HSS).
HPSS has been jointly developed by IBM with the US Department of Energy's (DOE) national labs at Lawrence Livermore, Los Alamos, Sandia, Lawrence Berkeley, and Oak Ridge. The labs have some of the world's highest performing supercomputers that also generate petabytes of archive data.
According to DKRZ, its current archive of 40 petabytes is set to double over the next five years. While it's the world's largest store of climate data, according to IBM, it's well within the 500 petabyte capacity of its HPPS and HSS combination.
Besides the HPPS, DKRZ's archive systems consists of seven Oracle/StorageTek SL8500 tape libraries, catered to by 56 robots, and also has a total capacity of 190 petabytes.
IBM's contribution to the setup will improve throughput between its supercomputers and the archive, which flows in and out at at a sustained rate of 12GB per second with a planned upgrade later this year that will support peaks of 18GB per second.
DKRZ said it expects to push up towards the 500 petabyte limit once its replaced its current Blizzard supercomputer with HLRE-3, a supercomputer to be delivered by Bull at an initial cost of €26m.
The HLRE-3 is expected to blitz Blizzard once the first configuration is delivered, and by the time it is complete in 2016, will have a peak performance of about 3 petaflops per second (3 quadrillion floating point operations per second). Despite a significant boost in performance, on the world stage, it will be dwarfed by the China National University of Defense Technology Tianhe-2's 33.86 petaflop per second performance and the DOE Cray-made Titan, which is capable of 17.59 petaflops per second.
According to IBM, the archive will support research on climate change by the Intergovernmental Panel on Climate Change; simulations to predict the impact of oil spills; assess the impact of climate change on West Africa, the Mediterranean, Central Europe, Indonesia; analyse air traffic routes; develop 3D models for more efficient city construction; and weather prediction.
Besides DKRZ, other organisations that will use the archive include European Centre for Medium Range Weather Forecasts, the Karlsruhe Institute for Technology, the German Meteorological Service and the computing center Rechenzentrum Garching of the Max Planck Society.
Read more on storage