BOM gets Cray supercomputer for weather prediction

The Bureau of Meteorology's (BOM) new Cray XC-40 supercomputer has been successfully commissioned, bolstering the organisation's ability to predict the weather.

The supercomputer had already been in use internally by the organisation, with the BOM confirming that the Australian public should start to see improvements in forecasts as of January next year.

BOM deputy director of information systems and services Dr Lesley Seebeck said the supercomputer marks a major milestone for the bureau, noting that the successful commissioning also demonstrates the BOM's capacity to deliver large-scale IT initiatives.

"The increased computing power will allow the bureau to undertake a program of improvements to enhance the frequency, accuracy, and certainty of forecasts in the coming years," Seebeck said.

The Cray XC-40 supercomputer runs on a Linux-based operating system, specifically designed to run large, complex applications and scale efficiently to more than 500,000 processor cores.

It houses 2,160 compute nodes, with 51,840 Intel Xeon cores, 276TB of RAM, and a usable storage of 4.3PB.

According to Seebeck, the new system will allow the BOM to take on board a lot of the data that is coming in from new and upgraded data sources, as well as provide a greater resolution.

"By 2018, we're going to move into a 1.5km grid, we're going to get right down to fine resolution in the cities, versus the 4km we currently see," she said.

The bureau announced in June last year that it had signed the AU$77 million contract with Cray to replace the ageing Sun Microsystems machine that was commissioned in 2013.

The BOM originally revealed its intentions to upgrade its supercomputer in August 2014, with the bureau's now-CEO Dr Rob Vertessy telling ZDNet at the time that implementing the new supercomputer will address two main issues of weather forecasting, and provide the system resilience that the organisation needs.

"For us to do a weather forecast for Australia, we have to collate the entire world's weather observation into one place. So that's problem number one, where you've got an immense amount of observational data being generated, but the volume of that data is ballooning. We're creating about a 1TB of data a day, and that's going to probably multiply by about 10 times in the next six to eight years," he said.

"Issue number two is the weather forecasting models that predict the weather -- as the new supercomputer power becomes available, we can run those models with greater detail at higher resolution, and run them more frequently. Those models produce immense amounts of output as well."

The Cray system is the ninth supercomputer to be commissioned by the bureau, with the XC-40 expected to deliver about 16 times the capacity of the previous one.

The new machine should also see a lifespan of about three years, with a mid-life upgrade scheduled for 2018.

The bureau received funding in the 2014-15 federal Budget for the supercomputer program to invest in both the new computing hardware and software improvements to numerical weather prediction modelling and forecast products.

In December, the BOM said its systems were fully operational and reliable in response to concerns that the weather bureau had suffered from a large breach.

"The bureau does not comment on security matters," it said at the time. "Like all government agencies, we work closely with the Australian government security agencies."

When BOM officials faced the Senate Environment and Communications Committee in February, they were probed with questions pertaining to the breach, but remained tight-lipped on the details of the event.

"I can say a few things, the first is that there have been no security-related disruptions to our service delivery, to our ICT systems at all -- that's the first thing," Vertessy said.

"The second is that it is well-known throughout the internet and the systems that we all run in government and business that there are a range of threat actors out there that require gradually improving security posture for those agencies to minimise the risks of the violations.

"The bureau, like all agencies, has an active program of improving its ICT security posture and we are in the fortunate circumstance because we're rebuilding some of our ICT infrastructure chiefly around the supercomputer. We've got the ability now to redesign the architecture of our systems as such that we have improved ICT security."

When handing down his defence whitepaper in April, Prime Minister Malcolm Turnbull confirmed that the BOM was in fact a victim of a cyber attack. BOM director Dr Andrew Johnson claimed during Senate Estimates in October that no sensitive information was taken.

"No personal information or sensitive data in our database has been accessed, but I stress that's to the best of my knowledge," Johnson said last month.

Charles Lim, industry principal of Cyber Security Practice at Frost & Sullivan, said in May that the attack was a deliberate attempt to cripple the nation's economy.

"[Australia has] a very big export base which is the food that is farmed in your country," he said. "If you get weather predictions wrong, that's going to affect your economy severely.

"These are the new areas cyber attackers are working on and we have to be concerned about that."

The BOM did confirm that it was the Australian Signals Directorate (ASD) that detected the initial intrusion.