Supercomputer to speed up BOM's weather predictions

The Bureau of Meteorology (BOM) will soon to go to market to purchase a new supercomputer and datacentre.

As an initiative announced as part of the 2014-15 federal budget, BOM's new supercomputer and datacentre will replace the existing Oracle supercomputer that is currently housed in a datacentre northwest of Melbourne.

The new supercomputer, which is expected to be up and running in two years time, will be 20 times faster than the current machine. It will be the eighth machine the Bureau has procured, with each typically having a lifecycle of approximately six years, and receiving a mid-life upgrade that often multiplies the running speed by two.

Bureau of Meteorology director Dr Rob Vertessy told ZDNet that implementing the new supercomputer will address two main issues of weather forecasting, and provide system resilience that the organisation needs.

"For us to do a weather forecast for Australia, we have to collate the entire world's weather observation into one place. So that's problem number one where you've got an immense amount of observational data being generated, but the volume of that data is ballooning. We're creating about a 1TB of data a day, and that's going to probably multiply by about 10 times in the next six to eight years," he said.

"Issue number two is that the weather forecasting models that predict the weather. As the new supercomputer power becomes available, we can run those models with greater detail at higher resolution, and run them more frequently. Those models produce immense amounts of output as well."

The supercomputer runs observational data collected from the global weather system — via satellites, aeroplanes, ships, ground station, as well as from installations in the ocean, such as buoys and moorings — through a numerical weather prediction system, a model used to forecast the weather. The model currently runs four times, or every six hours, per day producing hourly forecasts for the next seven days for all weather parameters such as temperature, wind speed, and rainfall.

However, given the expected speed of the new supercomputer, Vertessy said predications will run every hour, 24 times a day meaning the forecasts will be "very fresh".

Other expected benefits from the new supercomputer will include improved resolution the models currently run on, where they will be "much finer", and will be able to "pick up a lot of detail and weather phenomena that the current models don't".

BOM will also have the opportunity to run the models in limited areas on demand, Vertessy said.

"Let's say you've got a fire in a particular part of the country, we'll be able to setup a limited area model to run the forecast at even finer resolution than we do for the whole country, so we can give higher level of intelligence to emergency services that will be out there fighting the fires," he said.

"This is what we call high forecast precision for emergency situations. They could be fires, cyclones, or floods but whatever the danger, I think we'll now be in a much stronger position to give detailed guidance to the emergency services personnel."

The tender process for the supercomputer — which is expected to take some months before a successful tender is selected mid-next year — comes off the back of the Bureau's recent upgrade of its data link to a managed dark fibre service. The data link is now more than twice the speed of its previous link, increasing connection between the Bureau's two main datacentres from 80 gigabits per second (Gbps) to 200Gbps. The new data link consists of two fibre cables of 100Gbps each, which operate concurrently.

Prior to the upgrade, the Bureau was running a service a bundled set of 10Gbps links with Next-Gen, which Vertessy said "served us very well but had the disadvantage of not being scalable, nor offering any redundancies, so we knew with the supercomputer, we're going to have a hell a lot more of information flowing, so we needed a larger trunk capacity, but we also wanted a system that was fully redundant".

The initial capital outlay for the upgrade cost the Bureau AU$528,000, with operating costs of AU$275,000 per year.

"It's a significant investment, but absolutely critical to what is a mission critical intelligence system," Vertessy said.

In addition, BOM will soon have access — when it's launched at the end of the year — to a new Japanese weather satellite. The satellite, known as Himawari 8 — which means sunflower in Japanese — will become the operational satellite for the Bureau mid-next year for tracking cyclones, the rapid onset of thunderstorms, looking for volcanic ash that could affect aviation, or fog and low cloud.

"Before the supercomputer comes online, which is really convenient for us, this will be like going from black and white television to full HD colour," Vertssy said.

"So instead of getting satellite image every hour, we'll now start getting them every 10 minutes. They'll also be four times the resolution of the current satellite, and three times the spectral resolutions, meaning we'll be able to see more weather phenomena that current generation satellites can't see."

The data link upgrade and supercomputer project form part of a significant IT transformation that the Bureau is currently undertaking. Spearheaded by former Department of Finance CIO Lesley Seebeck, the transformation will include replacing BOM's weather forecasting and flood forecasting systems, the development of a new storm surge forecasting system and the introduction of several new water information products and services.

The organisation has also taken a more integrated approach to IT, Vertessy said.

"About a year ago we restructured how IT was done, previously it was done as a separate part of the organisation, but what we wanted to do was bring it together and take a stronger enterprise approach," he said.

BOM is also embracing mobile technology, where currently 54 percent of visitors to the Bureau's website do so from a mobile device.  By the end of the year, the Bureau plans to release a weather app for Microsoft Windows and Apple iOS platforms.

While there are already plenty of existing weather apps already available on the market, Vertessy said the key differentiator users will have access to is its MetEye feature that provides 6 kilometres-worth of gridded weather information for the whole continent.

"The app will be the foundation for new services we'll fold into the app over time. You won't get the full set of features all in one go but over time we'll be folding in new functionality into the mobile app. Our intention is not to duplicate out there, our intention is to provide something revolutionary," he said.

By summer, BOM also plans to trial Twitter for the first time, and use it for issuing out situational awareness.