UQ's new supercomputer is pushing the limits in analysing human skull models

The University of Queensland (UQ) is leveraging the power of its new supercomputer to analyse human skull models, with the work conducted being dedicated towards delaying the onset of one of the world's most debilitating illnesses -- Alzheimer's Disease.

Courtesy of Dell Technologies, UQ's new high performance computer (HPC) system is being used by the Research Computing Centre (RCC). The system, dubbed Weiner, is capable of processing massive amounts of computational tasks in parallel, including data visualisation and machine learning, which allows for the modelling of possible treatments for illnesses.

See also: Photos: The world's 25 fastest supercomputers (TechRepublic)

Speaking with media at the Dell Technologies Forum in Sydney on Tuesday, UQ RCC chief technology officer Jake Carroll said the centre boasts a wide range of employees, from physicists through to computer scientists, even those specialising in humanities.

"People from all walks of research need to be able to participate and integrate with these things," Carroll said.

He said UQ's challenge was massive; it has workflows that are of "colossal nature" -- petabytes upon petabytes of storage, and all of the problems that are associated with it.

"Data has weight so there's a consequence, I suppose. The compute that goes along with that data to leverage actual information, rather than just an unstructured data mess is probably core and critical," Carroll said.

Some of the more interesting use cases the university is tackling at the moment is around aging, specifically dementia and Alzheimer's, Carroll said.

The university's neuroscience research institute, the Queensland Brain Institute, has been using the Wiener system to model the behaviour of ultrasound using an analysis technique called Finite Element Method.

The modelling calculates what happens to each element of the brain when an ultrasound is passed through the skull. UQ is hoping that ultrasound can potentially be used to temporarily allow for the direct delivery of therapeutic drugs to the brain. This is currently not possible however, due to the presence of plaques that can disrupt brain functions and damage active cells and the blood-brain barrier which are hallmarks of Alzheimer's disease.

UQ now hopes to confirm its results in a study on sheep. This is because sheep have similar skull properties to humans.

Other use cases include climate modelling, psychological testing and learning, and disease identification.

"It's become a whole ecosystem," Carroll said. "Wiener has become a plethora of massive machine learning and deep learning capabilities in the organisation. It's the focal point of AI computing infrastructure at the University of Queensland."

UQ's School of Information Technology and Electrical Engineering is also using Weiner to develop new digital pathology techniques for faster blood samples results, while another machine learning algorithm has been created to diagnose the presence of skin cancer from histology slides with the accuracy of a trained pathologist, UQ said.

The supercomputer also supports the university's lattice light sheet microscope (LLSM), which uses an image restoration technique called deconvolution to provide clear, real-time 4D biology imaging.

Weiner, pieced together by Dell Technologies alongside UQ in November 2017, is currently the focal point for artificial intelligence (AI) and machine learning at the university, Carroll said.

The Nvidia Volta-based system, he said, has given the university a competitive advantage that no one else in the country had a few years ago. However, it has since needed to triple the size of its Weiner machine.

High performance computing and AI chief technology officer for Dell EMC in the Asia-Pacific and Japan (APJ) region, Andrew Underwood, told ZDNet that the original system had two phases, with the first consisting of two visualisation nodes -- PowerEdge R740 -- which were the computational building blocks for the supercomputer.

Phase two of the original system, which was driven by the growth in machine learning and deep learning, was built on Dell Technologies' PowerEdge C4140.

With the latest system, Underwood said, the specifications were designed around UQ's requirements; specifically, its heavy use of machine learning and AI workloads that are performed in real time.

The visualisation nodes are comprised of: Two Dell EMC PowerEdge R740 HPC nodes featuring two Intel Xeon Gold 6132, 28 cores per node; 1TB DDR4 RAM; three Nvidia Tesla V100 Accelerator units; 1.6TB Dell EMC NVMe flash storage; and 100Gbps Mellanox EDR InfiniBand HCA's.

The compute and analysis nodes boast 17 Dell EMC PowerEdge R740s, featuring two Intel Xeon Gold 6132, 28 cores per node; 384GB DDR4 RAM; two Nvidia Tesla V100 Accelerator Units; 1.6TB Dell EMC NVMe Flash Storage; and 100Gbps Mellanox EDR InfiniBand HCA's.

And the Phase 2.0 nodes are comprised of: 15 Dell EMC PowerEdge C4140, featuring two 2 x Intel Xeon Gold 6132, 28 cores per node; 384GB DDR4 RAM; four Nvidia SXM2 Tesla 32GB V100 Accelerator Units; 1.6TB Dell EMC NVMe Flash Storage; and 100Gbps Mellanox EDR InfiniBand HCA's.

"From an LLSM, running at full-tilt, we could probably get about 80TB a day of output data. We have put it at full-tilt a couple of times ... average datasets might be 7-8TB, which is still completely untenable for most normal environments to work in, hence you need supercomputing, networks, and big storage," Carroll said.

"We're in a world now that's far too big for that concept of [compute] back at the workstation," he added. "They [are] taking things from a human CT scan, MRI machine, through to the computational domain."

Carroll also told ZDNet that UQ is "quite convinced" that accelerated computing in all of its incarnations and iterations is "absolutely here to stay".

"It is the path to capability. We do not have enough CPU power in the world in our little environment to do these things anymore, we believe strongly enough in the ecosystem in accelerated computing and the associated capabilities of companies like Dell and Nvidia put together that we have invested heavily -- this will set us up for probably a three-year tranche," he explained.

"We will keep pushing; we have to keep pushing. If we don't, we get behind in our research competitive capability. We can't discover the way we want to, so we don't stop -- this is the nature of this game."

RELATED COVERAGE

• NCI boasts Australia's fastest supercomputer with AU$70m Gadi system
• CSIRO receives deep learning supercomputer from Dell EMC
• Pawsey gets AU$70m to upgrade two supercomputing clusters
• University of Sydney unveils AU$2.3m Artemis 3 AI research supercomputer  
• South Australia to house Defence's new AU$68m supercomputer centre
• Garvan Institute gets new supercomputer for genomic research
• It takes two to Tango: eRSA partners with Dell EMC for new research supercomputer
• Swinburne reaches for the sky with AU$4m astronomical supercomputer
• Medical imaging at the 'speed of light': Nvidia's Clara supercomputer