High-performance computing (HPC) uses parallel data processing to deliver the speediest possible computing performance. Whether it's supercomputers, such as the Exabyte fast Frontier HPE Cray supercomputer, or clusters of hundreds or thousands of servers, it's all about taking complex tasks and running them as fast as machinely possible.
They're also all designed to simplify life for high-performance software developers. With them, programmers can gain continuous integration, turnkey software stacks, and performance testing. This will make their lives much easier.
Key supporters of HPSF include Amazon Web Services, Argonne National Laboratory, CEA, CIQ, Hewlett Packard Enterprise, Intel, and NVIDIA, among others. The foundation will establish a technical advisory committee to manage working groups on various HPC topics, following a governance model inspired by the Cloud Native Computing Foundation (CNCF).
In short, Rick Stevens, Argonne National Laboratory's associate laboratory director for computing, environment, and life sciences, explained, "HPSF will provide a collaborative environment and access to resources that will enable the community to collectively develop the software needed to drive current and future generation HPC systems, and Argonne is honored to join as a founding member."
The DAOS Foundation focuses on the governance and development of the Distributed Asynchronous Object Storage (DAOS) project. Pronounced to rhyme with "boss," DAOS is a high-performance open-source storage system that has revolutionized performance for HPC and AI workloads. It offers a fully distributed key-value storage architecture, overcoming the limitations of traditional POSIX-based HPC storage systems.
DAOS supports high bandwidth, low latency, and high I/O operations per second (IOPS). Again, the name of the game is speed, speed, speed. This is a mature project that Intel started working on in 2012. Intel will donate the DAOS source code to the foundation and continue as a major contributor.
Like HPFS, DAOS comes with the support of a laundry list of HPC movers and shakers, such as Argonne National Laboratory, Enakta Labs, Google Cloud, Hewlett Packard Enterprise, and Intel.
Under the Linux Foundation umbrella, these two foundations signify a major leap in collaborative efforts toward advancing open-source solutions in high-performance computing and storage. They'll foster innovation, lower barriers to entry, and support the development of critical technologies in HPC and AI/ML.