InfiniBand could boost Linux supercomputing
Get Up to Speed on... Open source  Get the latest headlines and company-specific news in our expanded GUTS section. | ||||
The group, called the OpenIB Alliance, has begun work on unified Linux software support. Among those in the effort are established names such as Intel, Dell, IBM, Sun Microsystems and Oracle; InfiniBand specialists such as Topspin Communications, InfiniCon, Voltaire and Mellanox; and supercomputer users Lawrence Livermore and Sandia national laboratories.
InfiniBand is a high-speed networking technology that offers short delays and minimum processor effort when one computer needs to communicate with another. It arrived later than advocates hoped and isn't as widely used as once promised, but it's now an increasingly common feature of supercomputers made by connecting numerous low-end systems.
For example, InfiniBand is the interconnection fabric of the No. 3 system on November's Top500 list of supercomputers, Apple's System X with 1,100 dual-processor systems.
One person looking forward to the new effort is Don Becker, chief technology officer of Penguin Computing and a founder of the "Beowulf" idea of linking Linux machines together into a supercomputing cluster.
"I think the OpenIB effort is going to get the critical mass we need to make sure the vendors are working together, producing a working solution rather than battling over a market that won't occur unless they work together," Becker said. "In the next few months, I think things will be improving pretty rapidly."
The OpenIB alliance has been working on prototype Linux support since February. It plans to publish a technology schedule by the end of September and to release software within a year, said Allyson Klein, the initiative's marketing manager.
Linux in high-performance computing
Linux is catching on in high-performance computing circles. It can be obtained for free, its open-source nature makes it customizable, and it's largely compatible with the Unix expertise that's already widespread in the market. However, Linux rival Microsoft is angling for more supercomputing presence.
Linux companies such as Red Hat sell their products to high-performance computing customers, but Linux's openness permits those customers also to take matters into their own hands.
For example, two high-energy physics labs that use Linux clusters, the Fermi National Accelerator Laboratory (Fermilab) and the European Organization for Nuclear Research (CERN), are collaborating on software called Scientific Linux.
Scientific Linux is compiled from a modified version of the source code that underlies Red Hat Enterprise Linux, with a goal to be compatible with Red Hat's product.
Linux is also the operating system used by Blue Gene/L, an IBM system that's climbing the ranks of the most powerful supercomputers, as it moves from prototype to final status. Software running on the system sees it as a standard Linux cluster, but internally, Blue Gene/L is wired differently; Linux runs only on a small fraction of command processors that marshal the processing power of supporting chips that run a minimal operating system.
Open-source software is also used more broadly in supercomputing--for example, the Message Passing Interface (MPI) used to handle communications across a cluster. Augmenting that work, Ohio State University researchers have produced a version of MPI for InfiniBand that has been used on several Top500-ranked machines.
Merging with open source
But it can be awkward to bring commercial technology into the open-source realm--where detailed software must be freely available and modifiable.
One potential pitfall is stepping on the toes of existing programmer efforts--as IBM did when it released its mainframe Linux support that duplicated some work of an outside project. One Linux InfiniBand effort was already under way at the SourceForge site for collaboration on open-source programming projects.
But the new powers won't be imposing in this case, said Brian Stevens, vice president of operating systems development at Red Hat. "The project on SourceForge is essentially dead from lack of activity and adoption," Stevens said.
Another issue is reconciling different code bases. The OpenIB project began with software from Mellanox Technologies, the dominant seller of add-in cards that link servers to an InfiniBand network, but a Topspin programmer later launched his own effort, according to the OpenIB mailing list. The next day, Mellanox said it supported the software and would help.
Open-source license, copyright and intellectual-property issues also can complicate matters. The OpenIB Alliance will keep the copyright to its software, Klein said, but will accept outside contributions from others.
The software will be released under two licenses, the General Public License (GPL) used by Linux and the BSD license used by various Unix offshoots such as FreeBSD and parts of Apple Computer's Mac OS X.
And there's always the battle against time. InfiniBand isn't without competition: Several engineers who worked on InfiniBand now have directed their attention to overlaying a similar technology on the widely used TCP/IP standard that powers the Internet and countless corporate networks. In particular, Hewlett-Packard is an advocate of this technology, called Remote Direct Memory Access, or RDMA, over TCP/IP.