Linux heads for the data center

The project, being worked on by the Linux Cluster Cabal, aims to bring clustering to Linux. Clustering technology lets users harness multiple servers together to make one high-performance server. Originally created by Digital Equipment Corp. (NYSE:DEC) to give minicomputers the power of mainframes, it has extended into other arenas, including Unix.

The LCC is drafting a clustering architecture aimed at breaking the performance ceiling of today's commercial Unix-based clusters. The group aims to build a Linux cluster that could support up to 1,024 systems, or nodes, and do it much more cheaply than current high-end Unix clusters.

While there are other Linux clustering efforts underway, the LCC project will seek to match Digital's VMS-style clustering -- now marketed as Compaq TruClusters -- on a grand scale.

Movers and shakers
"There's no point working for clustering that stops at 100 nodes," said Cabal member Larry McVoy, who led Sun Microsystems Inc.'s first clustering initiative and is now president of BitMover Inc. "In eighteen months to two years, the dozen or so clustering initiatives on Linux will be dead," he predicted.

In addition to McVoy, the Cabal includes Linux heavy hitters Stephen Tweedie, who heads Linux file system development, and Peter Braam of Stelia Computing and leader of the group that built Carnegie Mellon University's Coda distributed file system.

They first met in secret in August to devise a clustering architecture that satisfies both commercial data processing and HPC (high-performance computing) requirements.

Although Linux has proved successful with the latter -- the Beowulf technical supercomputing project scales to several hundred nodes -- it has lacked the high availability features sought by corporate IT departments.

The Cabal takes advantage of the lack of kernel locks in Linux -- a virtue of its relative youth -- as the basis of its design. "Solaris has over 3,000 kernel locks. Whenever an OS gets that multi-threaded, you've made a pact with the devil," McVoy said.

Kernel locks are safety checks that ensure exclusive use of a computer's resources to one process. This hurts performance in some situations, and systems designers employ artful measures to minimize the disruptive effects of the locks on performance.

The power of Linux
One analyst agrees with McVoy that Linux may have advantages in clustering.

"The Linux kernel is simple and that is part of its strength," agrees Dan Kusnetzky, server analyst at International Data Corp. (IDC). "It's very fast and there's very little to go wrong with it in terms of functional capability. The developer can develop pretty much what he wants to."

At the same time, he says "it sounds like a very difficult thing to do."

Kusnetzky says that clustering has fallen in two religious camps: "shared nothing" adopted by NCR, Tandem, and Microsoft, and "shared everything", which relies on lock-management, and is the technology adopted by Compaq and Oracle. Finding a middle ground has eluded many architects, he said.

"If these people have come up with that scales to shared-nothing, but with the ease of development and configuration that you see in shared-everything, then they really have something. This would put Linux leading the single-vendor-creator operating systems."

But only, he noted, if the new architecture doesn't require application vendors to do major code rewrites.

Although scalable, such designs top out at 64 or 128 nodes, he says. The LCC design seeks to avoid such a ceiling by levering memory access capabilities used in symmetric multiprocessing (SMP).

The LCC architecture should minimize lock traffic in the region of common, or 'global' memory shared between machines on the cluster. One instance of the operating system is shared in a smaller unit, or "domain," of 2 to 8 CPUs.

The Cabal's design will accommodate multiple users on multiple operating systems all on the same cluster.

The design includes a VMS-style distributed lock manager, fail-over and recovery, and load-balancing. The group is also working with object-based storage vendors to incorporate advanced distributed file system technology.

Potential market
The project has won approval from power users worried by SGI's decision to sideline its Cray business.

"We in the high-performance computing community are very concerned about the future," said Robert Lucas, a scientist at the U.S. Department of Energy's Lawrence Livermore laboratory.

"While peak performance is going up, as is memory volume, by most other measures, we're going backwards right now," he said. SGI (NYSE:SGI) spent five years re-engineering the T3E [Cray], and is walking away from it, he added.

Lucas said though an open-source solution was viable: "Checkpoint [stopping a process and restarting it] and restart isn't going to be high on Linus [Torvalds, Linux kernel administrator] agenda, but it's top of ours."

But the Linux group must take heed of manageability features, warned IDC's Kusnetzky. For typical commercial users, "hardware and software costs combined are between 15 and 20 percent of the overall cost." The rest is the cost of employing administrators.

Public documents from the LCC are expected before the end of 1999.