That (other) f-word

In recent weeks, my writings about UnixWare andAIXhave brought me deeperinto the rather curious world of large-system Linux -- the attempts tomake open source operating systems competitive in environments wherebig boxes host arrays of CPUs and huge tracts of RAM. But the deeper we get,the more I wonder whether, when it comes to large systems, the Linux worldhas come to an impasse.

It's been tiring to hear the way both salespeople andinsiders at vendors of proprietary Unix systems sneer at Linux. Theconventional wisdom appears to be that Linux is many years behindmainstream Unix in areas of high-end scalability and performance. Sometimes such claims evenshow up in public. But even when the complaints are accurate I don'tsee the resolution being "many years away." Linux vendors say they're on top of it.

I heard similar whines about GUIs a few years ago, and from hereit looks like the Linux desktop has surpassed the tired Unix offeringsof Motifand CDE, both in usability and popularity. So let's turn to thearea of big iron, and see just how long it will take before Linux can play with the big boys.

Linux vendors know the path they need to take, but at thepresent time that path goes through a minefield. Here's the problem:Some current solutions geared toward making Linuxmore usable on very large systems exist, and they've even been offeredto the kernel development group. But they've been rejected, mainlybecause they demand tradeoffs that would hamper Linux performance onsmall systems. (A scheme that might allocate a bunch of megabytes of RAMfor a special caching technique wouldn't run too well on a boxthat only has a couple of megabytes of RAM, period.)

So, when code containing solutions to some big-iron problems was offered tothe kernel group by a well-known three-lettered hardware vendor, it wasrejected. Compounding the problem was that some in the kernel group didn't evenlike said solution for large systems, and other opinions held thatSMP-type multiple-processor systems were a poor answer to the issue ofhigh performance in the first place. Whether you can guess the three-lettered hardwarevendor or not doesn't matter, because the same problems apply to all big iron vendors.

Then there's the issue of trying to keep the core simple andstraightforward, which is a key goal of chief cat-herderLinus Torvalds. "A lot of the problems, especially with NUMA, are thatthe solutions tend to add complexity that simply isn't needed at all on'normal' machines," he said. (The NUMAclustering architecture is but one component that big-iron vendors wouldlike to bring into the Linux kernel.)

Should the Linux kernel group be dictating to hardware manufacturers howto architect their systems? Of course not. But neither do hardwarevendors have the ability to force the standard Linux kernel to maketradeoffs or implement "features" that kernel developers don't want.So how does a hardwaremanufacturer implement optimizations for their own high-speedarchitecture, or for large systems in general, that may conflict withthe best overall compromise position of the Linux kernel group?

Last week I was presenting at a conference where I met up with anexecutive of the previously mentioned three-lettered company, who felt stuck between rocksand hard places. The company wanted to get the most out of itshardware but also wanted to keep in the good graces of the open sourcecommunity. The company was concerned about what to do with its big-iron code that hadbeen rejected. Most importantly, the company wanted to avoid the use ofthat awful four-letter word beginning with "f":

Fork.

The GPL open source license used by the kernel inherently allows anycompany to make changes to the kernel, providing the changes themselvesare all open source. This is not the problem. Of far greater concern wassaid three-lettered company's fear that it would be perceived astrying to take a big-iron mutation of the Linux kernel into a differentdirection, never again to converge with the 'official' one maintained byLinus Torvalds & co. The creation of such a mutation would summon thedemons of Unix wars past, and we'd all be engulfed in eternal hot stuff.

It doesn't have to be that way. It's possible to create and maintain the large system stuff in a consistent manner, in the form of patches thatcan be applied to conventional distributions. As the distributionsprogress, the patches can be updated to work against them. That way thebig-iron system never diverges significantly from the core, it simplyprovides a parallel path that stays alongside conventional Linuxwithout drawing away from it.

We already have some examples of this, the best known being SGI's ProPack.Rather than do its own big-iron distribution, SGI chose to provide aset of packages that install on top of conventional distributions (in thiscase, either SuSE or Red Hat). This is the Right Way to do things and Ihope to see more of it. Maybe Caldera will see fit to contributeUnixWare code to the open source pool, so we can see just where thetradeoffs are and how much needs to be outside a general-purposekernel.

The process of non-standard kernel patches is just fine with Torvalds."On the whole we've actually tried to come up with compromisesthat most people can live with," he said. "It's fairly clear thatat least early on you will see kernel patches for specific uses --that's actually been going on forever, and it's just a sign of the factthat it takes a while toget to a solution that works for all the different cases." He continued:

"That's how things work in Open Source. If my tasteended up being the limiting factor for somebody, the whole point of OpenSource would be gone."

Indeed. I would also hope to see vendors agreeing on a consistent wayto develop big-system kernel modifications. After all, we already havea consortium formed tohandle many of the same issues in the opposite direction,for embedded Linux on small systems.

Unfortunately, it's been my observation that cooperative ventures betweenbig-iron vendors don't produce staggeringly productive results. (Need I gofurther than CDE itself?) That won't stop me from being optimistic here.Maybe the spirit of the Linux community will actually have some of these guysinspired to move -- albeit slowly and cautiously -- in the same direction: Forward.

Do you think Linux vendors are ready to make the leap into big iron? Let me know in the TalkBack below.