Linux: The forking fight-back
The nightmare scenario of any company thinking of investing or already committed to Linux goes something like this: as more and more vendors get behind the OS, commercial pressures will lead to fragmentation, and users end up stuck with an isolated Linux distribution.
It’s a vision that Microsoft is understandably keen to promote. In a January interview with BusinessWeek, Kevin Johnson, Microsoft's group vice-president for worldwide sales, marketing and services, gave a familiar assessment of Linux: "It has the potential to fragment like Unix did," he said.
But while the Linux community would dispute that fragmentation could ever happen, it isn't taking any chances. The Linux Standard Base (LSB), initiated in 1998, is designed not only to prevent fragmentation but to allow application makers to release a single software version certified to run on any LSB-compliant Linux distribution. For enterprises, the success of the LSB should mean the ability to switch to a different distribution and take all their LSB-compliant applications with them -- no modifications necessary.
The LSB passed a landmark in September 2004, with the release of the LSB 2.0, the first version to win the enthusiastic support of all the major Linux distributions. In November, a group of vendors called the Linux Core Consortium (LCC) announced they would jointly engineer a binary implementation of the LSB 2.0, which is to be used as the core of all the distributions of the LCC, and any other distributors who wish to take part.
But the real test will come this year, acknowledges the Free Standards Group (FSG), the organisation behind the LSB. Now that the operating system vendors are on board, ISVs must start to widely support the standard, if it's to have any real-world relevance. And months after the release of the LSB 2.0, not many big ISVs are in evidence. Jim Zemlin, executive director of the FSG, insists this year will be a turning point on that front -- he expects the body to double or quadruple membership this year, as large numbers of application vendors announce compliance plans.
Pressure from enterprises could also be essential. Some large companies, such as Credit Suisse First Boston (CSFB), have already announced they will only deploy LSB-compliant software, and those kinds of announcements could be the spur that ISVs need. "Organizations are usually well served when they adopt standards-based platforms. As the Linux suppliers offer LSB-compliant products, it will make interoperability much easier," says Dan Kusnetzky, vice president of systems software research at IDC.
Danger of fragmentation
So, what exactly is the LSB, and what does it mean for businesses? The project was initiated in 1998 under the chairmanship of Bruce Perens, with the aim of preventing Unix's fragmentation from spilling over into Linux. In 2000, the LSB and LI18NUX projects joined to form the FSG, and efforts over the next four years culminated in the release of the LSB 2.0. One of the primary motivating factors in all this has been the desires of ISVs to have a single unified scheme for porting, testing and certifying applications on Linux. With Unix, ISVs were forced to create a new port for every version, an expensive proposition; it is no surprise that Unix market share is steadily giving way to Windows and Linux.
"There was a big problem with the level of forking in the Unix community," says Red Hat analyst relations manager Nick Carr. "Applications wouldn't move around under any circumstances. It's not that it would take three months to port, it was that it wasn't going to happen."
The LSB builds on earlier efforts that attempted to prevent Unix fragmentation, such as the Portable Operating System Interface (POSIX) and the Single Unix Specification (SUS); it uses some of POSIX's source code standards and SUS' interface definitions. POSIX only defines programming interfaces and doesn't guarantee binary compatibility, while standards such as OSF/1, which aim for binary compatibility, were found to be too restrictive. The LSB aims to strike a balance between the two approaches - it includes a binary compatibility layer, but isn't as restrictive as OSF/1.
The aim of the LSB isn't to create a single, monolithic Linux operating system, but to
allow different LSB-compliant distributions to differentiate from the competition, while still running any LSB-compliant application.
LSB 2.0 was formed largely by ISV feedback, says Zemlin. Vendors wanted the specification to cover more of the libraries used in building applications, such components as additional runtime environments, cryptography, XML, PERL, PHP and Java. Version 2.0 added several significant features, such as support for SUS and an application binary interface for C++.
The standardisation project is necessary if application vendors and end users are to feel comfortable choosing open-source over a proprietary option, says Zemlin. "Without the standard, writing applications to multiple flavours of Linux becomes costly. By supporting the LSB, ISVs will achieve interoperability with a full range of Linux distributions, thereby reducing their costs for development and testing of their applications," he says. "Most significantly, the LSB brings assurance to Linux end users that they will have a broad choice of independent software for Linux."
Open source advocates often talk about the way that source-code visibility ensures better security and freedom from lock-in, but Zemlin points out that the development model is only part of the story. "You also need 'open standards' that offer the strongest way to ensure multiple, portable, interoperable implementations with fair access to all," he says. "The hidden secret in the world of open source is that without open standards all the promises of freedom are lost."
For this to work it needs the support of distribution vendors as well as ISVs. All the major Linux distributions are now LSB-compliant - about two-thirds of the LSB Certification Register is made up of various Red Hat and SuSE Linux versions. Not so many ISVs are in evidence. That's only to be expected, argues Zemlin. "Our standard - like all standards of this kind - has a common adoption cycle: once distribution vendors pledge their support and then begin certifying to the standard, ISVs begin to pledge their support," he says. "The response from the ISV community is overwhelmingly positive. This will save their organisations millions in development and testing costs."
ISVs of all sizes have something to gain from standardisation. "For large ISVs, even if they do not certify today, they will have a long term [plan] that will insure their Linux investments will be safe," he says. "For small ISVs the LSB presents a new opportunity for them to target Linux in a cost effective way."
A recent FSG survey of about 30 ISVs showed strong support for standardisation -- respondents said their biggest concern was to head off Linux fragmentation. While the survey is confidential, Zemlin says most major Linux application vendors are on the list.
The challenge is to turn that ISV interest into action. "There is lots of momentum around the LSB, with many key players in the industry supporting it, but still not enough active involvement from ISVs certifying their applications," says Dirk Hohndel, director of Linux and open source strategy at Intel, who is a member of the FSG's board of directors.
Partly, this is because ISVs have baseless fears about the cost and complexity around an LSB certification, Hohndel says. But the processes also need to be improved. "[Intel] is working with the key Linux OS vendors to include the LSB build environment and the verification tools in the software development kits, and to offer LSB certification as part of their own ISV certification efforts, to make this step easier for ISVs." Hohndel says. Intel is also providing marketing support, guidance and technical talent to help with implementation work. "After a couple of conversations, most of the ISVs see the significant value that LSB will provide to them in the long run," he says.
For application makers it comes down to a financial decision, according to Red Hat's Carr. "For many ISVs, the issue is of choosing a de facto standard versus a de jure standard. If the LSB is maturing from one release to the next, and Red Hat Enterprise Linux 3 is already established as a de facto standard, they might just choose to port their applications to something that is a known standard," he says. "For ISVs, it's a financial investment to support Linux, and they want the biggest possible return on their investment."
ISVs may be wary of fragmentation, but so far, LSB compliance hasn't come at the top of their list of priorities, says Carr. "They generally rank other issues, such as the development tool-chain, product support, features, and the overall ecosystem -- the presence of other ISVs and OEMs - first." This isn't as big a problem as it seems, he says, because Red Hat's close adherence to the specification guarantees a certain degree of LSB application support. "From a Red Hat viewpoint, we adhere to LSB standards and work closely with the LSB to drive the standards forward, keeping up with new developments in the open source community," he says. Still, Red Hat certification isn't the same as LSB compliance.
An initiative from several other Linux vendors is looking to eliminate the difference. The Linux Core Consortium (LCC), officially launched in November by a handful of major vendors, is collectively building a reference implementation of the LSB to be used as the core of all LCC members' operating systems. If LCC founder members Conectiva, Mandrakesoft, Progeny and Turbolinux have their way, an increasing number of Linux distributions will be based on a standard, LSB-compliant core. Linux makers such as Red Hat, SuSE and Debian have been invited to join, but are waiting to see the reference implementation before they commit.
The idea is that if an ISV certifies on an LCC-based Linux, it is practically identical to LSB certification. "All the applications that are certified against this reference implementation will be able to switch from one [OS] provider to another without doing anything," says Mandrakesoft chief executive Francois Bancilhon. "This is a strong enabler to the market."
If most Linux vendors are already agreeing to make their core technology meet LSB standards, the LCC argues, why not go a step further and use the same LSB implementation? "The need to differentiate is absolutely essential, but everybody agrees that this differentiation is in the top layer. There is a strong move toward convergence in the lower layers," says Bancilhon.
The group is learning from the mistakes of UnitedLinux, which commoditised both lower and upper layer software -- all UnitedLinux members' distributions were based on software licensed from SuSE, says Bancilhon. "One vendor controlled it, that was a mistake. Another was that they put the entire distribution in there. It should address only the commodity part of the technology," he says.
The LCC plans to deliver its first reference implementation in the coming months, and LCC-based distributions are due to arrive this year.
The FSG isn't standing still, either. Developments planned for the LSB this year include announcement of ISV support, more Linux vendor certifications, the first LSB certification lab in China, run by the Chinese government, obtaining ISO certification and perhaps most importantly, the release of the LSB 3.0.
The current LSB specification is a big improvement, but to stay relevant, the project must continue to evolve. "The LSB today covers only a part of what is needed for true compatibility and interoperability between different versions of Linux. It's an important step in the right directions but needs to be extended," says Intel's Hohndel. Key additions are more details around kernel extensions and APIs and ABIs higher up in the stack, for example around desktop environments or some managed runtimes.