Free software great and small

[The opinions expressed here are mine alone, and not those of Google, Inc. my current employer.]

Commentary--The OOXML document format war is over, and the good guys lost. The world will be a worse place because of it, for a long time to come. After being a lobbyist for many months, it was a great relief to get back to being a Samba coder. At least that's something I feel I have some competence in. The jury is still out on my lobbying career.

I recently got to attend a couple of conferences, both of them the really technical kind where I end up learning things. The first one was the Linux Foundation Collaboration Summit, in Austin, Texas. I got to spend time learning where Linux is going from some of the people creating it, and to meet a lot of old friends. Jim Zemlin, the head of the Foundation, has a vision of where Linux should go in the next ten years, and it is breathtaking in scope.

In brief, it's a kinder, gentler version of Linus Torvald's quote that Linux is headed for "world domination, fast". But make no mistake, it's just as compelling. Jim and the Foundation are working towards a world with Linux running on the smallest embedded devices to the largest supercomputer clusters, and all possible devices in between.

His vision is of Linux as a common platform for all the intelligent devices we already have, and are yet to invent. With a standard environment and common API's across all these systems, this mass of devices will become an irresistible target for new software from application developers. Bound together with the glue of the open Internet for communications, we should see some amazing new programs running on Linux that I can't yet imagine. If I could I'd probably be writing them myself, rather than just talking about them here.

I don't know if they'll succeed in this vision but it's certainly an impressive dream. Even with Linux becoming such a common platform I expect it to avoid the security dangers of the mono-culture we currently have with Microsoft Windows. The diversity of Linux distributions and different variants of the basic Linux architecture, not least the range of different processors that Linux runs on, should provide protection against a single attack threatening all installations.

People at the conference complained about the lack of desktop Linux success of course, but then again they always do. I replied with the same statement I always do on that subject, which is "wait". With Free Software it doesn't matter if it takes five years, ten years or even longer. Eventually the Linux desktop will become the mass market. It's just a matter of time and patience.

The second conference was SambaXP (the Samba eXPerience) in Goettingen, Germany. Closer to home in the programming sense, if not physically. This is the only worldwide conference dedicated entirely to Samba and as such is a must on my travel schedule. I get to drink good German beer with most of the Samba Team members, and meet people doing incredible things with the Samba code.

We had a very pleasant surprise this year, in the form of a large contingent of engineers from Microsoft who attended the conference. We used to have a great relationship with Microsoft engineering in the past, but that became somewhat clouded after the release of Windows 2000 and the instigation of the European Union antitrust lawsuit. I still recall the look of horror on the face of a Microsoft vice president at being heckled by angry engineers at a CIFS technical conference. He'd been talked by Microsoft marketing into giving a generic Windows XP marketing presentation instead of a technical one. He later admitted he felt like an endangered spotted owl in a bar frequented by loggers (which, in the US Pacific Northwest where Microsoft is based, is not a good thing to be).

This conference went much more smoothly than that, thank goodness. Microsoft gave some excellent technical presentations on the testing framework they are using to QA the protocol documentation they are releasing, and on SMB2, their new version of the Windows file sharing protocol. They presented some of the work that had gone into Windows Vista in order to make the protocols perform better on a high-latency wide area network, which was fascinating stuff -- at least to Samba programmers. (Maybe you had to be there.)

Other interesting meetings took place between the Samba Team and the OEMs who ship Samba in many different products. There are too many to list here, but the news that really caught my eye were the numbers from the IBM clustered Samba implementation. Ever since working at SGI, where we were able to announce Samba as the fastest SMB/CIFS server on the market (with numbers that look slow these days) I've been a speed freak. We've always prided ourselves on being a fast implementation, but the clustered Samba numbers were amazing. See the details here.

Their implementation was able to sustain SMB file reading throughput of 1.7 gigabytes per second to a server appearing as single IP address exporting a single share. This is with full Windows semantics; share modes, file locking, the works. The actual hardware was four clustered servers accessing a SAN (Storage Area Network) back end. What cheered me up most of all was the limitation in speed was the SAN network, not the Samba code running on the servers, so there's room for speed improvement with better hardware.

Before you worry I've turned into Jerry Pournelle (whose "Byte" magazine columns were notorious for being an advert for companies who gave him free hardware), the ctdb code being used for this is completely Free Software, even though the underlying clustered file system (IBM's GPFS) is not. There are other clustered file server vendors looking at adopting this code also, so expect to see people competing by trying to create improved versions of this product quite soon. If you want to experiment with completely Free Software, it also runs quite nicely on top of Red Hat's clustered Global File System (GFS) for the Linux kernel.

This is what I find to be the really motivating aspect of Free Software. People collaborating to make things together that they wouldn't be able to build in isolation. From the small embedded devices, to the super-scaling file server products I'm reminded of the "Bob the Builder" videos my two year son old loves to watch, and Bob's wonderful motto:

"Can we fix it?" "Yes, we can!"