Hardware's sinking beneath a software sea

COMMENTARY--There have been two dominant programming domains in the techworld, Unix and Windows. Both sides have theirown specialists, standard tools and programmingconventions. The two camps also operate in a worldpopulated by OEMs with very different business models.

On the one side are Windows OEMs, all of whom sharean operating system supplied by one company,Microsoft. This model offers a high level ofcompatibility across hardware, and thissoftware-inspired leveling is the thing which forcesOEMs, deprived of opportunities for hardwaredifferentiation, into fierce competition.

On the other side are Unix OEMs, who do not share asingle source operating system. Historically, Unixhardware shipped with an incompatible variant of Unix. That incompatibility, to be sure, is partly due todifferences in machine architectures (Solaris runs onSPARC chips, while HP-UX doesn't), but it also haslots to do with differences in API which makes writinga cross-UNIX software product more difficult.

That is all changing, however, with the explodingpopularity of Linux. If current trends continue,there is every reason to believe that Linux will sweepaway its Unix siblings to become the de facto centerof the Unix programming domain. Linux is not anoperating system exclusive to a particular brand ofhardware. In that sense, Linux fills the same marketniche as Windows, serving as a force for hardware commoditization as it becomes the standard to which Unix-style software is written.

The evolution of the software standard
For most of computing history, hardware defined theprogramming standard. Operating Systems did exist, tobe sure, but they offered far less functionality thanmodern operating systems, and more problematic, wereunique to a particular brand of hardware. ThoughCOBOL, as an example, was a standard of sorts, therewas no guarantee that COBOL written for an IBMmainframe would be compiled safely using the Burroughs compiler, and one could be sure that the Job Control Language (JCL) wouldn't work cross-platform.

Part of the reason for this was simple pragmatism. Processing speeds were much slower than they aretoday, meaning the advantages from a speed standpointof tuning an operating system to fit a particularhardware configuration were greater. The other part,of course, was an attempt at hardware differentiation. Since all the major computer makers were hardwarevendors, software played second fiddle to hardware andwas viewed more as a differentiator rather than aprofit center.

The "marginal" or per-unit cost of software shrinksas the product’s sales volume increase, since the costs are relatively fixed. This means that economic actors, who include buyerslooking to save money and sellers of software producthoping to boost revenues, have an interest in creatinga market that is as large as possible. In a worldwhere hardware defines the programming environment,therefore, there is a natural tendency for one vendorto become dominant, unifying the market throughubiquity.

Economics of scale, in other words, was the dynamicthat gave IBM its dominance of the IT landscape. IBM's products ended up as the base upon which aunified software economy was built. This allowed IBMto colonize the world with its mainframes andperipherals.

Hardware, however, was a messy thing upon which tostandardize. If you wanted to change to software thatran on a different brand of hardware, you had to throwout your existing hardware. As processing speedsimproved, however, it became possible to move to amore flexible foundation based on software. Softwareis virtually infinite in its variability, making itfar superior to a standard which relied on aparticular configuration of hardware.

The natural place for such a cross-brand softwarefoundation is at the operating system level. Anoperating system can expose the same external API toany application software, regardless of underlyinghardware. As more software targets the operatingsystem as opposed to hardware (or rather, hardwarewith a custom OS specific to that hardware), economicsof scale develop around generic software.

Windows as software standard
As is often the case, being in the right place at theright time makes a huge difference in terms of thesuccess of a company. That certainly applies to Gates& Co, for Microsoft was the company IBM chose toprovide an operating system for their new personalcomputer (PC).

IBM in 1980 was still the company to whom the worldturned for IT solutions. The world was changing,however, a change propelled by exponential increasesin processor speeds which made relying on a software middle-layer feasible. Seeing a change as occurs is a rare thing, however, and there is plenty of evidence that IBM didn't. How else to explain IBM's decision not to tie Microsoft to an exclusive supply contract? This left Microsoft free to license their operatingsystem to any hardware maker willing to pay, which ina world moving towards software standards, turned outto be a critical strategic error.

Microsoft, of course, reaped the whirlwind, turning an operating system they bought for $50,000 into the basis for a software market worth hundreds of billions of dollars. This unification around a software standard created fierce hardware competition underneath Windows, as OEMs raced to cut costs in a market with limited differentiation capabilities. The result is a market where PC hardware that is faster than many servers were a mere five years ago can be found for close to $500.

The quest for a standard Unix
Unix was introduced to the world in 1969 as a resultof work done at Bell Labs, a research division ofAT&T. For most of its history, Unix was a proprietary operating system--the rights to which were owned by AT&T (later sold to Novell, then to SCO, and finally ending up in the hands of Caldera). That control didn't last, however, mostly because AT&T was willing to license the Unix source to universities at almost no charge

What resulted was an explosion of innovation centeredaround Unix. Unix drove the first true "open source" revolution, as programmers around the world used access to the Unix source code to extend it into new areas of computing. The most famous outgrowth from all this was BSD (which stands for the "Berkeley Standard Distribution"), the operating system which became the core of Apple's Mac OS X. (For more Unix history, check out thisarticle by Ian F. Darwin, and Nick Moffitt's $7 history of Unix)

Unfortunately, the quest for a software standard basedon Unix hit a snag. Unix "forked," resulting in a proliferation of mini-standards similar in a number of facile features, but different enough to require workto ensure compatibility across variants. Part of theblame for this certainly goes to AT&T, whose legalbattles with clones of Unix forced many tore-implement code in order to avoid stepping on AT&T's intellectual property. Lacking a central gating authority certainly didn't help (something AT&T could have provided, had they not been so busy whacking heads), and networks weren't what they are today, making collaboration much more difficult.

Whatever the case, Unix became an operating systemlargely tied to the hardware upon which it was sold,thus perpetuating to a lesser extent the IBM businessmodel and allowing Unix-oriented hardware to avoid the commoditization which shook up the PC industry. This should be apparent to anyone who has experienced over the years the eye-popping prices of Unix servers.

Linux, the next step in Unix' evolution
Linux was born as a means to make a Unix-styleoperating system affordable in a market filled withhigh-priced offerings beyond the reach of normal(read: non-business) users. From such humblebeginnings, it has grown into an operating systemwhich offers the unification benefits that Unix wasintended to provide. Software running on Red Hat Linuxwill usually run on a comparable configuration ofMandrake (assuming the use of the same processor). Even where there are incompatibilities, however, aleader in the provision of Linux distributions hasalready arisen in the shape of Red Hat, offering thesame form of single-company guarantee of compatibilitythat Microsoft offers customers of Windows.

Today, the same forces of hardware commoditizationwhich shook up the desktop PC industry (andpractically shook IBM right out of it) now marchacross the Unix server landscape. A growing number ofhardware vendors ship with Linux, greatly lowering theprice of servers for people looking for a Unix-styleoperating system. Even better, Linux' low-costdevelopment past has made it compatible with hardwarefrom the fiercely-competitive PC business (hardware,mind you, made cheaper by Windows' unifying presence). Linux servers can run on Intel chips and have accessto the wide array of hardware options that exist forWintel systems.

This is a serious challenge to the business models ofcompanies accustomed to differentiating their hardwarewith custom variants of Unix. The path to Linux isone of low-margins and heated competition beneath astandard Linux base, as revealed by the market forWintel hardware. It's a path of immeasurable benefitto consumers, however, which is why the growth ofLinux should be welcome news to anyone who prefers theUnix way of programming.

Addendum
For all my talk of standards centered around software,one hardware company bestrides the industry as the defacto standard for the heart of most computers: Intel. As Windows and Linux battle for the hearts andminds of developers and users, Intel knows that it canmake money regardless of who wins (if either does),since both will need Intel chips (well, technicallyspeaking, Linux supports other processors, but themass market has shifted to Intel chips).

If history is any guide, however, even Intel shouldavoid feelings of complacency. Faster processors madereliance on a software standard versus a hardwarestandard more economical. The processor's place asthe last bastion of hardware-oriented standardization, therefore, remains only as long as it is not possible to use a virtual machine abstraction in all cases.

Java and .NET set the foundation for a shift fromreliance on a single processor standard. Few wouldconsider using Java or .NET for something as performance-intensive as games, but don't limit yourself to what is possible today. One day, these environments will be good enough for even the mostdemanding virtual world. Then even Intel will havecause to worry.

John Carroll is a software engineer living in Ireland. He specializes in the design and development of distributed systems using Java and .Net. He is also the founder of Turtleneck Software.