Longhorn: The nuts and bolts

I had this crazy idea that I might write something about the Microsoft Professional Developer's Conference (PDC) while the PDC was actually going on. However, between nine-hour jetlag, the all-day training sessions, the Irish contingent who ensured that I never got to bed before 2 a.m. each night, the mysterious blue liquid that turned a bunch of jetlagged geeks into disco-dancing, cigar-chomping, shop-talking (heck, we're still geeks) party animals, and other events I'll leave to your imagination, I'm lucky I'm not dead.

The PDC is a bit like sticking your brain in a taffy pull, both in terms of knowledge absorbed and the unnatural things you do to it. I'll leave meeting deadlines to people who actually get paid to write this stuff.

A PDC is where Microsoft tends to announce the technology that will shape development within the company for the near to medium-term future. In 2000, the PDC was all about .Net, at the time a product about as far from release as Longhorn, the next major version of Windows and the subject of this year's PDC, is from its shipping date.

Longhorn beta one is slated to appear in summer of 2004, which means a release version won't see the light of day until at least early to mid-2005. Releasing the code early, however, is particularly important for a product as ambitious as Longhorn. Not only is it a massive upgrade on core Windows technologies, such as user interface rendering and access to data on a file system, but it is the culmination of the .Net integration I've discussed in past articles.

In short, Longhorn is the .Net version of Windows which finalises the replacement of the WIN32 API with a managed system, as well as a massive rethink as to the way developers write applications for Windows and consumers interact with their computers.

.Net, .Net and more .Net
A year ago, I wrote a three part series (part one, part two and part three) where I argued that one of the reasons .Net would conquer the world was that Microsoft would convert its entire product library into .Net applications. Well, based on the PDC roadmap, it turns out I was right, so ha, ha, ha, ha. (Pay no attention to that, it's my jetlagged alter-ego talking.)

Longhorn will be the first operating system where ALL functionality is designed to be accessed through managed code. WIN32's reign as the Windows API has ended, replaced by managed .Net APIs.

Such a move is unique in the industry. No one, including Sun Microsystems, has made Java THE API for programming a particular OS, much less declared that, going forward, all new features would be offered through Java. Microsoft's decision to do that with .Net is an important advance, as it FORCES developers to write more secure code, simply because they can't make some of the coding errors (such as buffer overruns) that form the lion's share of security flaws.

WIN32 will still exist for backwards compatibility, of course, and native access APIs will exist for those applications which need it (though I suspect that many of them will call the managed APIs through COM Interop). However, Microsoft intends to ensure that all Longhorn functionality is accessible from a 100 percent managed program.

A side benefit of the transition to .Net is that it has given Microsoft an opportunity to revisit all aspects of the Windows API. The new "refactored" Windows API is more internally consistent than WIN32, an API that still bore the legacy of design decisions 10 to 15 years out of date, and felt strongly like an API built by many different groups within Microsoft (which it was). The new, cleaner Longhorn API simplifies development considerably, and makes the platform much more approachable to new and existing Windows developers.

Returning to the point I made at the start of this section, where Longhorn leads, the rest of Microsoft's application library will follow. If the Longhorn API is .Net, then it doesn't take a rocket scientist to figure out that a sizable percentage of Microsoft's applications released in Longhorn timeframes will largely be built with .Net. That's a large amount of code transitioned to a particular managed runtime environment, and will pull even more developers down the .Net path (that is, those not already pulled by the higher rates offered to .Net programmers).

The New Graphical Interface: Avalon
I've noticed before that it is much easier to create reasonably complex user interfaces in HTML than in WIN32. For instance, it's far easier to write a "skinnable" Web site than it is to write a "skinnable" WIN32 application. Granted, you could do practically anything you wanted in WIN32, but if you wanted to escape the look and feel imposed by WIN32 controls, you had to perform a bunch of programming gymnastics.

Avalon is a complete upgrade to the process of writing Windows applications. In a way, Avalon turns Windows development into a more advanced and feature-rich version of Web-style development. This constitutes more than just a conceptual similarity. One of the means by which Longhorn applications can be consumed is by accessing them from a Web server using a browser (obviously, Internet Explorer), causing them to run within a "sandbox" managed by the .Net runtime. These downloadable applications act like more functional Web pages, as they have access to the full set of rich user interface controls offered as part of the Longhorn operating system. Longhorn also makes it easy to integrate common Web-paradigm concepts into desktop applications, such as page forward / page back logic, and "page history" functionality.

Windows applications still follow the same model seen in .Net WinForms and even Java's JFC, wherein controls are constructed within a class and laid out on screen either manually or through some programmatic layout functionality. However, the goal of Avalon is to make it possible for many user interfaces, even complex ones with a custom look and feel for individual controls or dynamic background content, to be defined in eXtensible Application Markup Language (XAML). XAML (pronounced "zamel") is an XML grammar oriented around painting user interfaces and laying out controls that maps directly to the Longhorn application object model. XAML isn't just an updated version of a Windows resource file, however. XAML enables the creation of complex user interfaces. You can define the drawing of polygons on the screen, and alter the rendering behaviour of those polygons by applying transformations to them. You can specify video that will be used as background to a particular control, and alter the display of that background video, again, using transformations. You can specify layout rules for controls within a particular container. In short, XAML is a complete XML grammar for user interface rendering that goes far beyond anything seen in old-fashioned WIN32 resource files.

The XAML approach to user interfaces creates a more complete separation of look and feel from code that makes the user interface functional (business logic, etc.) than was possible with WIN32. User interfaces defined in XAML can be generated using design tools, such as a future version of Adobe AfterEffects demoed at the opening keynote. The output of these tools can be passed to programmers responsible for hooking into events generated by the Avalon object model in order to make the user interface dynamic.

The traditional model seen in WinForms and JFC, though necessary, places too much responsibility for layout in the hands of the programmer. With a cleaner separation between layout and code, a clearer division of labour arises which leverages what designers and developers individually do best.

Avalon works best with fast graphics accelerators, as it takes hardware acceleration deep into the heart of the Windows rendering system (better leveraging of DirectX/Direct3D). With GDI, only text rendering benefited from hardware acceleration, according to one of the session presenters. With Avalon, the whole windowing system will rely heavily on hardware acceleration to enable rapid construction of dynamically generated and complex user interfaces. Certain amounts of performance tuning between now and the final release will certainly ameliorate things for those with older graphics cards (in such cases, the CPU will have to work harder), but people with high-end video cards will have the best experience of Longhorn.

The Object File System: WinFS
The notion of files and folders as a manner in which data is logically organised on a disk is an old concept in computing. Data blobs constitute what are understood as "files", and these files are categorised into hierarchical folders. This model works well when you don't have too much data on your system. As hard drives and the amount of data we store on them grow inexorably larger, however, the model shows signs of strain. I am something of a data packrat, and quite literally, there are files squirrelled away somewhere on my hard drive that I have not seen in years simply because I have forgotten they exist.

A better approach comes from an analysis of how users tend to access their file system. Usually, users hunt for specific categories of data. For instance, you look for all the audio media on your system, irrespective of folder or specific audio type, because you want to play them. You might look for all documents, regardless of type (Word, WordPerfect, etc.), that are "word processing" files. The folder model can be a hindrance in this case, unless you make the effort to properly categorise folders to ensure that all your media files and documents are in one place.

Furthermore, what if you wanted to look for all files associated with a particular court case regardless of folder location OR media type, or wanted to locate all the files (jpg images, text files, movie files, audio files) associated with a particular music group? Doing this is much harder in the file/folder model, and the "contains text" search that currently exists for folder searches is only an inexact approximation of what we are trying to do.

Enter the notion of schema-based data objects. A programmer defines a schema for the data he or she is going to store which details what kind of properties and operations it will support. He or she then registers this schema with the system, which enables the creation of objects of that type in WinFS. Schemas for audio files, word processing documents, and others will ship as a standard part of the Longhorn WinFS system, which programmers can reuse in their own applications.

Any kind of value can be associated with a schema object, including objects which adhere to another schema, leading to object relationships. For instance, a custom object type of "PDCAttendee" can be created and registered with the system, enabling a list of PDCAttendee objects to be created on disk. This list can be retrieved and displayed, and those individual PDCAttendees can be associated with other schema-based objects on the system, enabling searches that retrieve objects based on their association with a particular PDC Attendee.

This search capability is important given the replacement of the notion of a "folder" into which data is categorised with the WinFS concept of "views". Views are displays of data retrieved from the WinFS file system based on the result of a particular search. Data relationships can be used in searches, which are composed using a new, object-based, SQL-style search language for accessing these objects (users likely won't see this language, but developers certainly will). These views are updated as the data underlying the elements gets updated. Standards sets of views exist on the system, but the ability exists for custom views based on custom searches to be created by end users.

One of the advantages of the WinFS approach to data management is that data formats are completely transparent. A schema defines the characteristics of the data, making it available to anything on the system that wants to talk to it. If other email client providers such as Eudora decide to adhere to the "Mail" standard schema, that data will be accessible from Outlook Express, and vice versa.

In short, WinFS replaces the notion of opaque data blobs that contain an unknown "something" with a richer model wherein data conforms to a particular schema type, enabling easier location and more transparency in data formats.

The Longhorn Communications Stack: Indigo
The PDC broke the essential advances of the Longhorn OS into UI (Avalon), Data (WinFS), and Communications (Indigo) in reflection of the importance that each represents to Longhorn. Indigo's centrality of place shows the importance Microsoft ascribes to network communications.

Indigo, however, does not involve as great a leap philosophically as Avalon and WinFS. Microsoft's Indigo communications stack will include libraries for building peer-to-peer networks and constructing secure and encrypted Web services on both server AND client systems, among other things. Such technology is already common, both upon Windows and other platforms.

Indigo, however, is a refactoring of Windows communication technology as it currently exists, designed to create a system that is less fragmented, more extensible and more standardised from an API standpoint. This is in keeping with the greater flexibility and consistency already present in technologies such as .Net Remoting or in ASP.Net's Web services architecture.

Indigo also represents a move away from the notion of remotable objects. In a presentation of the design philosophy followed by the Indigo development team (PDC attendees might remember it as the one where Mr. Box wrapped his legs around a hapless volunteer from the audience), Don Box declared that Web services are a better manner in which to exchange information between machines than remotable objects. Forcing programmers to think of inter-machine communications as truly a call to a SEPARATE machine (and not just to a local object in your namespace that looks, for all intents and purposes, like a local object) prevents bad inter-machine communication design decisions.

For that reason, Indigo will operate along the Web service model seen in ASP.Net .asmx files (which are ASP.Net's Web service definition files), and not the object remoting model seen in DCOM, CORBA or even .Net Remoting.

Why is Longhorn code being released so early?
There are a number of features I don't have time to deal with here, such as Microsoft's new XML-based build tool (msbuild) that will be common across all their SDKs and integrated into Longhorn, the new NGSCB / DRM functionality that will make Longhorn a more secure platform and open up new revenue-generation possibilities for small ISVs, the new controls that will ship with Longhorn, and a host of details relating to Avalon, WinFS and Indigo.

Likewise, there are a number of questions I have yet to answer, such as how they are going to handle interoperability between other systems (Pocket PC devices, or older versions of Windows) and WinFS, or how WinFS objects can be copied to another machine (remember, those schema objects have linkages to other pieces of data on the source system).

This explains, however, why Microsoft chose to release Longhorn so early. The large feature set and the new approaches to application development offered by Longhorn will take some time to digest. Every PDC attendee walked away with a working copy of Longhorn, even if pre-alpha, allowing them to start writing experimental applications today (and, I might add, negating claims of the vapourware status of Longhorn). This allows information about Longhorn to flow out of Microsoft into the developers that will write products for it, as well as allow developers to influence the final features that will exist in the released product.

Longhorn at this point is not a product that is set in stone. Developers can influence the final shape of Longhorn, a fact hammered home multiple times over the course of the PDC. This, to my mind, is the number one reason to release code so early. Open-source programmers have access to the latest build of Linux. Programmers of proprietary operating systems have access to early release versions of next generation operating systems. The two are flip sides of the same coin. Which you prefer depends on your programming philosophy.

Parting Thoughts
In closing, I offer a warning to fans of other operating systems. If history is any guide, much effort will be expended trying to downplay the importance of Longhorn, if not suggest that no one will, in fact, buy the product. Besides the fact that history seems to counter such naysayers (.Net is popular, as is PocketPC, XBox, SQL Server, Windows XP and the various components of Office), it provides a rationale for a lot of people to do nothing to respond.

Consumers don't tend to be privy to the ideological warfare that shapes so much of the dialogue between programmers in the various programming domains, and are thus unlikely to be swayed by such arguments. Mark my words, Longhorn will be immensely popular once it is released, because Longhorn is revolutionary technology that makes desktop computing better.

Proper competition demands that you reject the comforting fantasy that Microsoft never does anything right, and deal with the reality of what their technology offers consumers. So, as a final point (in the "sharp stick in the backside" sense), it's time to start learning from Longhorn so as to plan an adequate response to it.

John Carroll is a software engineer now living in Geneva, Switzerland. He specialises in the design and development of distributed systems using Java and .Net. He is also the founder of Turtleneck Software.