Microsoft: We were railroaded in Massachusetts on ODF

Warning. If you read any further, you will find yourself at ground 0.0 of the biggest battle the computer industry has ever seen. It is where the biggest warriors from the proprietary software world, the open standards world, and the open source world are engaged in hand-to-hand combat. At least for the moment, the open source and open standards worlds (the Rebel Alliance) appear to have joined sides against the proprietary warlords, led by Microsoft. Both on and off the field (where negotiators have failed to broker a truce), the engagement has not been pretty. But this is war. For the first time in the battle's history, the Rebel Alliance has dealt the warlords a stunning blow. 

"We were railroaded."

Those were not Microsoft's exact words, but if you were a fly on the wall during my recent series of correspondences with Microsoft's Alan Yates regarding how Microsoft's XML-based Office file formats ended up off of Massachusetts' list of approved file formats (essentially pulling the state's plug on future usage of Microsoft Office), it would be difficult to summarize his opinion in any other way.

To the untrained eye, the Massachusetts decision-- formally known as that state's Enterprise Technical Reference Model (henceforth referred to as MA ETRM)-- looks like one of those small open source victories for some European municipality looking to establish independence from big bad proprietary American technology. Such victories are important, no doubt, to that town, city, or country and even to perhaps to certain technology communities in general (e.g., open source) -- but largely peripheral to the bigger battle. 

David Berlind blogs regularly in Between the Lines and Microsoft Vistulations.

But, MA ETRM is about far more than open source. Industry historians will later view it as one of the most brilliant chess moves by a handful of industry titans with a common interest in breaking Microsoft's dominant grip. The MA ETRM proceedings are where some of Microsoft's biggest competitors (IBM, Sun, HP, Novell, and Adobe) gathered to make sure that Microsoft was checkmated with a devastating weapon that they themselves have been unable unleash on the American chessboard: Democracy.  

The stakes are huge. As far as customers go, Massachusetts is a good catch. It has over 80,000 employees on its payroll. Not only could Massachusetts future statewide procurements be affected by the state's new IT policy, so too could the IT purchases of many contractors, companies, and organizations that must interact with state's 173 agencies. The loss of the Massachusetts IT "ecosystem" is hardly enough to hurt the fortunes of a single large IT vendor, but a bigger problem looms for Microsoft. Not only are many US-based government organizations at federal, state, county and municipal levels keeping an eye on the Massachusetts situation as it unfolds, so too are the advisors to many countries. For example, Susie Lindsay, a research fellow at the Harvard Law School's Berkman Center for Internet and Society that's been working with 13 countries, including Brazil, China, Thailand, Brazil, and South Africa, on issues similar to those faced by Massachusetts, has been keeping close tabs on the situation (not only was she present at the last public meeting regarding the issue, she spoke at it).

Alan Yates is the business strategy general manager for Microsoft's Information Worker Product Management Group (part of its Office group). He is also the author of the comments that were filed with the Commonwealth of Massachusetts just ahead of that state's last semi-public "meeting" on Sept.16, 2005. (The press was not permitted to attend but a link to the audio of the event can be found here.) 

The organizers of that meeting--the Massachusetts Technology Leadership Council--may not have been aware that the meeting (along with the audio record of it) played a critical role in the overall drama. First, by participating in a public forum that it didn't have to attend (even though it did so, the Massachusetts IT department is not bound to use hearings or democratic processes), the state demonstrated a degree of transparency that confidently put the accountability of its officials on the line. Second, given that representatives from companies like IBM, Sun, and Adobe were allowed to both attend and speak at the meeting (which they did), it was clear that Microsoft's competitors were going to take every opportunity they could to weaken Microsoft's position (which they have routinely done throughout the process).

One week after that meeting, on September 23, MA ETRM (officially, version 3.5) was officially ratified. On the heels of such a stunning blow and with so much at stake, Microsoft is still looking to shape public opinion in hopes of containing and perhaps even reversing the damage.  Since I've devoted a fair amount of coverage to the MA ETRM deliberations (What ever happened to 'The customer is always right?' and Did MS send the wrong guy to Mass' ODF hearing?), Yates asked if I'd be open to hearing Microsoft's side of the story. 

That previous coverage concluded that Microsoft made some questionable decisions during the home stretch leading into the ratification of MA ETRM.  Yates disagrees and suggests that excluding Microsoft's file formats from the MA ETRM standard was not based on the technology's degree of openness (or lack thereof), but rather on a lack of due process along with a little inside baseball that may have afforded the alternative--OpenDocument Format (ODF)--an unfair advantage. In his Sept. 8, 2005 letter to the Commonwealth, Yates wrote:

The Information Technology Division (ITD) of the Executive Office for Administration & Finance of Massachusetts] did not provide sufficient time for review and comment on the proposed policy, nor a robust process for addressing comments. Due process requires much more, particularly given the unprecedented nature of the proposal and the potentially adverse consequences it could provoke. 

Falling short of accusing any particular party of tainting the process, Yates says the decision took Microsoft very much by surprise given the fact that its specification--formally referred to  as Microsoft's "Office XML Reference Schemas"-- had, to the best of Microsoft's knowledge, officially "made the cut" earlier in the year after Microsoft worked closely with the Commonwealth to tweak the language found in the company's patent license for those schemas. 

"Back in the January/February timeframe, we had a discussion with [Massachusetts] Secretary [of Administration and Finance Eric] Kriss and [Massachusetts CIO] Peter Quinn about our Office XML File Format license and we agreed to make a small change.  We had their agreement that our license met their test for openness," Yates said during a phone interview,  During the Sept. 16 meeting, Yates' surprise and frustration was echoed by Microsoft national technology officer Stuart McKee when, at 35:10 of the audio of the meeting during his second opportunity to speak, McKee said:

We do have some concerns that we're now not on the list and in fact I think you stood before this body and talked about us being on the list and in the conversations we were having. So, I guess the question is how does this policy evolve over time, what can we expect when we are on the list, off the list [and] can we get back on the list?

Although it isn't clear what Massachusetts' test for openness was back in the early 2005 timeframe, it is clear that "the test" was officially revised to the point that by the time MA ETRM was ratified on Sept. 23, Microsoft's file format no longer satisfied the Commonwealth's criteria for openness.  At 1:05:37 into the audio, Massachusetts Information Technology Division general counsel Linda Hamel could be heard summarizing the state's current thinking when she said:

There's a continuum of openness when you think about licensing and at one end of the continuum, you might put proprietary licenses.  At the far end of the continuum at the other end, you might want to put the General Public License.  There are a lot of licenses in between and when I look at the Microsoft license for the XML Reference Schema, I don't see it as an extreme on the proprietary end. I think it has a degree of openness to it. It's not as open as the license for PDF.  So, what we did when we were coming up with the standard was we kind of drew a line along the continuum of openness that we're talking about and that line in the final version happened to get drawn so that in the version of ETRM you see posted today (3.5), the Microsoft Patent License doesn't fall within that spectrum which isn't to say it doesn't have an element of openness to it.  So the question for the government has been where along the spectrum of openness we want to be. When we originally came out and included the Microsoft XML Reference Schema among the acceptably open components that we would look at, we got a firestorm of commentary from many communities saying it's just not open enough.

So, somewhere between the January/February timeframe when Microsoft felt as though it reached an agreement with Massachusetts and the Sept. 16 meeting, Microsoft's license was reassessed. It went from passing the Commonwealth's test for openness to failing it.  Exactly how and when that happened, Yates says he doesn't know. But his feeling that Microsoft might have been railroaded by influential forces behind closed doors was palpable during our call.  For example, Yates said that leading into that final meeting, many of the Microsoft-favorable comments that were filed with the Commonwealth were kept under wraps. In contesting my suggestion in a recent blog that MA ETRM's ratification was a fight to the finish, Yates thinks the meeting and the comment period that led up to it were moot. Said Yates:

We didn't feel as though there was any big conflict between us and the State of Massachusetts for the longest time.  But then all of the sudden, given this change in the policy, we were given a short time to comment on the change in the policy.  And then [Secretary Kriss] made the policy final.

Although it is unclear precisely when state officials made their final decision, Massachusetts gave the public two weeks to comment on a new draft of MA ETRM (that was published on Monday August 29).  In the Sept.16 meeting audio, Kriss can be heard saying that the deadline for comments was "last Friday" (in other words, the Friday before the meeting) which was Sept.9.  Pointing to a voluminous document of public comments (both for and against) that he says was finally published online after the hearing, Yates continued:

We were shocked that all of the comments had not been released.  Only five of the comments were released.   Four were against us and one was for us (ours).  The selective inclusion of comments was evidence that the decision was a foregone conclusion.

Microsoft wasn't alone in this feeling.  Heading into that meeting,  the press (including me at the time) discussed the Commonwealth's decision to go with ODF as though it were a done deal. But the truth is that it wasn't and maybe still isn't. Not only did I posit in the aforelinked blog that Microsoft may have sent the wrong man to the Sept. 16 meeting, it's clear from Secretary Kriss' own words (at 1:09:55) that the door is actually still open:

I think the state would be reviewing this format differently in this conversation if you did basically three things. If you dropped the patent entirely with regard to it; if you really did publish the standard.  By the way, my definition of a peer group is not users. It's people who are familiar with the technical aspects of the format itself. So, my mother who uses Microsoft Word I don't consider to be a member of the peer review community as to the file format. So, I just wanted to be clear about what I mean by peer. I don't mean that it's the user base. If you were to publish the standard and then make provisions for future changes to that standard to be part of a joint stewardship that is no longer solely controlled by Microsoft Corporation...if you were to do that, we would be delighted to begin a true technical comparison of your standard with the OpenDocument standard and go from there. But I think the issue is that those impediments that have risen in importance as we have gone through this process....Although you have made definite steps in the modification of your license, it's all a moving target. If you were to think about those aspects of it, I think we would be very interested in looking at the Microsoft definition. 

Had someone with the necessary authority been there to make the proverbial executive decision, that technical comparison might well be underway at this moment. As far as can be told from the  public record, the decision was not fait accompli and was in fact still open for discussion on Sept.16, seven days before MA ETRM (version 3.5) was ratified. 

Yates argues that even if the state were still open to discussing the matter, having a different executive at the meeting other than Microsoft national technology officer Stuart McKee would not have made a difference to the outcome. Citing at least two reasons, Yates claims the process was deliberately biased to assure that ODF made the cut and Microsoft's Office XML formats did not. First, pointing to the fact that Massachusetts had concerns about the license for ODF, his only explanation for the way that ODF was added to the Aug.29 MA ETRM draft at the same time that the Office XML formats were dropped is that behind closed doors Massachusetts officials must have received assurances from potential ODF licensor Sun that those concerns would be addressed. (Sun still maintains it has no patents that apply to ODF, so there's really nothing to license.) 

Meanwhile, Yates claims that Microsoft's only opportunity to officially respond to the fact that it had been dropped was during the two week public comment period that followed the draft's publication. Second, Yates and Microsoft are so convinced that the Office XML Reference schema is open enough that Massachusetts must be biased to see it any other way.

Both claims are complicated, so I've decided to tackle them in separate sections.

Behind closed doors
As it turns out, Yates is right about the chronology of events. In other words, for Massachusetts to have so confidently added ODF to the Aug. 29 draft, those assurances had to have been received ahead of the draft's publication.  Meanwhile, Yates claims that Microsoft wasn't afforded a similar pre-draft opportunity to discuss Massachusetts' concerns about Microsoft's formats.  Do the two taken together make solid grist for the conspiracy mill?  Here's what I have so far.

As pointed out earlier, Massachusetts Information Technology Division general counsel Linda Hamel noted that the state received a "firestorm of commentary" in response to the early 2005 MA ETRM draft that listed Microsoft's file formats as the supported "open" standard. According to various sources, the response was so opposed to the inclusion of a Microsoft format in what was otherwise an open initiative that it caused Massachusetts officials to take a deep breath and reconsider its test for openness.

The state was so overwhelmed with public comments that it decided it had to "take it to industry," according to Sun standards manager Doug Johnson, who has represented Sun's interests throughout the process. That decision led to a Massachusetts organized meeting on June 9, 2005 that was attended by Sun, IBM, Adobe, and many others. Representing Microsoft at the meeting were McKee, state government affairs manager Brian Burke and account technology specialist Leslie Tan. The meeting came on the heels of ODF 1.0's ratification by the Organization for the Advancement of Structured Information Standards (OASIS) and, according to Johnson, the tone of that meeting was that XML was the way to go and that virtually everyone in the room but Microsoft was behind ODF.

According to Dan Bricklin--Massachusetts resident, co-inventor of the electronic spreadsheet and moderator of the Sept.16 meeting--the state was clearly in pioneering territory (as governments go), flush with feedback similar to what developers get during the software usability testing process. "One way I now see what's going on looking in from the outside, the Commonwealth had a general need in terms of wanting there to be a wide range of producers and products," Bricklin told me, referring to solutions that could read and write whatever format Massachusetts settled on in order for the Massachusetts public to have unfettered access to the state's public documents.  "[Massachusetts] could have looked at it from their own viewpoint of what they thought was needed legally. But it didn't matter what they thought. What mattered was how developers saw that. If too many developers or their lawyers had any discomfort with the license, then that meant there was enough of a question as to the specification's openness.  It meant there was a risk to the availability of solutions."

Although Sun's Johnson doesn't suspect Microsoft of turning the state on to any potential problems with ODF's license, Johnson admitted in a phone interview that Massachusetts officials asked Sun for some clarifications regarding ODF's licensing requirements and that Sun was given an opportunity to address those concerns. "In late June [2005], Massachusetts started asking for the licensing language to ODF" Johnson said,

What no one knew is that Sun had long been working on a way to issue what amounts to "licenseless license -- a type of patent non-assertion covenant." Just the culture shock alone of such an idea to any technology company that depends on intellectual property (as Sun president and COO Jonathan Schwartz says his company does) would probably take the company's executives and lawyers a year or more to digest. 

What made Sun's idea of a non-assertion covenant unique is that it's the sort of covenant that mentions no patents in particular, but that gives developers carte blanche access to any of the issuer's patents they need to implement a particular specification without having to agree to or sign a license. Thus, it's sort of a license-less license. 

The idea is so progressive that some believe it could be more than just a next step towards non-proliferation of open source licenses (in other words, if no licenses are required, there's nothing to proliferate), but also the next step towards a different attitude when it comes to the software patent issues that currently plague the industry. 

Although Sun still had its license-less license under development, the request for clarification from Massachusetts represented the first opportunity for Sun to test its viability. So, Sun decided to apply it to ODF.  "Our guys got together with the Massachusetts guys and explained what the language will be and sometime around July or August, we gave the non-assertion covenant language to them," Johnson said.

Around the same time, Yates says that Microsoft and Massachusetts did have a meeting, but that Microsoft did not get a similar request for license clarification.  "At the end of July, two of our Government Affairs people -- Stuart McKee and Brian Burke -- were asked to meet privately with Secretary Kriss, Peter Quinn and Linda Hamel.  At that time, Secretary Kriss told them that he was moving toward a policy of naming OpenDocument as some kind of state standard. He said that there were multiple ways for Microsoft to work with it: a) natively; b) Microsoft-sponsored 3rd party support or;  c) natural 3rd party support.  He did not ask us for any changes to our license, etc. nor did he press for an 'answer' about our approach, or give a timeline.  We were not sure what precisely was going to happen," Yates wrote in an e-mail.

Yates also wrote, "There was also a Linux conference in San Francisco late in August I believe, where Peter Quinn was a panel member. At that time I believe that there was some mention that MA was going to do something important in favor of OpenOffice. We contacted Peter to try to learn what his comment meant, but did not find out exactly until the policy was posted on the website."

The policy Yates was referring to is the Aug.29 MA ETRM draft, published sans Sun's non-assertion covenants. Within two days, on Aug.31, Microsoft's position on the ambiguities in the ODF license du jour and a subsequent warning that ODF might not be as open as some were making it out to be appeared in a blog entry authored by Brian Jones, one of the program managers working on Microsoft Office.  Jones wrote in that post:

While we're on this topic, I think it's important that you all take a look at the comparable situation with Open Document. A lot of folks just seem to assume that since it's a standard, there are no IP issues and everything is very straightforward. Well, take a look at this: http://www.oasis-open.org/committees/office/ipr.php. Sun seems to be saying that it may have IP in the Open Document spec. While Sun says it is willing to provide a royalty-free license, one would still need to ask Sun for a license. The license is not posted.

From Microsoft's perspective, something was amiss. Yates feels that Microsoft, which up until that point had been forced to publicly post a new license including certain concessions that Massachusetts had asked for, was being subjected to a double standard.  According to Yates, "At that point, we weren't clear what the OpenDocument license was." 

Microsoft's position on Sun's disclosure however drew a torrent of criticism and has been labeled by some as an intentional misread of the legalese designed to disingenuously raise suspicions and cast dispersions on the fairness and objectivity of a state government's process.  For example, in the section entitled "Sun Steps Up to the Plate and Hits a Home Run" in one of her recent posts, Groklaw editor Pamela Jones who is not an attorney but who has historically proven herself to understand the intricacies of software and patent licensing, examined the original Sun disclosure and concluded it didn't "say what Jones said it does. Not by a mile."  In that post, Jones also points to a response from Sun's chief open source officer Simon Phipps who openly posited that Jones' post was a deliberate misread, saying "Brian's position is 100% pure FUD."

But the drama wasn't over. "Then we saw in these other blogs that the Sun guy talked with [Massachusetts CIO] Peter Quinn and they either finished or hurried up and fixed the license," Yates wrote.

As it turns out, Yates wasn't referring to a blog, but rather to an OASIS bulletin board thread involving OpenOffice.org representative Gary Edwards and the new OASIS chairman of the board -- Sun's Eduardo Gutentag. In that Sept.26 thread, Edwards is clearly looking to mount a response to Jones' assertions and Gutentag is saying not too worry because the non-assertion covenant, which will make all of Microsoft's claims moot, was coming in a couple of days.  While Edwards is seen saying that he's going to try to assure Massachusetts CIO Peter Quinn that Jones' post was off base, what Edwards clearly didn't know was that Massachusetts state officials already had a copy of the new non-assertion covenant (the "license-less" license).  Three days later, on Sept.29, Sun released the covenant under the title Sun OpenDocument Patent Statement.

Regardless of the dueling interpretations of Sun's original disclosure, Yates' feeling that the process was biased against Microsoft was further tweaked by the fact that  Sun issued the far more liberally worded (and aforementioned) patent covenant approximately one week after Massachusetts officially ratified its decision on Sept.23.

Even I have to admit the relevant events as well as their timing could make enough grist for the conspiracy grill to claim "fishy" if not "foul."  Did Massachusetts move forward on ODF with a simple assurance from Sun that its work-in-progress license-less license would be applied to the document format, whereas Microsoft was forced to amend its license to no avail and then dropped from the list without an opportunity to redress? If what Microsoft says about the content of its late July meeting with Kriss, Quinn, and Hamel is true, then the answer is yes.  But, is this chain of events, with an implication that the Commonwealth had some other agenda--perhaps an anti-Microsoft bias, a proverbial smoking gun? 

Although Yates argues that Microsoft wasn't given a similar opportunity to redress, Sun's Johnson suggested that the public nature of Massachusetts' deliberations-- including the inundation of comments in 2005H1 and the clear tenor of the June 9 meeting between state officials and "industry"-- was enough writing on the wall for Microsoft to know that it had a problem on its hands and what the sticking points might be. According to Johnson, Massachusetts initial approval of Microsoft's file formats in early 2005 was still subject to a public comment period, and it would have been premature for anyone to assume that any decisions were final.

In a telephone interview on Oct 12, 2005, Massachusetts CIO Peter Quinn made it clear to me that none of ETRM was set in stone back in early 2005. "We put ETRM out for public comment in January. We did that for the entire version 3.5," Quinn said. "We don't have to put anything out for public comment. But we always put things out anyway because we always get valuable insight." Referring to the fact that MA ETRM covers more than just file formats, Quinn said, "Because of all the feedback we got about the file formats, we ratified 3.5 without the formats."

Whereas Johnson clearly suggested that the writing was on the wall, Yates told me, "We were a bit naive."  Was it naivete? Or, was it the job of Microsoft's National Technology Officer Stuart McKee-- the man at Microsoft who is responsible for interfacing with governments like the Commonwealth of Massachusetts-- to not only do a better job of reading the situation, but to know what the stakes were in dealing with a government that answers to its public rather than a board of directors?  

There's no question that Microsoft's Office XML Reference Schema was once on Massachusetts' "list," and then, as a matter of public discourse--some might say democracy-- removed from that list. Going back to the notes from Massachusetts' June 9, 2005 meeting with industry-- referred to by Quinn as the Open Formats Summit--it would be impossible for anybody, Microsoft included, to assume that any format that was once on the "list" was guaranteed of staying on that list. In the very last section of those meeting notes, the next steps were clearly defined as:

• Identify a continuum of acceptable open document standards for the Commonwealth.
• Revise the standards and publish the revision for public comment; then finalize the standards.

These action items practically scream out that, at the time, Massachusetts was undecided on a standard and that the due diligence for choosing one was far from over.  Clearly, a new test for openness was being developed and Massachusetts had already started researching the matter. In fact, Microsoft's team on the job--Stuart McKee, Brian Burke, and Leslie Tan--didn't have to look far to see some of the criteria for openness that the State was considering. It was displayed in a footnote on the bottom of the June 9, 2005 meeting notes and stated as follows:

Specifically, [Ken] Krechmer, [Fellow, International Center for Standards Research, University of Colorado] notes, standards creators typically consider a standard to be open if the creation of the standard follows the tenets of open meeting, consensus and due process; implementers of an existing standard would call a standard open when it serves the markets they wish, it is without cost to them, does not preclude further innovation (by them), does not obsolete their prior implementations, and does not favor a competitor; and users of an implementation of the standard would call a standard open when multiple implementations of the standard from different sources are available, when the implementation functions in all locations needed, when the implementation is supported over the user's expected service life and when new implementations desired by the user are backward compatible to previously purchased implementations. 

With the writing on the wall, the question that sticks out in my mind is why didn't Microsoft sound its internal fire alarm sooner?  It appears that Microsoft left the fate of such an important initiative in the hands of a schedule and process over which it had no control. You have to ask why Microsoft didn't immediately establish an independent council to comb through all of the available information including public comments, minutes from the June 9 meeting, and other work regarding the measurement of openness in order to come up with an independently developed stringent test for openness, free from bias.  Knowing that Adobe's Portable Document Format (PDF) was also in contention, the council could have also figured out why exactly PDF was under consideration.

One reason no such fire alarm was sounded could be that Microsoft remained convinced-- as it still does today-- that no matter what anybody's test for openness could be, the license for its Office XML Reference Schema should pass with flying colors. Unfortunately, in Massachusetts' case, things couldn't be further from the truth.

Not only does Yates feel as though Microsoft's formats were denied equitable treatment as a part of the due process, he and Microsoft view the license to the Office XML Reference Schema as being sufficiently open for any developer-- even some open source developers-- to implement it in their software.  So convinced of this is Microsoft that the company and its representatives (McKee, Yates, etc.) continue to focus on the "we're open enough" argument rather than "what must we do to get on the list?" 

From Microsoft's point of view, something other than a fair assessment of Microsoft's license has played a role in the outcome. When you step back and listen to both sides, you can't help but wonder if Massachusetts is from Mars and Microsoft is from Venus.  So close, yet so far apart. Regardless of what Microsoft has to say, Massachusetts officials say that Microsoft's license isn't open enough (even though Microsoft made several concessions in the wording of the license over the last year just to appease the state). With no standard test for openness (an issue that I've blogged about before), Microsoft and Massachusetts have each arrived at their own test for it and the two are simply not the same.

In our phone interview, Yates went into detail regarding why he believes that Microsoft's patent license make the formats as open as they need to be.  One of the biggies that's on everybody's test is whether the license is royalty-free (RF) or not (Microsoft's license is). In fact, for a long time, the RF nature of a specification was practically the only test of whether it was open or should be considered as a standard.  But, as Kriss said in the meeting (see quote above), it's a moving target. Over the last few years, if the open source movement and patent quagmire have taught the world anything, it's that developers are far less free to do anything in software than originally thought to be. In that context, degrees of openness are very much about degrees of developer and end-user freedom.

On the basis that no specific patents are mentioned, Yates believes that Microsoft's license offers more such freedom than Adobe's license for the Massachusetts-approved PDF.  Indeed, in a blog I just wrote about Sun's recent patent covenant for developers looking to create OpenDocument Format-compliant implementations, one of the covenant's attributes that I cited as being unique was its lack of specificity regarding the patents needed to create such an implementation. Like Sun's covenant, Microsoft's patent license lacks the same specificity, which means that licensees/developers are free to use any Microsoft patent they need to in order to create software that complies with Microsoft's Office XML Reference Schema. When patents are specified, as they are in the case of Adobe's license for PDF, licensees must bear the additional burden of making sure that their implementations don't run afoul of an Adobe patent that's not listed on the license.  "We took a more inclusive approach by not naming any patents,"  Yates said.

On that front, Microsoft's license is more open than is Adobe's. But, whereas Adobe's license met Massachusetts' overall test for openness, Microsoft's license did not. That apparent (to Yates) application of a double-standard triggered Yates' suspicions that other forces were at work, perhaps drawing the democratic nature of the process into question.  According to Yates, "[Secretary Kriss] didn't feel that our license was open enough. It remains confusing to us that the Adobe license is, and ours isn't." 

As it turns out, Microsoft's argument omits other attributes of Adobe's license that impact the openness of PDF.  For example, Yates neglected to address Massachusetts ITD general counsel Linda Hamel's declaration during the September 16 meeting that Microsoft's license was less open than Adobe's for PDF.  Had Microsoft contacted Adobe for its perspective on openness or done what I did -- which was to contact software licensing expert Larry Rosen (who literally wrote the book on open source licensing)-- he might have learned that Adobe's patent clarification clears the way for any open source developer to make software that can read or write PDF files. Part of Massachusetts' test for openness is that any developer, including open-source developers, have no infringement concerns when it comes to developing software that complies with state standards.  Adobe's clarification states:

Adobe desires to promote the use of PDF for information interchange among diverse products and applications. Accordingly, the following patents are licensed on a royalty-free, nonexclusive basis for the term of each patent and for the sole purpose of developing software that produces, consumes, and interprets PDF files that are compliant with the Specification.

As said earlier, the clarification goes on to list the specific patents.  Commenting on that clarification, Rosen said the following via e-mail:

The Adobe Patent clarification notice looks fine to me. The patent licenses are limited in scope to compliant implementations, but that alone doesn't make them non-open, at least in my view, because the licenses don't actually prohibit non-compliant implementations (they just *don't license* them).

Had Microsoft contacted Adobe, it might have also heard Adobe's position on the issue of openness, which was expressed at 1:03:15 into the Sept.16 meeting by Michael Engelhardt, Adobe's senior director of public policy:

The license for PDF from the beginning has been a standard that we have published and made freely available to anyone who wishes to use it and there are over a couple thousand, I think, implementers of tools and you can go and, OpenOffice has a PDF builder built into it, Apple's word processor has a PDF maker built into it....You can download [the PDF specification for free] or buy the spec in book form on Amazon.com. ..You can do whatever you want with it.  The only licensing restriction if you can even call it that which says that if you build something that you call a PDF, it has to be a PDF....You can't call something a PDF that isn't in fact a PDF. It has to be compatible with all the other PDFs and all the other readers and everything else.  In other words, what you built has to work. Otherwise you have to call it something else. You can still build [something based on a derivative of PDF].  But, if it isn't a PDF that works like a PDF, it's not PDF. That's the only licensing restriction.  Also to address the question of openness and the criteria that you list, there actually are flavors of PDF that are pure open standards [and] ISO ratified.  But most importantly I think, in the context of what you want to use PDF for, PDF A for archiving was recently ratified by ISO and there's also PDF for accessibility that's being developed by AIIM under the ISO.  So we are working even more in the direction of openness than we've been in the past.  But right now, PDF is completely published, you can do whatever you want with it, everybody does

The derivative issue is an important one with respect to openness. What Engelhardt basically said is that developers are free to do whatever they want with Adobe's PDF specification. For example, they can break it apart or remix it with other specifications.   The only restriction on this activity is that if the final output of the software isn't 100 percent compliant with the PDF specification, the developer cannot say that the software or the documents it produces are "PDF."  The freedom to remix Adobe's work lies in stark contrast to Microsoft's license which says: "A 'Licensed Implementation' means only those specific portions of a software product that read and write files

that are fully compliant with the specifications for the Office Schemas."  In other words, if developers want a license, then they are bound to developing implementations that are compliant with Microsoft's specification.

In the bigger picture, Engelhardt's attendance to the Sept.16 meeting and readiness to address the issue of why Adobe's license is open is significant because of how Microsoft's key competitors were clearly lining up in support of Massachusetts' inclination to go with ODF. Engelhardt wasn't alone in doing this. IBM's Doug Heintzmann also got up to remind Massachusetts officials of how forcing the issue of compliance stifles the opportunity for a specification to be improved through innovation. Microsoft's enemies were clearly lined up at this meeting to make sure Massachusetts had no compunction about moving forward with ODF.

Clearly, we're deep in the legal forest of openness at this point, but it's exactly these lengths that Massachusetts went to in an effort to develop its own modern day test. The Microsoft license may be more open-ended because it forgoes a list of specific patents in favor of whatever of Microsoft's patents it takes to implement its Office XML Reference Schema.  However, when it comes down to brass tacks, Adobe would have been called to the mat already if the patents in its license weren't enough to cover the thousands of implementations of PDF that Engelhardt mentioned. Between that, the remix issue, and the range of developers (all) who can implement PDF, it would be difficult for me to argue in favor of Microsoft's position that its license is more open than Adobe's.

With the Adobe argument dispatched, we now turn our attention to Massachusetts' test for openness. As can be heard from one of the Secretary Kriss' earlier quotes during the Sept.16 meeting, Massachusetts explained its test for openness as follows:

• It must be published and subject to peer review
• It must be subject to joint stewardship
• It must have no or absolutely minimal legal restrictions attached to it

Sound familiar? It should. The three item list reads like a layman's summary of the primary points found in the aforementioned Ken Krechmer-related footnote at the bottom of the meeting notes for Massachusetts' June 9 Open Formats Summit.

During the Sept.16 meeting, Microsoft's McKee did not dispute the three-point test, but instead challenged Massachusetts' interpretation of it which, naturally, positioned Microsoft's license to pass. Even though the odds were heavily stacked against Microsoft going into that meeting, it was probably not a good time for a vendor to tell a customer with 80,000 employees that it doesn't like the customer's application of its own test. If there was any chance left for Microsoft on Sept.16, McKee was squandering it.

For example, on the subject of peer review, McKee cited the 400 million existing users of Office as a body of peers that Microsoft must pay very close attention to. But, to any outside observer who is familiar with this industry's consortia and standards bodies, it is clear that that was not what Massachusetts' meant by "peer review."  Although just about any vendor would lead you to believe otherwise, a vendor's customers are not its peers.

Likewise, joint stewardship is unambiguous. The implication is that the specification in question is not subject to the control of a single person or vendor. Even though Microsoft listens to its customers and often responds with features and functionality that it often claims were developed at the request of its customers, that is not the same as joint stewardship. Nor does the exposure of source code to selected organizations (such as governments)-- something Microsoft does under its shared source program-- fall within the category of "stewardship."

In my own considerations of what it means for a specification to be open (a.k.a., an open standard), I have often discussed how it must belong to an officially recognized standards body such as the World Wide Web Consortium (W3C) or the International Organization for Standards (ISO).  In using the language "joint stewardship," the three-point test from Massachusetts leaves room for specifications that may not necessarily have been ratified by such a standards body. Microsoft, as a member of several such consortia including ODF-chaperone OASIS, is no stranger to this concept. 

As can be seen from the aforementioned quote (at 1:09:55 in the audio from the meeting) where he mentioned his mother (whom, as a customer of MS Word, did not qualify as a peer in his mind), Secretary Kriss was quick to invoke the more broadly accepted definitions of peer (as in peer review) and joint stewardship.

On the issue of legal considerations, McKee (at 1:08:04 into the audio) tried to convince Massachusetts that Microsoft's patent license meets the state's test for openness when he said:

The Microsoft Office Open XML License is perpetual and royalty free.  And particularly related with the comments of the State of Massachusetts, it's a very very simple license. The publishing and distribution of it is on the Web site....but we also very specifically related to this issue included the words "public records" because we do understand there's a very specific issue related to public records.

However, at 1:08:46, Bricklin, who was moderating, was quick to point out at least one problem.  Referring to the section that Microsoft added regarding public documents, Bricklin said:

It says Read Only. It doesn't say read and write of public records on the license. It's got some legalese there.

McKee responded:

Correct. Correct. But the issue about the legal simplicity is something that we attempted, or legal issues, is something we attempted todress with perpetual and royalty free and specifically incorporating public records we attempted to make it very simple because these legal issues are very complex.

The section in the license to which Bricklin and McKee were referring says the following:

By way of clarification of the foregoing, given the unique role of government  institutions, end users will not violate this license by merely reading government documents that constitute files that comply with the Microsoft specifications for the Office Schemas, or by using (solely for the purpose of reading such files) any software that enables them to do so. The term "government documents" includes public records.

Today, well after the Sept.16 meeting and after MA ETRM was ratified on Sept.23, Microsoft is still steadfast in its efforts to convince observers that its license is as open as it needs to be.  During our interview, Yates said:

Massachusetts wants open access to their documents and their data in perpetuity.  We feel we took the steps to guarantee them that. Inclusive of the different ways to get things done. Regarding the tax-free access mentioned in your blog [editor's note: see Could PDF support in Office 12 quell and ODF uprising], we're trying to accomplish that. You can see that since were early in the curve of implementing the format--[Office 12] isn't even in beta yet]-- there could be some nervousness about third parties being able to support the format. But that's exactly what our license was designed to enable. Lots of third parties working with the format in perpetuity. 

Instead of challenging Massachusetts at the 11th hour, and now in post-mortem, perhaps Microsoft should have been asking Kriss, Quinn and Hamel for more details on the requirements that Kriss laid out as a prerequisite for entering Microsoft's Office XML Reference Schema into a technical bake-off against ODF.  Had Microsoft done that, the discussion might have started with an explanation of why usage of the word "reading" (in the aforementioned license excerpt) to the exclusion of the word "writing" was a red flag. 

If Bricklin, who was simply the moderator (and not a lawyer) picked up on that, software licensing lawyers would no doubt red-flag it as well. In a separate interview, Bricklin said, "The first question that raises is in a developer's mind is €˜Why would they go to the trouble of mentioning reading without also mentioning writing?'"

If there's a hint as to why that may be, then perhaps it appears in the line just before that proviso in Microsoft's license that says, "You are not licensed to sublicense or transfer your rights."

To the untrained eye, this statement seems rather innocuous. Who cares, right? As long as the rights to create software that can read and write Microsoft's formats are equally available to everyone on a royalty-free basis, then why should Massachusetts or any other developer be concerned?

Two words: Open source.

Although it hasn't made any sweeping decisions to adopt open source like other governments have, Massachusetts knows that it's an option, particularly when it comes to desktop productivity applications. During the Sept.16 meeting, Massachusetts CIO Peter Quinn discussed how he had been using OpenOffice.org. Going back to the Commonwealth's overarching goal to maintain its sovereignty (an important value to any United States state), the non-sublicensability and non-transferability of Microsoft's license is a deal breaker. 

Included in the notion of state sovereignty is the right of the state's agencies, employees, contractors and citizens to choose any type of software they want to read or write public documents. By not allowing its license to be transferred or sublicensed, Microsoft's patent license automatically prevents just about all open source software-- including OpenOffice.org--from supporting Microsoft's XML formats. 

Not only that, it also prevents open source developers from building the freely distributable and transferable programming tools (in all sorts of languages-- C++, Perl, Python, Java, etc.) that would turn incorporation of Microsoft format support into child's play for most developers. Such libraries are often the catalyst for rapid and viral adoption of a new specification in the open source community. In the short time since ODF was ratified by Massachusetts, the alpha version of a C#-based open source library called AODL has already shown up on SourceForge.net (one of the Net's leading open source repositories where developers gather to collaborate on projects).

According to Larry Rosen, provisions that prevent sub-licensing and transferability are antithetical to open source. "[The Microsoft license] not only prevents transfer or sublicensing of the patent rights," said Rosen, "but it also requires that open source developers put Microsoft's patent notices in our licenses."  These are terms that open source developers find to be unacceptable.  In a separate e-mail, Rosen said, "Open source depends on the right to sub-license."

When I took the issue back to Yates, he concurred that transferability is off-limits when he wrote:

By way of clarification of the foregoing, given the unique role of government institutions, end users will not violate this license by merely reading government documents that constitute files that comply with the Microsoft specifications for the Office Schemas, or by using (solely for the purpose of reading such files) any software that enables them to do so. The term "government documents" includes public records. There is a distinction between the right to create a program under the license and distribute that in source code form through all means of distribution (which is permitted by the license) and the right to grant downstream IP licenses divorced from the code - to transfer intellectual property - It is only this last right that we are restricting.  Our license says, "We want to continue to own our IP.  We are willing to give anyone in the world a license to that IP for purposes of developing software, but we don't want to put other people into the business of licensing our IP to others separate from their code. If anyone wants to modify or use the software they get from someone else that has used our IP, they can freely take the IP license from us again, to use or make changes.   Does this make sense?  It should not inhibit anyone from getting software done.

Then, pointing back to Brian Jones' blogs, Yates wrote, "Our license may not be compatible with the GPL, but it is compatible with many other open source licenses, and certainly can be used with the OpenDocument license."

Rosen disputes Yates' contention about the license compatibility. After reviewing the quote via e-mail, Rosen wrote:

Among the licenses that are explicitly sublicenseable are the MIT, MPL, CPL, Apache 2.0, OSL/AFL, and all licenses derived from them. That's most, I believe.  Microsoft's patent license is incompatible with all of them.

When confronted with Rosen's response, Yates offered a clarification of the original statement, saying, "While it is beyond my capacity to analyze [all of the open source licenses listed on the Open Source Initiative's Web site], we think that there is no problem with the two most used, key alternatives to the GPL; the LGPL and the BSD licenses."  

The discrepancy lies in the differences between the "most non-GPL projects" referred to in Yates' clarification and the "most open source licenses" referred to in the original statement--two very different scopes.

At a time when Microsoft is trying to convince the world that it was not only denied due process, but that it knows better than one of its largest customers, it's hard to know what to make of important semantic disputes of this nature. Long time Microsoft detractors will probably argue that it was a blatant attempt at FUD (fear, uncertainty, and doubt--a propaganda tactic often used by tech titans to distract customers and the press from alternatives or the truth).  Long time supporters of Microsoft would argue that it was just an honest mistake or that the quote was taken out of context. 

Rosen, who dedicates several pages to BSD's cryptic sublicensing terms in his book on open source licensing, disputed Yates' interpretation. "The Microsoft license is incompatible with any open source license that explicitly authorizes sublicensing and is incompatible with open source processes that as of matter of practice do sublicensing. Every open source project operates on the basis that sublicensing is allowed. That's how open source works, even if not every license says so explicitly," Rosen said in a phone interview. Rosen went on to say, "The BSD license is almost always used for software that is going to be incorporated into lots of other software operating under lots of other licenses. Everyone ignores the BSD's sub-licensability issue because they know that the license is ambiguous, short, and outdated."

If recent history is any indicator of whether open source developers see sub-licensability or transferability as being a negotiable item, then the answer is that it clearly isn't. A little more than a year ago, the Internet Engineering Task Force's deliberations regarding potential e-mail sender authentication standards (to combat spam) fell to pieces after open source developers including the Apache Software Foundation took umbrage at the non-transferability terms found in Microsoft's license to one of the foundation technologies under consideration. Microsoft refused to remove the terms. 

Earlier this year, non-transferability once again reared it's ugly head when the Apache Software Foundation red-flagged the license for another OASIS specification: WS-Security.  The impasse, which requires both Microsoft and IBM to give in on WS-Security's licensing language, has yet to be resolved. 

Why does any of this matter?  In the scheme of rating openness, knowing that open source developers are comfortable enough with Adobe's language to develop PDF-compliant software (whereas they lack that comfort with Microsoft's language) is enough to make anybody (Massachusetts included) feel as though they'll be able to get their software from any source, regardless of whether it's open source or not. By keeping that option open, the Commonwealth is also guaranteeing itself that intellectual property issues will not stand in the way of public document access in perpetuity.  

Not only should Massachusetts desire that level of openness in the same way any organization should (for any specification), but as a government in the United States, the Commonwealth has to tread very lightly when doing anything that could be viewed by the public as fettering an open free market or endorsing restraint of competition.

Microsoft, of course, is free to argue that sub-licensability and transferability are not as critical as open source advocates such as Larry Rosen make it out to be. If you're Massachusetts-- an "organization" that must take its public's interests into account--Microsoft's position on what open source developers should be concerning themselves with is not as important as what open source developers (a part of "the public") are actually concerned with (as evidenced by the e-mail authentication and WS-Security snafus).

Even if Microsoft's clarification were technically correct, the error in the original statement (that Microsoft's license is compatible with many other open source license)--offered in the context of heavy persuasion--would leave most customers with a bad taste in their mouths. 

Setting aside for a moment Microsoft's explanation, the dialog with Yates makes it clear that, even during this post mortem period, Microsoft is still determined to prove that its test for openness is better than that of Massachusetts' and that some form of illegal procedure on behalf of Massachusetts played the primary role in the outcome . Not once since I started corresponding with Yates did I get the impression that Microsoft was interested in figuring out what concessions it should have made--or perhaps could still make--in order to get back on the list.

In fact, one important point that has so far gotten no attention in the coverage of the Massachusetts decision is that the door is actually still open for Microsoft's format's get back on the list. In a telephone interview, CIO Quinn made it clear that if Microsoft fixed its patent license to meet the state's requirements, the state would reconsider the Office XML Reference Schema for inclusion in its standards. "We would support multiple formats as long as they're open" said Quinn. "If Microsoft were to do that, I would expect that we would add it to the list."

Quinn also disputed Microsoft's assertions that it was surprised or blindsided by the decision and that the process was deliberately biased or driven by some sort of surreptitious agenda. "I'm sorry. I have real difficulty with the whole blindsided piece. We have had continuing discussions with Microsoft going back to 2004 about the whole issue of the XML Schema and a continuing dialog of how we wanted it to be open. We absolutely let them know what our thinking was, and where we were heading, and what our plans were" Quinn said. "We are not interested in having this continuing discourse with Microsoft. We work with them, we think they are a good partner, we think the world owes them a big debt in terms of what they've done in terms of innovation. But where we part ways is on the definition of open."

Quinn also addressed accusations that Massachusetts' decision to exclude Microsoft's file formats are in someway connected to the antitrust litigation that took place between the two and that the standard was designed to manipulate the state's procurement process in a way that prevents the purchase of Microsoft Office. "We picked an open standard which has nothing to do with any vendor whatsoever. We're not trying to climb into bed with any single vendor and this is not about procurement," Quinn told me. "We had [Microsoft] in and we let them know what our thinking was. In late July, we told them that we're going to be adopting the OASIS standard and asked them for their support. We've had many constructive conversations with Microsoft. But it's their choice [not to support the standard] we picked. We're not trying to lock them out."

Yates may feel as though the Microsoft's naivete got it railroaded, and I agree that if focusing solely on the chronology of events, including Microsoft's accounts of its private meetings with state officials in Summer 2005, that Sun and ODF may have had some opportunities to rectify its license that Microsoft did not.

But, disputes over who said what when or whether there was some form of illegal procedure involved are mooted by Secretary Eric Kriss' Sept.16 explanation to Microsoft regarding what it must do for its Office XML Reference Schema to be reconsidered as a Massachusetts standard. With Quinn now on record as saying the door is still open, it's clear that the ball is in Microsoft's court and that state is open to continued use of Microsoft Office.

Given the way so many other governments and businesses are sure to follow in Massachusetts footsteps (using the state's documentation of its decision as a template for their own), Microsoft has two ways to ensure that its Office productivity suite remains in contention for millions if not billions of dollars in purchasing decisions. It can either unencumber its license to the point that it meets Massachusetts' definition of open or it can choose to support ODF in Microsoft Office.

Whereas it's clear that Microsoft has no interest in the former, I asked Yates about the latter and, referring to the working group within OASIS that worked on ODF, he said, "Sun and IBM didn't exactly invite us." But, in the Sept.16 meeting, IBM's Doug Heintzmann can be heard saying that Microsoft was not only invited to participate, but that the invitation still stands and will stand in perpetuity. None of the Microsoft representatives in attendance stood up to argue the point. Heintzmann might as well have said "checkmate" as long as he had the microphone.

Given the way Microsoft just added PDF support to Office-- something that Yates told me took Microsoft a year to do-- it is rather vexing that Microsoft is so fearful of adding ODF support as well (it's free to do so). After all, if it added support for ODF, Microsoft would not only pave the way for customers like Massachusetts to keep buying its Office suite, it would also put the choice of file format where that choice belongs--in the customer's hands.

Microsoft doesn't even have to set ODF as the default format for saving documents. It can default to its own. If ODF is as inferior as its detractors say it is and it fails to satisfy end users' needs, then Microsoft needn't be so insecure about supporting the specification. Failing support of ODF, it will leave many like Massachusetts who want that option with no choice but to seek out Office alternatives.

When Massachusetts put the ball in Microsoft's court so openly and publicly by giving it multiple ways to address the states' needs and continuing to leave those doors open, any and all claims of impropriety became non-issues. In addition to exposing the true colors of all parties involved, the audio from the Sept.16 makes it clear that the Commonwealth of Massachusetts is clearly at the forefront of the open world.

Not only has Massachusetts developed a modern and virtually unassailable test (particularly for its requirements since virtually every part of the test can be connected to the state's need for sovereignty), it has fully leveraged the democratizing forces of government and technology to arrive at an informed decision that serves the best interests of its public.

Given the way this decision will be repeated in other halls of government, I think it can be said that Massachusetts has once again delivered a shot heard round the world. That Microsoft's biggest competitors were standing by the Commonwealth's side to help it pull the trigger is evidence of why Massachusetts ETRM 3.5 truly was ground zero in one of this industry's biggest and most important battles.