Microsoft: We were railroaded in Massachusetts on ODF

Warning. If you read any further, you will find yourself at ground 0.0of the biggest battle the computer industry has ever seen. It is where thebiggest warriors from the proprietary software world, the open standardsworld, and the open source world are engaged in hand-to-hand combat. Atleast for the moment, the open source and open standards worlds (the RebelAlliance) appear to have joined sides against the proprietary warlords, led byMicrosoft. Both on and off the field (where negotiators have failed to broker atruce), the engagement has not been pretty. But this is war. For the firsttime in the battle's history, the Rebel Alliance has dealt the warlords astunning blow. 

"We were railroaded."

Those were not Microsoft's exact words, but if you were a fly on the wallduring my recent series of correspondences with Microsoft's Alan Yatesregarding how Microsoft's XML-based Office file formats ended up off ofMassachusetts' list of approved file formats (essentially pulling the state'splug on future usage of Microsoft Office), it would be difficult to summarizehis opinion in any other way.

To the untrained eye, the Massachusetts decision-- formally known as thatstate's Enterprise Technical Reference Model (henceforth referred to asMA ETRM)-- looks like one of those small open source victories for someEuropean municipality looking to establish independence from big badproprietary American technology. Such victories are important, no doubt,to that town, city, or country and even to perhaps to certain technologycommunities in general (e.g., open source) -- but largely peripheral to thebigger battle. 

David Berlind blogs regularly in Between the Lines and Microsoft Vistulations.

But, MA ETRM is about far more than open source. Industry historians will laterview it as one of the most brilliant chess moves by a handful of industrytitans with a common interest in breaking Microsoft's dominant grip. The MAETRM proceedings are where some of Microsoft's biggest competitors (IBM, Sun,HP, Novell, and Adobe) gathered to make sure that Microsoft was checkmated witha devastating weapon that they themselves have been unable unleash on theAmerican chessboard: Democracy.  

The stakes are huge. As far as customers go, Massachusetts is a goodcatch. It has over 80,000 employees on its payroll. Not only couldMassachusetts future statewide procurements be affected by the state's new ITpolicy, so too could the IT purchases of many contractors, companies, andorganizations that must interact with state's 173 agencies. The loss of theMassachusetts IT "ecosystem" is hardly enough to hurt the fortunes ofa single large IT vendor, but a bigger problem looms for Microsoft. Notonly are many US-based government organizations at federal, state, county andmunicipal levels keeping an eye on the Massachusetts situation as it unfolds,so too are the advisors to many countries. For example, Susie Lindsay, aresearch fellow at the Harvard Law School's Berkman Center for Internet andSociety that's been working with 13 countries, including Brazil, China,Thailand, Brazil, and South Africa, on issues similar to those faced byMassachusetts, has been keeping close tabs on the situation (not only was shepresent at the last public meeting regarding the issue, she spoke at it).

Alan Yates is the business strategy general manager for Microsoft'sInformation Worker Product Management Group (part of its Office group). Heis also the author of the comments that were filed with the Commonwealth ofMassachusetts just ahead of that state's last semi-public"meeting" on Sept.16, 2005. (The press was not permitted to attendbut a link to the audio of the event can be found here.) 

The organizers of that meeting--the Massachusetts Technology LeadershipCouncil--may not have been aware that the meeting (along with the audio recordof it) played a critical role in the overall drama. First, by participating ina public forum that it didn't have to attend (even though it did so, theMassachusetts IT department is not bound to use hearings or democraticprocesses), the state demonstrated a degree of transparency that confidentlyput the accountability of its officials on the line. Second, given thatrepresentatives from companies like IBM, Sun, and Adobe were allowed to bothattend and speak at the meeting (which they did), it was clear that Microsoft'scompetitors were going to take every opportunity they could to weaken Microsoft'sposition (which they have routinely done throughout the process).

One week after that meeting, on September 23, MA ETRM (officially, version3.5) was officially ratified. On the heels of such a stunning blow and with somuch at stake, Microsoft is still looking to shape public opinion in hopes ofcontaining and perhaps even reversing the damage.  Since I've devoted a fair amount of coverage to the MA ETRM deliberations (What ever happened to'The customer is always right?' and Did MS send the wrongguy to Mass' ODF hearing?), Yates asked if I'd be open to hearingMicrosoft's side of the story. 

That previous coverage concluded that Microsoft made some questionabledecisions during the home stretch leading into the ratification of MAETRM.  Yates disagrees and suggests that excluding Microsoft's file formatsfrom the MA ETRM standard was not based on the technology's degree of openness(or lack thereof), but rather on a lack of due process along with a littleinside baseball that may have afforded the alternative--OpenDocument Format(ODF)--an unfair advantage. In his Sept. 8, 2005 letter to the Commonwealth,Yates wrote:

The Information Technology Division (ITD) of theExecutive Office for Administration & Finance of Massachusetts] did notprovide sufficient time for review and comment on the proposed policy, nor arobust process for addressing comments. Due process requires much more,particularly given the unprecedented nature of the proposal and the potentiallyadverse consequences it could provoke. 

Falling short of accusing any particular party of tainting the process,Yates says the decision took Microsoft very much by surprise given the fact thatits specification--formally referred to  as Microsoft's "Office XMLReference Schemas"-- had, to the best of Microsoft's knowledge, officially"made the cut" earlier in the year after Microsoft worked closelywith the Commonwealth to tweak the language found in the company's patent license for those schemas. 

"Back in the January/February timeframe, we had a discussion with[Massachusetts] Secretary [of Administration and Finance Eric] Kriss and[Massachusetts CIO] Peter Quinn about our Office XML File Format license and weagreed to make a small change.  We had their agreement that our licensemet their test for openness," Yates said during a phone interview,  Duringthe Sept. 16 meeting, Yates' surprise and frustration was echoed by Microsoftnational technology officer Stuart McKee when, at 35:10 ofthe audio of the meeting during his second opportunity to speak, McKee said:

We do have some concerns that we'renow not on the list and in fact I think you stood before this body and talkedabout us being on the list and in the conversations we were having. So, I guessthe question is how does this policy evolve over time, what can we expect whenwe are on the list, off the list [and] can we get back on the list?

Although it isn't clear what Massachusetts' test for openness was back in theearly 2005 timeframe, it is clear that "the test" was officiallyrevised to the point that by the time MA ETRM was ratified on Sept. 23,Microsoft's file format no longer satisfied the Commonwealth's criteria foropenness.  At 1:05:37 into the audio, Massachusetts Information TechnologyDivision general counsel Linda Hamel could be heard summarizing the state'scurrent thinking when she said:

There's a continuum of openness when you thinkabout licensing and at one end of the continuum, you might put proprietarylicenses.  At the far end of the continuum at the other end, you mightwant to put the General Public License.  There are a lot of licenses inbetween and when I look at the Microsoft license for the XML Reference Schema,I don't see it as an extreme on the proprietary end. I think it has a degree ofopenness to it. It's not as open as the license for PDF.  So, what we didwhen we were coming up with the standard was we kind of drew a line along thecontinuum of openness that we're talking about and that line in the finalversion happened to get drawn so that in the version of ETRM you see postedtoday (3.5), the Microsoft Patent License doesn't fall within that spectrumwhich isn't to say it doesn't have an element of openness to it.  So thequestion for the government has been where along the spectrum of openness wewant to be. When we originally came out and included the Microsoft XMLReference Schema among the acceptably open components that we would look at, wegot a firestorm of commentary from many communities saying it's just not openenough.

So, somewhere between the January/February timeframe whenMicrosoft felt as though it reached an agreement with Massachusetts and theSept. 16 meeting, Microsoft's license was reassessed. It went from passing theCommonwealth's test for openness to failing it.  Exactly how and when thathappened, Yates says he doesn't know. But his feeling that Microsoft mighthave been railroaded by influential forces behind closed doors was palpableduring our call.  For example, Yates said that leading into that finalmeeting, many of the Microsoft-favorable comments that were filed with theCommonwealth were kept under wraps. In contesting my suggestion in a recent blog thatMA ETRM's ratification was a fight to the finish, Yates thinks the meeting andthe comment period that led up to it were moot. Said Yates:

We didn't feel as though there was any big conflictbetween us and the State of Massachusetts for the longest time.  But thenall of the sudden, given this change in the policy, we were given a short timeto comment on the change in the policy.  And then [Secretary Kriss] madethe policy final.

Although it is unclear precisely when state officials made their finaldecision, Massachusetts gave the public two weeks to comment on a new draft ofMA ETRM (that was published on Monday August 29).  In the Sept.16 meetingaudio, Kriss can be heard saying that the deadline for comments was "lastFriday" (in other words, the Friday before the meeting) which was Sept.9. Pointing to a voluminous document of public comments (both for andagainst) that he says was finally published online after the hearing, Yatescontinued:

We were shocked that all of the comments had notbeen released.  Only five of the comments were released.   Fourwere against us and one was for us (ours).  The selective inclusion ofcomments was evidence that the decision was a foregone conclusion.

Microsoft wasn't alone in this feeling.  Heading into thatmeeting,  the press (including me at the time) discussed the Commonwealth'sdecision to go with ODF as though it were a done deal. But the truth is that itwasn't and maybe still isn't. Not only did I posit in the aforelinked blog thatMicrosoft may have sent the wrong man to the Sept. 16 meeting, it's clear fromSecretary Kriss' own words (at 1:09:55) that the door is actually still open:

I think the state would be reviewing this formatdifferently in this conversation if you did basically three things. If youdropped the patent entirely with regard to it; if you really did publish thestandard.  By the way, my definition of a peer group is not users. It'speople who are familiar with the technical aspects of the format itself. So, mymother who uses Microsoft Word I don't consider to be a member of the peerreview community as to the file format. So, I just wanted to be clear aboutwhat I mean by peer. I don't mean that it's the user base. If you were topublish the standard and then make provisions for future changes to thatstandard to be part of a joint stewardship that is no longer solely controlledby Microsoft Corporation...if you were to do that, we would be delighted to begin a true technicalcomparison of your standard with the OpenDocument standard and go from there.But I think the issue is that those impediments that have risen in importanceas we have gone through this process....Although you have made definite stepsin the modification of your license, it'sall a moving target. If you were to think about those aspectsof it, I think we would be very interested in looking at the Microsoft definition. 

Had someone with the necessary authority been there to make the proverbialexecutive decision, that technical comparison might well be underway at thismoment. As far as can be told from the  public record, the decision wasnot fait accompli and was in fact still open for discussion on Sept.16, sevendays before MA ETRM (version 3.5) was ratified. 

Yates argues that even if the state were still open to discussing thematter, having a different executive at the meeting other than Microsoftnational technology officer Stuart McKee would not have made a difference to theoutcome. Citing at least two reasons, Yates claims the process was deliberatelybiased to assure that ODF made the cut and Microsoft's Office XML formats didnot. First, pointing to the fact that Massachusetts had concerns about thelicense for ODF, his only explanation for the way that ODF was added to the Aug.29MA ETRM draft at the same time that the Office XML formats were dropped is thatbehind closed doors Massachusetts officials must have received assurances from potentialODF licensor Sun that those concerns would be addressed. (Sun still maintainsit has no patents that apply to ODF, so there's really nothing to license.) 

Meanwhile, Yates claims that Microsoft's only opportunity to officiallyrespond to the fact that it had been dropped was during the two week publiccomment period that followed the draft's publication. Second, Yates andMicrosoft are so convinced that the Office XML Reference schema is open enoughthat Massachusetts must be biased to see it any other way.

Both claims are complicated, so I've decided to tackle them in separatesections.

Behind closed doors
As it turns out, Yates is right about the chronology of events. In other words,for Massachusetts to have so confidently added ODF to the Aug. 29 draft, thoseassurances had to have been received ahead of the draft's publication. Meanwhile, Yates claims that Microsoft wasn't afforded a similar pre-draftopportunity to discuss Massachusetts' concerns about Microsoft's formats. Do the two taken together make solid grist for the conspiracy mill?  Here'swhat I have so far.

As pointed out earlier, Massachusetts Information Technology Divisiongeneral counsel Linda Hamel noted that the state received a "firestorm ofcommentary" in response to the early 2005 MA ETRM draft that listedMicrosoft's file formats as the supported "open" standard. Accordingto various sources, the response was so opposed to the inclusion of a Microsoftformat in what was otherwise an open initiative that it caused Massachusettsofficials to take a deep breath and reconsider its test for openness.

The state was so overwhelmed with public comments that it decided it had to"take it to industry," according to Sun standards manager DougJohnson, who has represented Sun's interests throughout the process. Thatdecision led to a Massachusetts organized meeting on June 9, 2005 that wasattended by Sun, IBM, Adobe, and many others. Representing Microsoft atthe meeting were McKee, state government affairs manager Brian Burke andaccount technology specialist Leslie Tan. The meeting came on the heels ofODF1.0's ratification by the Organization for the Advancement of StructuredInformation Standards (OASIS) and, according to Johnson, the tone ofthat meeting was that XML was the way to go and that virtually everyone inthe room but Microsoft was behind ODF.

According to Dan Bricklin--Massachusetts resident, co-inventor of theelectronic spreadsheet and moderator of the Sept.16 meeting--the state wasclearly in pioneering territory (as governments go), flush with feedback similarto what developers get during the software usability testingprocess. "One way I now see what's going on looking in from theoutside, the Commonwealth had a general need in terms of wanting there to be awide range of producers and products," Bricklin told me, referring tosolutions that could read and write whatever format Massachusetts settled on inorder for the Massachusetts public to have unfettered access to the state'spublic documents.  "[Massachusetts] could have looked at it fromtheir own viewpoint of what they thought was needed legally. But it didn'tmatter what they thought. What mattered was how developers sawthat. If too many developers or their lawyers had any discomfort with thelicense, then that meant there was enough of a question as to thespecification's openness.  It meant there was a risk to the availabilityof solutions."

Although Sun's Johnson doesn't suspect Microsoft of turning the state on to anypotential problems with ODF's license, Johnson admitted in a phone interviewthat Massachusetts officials asked Sun for some clarifications regarding ODF'slicensing requirements and that Sun was given an opportunity to address thoseconcerns. "In late June [2005], Massachusetts started asking for thelicensing language to ODF" Johnson said,

What no one knew is that Sun had long been working on a way to issue whatamounts to "licenseless license -- a type of patent non-assertion covenant." Just the culture shock alone of such an idea to any technologycompany that depends on intellectual property (as Sun president and COOJonathan Schwartz says his company does) would probably take the company'sexecutives and lawyers a year or more to digest. 

What made Sun's idea of a non-assertion covenant unique is that it's thesort of covenant that mentions no patents in particular, but that givesdevelopers carte blanche access to any of the issuer's patents they need to implementa particular specification without having to agree to or sign alicense. Thus, it's sort of a license-less license. 

The idea is so progressive that some believe it could be more than just anext step towards non-proliferation of open source licenses (in other words, ifno licenses are required, there's nothing to proliferate), but also the nextstep towards a different attitude when it comes to the software patent issuesthat currently plague the industry. 

Although Sun still had its license-less license under development, therequest for clarification from Massachusetts represented the first opportunityfor Sun to test its viability. So, Sun decided to apply it to ODF.  "Ourguys got together with the Massachusetts guys and explained what the languagewill be and sometime around July or August, we gave the non-assertion covenantlanguage to them," Johnson said.

Around the same time, Yates says that Microsoft and Massachusetts did have ameeting, but that Microsoft did not get a similar request for license clarification. "At the end of July, two of our Government Affairs people -- StuartMcKee and Brian Burke -- were asked to meet privately with Secretary Kriss,Peter Quinn and Linda Hamel.  At that time, Secretary Kriss told them thathe was moving toward a policy of naming OpenDocument as some kind of statestandard. He said that there were multiple ways for Microsoft to work withit: a) natively; b) Microsoft-sponsored 3rd party support or;  c) natural3rd party support.  He did not ask us for any changes to our license, etc.nor did he press for an 'answer' about our approach, or give a timeline. We were not sure what precisely was going to happen," Yates wrotein an e-mail.

Yates also wrote, "There was also a Linux conference in San Franciscolate in August I believe, where Peter Quinn was a panel member. At thattime I believe that there was some mention that MA was going to do somethingimportant in favor of OpenOffice. We contacted Peter to try to learn what hiscomment meant, but did not find out exactly until the policy was posted on thewebsite."

The policy Yates was referring to is the Aug.29 MA ETRMdraft, published sans Sun's non-assertion covenants. Within two days, on Aug.31,Microsoft's position on the ambiguities in the ODF license du jour and asubsequent warning that ODF might not be as open as some were making it out tobe appeared in a blog entry authored by Brian Jones, one of the programmanagers working on Microsoft Office.  Jones wrote in that post:

While we're on this topic, Ithink it's important that you all take a look at the comparable situationwith Open Document. A lot of folks just seem to assume that since it's astandard, there are no IP issues and everything is very straightforward. Well,take a look at this: http://www.oasis-open.org/committees/office/ipr.php.Sun seems to be saying that it may have IP in the Open Document spec. While Sunsays it is willing to provide a royalty-free license, one would still need toask Sun for a license. The license is not posted.

From Microsoft's perspective, something was amiss. Yatesfeels that Microsoft, which up until that point had been forced to publiclypost a new license including certain concessions that Massachusetts had askedfor, was being subjected to a double standard.  According to Yates,"At that point, we weren't clear what the OpenDocument licensewas." 

Microsoft's position on Sun's disclosure however drew a torrent of criticismand has been labeled by some as an intentional misread of the legalese designedto disingenuously raise suspicions and cast dispersions on the fairness andobjectivity of a state government's process.  For example, in the sectionentitled "Sun Steps Up to thePlate and Hits a Home Run" in oneof her recent posts, Groklaw editor Pamela Jones who is not an attorney butwho has historically proven herself to understand the intricacies of softwareand patent licensing, examined the original Sun disclosure and concluded itdidn't "say what Jones said it does. Not by a mile."  In thatpost, Jones also points to a response from Sun's chief open source officer Simon Phippswho openly posited that Jones' post was a deliberate misread, saying"Brian's position is 100% pure FUD."

But the drama wasn't over. "Then we saw in these other blogs thatthe Sun guy talked with [Massachusetts CIO] Peter Quinn and they eitherfinished or hurried up and fixed the license," Yates wrote.

As it turns out, Yates wasn't referring to a blog, but rather to an OASIS bulletin board thread involving OpenOffice.orgrepresentative Gary Edwards and the new OASIS chairman of the board -- Sun'sEduardo Gutentag. In that Sept.26 thread, Edwards is clearly lookingto mount a response to Jones' assertions and Gutentag is saying not too worrybecause the non-assertion covenant, which will make all of Microsoft's claimsmoot, was coming in a couple of days.  While Edwards is seen saying thathe's going to try to assure Massachusetts CIO Peter Quinn that Jones' post wasoff base, what Edwards clearly didn't know was that Massachusetts stateofficials already had a copy of the new non-assertion covenant (the "license-less"license).  Three days later, on Sept.29, Sun released the covenant underthe title SunOpenDocument Patent Statement.

Regardless of the dueling interpretations of Sun's original disclosure,Yates' feeling that the process was biased against Microsoft was furthertweaked by the fact that  Sun issued the far more liberally worded (andaforementioned) patent covenant approximately one week after Massachusetts officially ratifiedits decision on Sept.23.

Even I have to admit the relevant events as well as their timing could makeenough grist for the conspiracy grill to claim "fishy" if not"foul."  Did Massachusetts move forward on ODF with a simpleassurance from Sun that its work-in-progress license-less license would beapplied to the document format, whereas Microsoft was forced to amend itslicense to no avail and then dropped from the list without an opportunity toredress? If what Microsoft says about the content of its late July meetingwith Kriss, Quinn, and Hamel is true, then the answer is yes.  But, isthis chain of events, with an implication that the Commonwealth had some otheragenda--perhaps an anti-Microsoft bias, a proverbial smoking gun? 

Although Yates argues that Microsoft wasn't given a similar opportunity toredress, Sun's Johnson suggested that the public nature of Massachusetts'deliberations-- including the inundation of comments in 2005H1 and the cleartenor of the June 9 meeting between state officials and "industry"--was enough writing on the wall for Microsoft to know that it had a problem onits hands and what the sticking points might be. According to Johnson, Massachusettsinitial approval of Microsoft's file formats in early 2005 was still subjectto a public comment period, and it would have been premature for anyone toassume that any decisions were final.

In a telephone interview on Oct 12, 2005, Massachusetts CIO Peter Quinn madeit clear to me that none of ETRM was set in stone back in early 2005. "Weput ETRM out for public comment in January. We did that for the entire version3.5," Quinn said. "We don't have to put anythingout for public comment. But we always put things out anyway because we alwaysget valuable insight." Referring to the fact that MA ETRM coversmore than just file formats, Quinn said, "Because of all thefeedback we got about the file formats, we ratified 3.5 without the formats."

Whereas Johnson clearly suggested that the writing was on the wall, Yatestold me, "We were a bit naive."  Was it naivete? Or, was it thejob of Microsoft's National Technology Officer Stuart McKee-- the man atMicrosoft who is responsible for interfacing with governments like theCommonwealth of Massachusetts-- to not only do a better job of reading thesituation, but to know what the stakes were in dealing with a government thatanswers to its public rather than a board of directors?  

There's no question that Microsoft's Office XML Reference Schema was onceon Massachusetts' "list," and then, as a matter of public discourse--somemight say democracy-- removed from that list. Going back to the notes fromMassachusetts' June 9, 2005 meeting with industry-- referred to by Quinn as theOpen Formats Summit--it would be impossible for anybody, Microsoft included, toassume that any format that was once on the "list" wasguaranteed of staying on that list. In the very last section of thosemeeting notes, the next steps were clearly defined as:

• Identify a continuum of acceptable open document standards for the Commonwealth.
• Revise the standards and publish the revision for public comment; then finalize the standards.

These action items practically scream out that, at the time, Massachusettswas undecided on a standard and that the due diligence for choosing one was farfrom over.  Clearly, a new test for openness was being developed andMassachusetts had already started researching the matter. In fact,Microsoft's team on the job--Stuart McKee, Brian Burke, and Leslie Tan--didn'thave to look far to see some of the criteria for openness that the State was considering. Itwas displayed in a footnote on the bottom of the June 9, 2005 meeting notes andstated as follows:

Specifically, [Ken] Krechmer, [Fellow,International Center for Standards Research, University of Colorado] notes,standards creators typically consider a standard to be open if the creation ofthe standard follows the tenets of open meeting, consensus and due process;implementers of an existing standard would call a standard open when it servesthe markets they wish, it is without cost to them, does not preclude furtherinnovation (by them), does not obsolete their prior implementations, and doesnot favor a competitor; and users of an implementation of the standard wouldcall a standard open when multiple implementations of the standard fromdifferent sources are available, when the implementation functions in alllocations needed, when the implementation is supported over the user'sexpected service life and when new implementations desired by the user arebackward compatible to previously purchased implementations. 

With the writing on the wall, the question that sticks out in my mind is whydidn't Microsoft sound its internal fire alarm sooner?  It appears thatMicrosoft left the fate of such an important initiative in the hands of aschedule and process over which it had no control. You have to ask why Microsoft didn'timmediately establish an independent council to comb through all of theavailable information including public comments, minutes from the June 9meeting, and other work regarding the measurement of openness in order to comeup with an independently developed stringent test for openness, free from bias. Knowing that Adobe's Portable Document Format (PDF) was also in contention, thecouncil could have also figured out why exactly PDF was under consideration.

One reason no such fire alarm was sounded couldbe that Microsoft remained convinced-- as it still does today-- that nomatter what anybody's test for openness could be, the license for its OfficeXML Reference Schema should pass with flying colors. Unfortunately, inMassachusetts' case, things couldn't be further from the truth.

Not only does Yates feel as though Microsoft's formats weredenied equitable treatment as a part of the due process, he and Microsoft viewthe license to the Office XML Reference Schema as being sufficiently open forany developer-- even some open source developers-- to implement it in theirsoftware.  So convinced of this is Microsoft that the company and itsrepresentatives (McKee, Yates, etc.) continue to focus on the "we're openenough" argument rather than "what must we do to get on the list?" 

From Microsoft's point of view, something other than afair assessment of Microsoft's license has played a role in theoutcome. When you step back and listen to both sides, you can't help butwonder if Massachusetts is from Mars and Microsoft is from Venus.  Soclose, yet so far apart. Regardless of what Microsoft has to say,Massachusetts officials say that Microsoft's license isn't open enough (eventhough Microsoft made several concessions in the wording of the license overthe last year just to appease the state). With no standard test foropenness (an issue that I've blogged about before), Microsoft and Massachusetts haveeach arrived at their own test for it and the two are simply not the same.

In our phone interview, Yates went into detail regarding why he believes thatMicrosoft's patent license make the formats as open as they need to be. One of the biggies that's on everybody's test is whether the license isroyalty-free (RF) or not (Microsoft's license is). In fact, for a longtime, the RF nature of a specification was practically the only test of whetherit was open or should be considered as a standard.  But, as Kriss said inthe meeting (see quote above), it's a moving target. Over the last fewyears, if the open source movement and patent quagmire havetaught the world anything, it's that developers are far less free to doanything in software than originally thought to be. In that context,degrees of openness are very much about degrees of developer and end-userfreedom.

On the basis that no specific patents are mentioned, Yates believes thatMicrosoft's license offers more such freedom than Adobe's license for the Massachusetts-approved PDF. Indeed, in a blog Ijust wrote about Sun's recent patent covenant for developers looking tocreate OpenDocument Format-compliant implementations, one of the covenant'sattributes that I cited as being unique was its lack of specificity regardingthe patents needed to create such an implementation. Like Sun's covenant,Microsoft's patent license lacks the same specificity, which means thatlicensees/developers are free to use any Microsoft patent they need to in orderto create software that complies with Microsoft's Office XML ReferenceSchema. When patents are specified, as they are in the case of Adobe'slicense for PDF, licensees must bear the additional burden of making sure thattheir implementations don't run afoul of an Adobe patent that's not listed onthe license.  "We took a more inclusive approach by not naming anypatents,"  Yates said.

On that front, Microsoft's license is more open than is Adobe's. But, whereasAdobe's license met Massachusetts' overall test for openness, Microsoft'slicense did not. That apparent (to Yates) application of a double-standardtriggered Yates' suspicions that other forces were at work, perhaps drawingthe democratic nature of the process into question.  According to Yates,"[Secretary Kriss] didn't feel that our license was open enough. Itremains confusing to us that the Adobe license is, and ours isn't." 

As it turns out, Microsoft's argument omits other attributes of Adobe'slicense that impact the openness of PDF.  For example, Yates neglected toaddress Massachusetts ITD general counsel Linda Hamel's declaration during theSeptember 16 meeting that Microsoft's license was less open than Adobe's forPDF.  Had Microsoft contacted Adobe for its perspective on openness ordone what I did -- which was to contact software licensing expert Larry Rosen(who literally wrote thebook on open source licensing)-- he might have learned that Adobe's patent clarification clears the way for any opensource developer to make software that can read or write PDF files. Partof Massachusetts' test for openness is that any developer, including open-sourcedevelopers, have no infringement concerns when it comes to developing softwarethat complies with state standards.  Adobe's clarification states:

Adobe desires to promote the use of PDF forinformation interchange among diverse products and applications. Accordingly,the following patents are licensed on a royalty-free, nonexclusive basis forthe term of each patent and for the sole purpose of developing software thatproduces, consumes, and interprets PDF files that are compliant with theSpecification.

As said earlier, the clarification goes on to list the specificpatents.  Commenting on that clarification, Rosen said the following viae-mail:

The Adobe Patent clarification notice looks fine tome. The patent licenses are limited in scope to compliant implementations, butthat alone doesn't make them non-open, at least in my view, because thelicenses don't actually prohibit non-compliant implementations (they just*don't license* them).

Had Microsoft contacted Adobe, it might have also heard Adobe's position onthe issue of openness, which was expressed at 1:03:15 into the Sept.16meeting by Michael Engelhardt, Adobe's senior director of public policy:

The license for PDF from thebeginning has been a standard that we have published and made freely availableto anyone who wishes to use it and there are over a couple thousand, I think,implementers of tools and you can go and, OpenOffice has a PDF builder builtinto it, Apple's word processor has a PDF maker built into it....You candownload [the PDF specification for free] or buy the spec in book form onAmazon.com. ..You can do whatever you want with it.  The only licensingrestriction if you can even call it that which says that if you build somethingthat you call a PDF, it has to be a PDF....You can't call something a PDF thatisn't in fact a PDF. It has to be compatible with all the other PDFs and allthe other readers and everything else.  In other words, what you built hasto work. Otherwise you have to call it something else. You can still build[something based on a derivative of PDF].  But, if it isn't a PDF thatworks like a PDF, it's not PDF. That's the only licensing restriction. Also to address the question of openness and the criteria that you list, thereactually are flavors of PDF that are pure open standards [and] ISO ratified. But most importantly I think, in the context of what you want to use PDF for,PDF A for archiving was recently ratified by ISO and there's also PDF foraccessibility that's being developed by AIIM under the ISO.  So we areworking even more in the direction of openness than we've been in thepast.  But right now, PDF is completely published, you can do whatever youwant with it, everybody does

The derivative issue is an important one with respect to openness. WhatEngelhardt basically said is that developers are free to do whatever they wantwith Adobe's PDF specification. For example, they can break it apart or remix it with otherspecifications.   The only restriction on this activity is that ifthe final output of the software isn't 100 percent compliant with the PDFspecification, the developer cannot say that the software or the documents itproduces are "PDF."  The freedom to remix Adobe's work lies instark contrast to Microsoft's license which says: "A 'LicensedImplementation' means only those specific portions of a software product thatread and write files

that are fullycompliant with the specifications for the OfficeSchemas."  In other words, if developers want a license, then theyare bound to developing implementations that are compliant with Microsoft'sspecification.

In the bigger picture, Engelhardt's attendance to the Sept.16 meeting andreadiness to address the issue of why Adobe's license is open is significantbecause of how Microsoft's key competitors were clearly lining up in supportof Massachusetts' inclination to go with ODF. Engelhardt wasn't alone indoing this. IBM's Doug Heintzmann also got up to remind Massachusettsofficials of how forcing the issue of compliance stifles the opportunity for aspecification to be improved through innovation. Microsoft's enemies wereclearly lined up at this meeting to make sure Massachusetts had no compunctionabout moving forward with ODF.

Clearly, we're deep in the legal forestof openness at this point, but it's exactly these lengths that Massachusettswent to in an effort to develop its own modern day test. The Microsoft licensemay be more open-ended because it forgoes a list of specific patents in favorof whatever of Microsoft's patents it takes to implement its Office XMLReference Schema.  However, when it comes down to brass tacks, Adobe wouldhave been called to the mat already if the patents in its license weren'tenough to cover the thousands of implementations of PDF that Engelhardt mentioned. Between that, the remix issue, and the range of developers (all) who canimplement PDF, it would be difficult for me to argue in favor of Microsoft'sposition that its license is more open than Adobe's.

With the Adobe argument dispatched,we now turn our attention to Massachusetts' test for openness. As can beheard from one of the Secretary Kriss' earlier quotes during the Sept.16meeting, Massachusetts explained its test for openness as follows:

• It must be published and subject to peer review
• It must be subject to joint stewardship
• It must have no or absolutely minimal legal restrictions attached to it

Sound familiar? It should. The three item list reads like alayman's summary of the primary points found in the aforementioned Ken Krechmer-relatedfootnote at the bottom of the meeting notes for Massachusetts' June 9 OpenFormats Summit.

During the Sept.16meeting, Microsoft's McKee did not dispute the three-point test, but insteadchallenged Massachusetts' interpretation of it which, naturally, positionedMicrosoft's license to pass. Even though the odds were heavily stackedagainst Microsoft going into that meeting, it was probably not a good time fora vendor to tell a customer with 80,000 employees that it doesn't like thecustomer's application of its own test. If there was any chance left forMicrosoft on Sept.16, McKee was squandering it.

For example, on the subject of peerreview, McKee cited the 400 million existing users of Office as a body of peersthat Microsoft must pay very close attention to. But, to any outsideobserver who is familiar with this industry's consortia and standards bodies,it is clear that that was not what Massachusetts' meant by "peerreview."  Although just about any vendor would lead you to believeotherwise, a vendor's customers are not its peers.

Likewise, joint stewardship isunambiguous. The implication is that the specification in question is notsubject to the control of a single person or vendor. Even though Microsoftlistens to its customers and often responds with features and functionalitythat it often claims were developed at the request of its customers, that isnot the same as joint stewardship. Nor does the exposure of source codeto selected organizations (such as governments)-- something Microsoft doesunder its shared source program-- fall within the category of"stewardship."

In my own considerations of what itmeans for a specification to be open (a.k.a., an open standard), I have oftendiscussed how it must belong to an officially recognized standards body suchas the World Wide Web Consortium (W3C) or the International Organization for Standards (ISO).  In using the language "jointstewardship," the three-point test from Massachusetts leaves room forspecifications that may not necessarily have been ratified by such a standardsbody. Microsoft, as a member of several such consortia including ODF-chaperoneOASIS, is no stranger to this concept. 

As can be seen fromthe aforementioned quote (at 1:09:55 in the audio from the meeting) where hementioned his mother (whom, as a customer of MS Word, did not qualify as a peerin his mind), Secretary Kriss was quick to invoke the more broadly accepteddefinitions of peer (as in peer review) and joint stewardship.

On the issue of legalconsiderations, McKee (at 1:08:04 into the audio) tried to convinceMassachusetts that Microsoft's patent license meets the state's test foropenness when he said:

TheMicrosoft Office Open XML License is perpetual and royalty free.  And particularlyrelated with the comments of the State of Massachusetts, it's a very verysimple license. The publishing and distribution of it is on the Website....but we also very specifically related to this issue included thewords "public records" because we do understand there's a veryspecific issue related to public records.

However, at 1:08:46, Bricklin, whowas moderating, was quick to point out at least one problem.  Referring tothe section that Microsoft added regarding public documents, Bricklin said:

It saysRead Only. It doesn't say read and write of public records on thelicense. It's got some legalese there.

McKee responded:

Correct.Correct. But the issue about the legal simplicity is something that we attempted,or legal issues, is something we attempted todress with perpetual and royaltyfree and specifically incorporating public records we attempted to make it verysimple because these legal issues are very complex.

The section in the license to whichBricklin and McKee were referring says the following:

By way ofclarification of the foregoing, given the unique role of government institutions, end users will not violate this license by merely readinggovernment documents that constitute files that comply with the Microsoftspecifications for the Office Schemas, or by using (solely for the purpose ofreading such files) any software that enables them to do so. The term"government documents" includes public records.

Today, well after the Sept.16meeting and after MA ETRM was ratified on Sept.23, Microsoft is still steadfastin its efforts to convince observers that its license is as open as it needs tobe.  During our interview, Yates said:

Massachusettswants open access to their documents and their data in perpetuity.  Wefeel we took the steps to guarantee them that. Inclusive of the different waysto get things done. Regarding the tax-free access mentioned in your blog[editor's note: see Could PDF supportin Office 12 quell and ODF uprising], we're trying to accomplishthat. You can see that since were early in the curve ofimplementing the format--[Office 12] isn't even in beta yet]-- there couldbe some nervousness about third parties being able to support the format. Butthat's exactly what our license was designed to enable. Lots of third partiesworking with the format in perpetuity. 

Instead of challengingMassachusetts at the 11th hour, and now in post-mortem, perhaps Microsoftshould have been asking Kriss, Quinn and Hamel for more details on therequirements that Kriss laid out as a prerequisite for entering Microsoft'sOffice XML Reference Schema into a technical bake-off against ODF.  HadMicrosoft done that, the discussion might have started with an explanation ofwhy usage of the word "reading" (in the aforementioned license excerpt)to the exclusion of the word "writing" was a red flag. 

If Bricklin, who was simply themoderator (and not a lawyer) picked up on that, software licensing lawyerswould no doubt red-flag it as well. In a separate interview, Bricklin said, "Thefirst question that raises is in a developer's mind is €˜Why would they go tothe trouble of mentioning reading without also mentioning writing?'"

If there's a hint as to why thatmay be, then perhaps it appears in the line just before that proviso inMicrosoft's license that says, "You are not licensed to sublicense ortransfer your rights."

To the untrained eye, thisstatement seems rather innocuous. Who cares, right? As long as the rightsto create software that can read and write Microsoft's formats are equallyavailable to everyone on a royalty-free basis, then why should Massachusetts orany other developer be concerned?

Two words: Open source.

Although it hasn't made anysweeping decisions to adopt open source like other governments have,Massachusetts knows that it's an option, particularly when it comes to desktopproductivity applications. During the Sept.16 meeting, Massachusetts CIO PeterQuinn discussed how he had been using OpenOffice.org. Going back to theCommonwealth's overarching goal to maintain its sovereignty (an important valueto any United States state), the non-sublicensability and non-transferabilityof Microsoft's license is a deal breaker. 

Included in the notion of statesovereignty is the right of the state's agencies, employees, contractors and citizensto choose any type of software they want to read or write publicdocuments. By not allowing its license to be transferred or sublicensed,Microsoft's patent license automatically prevents just about all open sourcesoftware-- including OpenOffice.org--from supporting Microsoft's XMLformats. 

Not only that, it also preventsopen source developers from building the freely distributable and transferableprogramming tools (in all sorts of languages-- C++, Perl, Python, Java, etc.)that would turn incorporation of Microsoft format support into child's playfor most developers. Such libraries are often the catalyst for rapid and viral adoptionof a new specification in the open source community. In the short time sinceODF was ratified by Massachusetts, the alpha version of a C#-based opensource library called AODL has already shown up on SourceForge.net (one ofthe Net's leading open source repositories where developers gather tocollaborate on projects).

According to Larry Rosen,provisions that prevent sub-licensing and transferability are antithetical toopen source. "[The Microsoft license] not only prevents transfer orsublicensing of the patent rights," said Rosen, "but it also requiresthat open source developers put Microsoft's patent notices in our licenses." These are terms that open source developers find to be unacceptable.  In aseparate e-mail, Rosen said, "Open source depends on the right to sub-license."

When I took the issue back to Yates,he concurred that transferability is off-limits when he wrote:

By way ofclarification of the foregoing, given the unique role of governmentinstitutions, end users will not violate this license by merely readinggovernment documents that constitute files that comply with the Microsoftspecifications for the Office Schemas, or by using (solely for the purpose ofreading such files) any software that enables them to do so. The term"government documents" includes public records. There is adistinction between the right to create a program under the license anddistribute that in source code form through all means of distribution (which ispermitted by the license) and the right to grant downstream IP licensesdivorced from the code - to transfer intellectual property - It is only thislast right that we are restricting.  Our license says, "We want tocontinue to own our IP.  We are willing to give anyone in the world alicense to that IP for purposes of developing software, but we don't want toput other people into the business of licensing our IP to others separate fromtheir code. If anyone wants to modify or use the software they get from someoneelse that has used our IP, they can freely take the IP license from us again,to use or make changes.   Does this make sense?  It should notinhibit anyone from getting software done.

Then, pointing back to Brian Jones'blogs, Yates wrote, "Our license may not be compatible with the GPL, butit is compatible with many other open source licenses, and certainly can beused with the OpenDocument license."

Rosen disputes Yates' contentionabout the license compatibility. After reviewing the quote via e-mail, Rosenwrote:

Among thelicenses that are explicitly sublicenseable are the MIT, MPL, CPL, Apache 2.0,OSL/AFL, and all licenses derived from them. That's most, I believe. Microsoft's patent license is incompatible with all of them.

When confronted with Rosen'sresponse, Yates offered a clarification of the original statement, saying, "Whileit is beyond my capacity to analyze [all of the open source licenses listed on the Open Source Initiative'sWeb site], we think that there is no problem with the two most used, keyalternatives to the GPL; the LGPL and the BSD licenses."  

The discrepancy lies in thedifferences between the "most non-GPL projects"referred to in Yates' clarification and the "most open sourcelicenses" referred to in the original statement--two verydifferent scopes.

At a time when Microsoft is tryingto convince the world that it was not only denied due process, but that itknows better than one of its largest customers, it's hard to know what to makeof important semantic disputes of this nature. Long time Microsoftdetractors will probably argue that it was a blatant attempt at FUD (fear,uncertainty, and doubt--a propaganda tactic often used by tech titans todistract customers and the press from alternatives or the truth).  Longtime supporters of Microsoft would argue that it was just an honest mistake orthat the quote was taken out of context. 

Rosen, who dedicates several pages to BSD's cryptic sublicensing terms inhis book on open source licensing, disputed Yates' interpretation. "TheMicrosoft license is incompatible with any open source license that explicitlyauthorizes sublicensing and is incompatible with open source processes that asof matter of practice do sublicensing. Every open source project operates onthe basis that sublicensing is allowed. That's how open source works, even ifnot every license says so explicitly," Rosen said in a phoneinterview. Rosen went on to say, "The BSD license is almostalways used for software that is going to be incorporated into lots of othersoftware operating under lots of other licenses. Everyone ignores the BSD's sub-licensabilityissue because they know that the license is ambiguous, short, and outdated."

If recent history is any indicatorof whether open source developers see sub-licensability or transferability as being a negotiable item, thenthe answer is that it clearly isn't. A little more than a year ago, theInternet Engineering Task Force's deliberations regarding potential e-mailsender authentication standards (to combat spam) fell to pieces after opensource developers including the ApacheSoftware Foundation took umbrage at the non-transferability terms found inMicrosoft's license to one of the foundation technologies under consideration.Microsoft refused to remove the terms. 

Earlier this year, non-transferabilityonce again reared it's ugly head when the Apache Software Foundation red-flagged thelicense for another OASIS specification: WS-Security.  Theimpasse, which requires both Microsoft and IBM to give in on WS-Security'slicensing language, has yet to be resolved. 

Why does any of this matter? In the scheme of rating openness, knowing that open source developers arecomfortable enough with Adobe's language to develop PDF-compliant software (whereasthey lack that comfort with Microsoft's language) is enough to make anybody(Massachusetts included) feel as though they'll be able to get their softwarefrom any source, regardless of whether it's open source or not. By keepingthat option open, the Commonwealth is also guaranteeing itself thatintellectual property issues will not stand in the way of public documentaccess in perpetuity.  

Not only should Massachusettsdesire that level of openness in the same way any organization should (for any specification), but as a government in the United States, the Commonwealth has to tread verylightly when doing anything that could be viewed by the public as fettering anopen free market or endorsing restraint of competition.

Microsoft, of course, is free toargue that sub-licensability and transferability are not as critical as open source advocates suchas Larry Rosen make it out to be. If you're Massachusetts-- an"organization" that must take its public's interests into account--Microsoft'sposition on what open source developers should be concerning themselves with isnot as important as what open source developers (a part of "thepublic") are actually concerned with (as evidenced by the e-mailauthentication and WS-Security snafus).

Even if Microsoft's clarificationwere technically correct, the error in the original statement (that Microsoft'slicense is compatible with many other open source license)--offered in thecontext of heavy persuasion--would leave most customers with a bad taste intheir mouths. 

Setting aside for a momentMicrosoft's explanation, the dialog with Yates makes it clear that, even duringthis post mortem period, Microsoft is still determined to prove that its testfor openness is better than that of Massachusetts' and that some form ofillegal procedure on behalf of Massachusetts played the primary role in theoutcome . Not once since I started corresponding with Yates did I get theimpression that Microsoft was interested in figuring out what concessions itshould have made--or perhaps could still make--in order to get back on thelist.

In fact, one important point thathas so far gotten no attention in the coverage of the Massachusetts decision isthat the door is actually still open for Microsoft's format's get back onthe list. In a telephone interview, CIO Quinn made it clear that if Microsoftfixed its patent license to meet the state's requirements, the state wouldreconsider the Office XML Reference Schema for inclusion in its standards. "Wewould support multiple formats as long as they're open" saidQuinn. "If Microsoft were to do that, I would expect that wewould add it to the list."

Quinn also disputed Microsoft'sassertions that it was surprised or blindsided by the decision and that theprocess was deliberately biased or driven by some sort of surreptitiousagenda. "I'm sorry. I have real difficulty with the wholeblindsided piece. We have had continuing discussions with Microsoft going backto 2004 about the whole issue of the XML Schema and a continuing dialog of howwe wanted it to be open. We absolutely let them know what our thinking was, andwhere we were heading, and what our plans were" Quinn said. "Weare not interested in having this continuing discourse with Microsoft. We workwith them, we think they are a good partner, we think the world owes them a bigdebt in terms of what they've done in terms of innovation. But where we partways is on the definition of open."

Quinn also addressed accusationsthat Massachusetts' decision to exclude Microsoft's file formats are insomeway connected to the antitrust litigation that took place between the twoand that the standard was designed to manipulate the state's procurementprocess in a way that prevents the purchase of Microsoft Office. "Wepicked an open standard which has nothing to do with any vendor whatsoever. We'renot trying to climb into bed with any single vendor and this is not aboutprocurement," Quinn told me. "We had [Microsoft]in and we let them know what our thinking was. In late July, we told them thatwe're going to be adopting the OASIS standard and asked them for their support.We've had many constructive conversations with Microsoft. But it's their choice[not to support the standard] we picked. We're not trying to lock them out."

Yates may feel as though the Microsoft'snaivete got it railroaded, and I agree that if focusing solely on thechronology of events, including Microsoft's accounts of its private meetingswith state officials in Summer 2005, that Sun and ODF may have had someopportunities to rectify its license that Microsoft did not.

But, disputes over who said whatwhen or whether there was some form of illegal procedure involved are mooted bySecretary Eric Kriss' Sept.16 explanation to Microsoft regarding what it mustdo for its Office XML Reference Schema to be reconsidered as a Massachusettsstandard. With Quinn now on record as saying the door is still open, it'sclear that the ball is in Microsoft's court and that state is open tocontinued use of Microsoft Office.

Given the way so many othergovernments and businesses are sure to follow in Massachusetts footsteps (usingthe state's documentation of its decision as a template for their own),Microsoft has two ways to ensure that its Office productivity suite remains incontention for millions if not billions of dollars in purchasing decisions. Itcan either unencumber its license to the point that it meets Massachusetts'definition of open or it can choose to support ODF in Microsoft Office.

Whereas it's clear that Microsofthas no interest in the former, I asked Yates about the latter and, referring tothe working group within OASIS that worked on ODF, he said, "Sunand IBM didn't exactly invite us." But, in the Sept.16 meeting,IBM's Doug Heintzmann can be heard saying that Microsoft was not only invitedto participate, but that the invitation still stands and will stand inperpetuity. None of the Microsoft representatives in attendance stood up toargue the point. Heintzmann might as well have said "checkmate"as long as he had the microphone.

Given the way Microsoft just addedPDF support to Office-- something that Yates told me took Microsoft a year todo-- it is rather vexing that Microsoft is so fearful of adding ODF support aswell (it's free to do so). After all, if it added support for ODF, Microsoftwould not only pave the way for customers like Massachusetts to keep buying itsOffice suite, it would also put the choice of file format where that choicebelongs--in the customer's hands.

Microsoft doesn't even have toset ODF as the default format for saving documents. It can default to its own.If ODF is as inferior as its detractors say it is and it fails to satisfy endusers' needs, then Microsoft needn't be so insecure about supporting thespecification. Failing support of ODF, it will leave many like Massachusettswho want that option with no choice but to seek out Office alternatives.

When Massachusetts put the ball inMicrosoft's court so openly and publicly by giving it multiple ways toaddress the states' needs and continuing to leave those doors open, any andall claims of impropriety became non-issues. In addition to exposing the truecolors of all parties involved, the audio from the Sept.16 makes it clear thatthe Commonwealth of Massachusetts is clearly at the forefront of the openworld.

Not only has Massachusetts developeda modern and virtually unassailable test (particularly for its requirements sincevirtually every part of the test can be connected to the state's need forsovereignty), it has fully leveraged the democratizing forces of government andtechnology to arrive at an informed decision that serves the best interests ofits public.

Given the way this decision will berepeated in other halls of government, I think it can be said that Massachusettshas once again delivered a shot heard round the world. That Microsoft'sbiggest competitors were standing by the Commonwealth's side to help it pullthe trigger is evidence of why Massachusetts ETRM 3.5 truly was ground zero inone of this industry's biggest and most important battles.