Auditor inquisition could determine ODF's fate in Massachusetts

Auditor inquisition could determine ODF's fate in Massachusetts

Summary: At a hearing yesterday in room 437 of the venerable State House in Boston, Mass., a Commonwealth Senate oversight committee chaired by Senator Marc R.

SHARE:
TOPICS: Open Source
45

At a hearing yesterday in room 437 of the venerable State House in Boston, Mass., a Commonwealth Senate oversight committee chaired by Senator Marc R. Pacheco opened yet another chapter in the ongoing saga of Massachusetts' Enterprise Technical Reference Model -- a blueprint for the state's information technology, authored by the state's IT Department (ITD). 

Andy Updegrove, legal counsel to OASIS (the consortium that published the OpenDocument Format standard), has published his stenographer's notebook of the proceedings. Dan Bricklin, who at my request brought his recording gear in the event that mine failed (I was one of few who were given permission to record the session), has made the audio available in three parts (as it turns out, his gear was better suited to the environment than was mine). 

In his blog, John Palfrey, executive director at the Harvard Law School's Berkman Center for Internet and Society sets the expectation that I'll leave no important quote from the event unturned, which I intend to do in the coming days (there was much to digest).  But for starters, I wanted to provide a higher level overview of what I observed.

Although the committee has seven members, only two -- Pacheco and Senator Richard T. Moore (who sat to Pacheco's left) were present when the proceedings began and Senator Moore, who only spoke up a handful of times, left part way through the hearing.  For a majority of the session which started at 1pm and lasted approximately 3.5 hours,  the room was packed with observers to the point that it was standing room only. 

As evidence of how central the Massachusetts decision is to the technology industry's balance of power, several of the largest vendors  -- IBM, HP, and Sun just to name a few -- were present at the hearing, as were the representatives of several industry lobbies.  Also present were several people with a variety of disabilities (some testified, some did not) and numerous advocates of open standards and/or open source.  Although the ongoing viabilty of Microsoft Office to the state's executive branch has been a central issue throughout the state's ETRM deliberations, no one known to be directly affiliated with Microsoft was present for the hearing.

As was expected, the primary focus of hearing was to explore the process that led to a key part of the ETRM blueprint -- the part that establishes OASIS' OpenDocument Format (ODF) as the standard XML-based file format that the state's executive branch must begin using on January 1, 2007 for the creation, editing, storage, and retrieval of the state's public documents. 

In a line of questioning directed at several state officials, Pacheco raised questions about the constitutionality of the decision, the authority that the ITD has to establish such standards, to what extent the ITD was obligated to include others in the decision, and, subsequently, to what extent it fulfilled those obligations.  Although I'd say that "process" was Senator Pacheco's chief concern, the deliberations also included some significant vetting of the wisdom that went into the ODF decision as well as a discussion of the potential costs and benefits should the state move forward with its plan. 

As those invited to testify were dismissed, it was clear from Pacheco's comments that the matter would remain open for investigation.  He asked that written copies of the testimony be forwarded to his office and tasked the state's auditors with the job of combing through everything there was to comb through with a fine tooth comb to better understand every knob and lever that was turned or pulled in reaching the decision.  Except for the state's CIO Peter Quinn and ITD general counsel Linda Hamel who, sitting side-by-side before Pacheco and Moore testified first, the testimony was decidedly unfavorable to both ODF and the process that preceded ITD's decision to select it as a statewide standard. In fact, after Quinn and Hamel took their seats in the gallery, I don't recall anything being said in the decision's favor.

After Massachusetts Office on Disability director Myra Berloff testified that her office was not consulted during the decision making process, the decision took a drubbing from two men with disabilities -- Gerry Berrier (Bay State Council for the Blind) and John Winske (Disability Policy Counsel).  The concern is that, if enforced, the new policy will result in a step backwards when it comes to the accessibility of computers to state employees with disabilities (an issue I will cover in painstaking detail in another blog).  During the hearing, Pacheco embarked on several digressions, some of which admonished Quinn and Hamel for not adhering to the collaborative spirit of open source by including more people in the decision making process.  Pacheco was also noticeably disappointed by how the decision may have been rushed, citing numerous requests to slow the process down and give others more time to comment on the decision.

Given how lopsided the testimony was, some of those in attendance who favor ODF, including Updegrove, left the hearing feeling as though the deck was purposely stacked against Quinn and Hamel (Updegrove says as much in his blog).  Whereas such a hearing couldn't take place without hearing testimony from Quinn and Hamel, the rest of the invitations to testify were clearly issued to those who are dissatisfied with ITD's decision. Whether or not that was by design, I cannot say. The Senator did not publish the criteria for his selections (verbal testimony was by invite only) ahead of time.  Several pro-ODF parties volunteered to testify but were denied the opportunity and submitted written testimony instead.  

As I left the hearing, and knowing what I know about open standards and open source, I came away with several impressions.  The first of these is that the ODF decision in Massachusetts was a very complicated one that cannot possibly be fully-vetted in the course of a single four-hour hearing.   Four hours is just enough time for enough people to say some things but not others in a way that only part of the truth is revealed, and in a way that certain misconceptions get to be aired without opportunity for rebuttal.  For example, during yesterday's proceedings, Senator Pacheco used the term "OpenOffice" where he should have used "ODF" as though the two are interchangeable and they are not.  There was not sufficient opportunity for anyone to articulate the difference between the two and why that difference is so important to the deliberations at hand. 

At one point, Quinn went unchallenged when he said that IBM's Lotus Workplace was one of several ODF-compliant solutions on the market today and it is not.  IBM's Lotus Workplace Managed Client -- the part of IBM's Workplace offering that will support ODF -- is a product that IBM is currently testing with a handful of customers and it could be said that, by virtue of that testing, it is still in its beta phase.  In other words, it is not yet shipping. 

Another impression that I left with is that the ODF decision could end up scuttled for two reasons. First, much the same way that a person caught red handed committing a crime can get off on a technicality -- for example, improper application of his or her Miranda rights -- the ODF decision could also be blown up on a technicality as well. Whereas Microsoft has raised questions about the fairness of the process, Senator Pacheco never really brought that up as an issue.  Instead, questions about constitutionality, authority, and inclusion of other state officials where they potentially should have been included -- questions that I as a technology journalist cannot even begin to address -- must now be thoroughly considered. 

What I can say is that during the hearing, it appeared as though Senator Pacheco and ITD General Counsel were disconnecting on the scope of some of the questions.  For example, Pacheco would ask a question about ITD's authority to  make such a sweeping decision and Hamel would essentially agree that ITD doesn't have that authority, but that the decision wasn't as sweeping as it was characterized to be.  Later, similar questions would be asked as though that disconnect was not resolved.   The entire four hours could probably have been spent establishing a foundational understanding on which to base more questions and testimony. 

At this point, it's impossible to say whether there was some sort of violation in process.  What's clear is that enough doubt has been raised about the process to require a more probing investigation.  If there's a silver lining to that cloud, it's that the policy setters in other governments (state, local, federal, etc.) that are considering similar types of decisions -- the ones who are watching the Massachusetts affair very closely -- will incorporate better i-dotting and t-crossing into their playbooks so as not to have their decisions subject to a similar inquisition over technicalities. 

The second reason that ODF could end up scuttled has to do with Peter Quinn and Linda Hamel themselves.  Although I have interviewed Quinn, this hearing was the first  time I had the chance to shake their hands.  Although I didn't ask, neither strikes me as a politician.  As stoic as they were while testifying to Pacheco and Moore -- and they were very stoic --  I couldn't help but think "What would happen if either of them decided for personal reasons that this fight isn't worth the personal sacrifice that it has so far been?" Although they were questioned with respect, it was clear from the beginning that Quinn and Hamel were not exactly in the good graces of Senator Pacheco.  I can't imagine that Quinn or Hamel ever envisioned or hoped for a day like yesterday.  Given that they are now the lone champions of the ODF decision in Massachusetts, they will need to have the stamina and the courage it takes to stay in the fire.  I'm not saying that they don't.  I glanced their way multiple times during the hearing and never once did I see a "What the heck am I doing here?" look on their faces.  But if either one steps down, ODF's future in Massachusetts will end up in serious jeopardy.

Finally, my last impression -- as a Massachusetts resident -- is that when people immerse themselves in the trees as has been done here, sight of the forest can at times be lost.   Perhaps some parts of the processes could have been better executed.  Perhaps the costs and benefits should have been detailed in an unassailable 500 page study (and now, because the state auditors are involved, it may).  Perhaps when it comes to implementing ODF, January 1, 2007 is simply an unrealistic date for those with disabilities to begin working with as-of-yet unproven solutions (unproven in terms of how accessible they are to those with disabilities).  Ultimately however, much the same way a killer can get off on a technicality in prosecution, it would be a terrible miscarriage of information technology if the bigger goal for Massachusetts gets lost in politics. 

Guaranteeing the accessibility to the state's public documents in perpetuity --  for all of the state's citizens and all of its employees -- is a noble cause that can only be served by an open standard (regardless of what that standard is).  Today, there are certain public documents in Massachusetts that are not as freely accessible as they should be (a situation that is not exclusive to Massachusetts).  Ultimately, after the processes have been vetted, and the costs and benefits are better understood (both short and long term), and those with disabilities have been assuaged (perhaps by more inclusion and different deployment dates), a decision on how to improve the current situation will have to be made and the definition of open and how openness can best serve the people of Massachusetts will once again be the most important questions. 

In the next few days, I will attempt to ask an answer some of the same questions that those now charged with examining the ODF decision will be studying as well.

Topic: Open Source

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

45 comments
Log in or register to join the discussion
  • Good for MA.

    The good news is:
    Ultimately, after the processes have been vetted, and the costs and benefits are better understood (both short and long term), and those with disabilities have been assuaged (perhaps by more inclusion and different deployment dates), a decision on how to improve the current situation will have to be made and the definition of open and how openness can best serve the people of Massachusetts will once again be the most important questions.

    And when (a larger group of) MA officials have to make "a decision on how to improve the current situation" and define "open and how openness can best serve the people of Massachusetts", then I expect that the deliberations will be easier to approve.

    The chance for open source advocates to control the decision will have vanished. And that too is good for the State.
    Anton Philidor
    • Sigh. ODF # Open source. n/t

      n/t
      mosborne
      • Arguing open source advocates don't prefer ODF? (NT)

        .
        Anton Philidor
        • As opposed to...

          Microsoft's OpenXML? Certainly not. I believe open source advocates would much prefer ODF to Microsoft's OpenXML. Why would that be? Simple. ODF is licensed such that everyone, including open source and Microsoft, are free to implement it and compete based the quality of the implementation instead of the format itself.

          Microsoft's OpenXML license on the other hand is incompatible with any open source license which requires sublicensing. The most well known of these would be the LGPL and GPL. Microsoft is well aware of this.

          Microsoft's claims of openness are insincere. Their disdain of the LGPL/GPL licenses are well documented. The OpenXML license is perpetual and royalty free so there is no direct revenue stream. If Microsoft truly wanted to OpenXML to be open then they would offer the same patent release as Sun has done for ODF.

          Microsoft doesn't want OpenXML to be open. They want to be able to control it and be able to re-define it at will without any outside consensus. By doing so, they reserve many competitive advantages to only themselves. For example, no else can implement OpenXML until it is fully defined, but only Microsoft can issue that definition. The current specs are not complete (check their website). When will they be? After Office 12 is released to market? Makes perfect sense from Microsoft's point of view. Why help the competition?

          Everyone had equal access to the ODF specification as it was built, including Microsoft. Interested parties were free to join the technical committee and help guide the specification. Again, including Microsoft. That exists today as well since ODF is still evolving. The problem is that Microsoft doesn't want to join the team, they want to *be* the team.

          I don't blame them for wanting control, but they shouldn't be calling their format OpenXML because open it definately isn't.
          mosborne
    • really?

      ODF is used by two commercial packages, and Microsoft have commissioned an import filter.

      If MS XML is used instead of ODF, the only Authoring tool MA can use is MS Office. No other commercial package could compete.

      Do you think this would be right?

      The politicians said that ODF was "unproved" and that MSOffice was widely used. Except that the relevant version of MSOffice is not used at all, and definitely not proven since it DOESN'T EXIST YET (you need to wait until 2006).

      I think this is frankly totally incredible. The only sane solution bearing in mind MSXML's authoring restriction is to use ODF.

      This is based the predicate of a format that can be authored by any commercial package rather than being locked into one supplier.

      Explain you logic behind having a different view, I simply can't see it.
      hipparchus2001
  • Disabilities

    You can bet your sweet britches that whatever software the disabled demonstrator was using to read a document will either annonce support for odf and or a substitute will be built before they have a chance at another meeting. I would go so far as to say that when they do show this new assesibility feature that it will actually work unlike the one demonstrated during the meeting.

    The open source community has unlimited knowledge, resources and agility to make something like this happen extremely quickly.
    jjanks
    • Bet ?

      What makes you so certain ? There are many features in a lot of Open Source projects that I had wished (and sometimes voiced) to be part of it and has been ignored.

      So unless you're going to write this 'missing pieces' how can you be so sure that a usable solution will appear before the 2007 date ?

      Whose cool-aid you been drinking ?
      JJ_z
      • But you can always add it...

        if it takes off, all the better.
        If noone likes it, well, that's the evolution of it.
        Tha's why it's open.
        el1jones
  • That's funny

    "Given how lopsided the testimony was, some of those in attendance who favor ODF, including Updegrove, left the hearing feeling as though the deck was purposely stacked against Quinn and Hamel (Updegrove says as much in his blog). Whereas such a hearing couldn't take place without hearing testimony from Quinn and Hamel, the rest of the invitations to testify were clearly issued to those who are dissatisfied with ITD's decision. Whether or not that was by design, I cannot say."

    Isn't that pretty much the same claim that Microsoft and others have been making -- that the process was "stacked against" them from the start, and that those involved in the decision-making process were mainly Microsoft's competitors?

    Carl Rapson
    rapson
    • carl, by stacked against them...

      do you mean because MS didn't want to offer what they were looking for? Do you mean that because the proposal of what was wanted and ignored by MS considered stacked? I don't see how anything was stacked against MS. They were given what was wanted by the ITD, MS chose to offer their own decision on what they needed, and even after all of their tries, didn't fit that proposal originally published to the vendors. I know you're usually somewhat partial, but this time you are wrong. MS knew what was wanted, they simply chose to ignore it, so they were denied the account. Let me tell you this, if i were given the needs of one of my clients, no matter the size, i would go by exactly what they want, not by what i believe they really need. It's their mistake to make, they are the ones paying for it. MS could have drawn up their proosal the way MA wanted, then had a list of things thay could possibly happen in the future to show MA. Instead they decided to just go their own route. Common sense when dealing with a customer, follow their criteria, then offer ideas as to what may help them along side their respective needs.
      Monkey_MCSE
      • Nope...

        ...you're reading too much into my comment. By "stacked against them", I meant that was Microsoft's (and some supporters') claim, nothing more. I made no evaluation as to the validity of such a claim, as I do not make any evaluation of the claims of 'MS bribery' going on in relation to the current action. I just thought it was funny that David poo-pooed the idea that the system was stacked against Microsoft, even considering how most participants in the process were demonstrably anti-Microsoft and the process itself seemed to magically change along the way, virtually ensuring that Microsoft could not "win". Yet here seems to be the exact opposite situation, and David raises the exact same questions that were raised previously.

        I just find it ironic, is all. Don't read anything into my post that isn't there.

        Carl Rapson
        rapson
        • Reading stuff into posts that isn't there

          [i]Isn't that pretty much the same claim that Microsoft [b]and others[/b] have been making -- that the process was "stacked against" them from the start[/i]

          Who are the others? Was there another competing proprietary format or do you mean the other Microsoft supporters? Also, by implication, if the process is stacked in favor of Microsoft, which it would have to be if the requirements favored the format presented by the biggest supplier of office productivity software, wouldn't the process be stacked against everyone else?

          I guess my question is, how could the proposal have been stated to invite both a proprietary, not completely documented format for which no applications exist to use it, and an open format for which a complete specification is available to all to produce applications that can both read and write data in the format free of any license restrictions?
          Taz_z
          • Here

            By "others" I'm referring to several regulars here on ZDNet. I realize that the posters here aren't necessarily representative of the world at large, but then again, how many people outside of MA and the IT world really know much about this, or even care? I would imagine that even most MA citizens aren't aware of what's going on.

            As for your other questions, I don't claim to have any answers. I have no doubt that it will eventualy be resolved, albeit to someone's dissatisfaction. But that's always the case with political decisions. I have bigger things to worry about than whether or not Office can be used in Massachusetts, or whether or not MS adds ODF capabilities to Office.

            Carl Rapson
            rapson
        • Wasn't stacked against MS

          It is not possible by any stretch to say the process was stacked against Microsoft. Microsoft didn't lose, their XML format did. Stacking, if it occurred could only have been against the Microsoft *format*, not the company itself.
          mosborne
        • Re: Nope...

          [i][T]he process itself seemed to magically change along the way, virtually ensuring that Microsoft could not "win".[/i]

          What was magical about it?


          :)
          none none
      • MS and whiney arrogance.....

        Gotta love Microsoft. The government says "here is what we want". Microsoft says "no, here is what you will take and if you don't, we are going to whine about it like you wouldn't believe".

        Rather than just give MA what it wants, Microsoft continues in its typical arrogant ways and wants to dictate what is best for the customer. I for one hope Microsoft gets spanked badly on this one. Hard to say though. Guess it depends how much of a political frenzy they can generate and how many campaigns they "contributed" to.
        shawkins
      • BTW...

        "I know you're usually somewhat partial..."

        Not true. What I do is ask questions, admittedly sometimes playing "devil's advocate", but always with the intention of opening the discussion to alternate views (something many here seem reluctant to do). Seldom do I come down on any side in a debate. Others tend to categorize me according to their own biases, but that's their fallacy, not mine.

        If you check I think you'll find I haven't "chimed in" on this particular topic before this, on either side. In point of fact, I happen to disagree with Microsoft's stance on this - I see no reason why MS shouldn't add ODF capabilities to Office. There just has been no reason for me to "jump in" before now, as others have said it much better than I ever can.

        Carl Rapson
        rapson
      • Who decided what "MA wanted" in proposals?

        A few MA officials decided that Microsoft should be excluded. Among the points raised at the hearing are whether those officials should have been the only ones making the decision and whether the criteria considered were complete.

        The doubts raised show that there's a question whether the decision was made by the "customer", the State of MA, or by a few officials who behaved erroneously by not following expected and mandated procedures, let alone the factors they chose to consider in making their decision.

        Interesting to me that the two officials who testified were in an atmosphere that made them seem culprits. The question was less Why did you make that decision than How could you do that?

        These are, by the way, people with careers to pursue. This may not be a comfortable position for them.
        And Mr. Berlind's check on their expressions shows how well he recognizes that the two officials who testified were alone. He wonders what would happen if either (or both) left.

        Does that sound to you like the State of MA, the customer, resolved to follow a decision?
        Anton Philidor
        • Flaming, aren't we?

          Your subject title is interesting. I'll assume that you don't really believe that extremely large vendors of proprietary products should be consulted in order to find out what they would like to see in their potential customers' proposals. However, I am really curious about your first sentence:

          [i]A few MA officials decided that Microsoft should be excluded.[/i]

          First, I will assume you can find quotes attributed to "MA officials" that said Microsoft should be excluded. Second, I will also assume you can find anyone associated with MA in some capacity who said Microsoft should be excluded. I seem to remember reading or hearing that Microsoft was more than welcome to be considered if they would bring something to the table that met MA's requirements. Hell, even Carl agrees MS should just support ODF.

          [i]The question was less Why did you make that decision than How could you do that?[/i]

          Well, of course, the people doing the questioning are simply not capable of understanding the why or asking the right hard questions. They didn't know a file format from an application or that a particular IBM product didn't support ODF.

          [i]Does that sound to you like the State of MA, the customer, resolved to follow a decision?[/i]

          Hard to say now, this angle is just starting. I'm wondering where the 2 state officials pushing this now were while all of this was going on. I'll guess they couldn't have cared less until a Microsoft representative brought it to their attention. I think we have to wait and see now what approach is taken. Will they try to find some technically respected people to speak out for them against ODF and for Office XML? Will they try and change the rules by rewording the proposal to give Microsoft more of a chance? I am interested to see if they are going into any technical aspects at all and how they are going to counter the technical, licensing (or lack thereof), and economic advantages of going with ODF over XML. Even John Carroll's best argument is that Microsoft should be considered as a provider simply because they are the biggest.
          Taz_z
          • I did not expect so.

            My title referred to the fact that the State of MA did not make a decision about acceptable formats. The decision was not made by any of its branches of government, and not with extensive contributions from executive branch agencies.

            Instead, several MA officials made the decision.

            So any discussion should, I believe, refer to a decision by MA officials and not by the State of MA. This is about the exercize of discretion by those individuals, nothing more.


            You wrote:
            I seem to remember reading or hearing that Microsoft was more than welcome to be considered if they would bring something to the table that met [the] MA [officials]' requirements.

            This does show that Microsoft has been excluded, whether validly or not.


            In connection with the tone of the questions, you wrote:

            Well, of course, the people doing the questioning are simply not capable of understanding the why or asking the right hard questions. They didn't know a file format from an application or that a particular IBM product didn't support ODF.

            Are you sure you meant "simply not capable of understanding"? That's worse than anything I've said about the MA officials.

            And you wouldn't draw such conclusions about "the people doing the questioning" on the basis of lack of knowledge of technical details, would you?
            The fact that an IBM application is still in beta seems far less important than whether the procedure used in making a decision followed the rules in MA.

            I think we've agreed that the decision will be revisited. And that's significant.

            You wrote:
            I am interested to see if they are going into any technical aspects at all and how they are going to counter the technical, licensing (or lack thereof), and economic advantages of going with ODF over XML.

            I'll take that as looking for an opinion.

            I think they'll stay away from judging as much as possible. The eventual language will leave it to the agencies concerned, within guidelines based on goals and intentions and identified quoted mandates.

            The battle will be fought at a lower, less public level.

            The justification is that circumstances change, and documents in force should not have to keep up with changes in products, markets, laws, etc.

            That actually makes sense.
            Anton Philidor