Tug-o-war over Mass Bay Charter: Harbinger of things to come?

While reading yesterday's Boston Globe, I read with great interest a story about the tug-of-war over the Massachusetts Bay Charter.  Written in the early 1600's, the Massachusetts Bay Charter came from the throne of England and gave the settlers rights to create their own government.  You can't read the story online unless you're a Globe subscriber (a bad idea which has been beaten to death -- usually to no avail -- in the blogosphere).  The State must take extraordinary measures to minimize any and all barriers to permanent document accessibility. The tug-of-war involves the current curator of the document -- the Salem Athenaeum -- and how it plans to auction the document to the public.  According to the Globe's story, the document could fetch more than $10 million. 

Meanwhile, now that there's a chance that such an important public document could fall into the hands of a private citizen, all sorts of alarms are going off (and stories are being written).  Why write about it here? Because there's relevance to the OpenDocument Format controversy that has still yet to completely resolve itself here in the Commonwealth.  I'm not going to rehash the details of the controversy other than to say that I agree with those who see this important historical document as a document that should now and forever be accessible to the public.  It represents one of the most defining moments for the state as well as for the country and for it to end up on the wall of someone's house (I'm exaggerating; it needs special storage systems to properly preserve it, but you get the picture) would be a travesty.  Simply put, this is not a proud moment in Massachusetts' recordkeeping history and it's not a history that should be repeated regardless of the relative triviality of the documents being generated today vs. that of the Massachusetts Bay Charter.  To add an element of surrealism to the story, in an extremely strange robbing Paul to pay Peter twist of irony, the Salem Athenaeum has said that it will use the proceeds to repair its collection of crumbling books. 

What lessons can be learned from this?  Is there a point in the future where written works -- private or public --  will be irretrievably lost?

So, looking at the State's existing policies for electronic storage and retrieval of its public documents, it's clear that in order to keep history from repeating itself -- in other words, to keep public documents from becoming inaccessible to the State's citizenry in perpetuity -- the State must take extraordinary measures to minimize any and all barriers, technological or otherwise, to permanent document accessibility.  The same should be true for every local, state, and federal government and there's a limited amount of room for commercial interests in the issue.  It's not about money. It's about history and free society.   The Massachusetts Bay Charter is nearly 400 years old.  Are we foolish enough to believe that any of today's file formats or their purveyors will be around in 400 years? These values inevitably lead us back to the debate of what is open and what is not.  But how about what is transparent and what is not?

Today, there are two office file formats competing for the hearts and minds of governments.  Both can trace their roots back to specific products whose code was primarily overseen or developed by one software vendor. Although Microsoft's formats are are behind in the race to become ISO-ratified standards, both formats (Microsoft's and the OpenDocument Format) have ultimately found their way into the stewardship of a multivendor consortia.  The OpenDocument Format is at OASIS and Microsoft's Open XML format is at Ecma.  Backers to both formats have pulled out all the stops more recently forming additional promotional alliances to drive adoption.  Last month, backers of the OpenDocument Format led the formation of the ODF Alliance. Then, while I was out getting back surgery, Microsoft led the formation of openxmldeveloper.org which launched with 39 members including Florida's State House of Representatives. Proving how important this beachhead is in the industry, the organization's formation was announced by Bill Gates at Microsoft's Office Developers Conference. Worth a bit of hype-busting was some text in the press release that says that the Open XML file formats were developed by Ecma.  Says the release:

The Open XML Formats Developer Group is being formed as a community for developers to exchange information with each other regarding the usage of the Ecma-developed Office Open XML file formats. The community will serve as a technical resource for Open XML developers to submit and answer technical questions and to share tools and ideas around Open XML Formats-based solutions. 

To say the formats were developed by Ecma is almost as laughable as saying that the OpenDocument Format was developed from the ground up by OASIS.  Refined by OASIS?  Sure.  Developed from the ground up?  Not. Is there a later date at which either specification may have more consortia blood in them than not?  Maybe.  And thus my comment about transparency.  Assurance to governments that their electronic documents will now and forever be available to their peoples lies directly in the stewardship and public domain nature of the specifications in question.  From a legal perspective, open source code is often confronted with the same challenge and in response, open source developers have improved their diligence in developing code that's free from potential proprietary ties.  It's not enough for governments to know that the specifications in question are open enough to the point that they can take over their development should they one day be left to whither on the vine while hundreds or thousands of public documents remain encoded in those formats.  Governments need to know more than the fact that the formats are open and stewarded by organizations with lots of paid vendor members who can often be little more than silent partners or sycophants.  They need to know that those members are not just behind the specification, but actively involved in their development so as to assure all customers that, at the heart of the matter, there aren't just one or two driving forces under the hood.   When strong, vibrant developer communities are truly and actively engaged in specification development, that's what gives those specifications their real sea legs and that's what gives governments as well as other organizations the assurances they need in order to identify the least riskiest direction to go.

I am not condoning or condemning either format.  What I am saying is that for the backers of either one to better make their case now and in the future, they must be prepared to very clearly demonstrate how the specifications are truly the work of an actively engaged community rather than specific vendors with undeniable motive.   I'm not sure how something like this is done.  But the innuendo, embellishments, and exaggerations do nothing more than to undermine trust.  At some point, the specifications and the processes behind them must do the real talking and speak for themselves.