Democracy and standards

David Berlind recently completed an interesting in-depth analysis of the events that led to Massachusetts decision to jettison Office XML from its approved list of document standards. Assuming the decision stands, this will affect a lot of people besides workers at government agencies.  Contractors that work for the state or regular citizens required to send digital documents to government offices will have to hew the line drawn by state technology authorities.

Berlind closed his analysis with the following paragraph:

Not only has Massachusetts developed a modern and virtually unassailable test (particularly for its requirements since virtually every part of the test can be connected to the state's need for sovereignty), it has fully leveraged the democratizing forces of government and technology to arrive at an informed decision that serves the best interests of its public.

Let it be said that I fully respect Berlind's opinion, his journalistic skills, and his integrity. I have long been one of his "fans," and read almost everything he publishes on ZDNet. That doesn't mean that I always agree with him, and today is one of those times that I don't. Over the course of this week, therefore, I'll outline why I disagree with some of the conclusions presented in his analysis.  I'll start today with his claim that the process demonstrated the power of democracy - and by implication, government - in the creation of software standards.

Let's step back a moment to remember that the central issue in this discussion is one of control over GOVERNMENT procurements.  When is the last time you looked to GOVERNMENT procurements and thought "if only I could spend my own money as efficiently as government spends its tax revenue?"

Governments have a very hard time efficiently allocating resources because governments aren't using the same procurement rules that businesses and inviduals use. Businesses and individuals are spending their own money. They worry about making that money go as far as possible.

Costs matter to businesses and individuals in ways they don't to governments. If you ever had any doubt about that, remember that government is also famous for purchasing $10,000 toilet seats or building a bridge in Alaska that goes nowhere.

Governments spend money "democratically," if "democratically" is read to mean spending it in a way that results in the least bad press or satisfies the interests of well-organized groups. Governments tend to respond to the loudest voice. That's why lobbying is such an important activity, and is also the reason political interest groups often have power out of proportion to their numbers.

Ask yourself why teachers unions in California can tie the Sacramento Legislature in such knots even though high deficits and poor test results prove that California is spending its money badly, or why farmers that represent tiny fractions of working populations in rich countries have convinced governments to pour huge sums of money into a vanishingly small sector? The loudest voice gets heard first, and few voices are as loud as California teachers unions or farm lobbies in Western countries.

The same principle applies in Massachusetts' battle over a technology standard.

Microsoft's Office XML specification was once on Massachusetts’ approved list (and I'll talk more about the details in future installments). That all changed, however, after a period of public comment, as explained on page 3 of David's article...

As pointed out earlier, Massachusetts Information Technology Division general counsel Linda Hamel noted that the state received a "firestorm of commentary" in response to the early 2005 MA ETRM draft that listed Microsoft's file formats as the supported "open" standard. According to various sources, the response was so opposed to the inclusion of a Microsoft format in what was otherwise an open initiative that it caused Massachusetts officials to take a deep breath and reconsider its test for openness.

Receiving bagfulls of email in response to a proposal to include a Microsoft technology on an approved list of document formats shouldn't surprise anyone that has participated in debates between proponents of open source and proprietary software. Open source advocates, a group who are are VERY well organized and have VERY loud voices, did the same thing to Unisys over it's LZW patent, to Kollar-Kotelly in comments relating to her proposed remedy to the Microsoft antitrust case, to the EU Commission in its battle over software patents, or on any other front in the war declared by open source on proprietary software. It really isn't anything unexpected.

What such feedback definitely does NOT do, however, is put Massachusetts in the position of a group flush with feedback similar to what developers get during the software usability testing process. That was the opinion of Dan Bricklin, co-inventor of the electronic spreadsheet. A few moments in the feedback section on Slashdot for an article that says anything remotely nice about Microsoft should disabuse anyone of that notion.

Of course, it wasn't just the open source movement that ensured an outcome that didn't include a Microsoft-backed technology. Another group which can be counted upon to oppose initiatives backed by Microsoft are competitors.  In other words, it doesn't surprise me in the least when the following happened:

Page 3: The meeting came on the heels of ODF 1.0's ratification by the Organization for the Advancement of Structured Information Standards (OASIS) and, according to Johnson, the tone of that meeting was that XML was the way to go and that virtually everyone in the room but Microsoft was behind ODF.

That meeting was attended by Sun, IBM, Adobe, and many others, groups in direct competition with Microsoft. They also backed a "standard" that accounts for a small fraction of the market. They have nothing to lose in doing so, simply because they have already lost in the wider market for Office products (PDF is popular...as popular as Word formats?). How better to undo that outcome than to back a standard that is used by very few?  Even if ODF doesn't acquire critical mass, it never hurts to throw confusion into a market already consolidated around a competitor's product.

A group of Microsoft's opponents backing non-Microsoft technology is like a council meeting with an audience filled with partisans of the losing party. Should it surprise anyone when that audience loudly opposes policies backed by the winning party? The resulting arguments aren't a promotional poster for the power of democracy, but the power of interest group politics.

Microsoft is a large company, and can certainly speak loudly. Every Microsoft press release is scrutinized by armies of pundits, and even if those press releases are spun in a negative direction, their voice is at least heard. That doesn't mean, however, that Microsoft can shout louder than the assembled voices of its opponents, nor alter the tone of an industry that views hatred of Microsoft as an entry requirement into the technology intelligentsia.

Berlind thinks the Massachusetts government has made a brilliant decision, in spite of the fact that it backed a technology that is mostly a no-show in a much larger democratic process (the global marketplace) that has standardized on Microsoft Office and its associated technologies. He thinks it reveals the power of democracy, even though that "democratic power" is the same force that creates laws banning interstate wine shipments (in order to protect local liquor wholesalers and tax revenues), or spends huge amounts of money to buy product from non-competitive local companies (the likely explanation for stories of $10,000 toilet seats).

In my opinion, governments should endeavor as much as possible to match the choices made by the larger marketplace, simply because that marketplace is likely to be fairer, and less skewed, than council meetings or government-assembled standardization committees can ever hope to be. The best way to do that is to let individual agencies make decisions based on needs. Given that government is supposed to be a FACILITATOR and not a TREND-SETTER, agencies that are effective will tend to use technology that its customers - the voting public - have chosen to use in their business lives.

If ODF swept the business world and became a new, de facto standard for digital documents, I would have zero difficulties accepting the new world order.  Where I have a problem is when special interests attempt to shoehorn a new standard onto an unwilling marketplace by playing the game of power politics.  

Government is both absolutely critical to the proper functioning of a market economy, as well as its biggest threat.  Massachusetts' attempt to redesign the shape of the marketplace by using its giant procurement club falls into the latter category. 

More tomorrow (or sometime this week).