The spreadsheet love affair

I never thought I'd go on a tear about Excel in these pages, but Josh Greenbaum's correct assertion that Excel is pretty much everywhere and is probably the software industry's most successful product provides the perfect foil. Josh concludes:

So, like the floppy disk icon that never dies, the Excel spreadsheet lives on and on, despite advances in technology that should have buried it a long time ago. This ubiquity and staying power says volumes about what users want from enterprise software, and their continued votes in favor of a 20-plus year old user experience should give everyone who believes that the best technology deserves to win a deserved pause. Excel works well-enough for millions of users all day long, and learning to live with it is a strategy that everyone, from CEOs to managers to software developers, needs to keep in mind.

I'm not going to disagree. As Josh alludes, there's very little point trying to roll rocks uphill. What Josh doesn't expose though are the risks that go with spreadsheet use.

Each year I pen a lament to spreadsheet use for my accounting colleagues, usually prefacing with some tale of spreadsheet woe. I have a bag full of them. Everything from the mortgage provider that overpaid some $270 million for a debt book, through to energy futures overpaid by $9 billion down to the $2 million a month interest calculation error. Heck, there's even an annual European spreadsheet risk conference. Ray Panko of the University of Hawaii has been tracking the issues for many years. On its website, the University notes:

Initially, spreadsheet research focused on errors, including typing errors, pointing errors, logic errors, and omission errors. More recently, regulatory compliance pressures have focused a great deal of attention on how corporations are doing financial reporting and other critical corporate processes. What they are finding is lots of spreadsheets, often hundreds arranged in manually-operated webs. Suddenly, spreadsheet error and security research is no longer "just academic."

It never has been 'just academic' but companies, CFOs and users seem so enamored of spreadsheets that their inherent dangers are often simply ignored.

I've always held the view that the spreadsheet was never designed for the sophisticated uses to which companies continue to put it. At best it is a development envronment that is rarely documented because users are not trained as developers. The net result is that when things go wrong, errors are notoriously difficult to find. What's more, there seems to be a fundamental lack of awareness around the extent of spreadsheet error. Some studies I've seen suggest it is as high as 95%. Panko's latest research asserts:

In general, errors seem to occur in a few percent of all cells, meaning that for large spreadsheets, the issue is how many errors there are, not whether an error exists. These error rates, although troubling, are in line with those in programming and other human cognitive domains. In programming, we have learned to follow strict development disciplines to eliminate most errors. Surveys of spreadsheet developers indicate that spreadsheet creation, in contrast, is informal, and few organizations have comprehensive policies for spreadsheet development. Although prescriptive articles have focused on such disciplines as modularization and having assumptions sections, these may be far less important than other innovations, especially cell-by-cell code inspection after the development phase.

The broader question then is why business continues to use this most elementary of tools instead of the wholesale embracing of specialist analysis tools and products? The only conclusion I can come to is that the spreadsheet is seen as convenient in a way that other applications are not and that the learning curve is sufficiently shallow for anyone to pick up the basics and do something useful. It's also cheap, often pre-installed on user machines at low cost in bulk deals.

In the meantime, the software industry continues to find ways of patching up an old horse that should, in my opinion, have been put out to grass a long time ago. And it doesn't go unnnoticed that the most popular topic of conversation on the UK's community site for accounting professionals is: the spreadsheet.