Testing times for open source

One of the problems people who wear ties have with open source is the question of why anyone bothers to write it. Linus Torvalds famously said that if ten people collaborate on a project for an hour, everyone gets back nine times more work than they put in -- a return on investment that should satisfy the most avaricious suit., There's more to it than that, though: the very human desire to create something of worth and be credited with it is highly motivating, and open source software is much better at this than work hidden behind the proprietary shield.

Yet software isn't just about creativity. The stuff needs to be tested -- and as anyone who's been involved will testify, this is unglamorous grunt work done to pay the rent. Open source is notoriously bad at getting people to do the dull bits and spectacularly underpaid: combine that with the anonymity of testing and you have a recipe for dangerous lacunae in the production process. The alternative to rigorous testing during design and implementation is debugging in the field -- while more fun and better for the ego, it's not normally seen as beneficial in business critical systems.

Proprietary systems aren't immune either. Even in the command economy of a typical commercial software company, testing tends to be the poor cousin when it comes to resources and staffing. Typically, it's left until last, under-resourced and under-managed for most of the design cycle then over-resourced and over-managed during that last panicky stage. Even when lessons are learned, they are kept inside the company and then often forgotten. Who now remembers the experiences of Y2K?

This is where open source can turn the problem into an opportunity. Testing works badly because the whole software design process is badly engineered: companies have the option of patching the issue by throwing money at the weak spots, and thus are less likely to take the time to step back and consider the conceptual and human aspects at the heart of the problem.

Open source does not have the luxury to overlook this. It must find ways to create a distributed testing regime that produces reliable results from people who find it an involving and satisfying process. By thinking about the human factors first, open source can invent ways of approaching this problem which will be valuable for everyone involved in the engineering of complex systems. Innovation here may have more important and more profound effects than merely writing better software, and it deserves serious attention.