Stop brain-dead software testing!

Lousy software testing damages many IT projects. When testing doesn't uncover obvious problems that prevent software from operating as users expect, then failed projects are inevitable.

The consequences of brain-dead testing can be severe. For example, clothing retailer J.Crew recently had serious customer relationship problems when it deployed poorly tested software:

J.Crew reported weaker than expected second-quarter earnings due to severe problems following a website and call center upgrade. The fiasco caused cost the company $3 million in addition to lost sales and dissatisfied customers.

• Related: J.Crew: Failed upgrade hits financial performance

Last April's massive IT failure at Heathrow Airport's terminal T5 was also caused by brain-dead testing. In a moment of understatement adding insult to injury, British Airways CEO, Willie Walsh, commented:

We are working hard to tackle the difficulties we have had with the terminal’s baggage system. From time to time problems have developed that were not encountered during the extensive trials.

• Related: IT failure at Heathrow T5: What really happened

For another example of testing madness, look no further than Georgia Blue Cross and Blue Shield. Last August, the company wrongly released 200,000 medical records into the wild because of poor testing. A spokesperson told me:

There was a system change that was not comprehensively tested. We have already made changes to prevent it from happening in the future.

• Related: Failed IT causes major Georgia Blue Cross health privacy breach

Click for five important tips to improve implementation testing. -->

From a testing point of view, the question relates to unforeseen failures, or failures that reasonable and adequate testing failed to identify.

• Prioritize what your software does. Pick out the most important features that offer the most value to your end user.
• Identify the impact of unforeseen bugs and failures in those high priority components. What could possibly go wrong if those components failed or acted incorrectly?
• Decide if it actually matters. Not everything out there deserves herculean planning efforts for failure management.
• Once you know what matters and what doesn’t, plan for the impact and not the failure. That is, plan for the outcome of the failure and not the failure itself. In the event of a nuclear meltdown, nobody cares what you do with the unforeseen thrown exception that halted the control systems. They want plans for stopping a catastrophe.
• Work with the customer to make sure they understand failure impacts and how to manage them appropriately.

The comments present a balanced, business-oriented view on software testing. Since absolute perfection is a practical impossibility, examining priorities from a user perspective is correct.

On the other hand, all this assumes the organization has really set aside time for consistent testing. If your team doesn't perform basic levels of systematic testing, then your project will fail.

[Image via dst-corp.com. Erik Hollnagel's homepage.]