in reality, its not usually software testing that's the problem. it's the project stakeholders who don't plan enough time to do adequate testing. it's the programmers who go way over their time estimates. it's the program managers (or similar role) who decide to ship a project even though it's not ready, and fix bugs later. it's the scope creep without schedule adjustments. it's companies who don't realize the value software testing gives.
In short, testing gets the shaft because of poor planning, poor resource management, and an unwillingness to delay projects or cut features when issues are found.
IT Project Failures
Michael KrigsmanStop brain-dead software testing!
Summary
Lousy software testing damages many IT projects. When testing doesn’t uncover obvious problems that prevent software from operating as users expect, then failed projects are inevitable.
Topics
Blogger Info
Michael Krigsman
Biography
Michael Krigsman
Michael Krigsman is CEO of Asuret, Inc., a consulting company dedicated to reducing technology implementation failures. Asuret's suite of software tools improve the success rate of enterprise software deployments by quantifying and measuring governance issues that cause most project failures. Michael led the research effort underlying Asuret's model of collective intelligence and its practical application to reducing IT failures in consulting environments. He is a recognized authority on the causes and prevention of IT failures and is frequently quoted in the press on IT project and related CIO issues. He is considered an enterprise software industry "influencer" and provides advice to technology buyers, vendors, and services firms.
Previously, Michael served as CEO of Cambridge Publications, which develops tools and processes for software implementations and related business practice automation projects. Michael has been involved with hundreds of software development projects, for companies ranging from small startups to Fortune 500 organizations. Michael graduated with an M.B.A. from Boston University and a B.A. from Bard College. He is a Board member of the America's Cup Hall of Fame and the Herreshoff Marine Museum in Bristol, RI.
Lousy software testing damages many IT projects. When testing doesn’t uncover obvious problems that prevent software from operating as users expect, then failed projects are inevitable.
The consequences of brain-dead testing can be severe. For example, clothing retailer J.Crew recently had serious customer relationship problems when it deployed poorly tested software:
J.Crew reported weaker than expected second-quarter earnings due to severe problems following a website and call center upgrade. The fiasco caused cost the company $3 million in addition to lost sales and dissatisfied customers.
- Related: J.Crew: Failed upgrade hits financial performance
Last April’s massive IT failure at Heathrow Airport’s terminal T5 was also caused by brain-dead testing. In a moment of understatement adding insult to injury, British Airways CEO, Willie Walsh, commented:
We are working hard to tackle the difficulties we have had with the terminal’s baggage system. From time to time problems have developed that were not encountered during the extensive trials.
For another example of testing madness, look no further than Georgia Blue Cross and Blue Shield. Last August, the company wrongly released 200,000 medical records into the wild because of poor testing. A spokesperson told me:
There was a system change that was not comprehensively tested. We have already made changes to prevent it from happening in the future.
Click for five important tips to improve implementation testing. –>
Disclosure
Michael Krigsman
Michael Krigsman writes and speaks about technology in a manner that most observers consider to be fair and balanced. Michael believes that writing about IT failures, which often have complex causes, creates a unique obligation to be reasonable and accurate in both reporting and analysis.
Michael maintains active personal and professional relationships with enterprise technology buyers, vendors, analyst firms (or individual analysts), consultants, and system integrators. As CEO of Asuret, Michael sells and delivers paid services to members of these same groups.
Vendors regularly reimburse Michael's out-of-pocket travel expenses to attend industry conferences and events. Conference organizers frequently waive entry fees when Michael attends industry events. Michael often speaks at industry conferences and events.
He is a member of the Enterprise Irregulars, a loose association of consultants, investors, industry representatives, analysts, and users of enterprise software.
For daily updates on Michael's activities, follow him on Twitter.
Biography
Michael Krigsman
Michael Krigsman is CEO of Asuret, Inc., a consulting company dedicated to reducing technology implementation failures. Asuret's suite of software tools improve the success rate of enterprise software deployments by quantifying and measuring governance issues that cause most project failures. Michael led the research effort underlying Asuret's model of collective intelligence and its practical application to reducing IT failures in consulting environments. He is a recognized authority on the causes and prevention of IT failures and is frequently quoted in the press on IT project and related CIO issues. He is considered an enterprise software industry "influencer" and provides advice to technology buyers, vendors, and services firms.
Previously, Michael served as CEO of Cambridge Publications, which develops tools and processes for software implementations and related business practice automation projects. Michael has been involved with hundreds of software development projects, for companies ranging from small startups to Fortune 500 organizations. Michael graduated with an M.B.A. from Boston University and a B.A. from Bard College. He is a Board member of the America's Cup Hall of Fame and the Herreshoff Marine Museum in Bristol, RI.
More from “IT Project Failures”
Talkback Most Recent of 15 Talkback(s)
-
tester_guy10/03/2008 09:27 AM -
ZDNet Blogger
Fair point
Although not always true either. It's difficult to generalize because every possibility under the sun can come into play.
mkrigsman@...10/03/2008 09:55 AM -
Sometimes It's Not Project Management or QA
Once upon a time, I worked at a company as a project engineer where the sales people frequently didn?t pass sales orders down to application engineering for 2-3 months (no one knows why). So, sitting in a planning meeting with my boss, my counterparts, and other middle managers from the other relevant departments, we would plan out when customer delivery would occur. And to everyone?s horror, we would project, for many orders, a delay of one quarter.
Solution? Go back and make wild assumptions as to when things would have to occur to come within a few weeks after the target FAT date.
Of course, being new at this and na?ve, I would ask if this was really feasible. Upon leaving the meeting, my boss reprimanded me saying "By your question, you just called everybody a liar."
Result? When the customer came for FAT, we would have to jury rig the system to appear to be "working" and then pray we could get it working for SAT. Of course delivery was way late, and we ended having to do both development and testing onsite at the customer?s site (not pretty). Project manager, as always, is the one holding the bag. Customer, as always, was way pissed off.
Moral. Sometimes the problem is not the project managers or QA.
elizab(Edited: 10/03/2008 03:20 PM) -
Amen!
Agreed. Just read an interesting article highlighting the fact that 40% of CIO's are indifferent about the QA of their own software.
www.testcommon.com10/08/2008 12:14 PM -
Not that easy
Concentrating on the "key" issues is all well and good but its usually some "little" thing that causes a "big" failure.
No_Ax_to_Grind10/03/2008 11:23 AM -
Sign at Software Engineer firm reads...
SWEAT THE SMALL STUFF
User0773410/05/2008 07:47 AM -
RE: Stop brain-dead software testing!
Michael, thanks for your kind comments! Your post gave me some more food for thought, especially about getting the most "value" for your testing time. Software testing is in many ways all about finding bugs, not proving perfection.
The problem is that with so much variability in platform configurations, it's getting harder and harder to find all of the bugs with every possible setup.
At uTest, we're really trying to make it easy to throw a broader net and catch more of these kinds of bugs. For one of our customers, we discovered that their installer wouldn't run in certain foreign language configurations. This was a big problem because their product was targeted to non-English speakers. We saved them a lot of time and money not having to fix the problems later.
Good testing doesn't have to be difficult, it just takes strategy, discipline, and customer focus.
spchampion10/03/2008 11:41 AM -
COTS software
A big challenge is current IT is that most of the software is COTS (Commercial-off-the-Shelf for the acronym impaired).
If you have a piece of bespoke software, and you discover something that clearly doesn't meet the requirements, it's a bug - no questions asked.
If you have a COTS application, it's a feature request to be prioritized in some future release, if you are lucky.
The obvious problem this creates is that you end up stuck with a bug, but it is really far more insiduous. When your bug list is topped with things that your team has little power to address it is very demoralizing. Why should the team put all this effort into testing, when it will be a over a year before anything will be done with most of the results.
...and enterprise software vendors wonder why companies are so slow to upgrade...
No amount of testing will overcome this. You just end up standing there between a rock and a hard place. You have put the project in sleep mode until the issues are addressed, you can "accept the risk" and move forward, or you can swap out the problem component at considerable cost and expense - assuming that is even politically possible.
...or you can just not bother to do thorough testing and act surprised when things go wrong.
I'm not saying I don't believe in thorough testing. By god I do. But it clearly isn't rewarded.
Erik Engbrecht10/03/2008 02:41 PM -
So is open source an answer, then?
(note: not "the" answer... not a zealot, no axe to grind,
etc.)
For shops getting burnt repeatedly by one piece (or type)
of COTS system, how many would benefit by looking for
some FOSS packages that did the job (or a good chunk of
it) and could then be tailored to needs? Sort of taking the
usual replace-bespoke-by-COTS fashion and (to a degree)
turning it on its head. The company wouldn't even need to
worry about releasing their modified code if it was only
used internally, to my understanding of most open source
licenses (IANALNDIPOOTV). The most significant headache
would occur with change management - when the
upstream package provider released a new version,
applying some policy to merge or reject updates to the in-
house custom version. It takes a somewhat different
mindset than many corporate managers are used to
dealing with.... which I strongly suspect to be one of the
major associated risks to the entire idea.
jdickey10/03/2008 09:52 PM -
Only if you have developers on staff
Most IT shops do not have developers they can devote to re-writting applications.No_Ax_to_Grind10/04/2008 10:39 AM -
RE: Stop brain-dead software testing!
Brain-dead reporting.
The author has given just a phrase: "brain-dead software testing" and then a few examples of the results of "brain-dead software testing".
What's lacking? A DISCUSSION ON "BRAIN-DEAD SOFTWARE TESTING".
Jeez.
plainstreet@...10/04/2008 05:17 AM -
ZDNet Blogger
Did you click page 2?
The discussion and analysis appears on page 2. Perhaps you missed that?mkrigsman@...10/04/2008 02:17 PM -
RE: Stop brain-dead software testing!
I think we should be more careful in our definitions.
As a tester of 30 years standing, I can tell when projects have been managed to involve testing and when they have not.
The writer of this report is unnecessarily scathing about "testing" and thus "testers", when he should know better.
A disappointing act on the part pf ZDNet to endorse such poorly created self-advertising.
simon@...10/06/2008 08:59 AM -
What?
I missed the bit about how all software testers got a negative review. I also missed the advertising.
seanferd10/06/2008 01:57 PM -
RE: Stop brain-dead software testing!
very good article, i really like it. I am doing a bit on research about "Software testing" and i found also macrotesting www.macrotesting.com to be huge source for software testing.
Thanks for your article
Regards,
Prem
dvlprem06/23/2009 05:37 AM
Talkback - Tell Us What You Think
Get it the way you want it
ZDNet Newsletters
Get the best of ZDNet delivered straight to your inbox
Vendor Showcase
Blog Roll
- All About Microsoft
- The Apple Core
- Between the Lines
- BriefingsDirect
- Collaboration 2.0
- Dev Connection
- A Developer's View
- Digital Cameras & Camcorders
- Ed Bott's Microsoft Report
- Emerging Tech
- Enterprise Web 2.0
- Five Nines: The Next Gen Datacenter
- Forrester Research
- Googling Google
- GreenTech Pastures
- Hardware 2.0
- Home Theater
- iGeneration
- India IT
- Irregular Enterprise
- IT Project Failures
- Laptops & Desktops
- Lawgarithms
- Linux and Open Source
- Managing L'unix
- The Mobile Gadgeteer
- On Sustainability
- The Semantic Web
- Service Oriented
- Smartphones and Cell Phones
- Social Business
- Social CRM: The Conversation
- Software & Services Safari
- Software as Services
- Storage Bits
- Team Think
- Tech Broiler
- Tom Foremski: IMHO
- The ToyBox
- Virtually Speaking
- The Web Life
- ZDNet Education
- ZDNet Government
- ZDNet Healthcare
- Zero Day
Blog Archive
White Papers, Webcasts, & Resources
- 10 Dying IT SkillsThere are some things in life, like good manners, which never go out of ... (Global Knowledge) Download Now
- WPA2 Security: Choosing the Right WLAN Authentication Method for Homes and EnterprisesAsk a hundred CIOs what three things about WLANs (wireless LANs) strike ... (Global Knowledge) Download Now
- 2010 IT Skills & Salary ReportAre you wondering how your salary compares to your colleagues? Or how ... (Global Knowledge) Download Now
