When projects fail...

In UK, the Royal Academy of Engineering and the British Computer Society tell us that a lack of professionalism in software engineering is a major cause of money wasted on poor IT systems. Well, who could argue with that?

But they go on to claim that what is needed is a system of registration for people working on safety-critical or banking systems. Now why would they think that? Cynics might suggest that both organisations are in the business of running exactly the kind of membership organisations that would benefit from a requirement for registered professionals.

The BCS has for decades campaigned for a professionalised IT industry, and asserted its claim to be the arbiter of suitable qualifications. Yet there has never been any evidence to link any particular examination with practical achievement. No doubt many BCS members have done sterling work, but so have lots of other people.

Hankering after the supposed status of engineers has also been a popular theme for decades. Odd really, when you consider that many engineers regard themselves as unappreciated and underpaid. Despite this, the idea is constantly put forward that if only IT people worked like engineers, shiny new software systems would come rolling off a production line.

If we look at actual engineering projects, though, life is not so simple. The Millennium Footbridge across the Thames was a huge embarrassment to the engineering firms involved, as it swayed its way to closure. Was it designed by unqualified engineers? Of course not. The project was doubtless staffed by people with plenty of certificates and memberships of professional institutions.

Or what about a much larger project, such as the creation of Concord? The final product has been hugely admired as a technical and aesthetic achievement. But was it built on time and to budget? Absolutely not. It was a disaster in both respects and struggled to completion as a result of political commitment rather than some magic provided by certificates in project management.

Things would not be so bad if we knew what software developers should know. By and large, we don't. Certainly it is helpful for them to know at least one programming language. By the same token, it is as well for engineers to be able to count. Where do we go after that?

Academics have always made a contribution to the development of ideas in software. So maybe that would support the idea that educational bodies know how to decide what skills are really needed. That argument is unpersuasive when we look at, say, the very simple idea of programming using logical structure rather than jumps.

It all came from an article called "Goto considered harmful" by Edsger Dijkstra. The background is interesting, though. Although Dijkstra was a shrewd theoretician, he did not invent the idea that goto might be harmful. He observed that successful programmers avoided its use and concluded that there was a link. There probably is, although it has never been proved that pointing out the harmfulness of goto has dramatically improved the output of unsuccessful programmers.

So why do projects fail? One reason is that skills established decades ago in the creation and testing of software are often overlooked nowadays. Sadly, the canard that software companies think that if you have users, testing is superfluous has some truth. Software is now rushed out to meet market pressures. Feature after feature is added without any review of the basic structure of the resulting masses of software.

The problems are often outside the remit of the skilled software developer. They relate to the conflicting goals of consulting firms that need to keep armies of staff busy and the often unrealistic time pressures of clients. They are exacerbated by political issues such as reluctance to admit that part of a development project should be scrapped and started again. And as we constantly push the boundaries of what can be achieved with computers, problems of this kind just keep coming back.

Martin Brampton is founder of Black Sheep Research (www.black-sheep-research.co.uk), an independent consultancy providing research, writing and speaking services on a wide range of business and technology issues. Martin was previously a director at Bloor Research, and has worked with IT as a user and analyst for over 20 years. He is a long-term contributor to silicon.com through videoed debates and his weekly column, which tackles a wide range of issues. He can be contacted through his website.