Devil's Advocate: Building software as much art as science

Programmers must be creative

It is sometimes thought that writing software is a technical matter that requires analytical skills. There is an element of truth in this, but it is by no means the end of the story. Building quality software involves creativity quite as much as analysis.

In fact, the modern world tends to exaggerate the role of analysis. Perhaps this is because of the central place we give to science as a way to understand human experience. We are inclined to think that science is the best example of clear thinking and a rational approach to questions.

In a way, this is true, for as the western world has over the last few centuries placed less emphasis on authority and more on reasoned criticism, so we seem to have made great strides. Yet the view that science simply discovers natural laws has often been criticised, both by philosophers and by thoughtful scientists.

Physicists have generally led the way. When Newton discovered simple mathematical rules that appeared to explain all the complexities of the astronomical world, people could be forgiven for thinking that the world was absolutely governed by such rules, if only we could discover them. A few centuries later, studies of phenomena at the atomic level turned all this upside down.

Heisenberg told us that atomic events are intrinsically uncertain and that the more we try to measure accurately, the less sure we can be of the results. It even seemed that the observer had an essential part to play in atomic events, exemplified by the notorious and uncomfortably placed Schrodinger's Cat.

Nowadays, physicists are much more inclined to think they are creating interesting models that give us insight into the world. They have handed over the belief that science tells us all about the essential laws that govern the world to biologists, who frequently take a rather literal approach to scientific theories.

What has this to do with software? Good software actually creates a new aspect to the world; it generates new possibilities. Only the simplest of systems are merely an automation of something that already exists. Indeed, modelling systems on already existing practices is often a quick way to failure.

A simplistic description of the software development process suggests that all the interesting thinking takes place at the design phase. The completed design can then be mechanically turned into code. But this hides some of the most important processes. It is only when the design comes into contact with the unthinking computer that its quality becomes apparent.

There are many different ways to create a computer process that will assist a particular activity. Some are far better than others, in terms of their economy of operation and their ability to give desired results in all circumstances. The problem does not determine the solution, leaving huge scope for the creative talents of software developers.

There are many practical implications to this. One is that outsourcing of software development is not as simple as it is often portrayed. While there are undoubtedly talented developers among the outsourcers, they are necessarily more remote from the environment into which the software is to be deployed. This can severely limit their ability to create a solution that benefits the client organisation.

Another is that recruitment practices often place too much stress on skills such as knowledge of programming languages, and too little on creativity, innovation and communication. Software developers should have the talents to contribute to the development of an organisation, rather than being regarded merely as mechanics applying a technical skill.

Yet another is that we should recognise that IT is unlikely to have reached a terminus. The great fascination of software is that it has constantly revealed new and unexpected possibilities. There is no reason to suppose that this is at an end. Thank goodness.