'Wicked problems': collaboration, risk, and failure

Achieving project goals requires that IT work with lines of business, external vendors, and so on. As a result, virtually every major enterprise IT initiative crosses organizational boundaries. Consequently, collaboration plays a fundamental role in driving success on large projects.

• Related: IT failure and collaboration: Ten big symptoms

Despite the obvious importance of collaboration, many project management techniques and methodologies do not recognize human interaction as a defining point of IT success. Even worse, command and control hierarchies sometimes view process and methodology as a panacea that solves all project ills. Project politics, poor communication, inconsistent goals and objectives, and other messy realities interfere with idealized notions of process.

Management perspectives and techniques that ignore the human element are doomed to fail. Methodology and the risk of wickedness. Most existing frameworks for controlling IT projects rely on the assumption that methodologies reduce risk by defining repeatable implementation steps. From this perspective, risk management is a quality control function that periodically checks project status against steps in the methodology. In theory, great process coupled with checkpoint reviews should yield project excellence. Unfortunately, that theory just doesn't hold up in practice. Despite these common practices, projects continue to fail, costs skyrocket, and user complaints remain high. Project techniques that ignore collaboration and the human element contribute greatly to the IT failures problem. To be clear: methodology, process, and risk control are essential attributes of all large project initiatives. However, existing tools do not sufficiently address human dimensions that interfere with efficient process. Stated accurately, the tools are not bad, however, they are incomplete. A recent article in project management journal, PM World Today, discusses an updated approach to risk that does consider broader aspects of collaboration. The view presented there is generally consistent with the body of thought presented in this blog.

• Related: Three truths of IT success

To highlight weaknesses with ordinary risk methods, the author, Dr David Hancock, compares "tame problems" to "wicked problems:"

Tame problems are problems which have straight-forward simple linear causal relationships and can be solved by analytical methods, sometimes called the cascade or waterfall method.... Wicked problems are characterized by high levels of behavioral complexity. What confuses real decision-making is that behavioral and dynamic complexities co-exist and interact in what we call wicked messes.... Many risk planning and forecasting exercises are still being undertaken on the basis of tame problems that assume the variables on which they are based are few, that they are fully understood and able to be controlled. However uncertainties in the economy, politics and society have become so great as to render counterproductive, if not futile, this kind of risk management that many projects and organizations still practice.

Although Hancock's view appears deceptively simple and logical, it exposes serious weakness in how most enterprise customers, vendors, and system integrators manage project vulnerabilities.

• Related: Early warnings: The IT project failure dilemma (part 1)

• Related: Early warnings: The IT project failure dilemma (part 2)

This table excerpt from the article presents differences between traditional views of risk and a broader concept that Hancock calls Risk Leadership:

Risk Management	Risk Leadership
Works to a defined scope, budget, quality and program.	Recognizes the possibility of different outcomes and tries to ensure that risk activities are directed towards making an acceptable set of outcomes more likely
Uses the instrumental lifecycle image of risk management as a linear sequence of tasks to be performed on an objective entity using codified knowledge, procedures and techniques, and based on an image of projects as apolitical production processes.	Uses concepts and images which focus on social interaction among people, understanding the flux of events and human interaction, and the framing of projects within an array of social agenda, practices, stakeholder relations, politics and power.
Manages process to ensure complicated projects of people and technology are kept running smoothly.	Develops behaviors and confidence in team through scenario planning and team building to identify and respond to risks and opportunities.
Establishes detailed steps, processes and timetables for risk management.	Understands the 'many acceptable futures' proposition and manages risk to produce the changes needed to achieve the acceptable outcomes.
Practitioners as implementers of the risk process. Training and development which produces: practitioners who can follow detailed procedures and techniques, prescribed by project management methods and tools.	Practitioners as reflective listeners. Learning and development facilitates the development of reflective practitioners who can learn, operate and adapt effectively in complex project environments, through experience, intuition and the pragmatic application of theory in practice.
Applies concepts and methodologies which focus on risk management for product creation or improvement of a physical product, system or facility etc and monitored and controlled against specification (quality), cost and time.	Applies concepts and frameworks which focus on risk management as value creation. Whilst, aware that 'value' and 'benefit' will have multiple meanings linked to different purposes for the organization, project and individual.
Attempts to control risk by monitoring results, identifying deviations from the plan and developing mitigation actions to return to plan.	Adapts the risk process to overcome major political, bureaucratic and resource barriers to develop change in behaviors through trust and managing expectations.
Based on the assumption that the risk model is (assumed to be) the actual 'terrain' (i.e. the actual reality 'out there' in the world).	Based on the development of new risk models and theories which recognize and take cognizance of the complexity of projects and project management, at all levels and that the model is only part of the complex 'terrain'.
Seeks predictability and order.	Has learned to live with chaos, complexity and uncertainty and leads through example to a successful conclusion..

My take. It's time for the world of enterprise software and IT to consider collaboration across silos and departments as a fundamental consideration for project success. This shift demands changes in mindset and thinking, updated methodologies, and new measurement techniques. Adaptable organizations that make the transition will see better projects, higher brand equity, and achieve generally higher levels of business success. Change is hard, but sometimes it is necessary. [Photo from iStockphoto.]