Project failures occur with stunning regularity across industries, technologies, and geographies. Although IT failures often appear rooted in technology, the real causes can usually be traced back to people and management, rather than to hardware or software.
Andrew Shimberg has conducted substantial primary research into the underlying and root causes of IT project failure. Andrew described his work in the July, 2007 issue of MIT Sloan Management Review in an article titled, How Project Leaders Can Overcome the Crisis of Silence. I discussed that article here.
Andrew is president of Advisory Services for BSG Concours, a provider of executive education, research and strategic advisory services. I interviewed him to learn why so many IT projects fail.
Andrew, please tell us about your research into failed projects.
The research underlying the MIT article is based on a study called Silence Fails, conducted jointly by BSG Concours and VitalSmarts, a performance training company. Over years of consulting, we kept coming across the same failure-inducing patterns in many organizations, despite all their attempts at interventions. Silence Fails was our attempt to systematically research these patterns.
While the article focuses on IT, the reasons underlying failure are very much the same across all kinds of large corporate initiatives.
What did you learn from the Silence Fails study?
We learned that certain key "levers" receive a disproportionate amount of attention during projects, yet are not necessarily where the problem patterns lie. Most people focus on levers such as the company's business strategy, the organizational and project structure, the management systems, and the processes inside the business. These represent the visible, organizational rules against which projects are planned and executed. We say these rules operate "above the water line," to use an iceberg analogy.
In contrast, many project decisions are shaped, and ultimately derailed, by unwritten rules that lie "below the waterline." These unwritten rules are influenced by the shared history of the organization, the economic climate, and cultural values inside the company. In a sense, they reflect the kind of advice you'd give a newly-hired friend, to help him or her be successful in the organization – including descriptions of how things really work in the company.
To fully understand a project, or an organization for that matter, one must consider what takes place above, and also below, the waterline.
How do these levers drive project success or failure?
Project failures often have roots in two distinct, emergent and dysfunctional behaviors arising from the unwritten rules I just described. These are:
- Silos and territorialism across the functions and business units
- Unhealthy conflict avoidance within the project or organization
When participants feel safe in their environment, they tend to speak out and bring problems to the surface, allowing those issues to be managed.
On the other hand, without a sense of trust and safety, participants are reluctant to voice concerns and risk conflict, which keeps problems hidden.
Denial is frequently a component of failure situations.
Organizational self-denial is common. As an example, let's look at "fact-free planning," which is described in the MIT article. Fact-free planning is one reason projects are launched with unrealistic expectations around timelines, budgets and so forth.
How does fact-free planning occur? Many organizations follow a top-down approach to project planning, in which senior management specifies key parameters, such as budget and timeline, before the project team is assembled. The newly-established team comes together, starts work, and then realizes the project will be more expensive than management had anticipated. Despite realistic input from the project team, management remains committed to the established budget. And so the cycle of failure begins.
As we discussed earlier, unwritten rules about avoiding conflict can mask problems, preventing them from being exposed and managed. All too often, these hidden problems grow into real, and very expensive, failures.
How do you stop the negative cycle caused by organizational self-denial?
The unwritten rules governing how information is communicated inside an organization must be explicitly examined. Top leadership should send a clear message, through both words and actions, encouraging participants to speak up. Embedded leaders, such as project managers, must also be a part of this process.
The real solution is building an environment where participants feel safe, and are encouraged to speak the truth in a constructive manner.
Executive sponsorship is another component of IT failure situations.
Lack of sponsorship generally arises from two causes: the appropriate executives have no time, or the sponsor has not assumed full accountability for his or her role.
The traditional, above the waterline, solution is educating sponsors regarding their project responsibilities, addressing specifics of their project role, and defining expectations around that role.
Below the waterline, the solution again involves creating an environment where the team feels comfortable speaking up. In addition, the team must be taught how to voice its concerns in an appropriate and constructive manner. These skills can be taught and learned.
How can project teams learn below the waterline skills?
Teams should participate in an action learning process, where they are trained to apply and practice these skills together as a group. Classroom-based learning alone won't work; it's essential to combine training and practice.
When intelligent, careful communication is encouraged inside a safe environment, the results are amazing. When top management is involved in this effort, you hit home runs.
Where do traditional project management techniques fit into this approach?
Traditional project management is an essential part of the equation; however it focuses above the water line. In contrast, the issues we've just discussed lie below the waterline.
While methodologies, processes, risk management, and so forth are necessary for running projects, factors below the waterline fall outside the realm of the project management discipline.
In my view, achieving a complete and balanced picture involves combining below the waterline thinking with traditional project management.
How can an IT manager make his or her project successful?
Leaders should do two things. First, make the environment safe so team members feel comfortable, and encourage them to speak up early and often; this can be harder than it appears. Second, teach the team to communicate and address issues effectively – it is possible to voice concerns without triggering combative or defensive reactions.
[I am grateful to fellow Enterprise Irregular, Susan Scrupski, for arranging this interview.]