This week, the Commonwealth of Massachusetts held a hearing, most likely the first of several, examining failed IT projects run by Deloitte Consulting. The hearing focused on Deloitte projects for the state's Department of Unemployment Assistance and Department of Revenue. I attended as an observer.
The complex nature of large IT projects makes untangling truth from fiction a challenging, often futile, exercise. Nonetheless, the significant economic waste and human suffering that arises from failed IT makes the analysis worthwhile and important.
Background. The Department of Unemployment Assistance project started in 2007 with BearingPoint as the primary contractor. That company went bankrupt and was taken over by Deloitte, which assumed an obligation to complete the project. Today, the $46 million initiative is two years late, $6 million over-budget, and the subject of complaints from users. The project replaced 11 different systems that were not integrated and could not exchange data.
Massachusetts cancelled its Deloitte project with the Department of Revenue earlier this year, despite having spent $114 million.
Currently, Massachusetts has an ongoing project with Deloitte related to the Registry of Motor Vehicles.
The hearing included testimony from:
- Joanne Goldstein, Secretary for the Executive Office of Labor and Workforce Development
- Michelle Amante, Director of Department of Unemployment Assistance
- Amy Pitter, Commissioner of the Department of Revenue
- Navjeet Bal, former Commissioner of the Revenue Department
- John Letchford, Massachusetts state CIO
- Mark Price, the Deloitte executive who oversees the firm's business with Massachusetts
Analysis. We must view the Massachusetts situation in light of Deloitte's history of troubled projects, such as the state of California, Marin County, Los Angeles school district, state of Florida, Levi Strauss, and others. The multiple Massachusetts failures are not isolated examples.
In fairness to Deloitte, one could argue that their large volume of business naturally leads to a certain number of difficult projects; however, other system integrators do not seem to pop up on my IT failures radar with Deloitte's regularity. In addition, that argument assumes that a certain number of failures are okay, which I believe is wrong.
The hearing centered on three key points:
- Ascertaining details of the projects and background
- Determining the extent to which the projects actually failed
- Trying to understand lessons learned
During questioning, both Secretary Goldstein and Deloitte's Mark Price indicated the unemployment assistance project was a success. Goldstein said, "I am happy with the launch" while Mark Price claimed [registration required], "We fully believe that the system is a success.’’ Although Price acknowledged there were some "issues and challenges," he said the "systems are working well" in Massachusetts and other states.
When asked about lessons learned to improve the project, Price appeared to offer little of substance, bringing to mind the term "dissemble." The Boston Globe elaborates:
State Senator Cynthia Stone Creem, the Newton Democrat who is chairwoman of the committee, expressed disbelief at Deloitte’s view of the system. She said she would schedule another hearing to question Deloitte officials further.
“As I sit here today, I haven’t heard anything you think Deloitte might have done differently,’’ Creem told executives of the firm called to testify. “I’m just at a loss.”
State senator Creem did an admirable job in relentlessly questioning the witnesses, despite her lack of experience with large-scale IT projects. Still, numerous questions and issues remain open, including:
- What precisely caused these projects to go late, over-budget, and deliver inadequate results?
- What was the role of the state versus Deloitte in creating the troubled outcomes?
- How precisely did the state's obligations and responsibilities overlap with Deloitte's? In other words, who was responsible for what, and where were the dependencies on both sides toward the other?
- How does Deloitte define success or failure — at what point do issues on a project rise to the level of being called a problem?
- How can a canceled Department of Revenue project that wasted over $40 million be considered a success?
- Why do Deloitte and the Department of Labor call their project a success despite numerous reported problems associated with the go-live?
- Where does the state need to improve its project management, procurement, and contract negotiation capabilities for IT-related matters?
- To what extent did Deloitte have an obligation to guide the state to prevent delays, cost overruns, and ensure the rollout went smoothly?
- Deloitte's testimony focused on process rather than outcomes. For example, Price stressed he was "proud" of the collaborative and "comprehensive" testing effort on the unemployment project. Yet, he failed to address why testing did not uncover the issues that eventually caused the go-live problems. What was Deloitte's responsibility in helping the state define metrics, measurable outcomes, and other parameters necessary to determine success?
- Prior to submitting a final bid for the unemployment contract, Deloitte reduced its price by 19 percent, without cutting back project scope. Does this indicate the original price was artificially high? Did Deloitte anticipate it would make up the price difference on downstream changes of scope? What are reasonable profit margins on such a large project?
- What was Deloitte's responsibility in helping the Department of Revenue balance significant business process change or customize the software extensively?
- Why did Deloitte sell the Department of Revenue packaged software without first defining the extent of customization that would be required? It appears the need for extensive customization drove the department's decision to abandon its project prior to completion, despite the high cost of doing so.
Final thoughts. It is striking to note that most personnel testifying were not present when the projects were begun, due to the long time frames. Therefore, the business case and underlying rationale were handed down, almost like folklore, from one administrator to the next, and metrics were ill-defined or non-existent. The lack of clear metrics governing expected outcomes helped Deloitte argue that these projects were a success.
In general, the relationship between system integrators and clients is fraught with difficulty, because the integrator may be the key financial beneficiary when a project runs over-budget. This dynamic creates a potential conflict of interest between the integrator's desire to serve customer needs and the possibility of gaining greater revenue from the late project. The IT Devil's Triangle concept describes this conflict.
Having studied and analyzed IT failures for years, I find it disingenuous for Deloitte to claim success on projects where the go-live is flawed, the project is canceled, or the deliverables are years' late. At the same time, it appears the state made a variety of uninformed choices and bad decisions, from software selection to poorly managing the Deloitte relationship.
Major IT failures virtually always result from a combination of missteps on both the customer and system integrator sides. Therefore, Massachusetts lawmakers should remember that pointing the finger at one side usually implicates the other. These projects are complex undertakings made even more difficult by politics, long duration, and even, perhaps, opportunism.
Senator Creem and her staff are doing the right thing by studying these IT failures. Let's hope they have the fortitude and political will to continue.