Minnesota HealthMatch: A perfect storm for IT failure

Minnesota HealthMatch: A perfect storm for IT failure

Summary: When poor judgment and lousy communication meet genuinely bad technology, it creates an awful perfect storm where IT failure is almost certain. Minnesota's Department of Human Services (DHS) just closed out such a project.

SHARE:
TOPICS: CXO
13

Most projects fail due to shifting requirements, inattentive management, poor communication, lousy business fit, politics, and other assorted errors in judgment and organizational maturity. Only in the most unusual circumstances does bad technology dooms a project to failure.

In those cases where poor judgment and lousy communication meet genuinely bad technology, it creates an awful perfect storm where IT failure is almost certain. Minnesota's Department of Human Services (DHS) just closed out such a project, called HealthMatch, in which everything seemed to go wrong, the organization wasted millions of dollars, and little or nothing remains to salvage.

HealthMatch was an automated system designed to match Minnesota residents with appropriate state-run health programs, based on eligibility. The variety of residents' needs and situations, together with the complexity of state programs and 16,000 eligibility rules, made this a daunting task.

HealthMatch started in 2002, with a budget of $13 million and anticipated completion date of two years. The project included three main components: a health care system eligibility rules engine, employee workflow, and a client portal for state residents to initiate cases into the system.

A 2003 analysis by the Minnesota Legislative Audit Commission reported that HealthMatch faced problems such as: "income determination errors, DHS’s income estimates not matching actual income, misreported insurance information, and oversight weaknesses".

Minnesota Public Radio reported:

Programming outsourced to India had so many errors it was scrapped.... A subcontractor with expertise in Medicaid rules dropped out. Three project managers and five deputies cycled through HealthMatch in four years. The Legislature tinkered with eligibility rules. The department dragged its feet on developing system requirements and made changes that required redoing completed computer code.

Almost ten years after the project was conceived, Minnesota has now closed the final chapter and agreed to pay the software developer, Affiliated Computer Services (ACS), $7.25 million, to settle a lawsuit for unpaid invoices. Altogether, the state spent over $41 million on HealthMatch.

SHARED RESPONSIBILITIES

The IT Devil's Triangle concept tells us that every major software implementation project includes three parties: enterprise customer, system integrator, and software vendors; most failures involve contributions from all three groups. In this case, there was no system integrator, because HealthMatch was a custom product, with the vendor, ACS, presumably responsible for both technical development and integration.

According to a 2007 analysis (PDF download) performed by the Minnesota Legislative Audit Commission, both the Department of Human Services and ACS made significant errors that contributed to this failure. The $7.25 million financial settlement supports this view; Minnesota Public Radio says that, "ACS asked for an additional $19.4 million for partially completed work. The Department of Human Services offered $3 million."

The 2007 report explicitly describes shared responsibility for problems on this project:

[T]he system’s design was substantially modified twice, and each change lengthened HealthMatch development. However, the project has also repeatedly failed to meet scheduled benchmarks, even when these scope changes are taken into account. In our view, both DHS and its contractor, [ACS predecessor company] Albion, share responsibility for project setbacks.

Despite definite assessment of shared responsibility, the report describes how the parties cast blame on each other:

Although each has accepted some responsibility, DHS and [ACS predecessor company] Albion administrators disagree vehemently over the relative importance of various factors in causing delays....

NUMEROUS POINTS OF FAILURE

in 2008, the Legislative Audit Commission issued another assessment report (PDF download), further examining reasons for delays, cost-overruns, and poor deliverables. The report presents a scathing condemnation of the Department of Human Services, external vendor ACS, and even the core technology underlying the eligibility rules engine.

The 2008 report describes a litany of reasons this project failed; including basic aspects of technology, management, staffing, business requirements definition, and so on. These snippets, from the document, give a flavor for the mismanagement and errors related to HealthMatch:

  • DHS received free, perpetual license to use the @Vantage software [PDF download of current @Vantage brochure] upon completion of the design phase of the project. Shortly after the HealthMatch contract began, the vendor dismissed the Fox Systems personnel assigned to the project, thereby severing the relationship with Medicaid experts who had enabled the vendor to meet minimum DHS project qualifications and leaving a significant gap in the requisite skill set.
  • The most critical step the team made in assessing continuation options was to come to a recommendation regarding use of the @Vantage rules engine....The analysis findings finding resulted in a unanimous recommendation to pursue...discontinuing use of the @Vantage rules engine and developing a new health care system. This recommendation is predicated on the high risks associated with the @Vantage system in terms of development time frames, ability to meet critical success factors, and alignment with federal, State, and agency strategic IT direction.
  • [F]rom a technical viewpoint, some of the @Vantage components have features that are incomplete, error prone, and not always efficient to use. Compounding issues with @Vantage, the vendor implemented HealthMatch with poor quality, a lack of technical management, and times a lack of correct use of @Vantage features.
  • Because many HealthMatch requirements were captured in the context of @Vantage limitations, existing requirements need to be reviewed.... Hire Requirements Analysts who possess the skill-set to write and manage changes to requirements.
  • Best practices were not being enforced or adhered to consistently across all project management process areas (project governance, planning, schedule, scope, cost, management, quality management, communications management, and risk and issue management.)
  • HealthMatch staffing and position descriptions for HealthMatch were not adequately defined or supported to meet business requirements and timelines.
  • [D]iscrepancies were found during the assessment between the level of completion that had been indicated in the project plan and the actual findings while analyzing the project artifacts. In some cases [coding] is several versions behind the current design version even though the project plan tasks had been marked at 100% coding completion.

STRATEGIC ANALYSIS

Major failures generally arise as the confluence of multiple issues coming together to form a larger problem. Three foundational elements came together to cause the HealthMatch failure:

  1. Undisciplined and inexperienced program management in the Department of Human Services
  2. An inconsistent, and probably over-extended, vendor
  3. Immature and poor tested technology

Each of these factors is so basic, that weakness in any one could have put the project at significant risk. All three occurring together make HealthMatch appear like an expensive, ill-fated comedy of errors in which failure was inevitable.

Despite spending $41 million, the Associated Press reports that Minnesota DHS, "still uses aging systems developed decades ago and paper applications processed by state, county and tribal workers. It's a system tailor-made for errors, with one audit finding rates of improper denials and approvals of 10 to 14 percent."

In addition, this past Friday, a Minnesota lawmaker introduced a bill directing "the Minnesota Department of Human Services to find a contractor to develop software that would match applicants for health care, welfare and food stamps with the appropriate programs." Déjà vu, anyone?

Advice to CIOs: Significant and chronic problems, such as those experienced on HealthMatch, usually indicate deep structural issues on a project. In such cases, completely restarting the project may be the best solution, despite the immediate costs and political challenges. Although recommending termination of a strategic project is difficult, wise CIOs take decisive action before a situation escalates out of control.

Related: 7 tips to safely kill zombie projects

Innovative CIOs collaborate with lines of business, including their organization's CFO, to place long-term objectives above parochial local politics. While this may create short-term anxiety, it is the right thing to do and ultimately separates ordinary CIOs from those who are truly great.

Thanks to Chris Kanaracus for bringing HealthMatch to my attention. Image from iStockphoto.

Topic: CXO

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

13 comments
Log in or register to join the discussion
  • Message has been deleted.

    Unusual1
  • So, a government program utterly failed in its mandate

    while managing to run way over budget. The only surprise in this is idiots who actually think government run healthcare will be any different.
    fr_gough
    • RE: Minnesota HealthMatch: A perfect storm for IT failure

      @frgough@... You awful teabagger Republican Nazi!! How dare you question unicorny progressive genius! /sarcasm

      Now you'll probably get some of the death threats those charming union folks in Wisconsin have been sending out to everyone like junk mail.
      SgtSpork
    • RE: Minnesota HealthMatch: A perfect storm for IT failure

      @frgough@... Yep, let's dump Medicare and the VA. No one ever gets any care. My parents are dreaming to think that Medicare treats them pretty good.
      Regulator1956
    • RE: Minnesota HealthMatch: A perfect storm for IT failure

      @frgough@... <br><br>As someone who has worked for large projects for both government and private companies, I'll let you in on a secret - there's absolutely no difference.<br><br>I've seen millions wasted in both camps, usually due to their IT departments saying "Sure we can do it" or people determined to reinvent the wheel from the ground up (usually with Linux).<br><br>The one real difference between public and private is that public is always cheaper due to lower loan interest and no pressing need to make a profit. Unfortunately the unthinking hatred of government prevalent in the US, means that few governments either state or federal, are courageous enough to make that choice.<br><br>Health, education, communications and insurance are all too important to be left in the hands of the jackals of private industry. Competition as it stands now is really the law of the jungle, which is great for the predators, but not good for the prey - which seems to be the majority of us.<br><br>Everytime someone brings out the old chestnut of inefficient government, I hear banjos <img border="0" src="http://www.cnet.com/i/mb/emoticons/wink.gif" alt="wink">
      tonymcs@...
      • RE: Minnesota HealthMatch: A perfect storm for IT failure

        @tonymcs@...

        Public is rarely cheaper, due to the rules and legislated restrictions on a given project.

        I've been on both sides of this, as a Government IT manager and as a software vendor.

        In my experience public sector projects are always more expensive and take longer to complete.

        The vendors do need to make a profit. The government does not need to make a profit and is wasteful by design. The "use it or lose it" budget model is a prime example of built in waste. The procurement process all but ensures that products will cost much more than if they were purchased in the private sector. Private sector can solicit bids from anyone and make decisions based on actual price / performance. Public sector projects are generally constrained to "approved vendors" and difficult bid processes.

        The paper work and associated administrative requirements placed on vendors wishing to do business with the government automatically increases the costs.
        woot!
    • RE: Minnesota HealthMatch: A perfect storm for IT failure

      @frgough@...
      Yup.

      The obvious only failure in the above case study is that the proper spin doctor had not been employed. Government is exceptionally good in this area, that is why they work so hard for you...
      mist42nz
  • The minute I read that government was involved....

    ...I knew it was another government fiasco. Making war and oppressing its citizens are the only things government is good at.
    sissy sue
  • RE: Minnesota HealthMatch: A perfect storm for IT failure

    Unfortunately another example of how exception driven business processes cannot be converted into an IT project like this.

    Chuckle, and yes frgough, most government is exception driven.......
    ;)
    rhonin
    • RE: Minnesota HealthMatch: A perfect storm for IT failure

      @zenwalker rules, rules, and more rules.... so many that we should invent some new rules to organise them all.
      mist42nz
  • RE: Minnesota HealthMatch: A perfect storm for IT failure

    So thats where my money went. Minnesota is run by a bunch of clowns.
    MarkWB
  • The tell tail buzzwords are all in the brochure...

    "The @Vantage architecture contains a
    set of loosely coupled Java services
    that can be reused in different contexts
    by different systems. The underpinning
    of these services is a powerful and
    customizable meta model that describes
    the mapping of data from XML
    messages to Java Beans to storage as
    rows in a database."

    Anything that has this sort of stuff in it must by its very nature be riddled with errors and inconsistencies - when will people ever learn?
    It's the disasterous syndrome of shoving the data through as many different incompatible representations as you can - until its wrong, or lost.

    It also claims to be "state of the art" - another clear signal that the software is of dubious quality and is probably resurrecting old techniques under new marketing.

    I'm not getting at Java in particular by the way, you can create this kind of multi-tier garbage just as effectively in .NET.
    jorwell
  • The Impact of Complexity

    Time and time again, common sense has shown us that complexity creates high opportunity for failure. <br><br>In this case, the highly complex rules that govern eligibility that then have to be incorporated into the rules engine. These rules are probably generating confusion with the humans who need to interpret them even with the manual systems. With this much complexity, I would <i>not</i> be surprised to find that some rules conflict with others. <br><br>This is the unintended consequence of mandating so many rules to serve so many different policies and laws.
    elizab