Texas warns IBM on failed data center consolidation

Texas warns IBM on failed data center consolidation

Summary: In a classic case of the "blind leading the blind," Texas accused IBM of failing to deliver on an $864 million IT initiative.


The Texas Department of Information Resources (DIR) sent IBM a "Notice to Cure," accusing the large system integrator of failing to perform its obligations on a data center consolidation contract worth $863 million.

According to an internal report prepared by the department, this is a case of the "blind leading the blind," with both parties at fault.

The press release issued by DIR includes sabre-rattling language from the DIR, as the department tries to force IBM into compliance with its requests:

The accumulated effect of under-investment by IBM, poor performance, and continual disregard for the protective obligations of the MSA, has resulted in harm to state agencies, exposure to unnecessary risks, and failure to achieve the objectives set and agreed by IBM.

The Notice to Cure is DIR’s response to repeated performance failures over an extended period of time, and follows months of unsuccessful attempts to find a collaborative solution with IBM.

The Notice to Cure gives IBM 30 days to remedy identified deficiencies. DIR expects this action to result in immediate and substantial performance improvements.

The Notice to Cure (I embedded the entire document at the bottom of this post) offers a harsh indictment of IBM's performance to date. It blames IBM for failing to meet contractual obligations in numerous different areas.

Here are the top six allegations:

Transformation. IBM has failed to complete Transformation Services in accordance with the terms of the MSA, including Exhibit 20. Specifically, IBM failed to complete Transformation of the twenty-seven (27) agencies into the consolidated data centers within the applicable timeframes. To date, only five (5) agencies are completely transformed.

Backup and Recovery. Despite the essential nature of backup and recovery services. IBM frequently fails to perform all required backups on systems under IBM management. including systems implemented in the consolidated data centers. IBM also fails to provide verification for all managed systems whether backups are in fact performed as scheduled and completed successfully.

Disaster Recovery. Significant deficiencies noted in the latest rejection included, among other things: runbooks and technical recovery guide sections are populated with boilerplate language not specific to the agency. the server or the application; applications installed and business functions supported on a server are missing from runbooks and recovery guides; recovery and dependency infonnation related to applications is typically not documented;. and recovery details associated with backup and recovery are not sufficiently documented to enable step-by-step recovery.

Staffing. IBM has failed to provide sufficient and suitably qualified personnel to perform the services, causing severe procurement, project and service request backlogs that have materially and adversely affected the ability of the agencies to serve their constituents and fulfill their missions. IBM has continued to remove staff from this account despite the growth of the backlogs.

Security. IBM has breached these contractual obligations by repeatedly failing to adequately track and monitor its activities associated with off-hoarding IBM personnel from this account. IBM has also failed to fully implement required Antivirus Software and to timely and consistently apply software patches for all managed systems....

Service Level Failures. IBM's chronic failure to attain service levels, disregard for required corrective actions and failure to implement the tools and processes necessary to measure all Service Levels places IBM in material breach of its obligation to provide the Services in accordance with the Service Levels.

According to Government Technology, IBM denies the allegations:

"IBM has fulfilled its obligations under the contract and today's action by DIR was unnecessary and unjustified," said company spokesman Jeff Tieszen. "IBM has worked in cooperation and good faith with DIR to provide benefits and improvements to all citizens of Texas. IBM very much regrets the state's action and will aggressively protect its interests going forward."

Strategic Analysis

This failure is another example of poor alignment between customer expectations and service provider delivery. Despite extensive project contracts and documentation, much of which is available publicly, the parties cannot agree on what is actually required under the contract.

Lack of agreement is a common source of problems on large or complex projects. Other recent examples include the EDS v. BSkyB lawsuit in the UK and Marin County's lawsuit against Deloitte Consulting. In both those cases, the enterprise customer claimed fraud against the service provider.

Related: EDS to pay $460 million over CRM failure

Related: Marin County sues Deloitte: Alleges fraud on SAP project

My take. I suspect the key to this case lies buried in an obscure review document commissioned by DIRS:

It is unlikely that either DIR or IBM could have anticipated this misalignment during the contracting process. DIR had little to no prior exposure to either large-scale managed services or complex multi-agency infrastructure projects. And IBM had limited experience in conducting such arrangements in the context of a state government environment such as the one operated in Texas.

In other words, according to the Department's own report, neither party had sufficient experience to implement a program of this type and scale successfully. Yes, it's a blind-leading-the-blind scenario.

Projects of this size are usually doomed to cost and schedule overruns. Nonetheless, the seduction of theoretical benefits causes otherwise rational people to go down the path toward failure.

What human tendencies drive us into situations of inevitable failure. Please add your voice in the comments!

Full text of Texas-IBM Notice to Cure

Photo from iStockphoto

Topics: Hardware, Data Centers, IBM, Storage, Virtualization

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • IBM has the governor firmly in hand, nothing will happen

    However IBM may have skimped on the contract, they did not cut any corners when it comes to scratching the backs of Rick Perry and his cohorts. So nothing will come from the DIR public posturing, except that some low-level staffers will take career hits.
    terry flores
    • RE: Texas warns IBM on failed data center consolidation

      @terry flores

      Are you another one of these guys that just likes to flame public figures so it sounds like you are credible? As you may have read (or should have read since you have so boldly staked your reputation on your bantering) Texas has solicited rebids for the entire contract in January of this year. In addition, in 2010 Rick Perry issued a stop work order to IBM until they addressed rumors that they had failed to adequately back up data for 20 other government agencies. Does that action sound like someone that is "on the take?"
    • Failed data center consolidation

      @terry flores In the wake of seemingly ever-increasing cyber threats and violent natural disasters, businesses need to ensure their <a style="text-decoration: none; color: #333333;" href="http://www.sfbayarealowcostdatarecovery.com/">hard drive data recovery</a> and business continuity plans are up to scratch.
  • Any chance you could go back and put spaces where they belong?

  • RE: Texas warns IBM on failed data center consolidation

    Not the first time managers on the customer side and sales people on the service side sign agreements, not realizing each side, including lawyers, have a different understanding of the contract.

    If everyone would be grownups, and each give the opportunity to the other to explain their understanding of the contract and problems encountered, then take a deep breath and figure out a way to get it resolved instead of stupid posturing and lawsuits which will only cost IBM and Texas taxpayers even more dollars.
  • RE: Texas warns IBM on failed data center consolidation

    Big deal1
    I was a consultant for the Commonwealth of Pa and noticed that the IBM contractors were in the 5th year of a 3 year contract for a Lotus Notes development database.
    Nothing new or out of the ordinary for an IT project.
  • This wreaks of what I used to see at Andersen Consulting (now Accenture)

    Long on sales, short on delivery. While it goes without saying you have to be confident to succeed in business, so much of what I observed within Andersen once upon a time just struck me as "talk is cheap".

    In addition the practice of taking people with high grade point averages (not even CS degrees) and thinking they could produce good solutions for technology problems never made sense to me. And it still doesn't. There's something to be said about having a passion for technology at some stage in your life, which with some of the people I worked with, never seemed to have been the case.

  • RE: Texas warns IBM on failed data center consolidation

    It is not surprising that IBM has failed on a grand scale on this project. IBM has changed as an institution and now seems to employ mostly MBAs who can give sleek PPT presentations but are technically illiterate. I am aware of a situation where IBM sold a number of Colleges (including ours) some AIX servers with a variety of software that was never integrated and ran into numerous problems. The problem could not be resolved as the IBM divisions kept arguing whether it was a software problem or a hardware problem. On debugging I noticed some of the problems were due to Java?s perennial ?Exception thrown, class not found? genre. I was surprised that IBM was using Java for some of their system level functions. Java enables inept developers to skip on design and testing, by deferring design decisions to runtime (Think GC) . As a company, by using Java, IBM saves on development costs and end users pay the price of the non-scalability of the resulting java solution. So I won?t be surprised if the problems experienced by the Texas department stem from the abuse of Java as a development language by IBM to save on development costs.
  • RE: Texas warns IBM on failed data center consolidation

    I have been doing contract work here in the DC area for 5 years. The major consulting firms are called "Beltway Bandits" and the names are fitting. Some projects screwed up by various companies before they were thrown out are as follows.
    Affiliated Computer Services was the prime. EDS worked on the contract and walked away with Approximately $500 million before they were thrown out. Nothing worked and the money was lost At FannieMay the prime was an Indian company, no longer in business, and blew $500 million. Now they have allocated $600 million for a new contract. The same company blew more than $200 million over 7 years before they were thrown out of a Texas contract. Lockheed Martin is out of a contract at CMS (Social Security) when they spent all the money in 5 years on a 6 year contract and have nothing to show for it. I think that was $300 million.

    I could go on and list many more contracts. I have only covered $2 BILLION in wasted money here. The attitude of the companies is we did not finish but we still made millions of taxpayer dollars
    The Old Timer
  • RE: Texas warns IBM on failed data center consolidation

    This has more to do with the delusional belief of the Gov of Texas that HE and HIS staff don't really have to know a thing about running government services - and the free market (and outsourcing) firms MAGICALLY know something about running gov't services and the technology to do so - hey, because their ad agency says so.

    In fact, IBM is rather clueless themselves, but put two fast speaking, low-quality leadership teams together -- and you have the blind leading the blind, back-slapping each other on how smart they both are.

    Congratulations to the tax payers of Texas. You will pay the bill with little to show. Perry will walk away with chits for later pay-back - and claim the free markets work (yes, to pad his pocket!). And Big Blue will pocket millions accomplishing very little, but running ads telling the world just how smart they are.
  • RE: Texas warns IBM on failed data center consolidation

    This is the most accurate and more informatuve post I ever read so far. It gives a lot of information with regards to the matter that is very important to traders and newbie a like. I'm very grateful that I was able to read your post and know a lot of impressive information. gallbladder