Top 5 government IT disasters besides Healthcare.gov

A report released earlier this year by the US Department of Commerce revealed how one US agency, fueled by the paranoia of a nation-state attack,  spent US$2.7 million trying to destroy US$3 million worth of its own IT equipment, even though evidence of such an attack was never found .

This odd decision came out of a miscommunication between the affected agency, the US Economic Development Administration (EDA), and the Department of Commerce's Computer Incident Response Team (DOC CIRT). DOC CIRT had determined that two components on EDA's network were infected with malware. EDA somehow got the idea that in fact 146 components were infected.

EDA cut all their systems off of the networks. Confusion ensued about how widespread and sophisticated the infections on their systems were. Just to be thorough in the event that foreign infiltrators were working their way into the US Economic Development Administration, EDA's CIO ordered the destruction of all of EDA's IT components. This included printers, mice, keyboards, TVs and cameras.

Only a few weeks ago Xerox, the operator of the Electronic Benefits System (EBT) was conducting a routine test of their backup systems. For reasons which aren't clear, this lead to  the system shutting down for five hours.

EBT is the system for food payments to beneficiaries of the Supplemental Nutrition Assistance Program (SNAP) and Women, Infants and Children (WIC) program in 17 states. It's what is usually known as food stamps. Of course, they don't use stamps anymore, they use a special debit card.

During the five hour outage merchants are supposed to use an emergency voucher system, but some merchants decided to just keep accepting the cards, which effectively no longer had limits on them. Some cardholders went on a shopping spree.

Between 2001 and 2005, the FBI attempted and failed to build a system called the Virtual Case File.

Part of a larger project called Trilogy, Virtual Case File seems to have been designed as a case study in how not to run a large IT project. It experienced significant cost and schedule overruns. There was scope creep. The scheduling was driven by desired outcomes, not reality. There was lack of clear ownership.

Author Simon Moore says the heart of the problem is vague requirements. The requirements were ill-defined and changed throughout the project.

Image credit FBICollector.com

The Federal Aviation Administration's Advanced Automation System (AAS) project was supposed to provide a complete overhaul of the nation's major air traffic control computer systems: new tools and displays for controllers to improved communication equipment and a revamped core computer network. In the end, the FAA decided that $1.5 billion worth of hardware and software out of the $2.6 billion spent was useless.

The testimony of a GAO official before Congress in 1994 makes clear that it was bad management which caused all the problems:

AAS's cost and schedule problems have resulted from several technical and managerial factors. First, FAA and IBM's development and implementation plan, including cost and schedule estimates, was overly ambitious given the highly demanding requirements and the complex software architecture for this system. Second, FAA did not provide adequate oversight of IBM's performance, especially during the initial development of the key ISSS component. As a result, IBM's lack of progress did not always surface in a timely manner. Third, FAA was indecisive in resolving some issues about basic requirements, such as the format of new electronic flight data strips to be used by controllers. In our opinion, the above factors — not inadequate funding or federal procurement rules, as contended by some proponents of an air traffic control corporation-- have caused the AAS’s problems.

AAS is another in a long line of projects that underscore what IT is up against in trying to work in government: The procurement process took far too long and was far too political. The new system was supposed to be magically powerful compared to the old one, so expectations for it ran out of control. It also seems that IBM was as unrealistic in the design as anyone and didn't execute well.

NHS Connecting for Health, launched in 2002, was supposed to "...[deliver] the NHS National Programme for IT (NPfIT), an initiative by the Department of Health in England to move the National Health Service (NHS) in England towards a single, centrally-mandated electronic care record for patients and to connect 30,000 General practitioners to 300 hospitals, providing secure and audited access to these records by authorized health professionals."

Sounds ambitious and wouldn't it be cool if it worked! It didn't. Go to the Connecting For Health web site and you see a big banner at the top that says "NHS Connecting for Health ceased to exist on 31st March 2013. This website is therefore not being updated." It refers users to a different site.

The BBC reported that "[a] report by the influential Public Accounts Committee (PAC) concluded an attempt to upgrade NHS computer systems in England ended up becoming one of the "worst and most expensive contracting fiascos" in public sector history." The most recent estimate of money wasted is £9.8bn, but even that doesn't account for everything.

• Déjà Vu All Over Again: California's DMV IT Project Canceled — The state of California's Department of Motor Vehicles (DMV) is 0 for 2 now on IT modernization projects. In 1994 it canceled a project begun in 1987 after spending $44 million. Earlier this year, six years and $134 million into a second go at it, the state canceled it too. Doesn't anyone in California know anything about computers?
• SAM.gov - The System for Award Management — SAM, which is designed to integrate three acquisition data systems that store and make available information about contractors, went online in July 2012 and was taken off-line days later due to performance issues. It's up and running, but some claim it's still not running right.
• Federal Protective Service Risk Assessment and Management Program (RAMP) — program useless, massive cost overruns - The Department of Homeland Security (DHS)'s Federal Protective Service (FPS) Risk Assessment and Management Program (RAMP) — Enough Acronyms For You? (EAFU?) — is designed to facilitate the FPS's mission of securing Federal facilities and ensuring occupant and visitor safety. A report by DHS's CIO says that "RAMP’s development and deployment has been delayed for two years, and its Life Cycle Cost Estimate (LCCE) has grown from an initial estimate of $15.9M in 2008 to $183M in draft 2011 LCCE. RAMP’s LCCE growth is attributable to poor initial estimates, scope increases, reprioritization of capabilities, unplanned fixes to technical issues, and extended system lifespan."