Failed government IT: 'The mother of all databases'

The "IT system used to identify terrorist threats that has been crippled by technical flaws," according to a memo from the House of Representatives. The failed system is part of a central US government repository of data called the mother of all databases.
Written by Michael Krigsman, Contributor

The "IT system used to identify terrorist threats that has been crippled by technical flaws," according to a memo from the House of Representatives Committee on Science and Technology. The failed system is part of a "central US government repository of data on international terrorist identities...described by Vice Admiral (Ret.) John Scott Redd as 'the mother of all databases.'"

This enormous database, called the Terrorist Identities Datamart Environment (TIDE), is operated by the National Counterterrorism Center (NCTC) to support the "government's various terrorist screening systems or watchlists.”

My take. I was initially skeptical of the allegations described in the House "Inspector General memo" because it raises highly technical issues in a political context. However, my impression changed substantially after studying the more detailed "Subcommittee memo," which exhaustively documents the investigative sources forming the basis for the allegations.

Given the careful documentation, I believe the memos accurately portray current project status. While I have no opinion regarding specific descriptions of misappropriation of funds, the project management and contractor oversight flaws certainly ring true. From a technical perspective, the allegations are sufficiently detailed to appear rooted in fact.

The official NCTC response, described at the end of this post, offers little reassurance to those concerned about government waste on IT projects. Apparently, even the nation's most substantial national security projects are subject to failure and allegations of malfeasance.

This isn't the first government IT failure and certainly won't be the last.


The House Committee on Science and Technology impact memo, written to the Office of the Directorate of National Intelligence (ODNI) Inspector General, frames the issue:

The Subcommittee has learned that the TIDE database is suffering from serious, long-standing technical problems. The Subcommittee has also learned that a critical NCTC initiative, named "Railhead," which is intended to replace TIDE with enhanced capabilities has suffered from severe technical troubles, poor contractor management and weak government oversight. As a result, potentially hundreds of millions of dollars have been wasted, delivery schedules have slipped, contractor employees have been laid off in order to restrain escalating costs, and the NCTC is now scrambling either to fix the technical troubles or possibly to abandon the program altogether. The end result is a current IT system used to identify terrorist threats that has been crippled by technical flaws and a new system that if actually deployed will leave our country more vulnerable than the existing yet flawed system in operation today.


Some Railhead insiders allege that a significant portion of the estimated $500 million dollars spent on Railhead has been inappropriately used to renovate a building of one of the prime contractors, The Boeing Company, into a Sensitive Compartmentalized Information Facility (SCIF) in Herndon, Virginia. These individuals have also questioned the technical solutions endorsed by the government to replace the current TIDE database, the qualifications of some of the Boeing subcontractors and potential conflicts-of-interest between the program director of another key Railhead contractor, SRI International, and the government's Railhead program manager because of their alleged close personal ties. In short, documents obtained by the Subcommittee suggest that, despite hundreds of millions of dollars invested in Railhead and years of development, the government has little to show for its efforts.


Like many of these programs, the flaws and failures on Railhead have been exacerbated by weak government oversight, poor contractor management and lack of contractor accountability for the program's performance. Turfbattles among contractors, particularly between the design team and development team, have hampered the sharing of critical technical data that has impaired the success of the Railhead program. In addition, one list of Railhead staff from January 2008 identifies a virtual army of 814 private contract employees from dozens of companies involved in Railhead and only 48 government officials keeping tabs on this mammoth and critically important national security program. In fact, an estimated one dozen government slots on Railhead have been vacant for more than one year. A combination of these management problems and technical troubles seems to have doomed the Railhead program to failure.


The Inspector General memo was based on worked performed by the Subcommittee on Investigations and Oversight. The more specific technical memo adds depth and detail to the allegations:

Among the largest and most expensive programs currently being funded by the ODNI is a program at the National Counterterrorism Center to improve and replace its current information technology systems, including the TIDE database, in order to enhance information sharing among federal agencies and improve access to counterterrorism intelligence data collected from more than 30 separate government networks that feed data into NCTC.


Documentation obtained by the Subcommittee points to a host of technical problems on Railhead, potential contractor mismanagement, contractor disputes, agency turf battles, poor government oversight and schedule delays that have hindered and hampered legitimate information sharing efforts on the program, have resulted in the potential waste of hundreds of millions of taxpayer dollars and placed the government's key counterterrorism information sharing initiative in jeopardy of failing.


But technical problems on the current TIDE database appear to be hindering those efforts, and its successor --Railhead -- is on the verge of collapse.

The original TIDE database, built by Lockheed Martin, replaced the Department of State's TIPOFF database, designed and built by The Analysis Corporation, in the wake of the 9.11 terrorist attacks to automate the terrorist watch list. The TIDE database was built in Oracle as a relational database management system (RDBMS). This original database, however, suffers from basic design, management and maintenance ' inefficiencies and problems. For instance, only about 60% of the data, including names and addresses, mentioned in CIA cables provided to NCTC are actually extracted from these messages and placed into the TIDE database.

The TIDE database has evolved overtime as both contractors and government employees have attempted to expand and enhance the database to improve their own use of the system. But none of them appear to have taken into account the overall design or engineering architecture of the entire system. As a result, there are now dozens of tables or categories for identical fields of information making the ability to search or locate key data inefficient, ineffective and more time consuming and difficult than necessary.

In addition, the TIDE database relies on Structured Query Language (SQL), a cumbersome computer code that must utilize complicated sentence structures to query the tables, rows and columns that encompass the TIDE database. Without proper documentation on whether a table contains information on names, addresses, vehicles, license plates or an individual's nationality, for instance, analysts have no valid mechanism to conduct a search of these "undocumented" tables.

Without a detailed index of the data stored in each table in TIDE, the SQL search engine is blindfolded, unable to locate or identify undocumented data. The current TIDE database is composed of data fields that are presented in 463 separate tables, 295 of which are undocumented, according to one internal Railhead document. As a result, critical terrorist intelligence in the TIDE system may not be searched at all. "Existing TIDE data model is complex, undocumented, and brittle," the document notes, "which poses significant risk to RLSI [Railhead Lead System Integrator] data migration and modeling."


The NCTC provided a vague and general response to the allegations, saying the conclusions are:

[I]nconsistent with the facts. The letter implies that there exists a risk to our nation’s security related to the implementation of NCTC’s information technology program, commonly known as Railhead. There has been no degradation in the capability to access, manage and share terrorist information during the life of the Railhead program.

Railhead is a multiple contract venue to support the operations and maintenance of existing IT systems; it replaces and builds new functions for the Center. Fundamentally, it is a series of technology (primarily software) upgrades implemented between now and 2012, rather than all at once to improve mission capabilities for many systems.

[Via an unnamed reader who referred me to the Ars Technica story; I'm always grateful for reader submissions of failed IT projects. Anonymous submissions are welcome. Requests for interview to both the ODNI and the Subcommittee were not returned.]

Editorial standards