FBI's data management headaches escalate with Patriot Act

A Justice Department audit found that the Federal Bureau of Investigation misused the USA Patriot Act to obtain personal information about people. Within that 126-page audit is evidence that the FBI is still struggling to manage its data.

In fact, I've seen the inability to keep the FBI databases current up close. We bought our house from an FBI agent four years ago. He retired in September, but the FBI didn't process the paper work until last month. His exit papers were sent to our address via FedEx. Yes, the FBI had an four-year-old address for one of its agents. Luckily, he didn't go far and we could point the FedEx guy in the right direction.

In his report, the Justice Department Inspector General Glenn A. Fine found that agents demanded personal data without proper authorization. And in the data management department the FBI didn't know how much data it requested via "national security letters," or NSLs, and then couldn't track the requests once issued.

To get these NSLs, which are used to get electronic communications, phone records and financial information, an agent has to ask permission electronically from a superior and then get a signed letter. Once that was in hand, information was gathered and then used in investigations and/or shared among other parties, such as a terrorism task force. The report notes that NSLs are a vital but mismanaged tool.

Once you read the report, you get the impression that paper-based processes are broken, converting data into an electronic form is tricky and tracking is even worse. What's also clear is that the FBI's primary business is information and managing it. But while thousands of companies--Wal-Mart, Procter & Gamble and Lowe's--have data warehousing down to a science the FBI botches basic data management.

The report also notes that the "FBI had no policy or directive requiring the retention of signed copies of national security letters or any requirement to upload (the letters) into the FBI's case management system, the ACS system." In other words, there is no uniform system to track national security letters manually or electronically.

Some FBI offices require documentation that the letters were received with information such as Social Security numbers and names. In these select offices where documentation is required it's uploaded to the ACS and then shared with other investigations.

But the ACS isn't the only database in town. The inspector general's report cites a number of databases that are supposed to hold information from national security letters. For instance, a database called Telephone Applications compiles information such as telephone billing records and subscriber information. The goal of that database is to track the calling patterns of a subject.

Another database is the Investigative Data Warehouse (IDW), which compiles other FBI data, such as photos, biographical data, financial data and physical location for "thousands of known and suspected terrorists." The IDW can be accessed by 12,000 users including FBI agents and analysts and members of joint terrorism task forces.

However, information from the ACS and Telephone Applications databases don't necessarily make it to the IDW.

There's also a database run by the FBI's Office of General Counsel National Security Letter database (OGC). This database is the one that's supposed to track the number of national security letters issued. The issue with this database: The Inspector General says it's inaccurate.

Using the FBI's stats, in calendar year 2000--before the Patriot Act was passed--the agency issued 8,500 national security letters. After the Patriot Act, these letters jumped to 39,000 in 2003, 56,000 in 2004 and 47,000 in 2005. But according to the Inspector General report, there were "incomplete or inaccurate information" entered into the OGC database on a number of letters issued.

As a result, the report surmises that there were 17 percent more letters than were recorded in the OGC database.

Meanwhile the Inspector General notes that, "we found that the FBI did not consistently enter the NSL approvals into the ACS in a timely manner." And when the information was in the OGC database "we found incorrect data entries" including "blank data fields, typographical errors and a programming feature that provides a default value of '0' for the number of 'NSL requests.'"

The real kicker: The Inspector General couldn't determine how inaccurate the database was because "an unknown amount of data relevant to the period covered by our review was lost from the OGC database when it malfunctioned."

There were other issues with the OGC database. For instance, the database can't filter NSL requests for the same person in the same investigation. Why? The database doesn't account for the possibility that John T. Doe and J.T. Doe might be the same person.

Given those aforementioned items the Justice Department recommends that the FBI improves or fixes the databases in question along with the processes surrounding them. That's easier said than done though. The agency has bungled technology implementations for decades and spent billions. Let's hope the FBI gets its data ducks in a row soon.