FBI shows off new cross-agency database

The FBI showed off its Investigative Data Warehouse - a database with 659 million records - to reporters yesterday, at least in part to address criticism that it is saddled with out of date and irrelevant technology, The Washington Post reports.

The data warehouse is an effort to "connect the dots" that the FBI was accused of missing in the months before the 2001 attacks, bureau officials said. About a quarter of the information comes from the FBI's records and criminal case files. The rest -- including suspicious financial activity reports, no-fly lists, and lost and stolen passport data -- comes from the Treasury, State and Homeland Security departments and the Federal Bureau of Prisons.

"That's where the real knowledge comes from . . . sharing information," said Gurvais Grigg, acting director of the FBI's Foreign Terrorist Tracking Task Force, who helped develop the system.

Privacy advocates like the Electronic Frontier Foundation reacted badly to the news.

"It appears to be the largest collection of personal data ever amassed by the federal government," EFF's David Sobel said. "When they develop the capability to cross-reference and data-mine all these previously separate sources of information, there are significant new privacy issues that need to be publicly debated."

Grigg said agents can now run 1,000 searches across 50 databases in half an hour. It would have taken some 30,000 hours to do the same task before 2002.

Officials did indicate that data tends to stay in the system unless there is a reason for removal.

Every data source is reviewed by security, legal and technology staff members, and a privacy impact statement is created, Grigg said. The FBI conducts in-house auditing so that each query can be tracked, he said.