DHS program tries to connect far-flung information, but at what privacy cost?

A largely overlooked story by Mark Clayton of the Christian Science Monitor details ADVISE, a massive data sweep program by the Dept. of Homeland Security.

The US government is developing a massive computer system that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity.

The system - parts of which are operational, parts of which are still under development - is already credited with helping to foil some plots. It is the federal government's latest attempt to use broad data-collection and powerful analysis in the fight against terrorism.

ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) is part of DHS' Threat and Vulnerability, Testing and Assessment (TVTA) program, which received $50 million in funding. Clayton writes:

What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as "entities" - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.

But ADVISE and related DHS technologies aim to do much more, according to Joseph Kielman, manager of the TVTA portfolio. The key is not merely to identify terrorists, or sift for key words, but to identify critical patterns in data that illumine their motives and intentions, he wrote in a presentation at a November conference in Richland, Wash.

For example: Is a burst of Internet traffic between a few people the plotting of terrorists, or just bloggers arguing? ADVISE algorithms would try to determine that before flagging the data pattern for a human analyst's review.

Plenty have serious privacy concerns about such a program, and some critics like the Electronic Frontier Foundation think it smells like TIA.

In 2002, news reports revealed that the Defense Department was working on Total Information Awareness, a project aimed at collecting and sifting vast amounts of personal and government data for clues to terrorism. An uproar caused Congress to cancel the TIA program a year later.

ADVISE "looks very much like TIA," Mr. Tien of the Electronic Frontier Foundation writes in an e-mail. "There's the same emphasis on broad collection and pattern analysis."

But Mr. Sand, the DHS official, emphasizes that privacy protection would be built-in. "Before a system leaves the department there's been a privacy review.... That's our focus."