Government datamining exploding at high rate
The Pentagon pays a private company to compile data on teenagers it can recruit to the military. The Homeland Security Department buys consumer information to help screen people at borders and detect immigration fraud.... The Education Department's Project Strikeback uses mining methods to compare its databases with the FBI and verify identities. The Defense Department's Verity K2 Enterprise program searches data from the intelligence community and Internet searches to identify foreign terrorists or U.S. citizens connected to terrorists. A Navy program analyzes data to try to predict where it might find small weapons of mass destruction and narcotics smuggling in the shipping industry.
Datamining industry executives and analysts say the government is blowing away the private sector in its purchases of datamining software and services. It's hard to know exactly how much is going on because so many projects are classified, but going back to 2004, 52 agencies had begun or planned 199 such projects. A GAO report released in April looked at four government agencies and found they spent $30 million on datamining, mostly for law enforcement and counterterrorism.
"The only question we would have is at what rate would the demand be increasing," Wayne Johnson, a financial analyst at Raymond James Financial Inc., said of the government's interest in buying commercial data and related software.
"What was surprising . . . was how aggressive and hot the intelligence and security market is for this," said William Donahoo, vice president of product management and marketing at Cogito, which sells its software to the National Security Agency.
Donahoo doesn't know how the NSA is using the software but believes it could reveal patterns about how people deal with one another just from their calling records, the Post said. "There are gatekeepers and bridges and collaborators and leaders that could be identified just by the nature of the communications among the groups," he said. "You do not have to know the content of the conversation to identify this."
While there's no doubt that the government is buying, is it money well spent? Some critics doubt it.
"What you don't want is to get into the Kevin Bacon game, which is to say that you show that everybody is six degrees of separation from a terrorist," said James B. Steinberg, former deputy national security adviser in the Clinton administration. "Out of pure resource allocation, it is so unlikely to provide something useful and so likely to provide dead ends and false leads that you are going to spend an enormous amount of resources on things that don't pan out," he said. "Before you start searching haystacks for needles, you've got to have some reason to believe that the needles are there.""This issue of using data to ferret out evildoers, many administration officials believe very firmly this is the way we should be going and that the barriers there should be overcome because it will result in a greater good," said another former official, who spoke on condition of anonymity. "It's a philosophy that if you have nothing to hide, why do you care if I know what movies you rent? Who you are talking to? If you live a godly life, a perfect life, you don't have worry about 100 percent disclosure."Homeland Security's CAPPS II project was a failure because it identified too many false risks.
"I am just not prepared to say that because someone can't get a mortgage, they are a terrorist threat to an airplane," said a former official, who spoke on condition of anonymity because he was not authorized to speak for the program. "These data aggregator products are used today in the financial world to identify certain things, and they're not designed to identify potential terrorist threats."Not everyone in government is so concerned about the impact of false positives.