NSA: Can it find signals over noise?

Strip away the politics and privacy debate and the NSA's penchant for hoarding data is a major big data problem.

The U.S. National Security Agency (NSA) is reportedly collecting millions of online address books and tracking instant messages. Like a pack rat, the NSA is hoarding data and communications in the hopes that somehow it'll find an evil doer, but the big question is whether it can ultimately find the right signals.

NSA Special Report

Data-driven analysis debunks claims that NSA is out of control (Special Report)

If these numbers were reported in a corporate situation, they would be considered an absolute triumph of big data management and implementation. UPDATE: Response/corrections/clarifications from Washington Post reporter.

Read More

According to the Washington Post, the NSA collects millions of e-mail address books globally. Tech and political outrage ensues almost on cue.

But the big takeaway here is that the NSA has too much data to mine. The NSA is collecting so much that you have to question whether all this spying yields all that much in sheer returns.

Of course, the NSA argument will be that it only needs to find one signal and prevent a terrorist attack to get a return. That debate and the balance between privacy and safety are key issues for society.

In the meantime, let's ponder the sheer volume of information the NSA collects. The NSA can monitor everything as numerous leaked documents have shown. The overall aim for the NSA's collection of email and buddy lists is to create a social graph of the bad guys and see relationships and connections for intelligence.

That's a lot of noise for a few signals and one helluva big data problem. Is it worth it?

nsa yahoo
Credit: Washington Post