NSA: Can it find signals over noise?

NSA: Can it find signals over noise?

Summary: Strip away the politics and privacy debate and the NSA's penchant for hoarding data is a major big data problem.


The U.S. National Security Agency (NSA) is reportedly collecting millions of online address books and tracking instant messages. Like a pack rat, the NSA is hoarding data and communications in the hopes that somehow it'll find an evil doer, but the big question is whether it can ultimately find the right signals.

According to the Washington Post, the NSA collects millions of e-mail address books globally. Tech and political outrage ensues almost on cue.

But the big takeaway here is that the NSA has too much data to mine. The NSA is collecting so much that you have to question whether all this spying yields all that much in sheer returns.

Of course, the NSA argument will be that it only needs to find one signal and prevent a terrorist attack to get a return. That debate and the balance between privacy and safety are key issues for society.

In the meantime, let's ponder the sheer volume of information the NSA collects. The NSA can monitor everything as numerous leaked documents have shown. The overall aim for the NSA's collection of email and buddy lists is to create a social graph of the bad guys and see relationships and connections for intelligence.

That's a lot of noise for a few signals and one helluva big data problem. Is it worth it?

nsa yahoo
Credit: Washington Post


Topics: Security, Big Data, Government

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Is there a back door for foreign countries too?

    It's not that big a data problem. Hoovering up everyone's address book is not a great amount of data to deal with. Then they collate it. The NSA has the means and the money to do it.

    The interesting part is that The Washington Post (and an earlier ZDNet article) said the Australian government is Hoovering up 300,000 address books a day. Most people in that country use Microsoft / Google / Yahoo mail. Does the NSA allow a foreign government access to all this?
    • "Does the NSA allow a foreign government access to all this ?"

      Yes, GCHQ.
      Alan Smithie
  • Is it worth it?

    Well, it all depends, doesn't it? When it comes to terrorist attacks, the ROI is certainly abysmal. But that supposes that is truly the main goal of these activities. Making the haystack as large as possible is not going to help find the needles. But what if what you are looking for is easy to find? It is trivial to sieve through emails coming from corporate senders and check if there isn't something that might be of interest to someone who has friends in high places and might give a nice campaign contribution to someone who decides the NSA's budget.