Boundless Informant: US gov't collects 100 billion surveillance records a month

Boundless Informant: US gov't collects 100 billion surveillance records a month

Summary: Meet the U.S. National Security Agency's global intelligence tracking tool, "Boundless Informant," the latest secretive system leaked by a London newspaper.

Screen Shot 2013-06-08 at 21.58.44
(Image: Screenshot via The Guardian)

The stream of leaks revealing the U.S. National Security Agency's (NSA) secrets carries on with the public outing of an powerful intelligence tracking tool.

On the back of key talks between Chinese president Xi Jinping and U.S. President Obama on issues of surveillance and cybercrime, the U.S. government's week has just gotten a lot worse. In a fresh wave of documents obtained by The Guardian, the details of the NSA's data mining tool "Boundless Informant" are laid out for the world to see.

Read this

PRISM: Here's how the NSA wiretapped the Internet

PRISM: Here's how the NSA wiretapped the Internet

The National Security Agency's "PRISM" program is able to collect, in realtime, intelligence not limited to social networks and email accounts. But the seven tech companies accused of opening 'back doors' to the spy agency could well be proven innocent.

The first story, which brought U.S. intelligence capabilities and surveillance to the media's attention, were claims that the NSA received a court order which allowed it to collect the telephone records of U.S.-based Verizon customers.

The order was issued by the Foreign Intelligence Surveillance Court (FISC), a secretive establishment which was created under the Foreign Intelligence Surveillance Act (FISA) 1978 and amended by the Patriot Act in 2001. The court order forced Verizon to hand over communications metadata on an "ongoing, daily basis" to the agency until July 19 this year, when the order expires.

After the court order came to light, details over NSA's internal computer system, dubbed PRISM, were leaked by The Washington Post. The report alleged that Prism was used to collect communications data from around the globe since 2007 under the NSA's Signals Intelligence Directorate, with "the assistance of communications providers in the U.S."

Seven firms allegedly involved in the program were named as Microsoft, Yahoo, Google, Facebook, AOL, Skype, YouTube, Apple, and PalTalk. One by one, the companies all denied knowledge of the system, and the Post quietly altered the report, which originally stated the tech giants "knowingly participated" in the scheme.

In response, U.S. Director of National Intelligence James Clapper issued a statement stating the system is "important and entirely legal," and the behaviour of media outlets disclosing details of the program was "reprehensible."

The intelligence chief released a PRISM factsheet (.PDF) on Saturday which claims that under Section 702 of FISA: "the United States government does not unilaterally obtain information from the servers of U.S. electronic communication service providers." In addition, data is only obtained following FISA court approval and with the knowledge of service providers.

In short:

"Section 702 facilitates the targeted acquisition of foreign intelligence information concerning foreign targets located outside the United States under court oversight. Service providers supply information to the Government when they are lawfully required to do so. The Government cannot target anyone under the court-approved procedures for Section 702 collection unless there is an appropriate, and documented, foreign intelligence purpose for the acquisition."

PRISM's existence has been recently confirmed by President Obama in a speech on Friday. Whereas the internal computer system collects data, Boundless Informant focuses on organizing and indexing metadata. In other words, the tool categorizes communications records rather than the content of a message itself, such as a text message or phone call.

A leaked fact sheet (.PDF) explains that almost three billion pieces of intelligence has been collected from U.S. computer networks in the 30-day period ending March this year, as well as indexing almost 100 billion pieces worldwide. Countries are ranked based on how much information has been taken from mobile and online networks, and color-coded depending on how extensively the NSA is spying on a country.

Image credit: Screenshot ZDNet/ The Guardian

Users of the tool are able to select a country on Boundless Informant's "heat map" to view details including the metadata volume and different kinds of NSA information collection.

Iran is top of the surveillance list with over 14 billion data reports categorized by the tracking tool in March, with Pakistan coming in close second at 13.5 billion reports. Jordan, Egypt and India are also top contributors.

Levels of country-specific surveillance are color-coded depending on severity; green the least and moving through the spectrum to red if a country is under heavy surveillance.

Example use cases include "How many records (and what type) are collected against a particular country?" and "Are there any visible trends for the collection?" for example.

The other leaked document (.PDF) says the tool is designed to give NSA officials answers to questions including what coverage the agency has on specific countries, how data collection compares in different regions, and how many records are being produced.

Both documents were protectively marked as "top secret" and "NOFORN," denying non-U.S. citizens from viewing them.

According to the documents, Boundless Informant is hosted on corporate servers and leverages open-source FOSS technology. Raw data is analyzed and processed in the cloud. The level of data categorized can also be broken down to determine which intercepts originate from the U.S., and this detail includes IP addresses — which can be tracked back to determine a user's country of origin, state and city.

In a March hearing last year, NSA director-general Keith Alexander has repeatedly denied that the U.S. government spies on its citizens. When asked by Rep. Hank Johnson (R-GA) if the NSA has the technological capacity to identify citizens based upon the content of their emails, Alexander commented:

"No no, we don't have the technical insights in the United States. In other words, you have to have something to intercept or some way of doing that either by going to a service provider with a warrant or you have to be collecting in that area. We’re not authorized to that nor do we have the equipment in the United States to collect that kind of information."

The exposure of the NSA's internal Boundless tracking tool — which is likely only used by the intelligence agency — and Alexander's previous comments appear to be in complete contradiction. The NSA has maintained its position and denies spying on U.S. citizens; a spokesperson for the agency telling The Guardian:

"NSA has consistently reported — including to Congress — that we do not have the ability to determine with certainty the identity or location of all communicants within a given communication. That remains the case. The continued publication of these allegations about highly classified issues, and other information taken out of context, makes it impossible to conduct a reasonable discussion on the merits of these programs."

Topics: Government US, Privacy, Security

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Amazing!

    I live in a country inside the green zone. I assume that only little or no data have been collected from us. I don't know if that makes me happy :(
    • Give a wink to the clouds/moon

      Thanks for the picture, US Government.

      There other ways to collect data/information not listed in the article.
  • Small Guide to hide yourself from Prism & Co. :

    Here are some recommendations: 1. Do not use Facebook or Google+. 2. Set the browser to delete all cookies after finishing the program. (Otherwise all search queries are stored virtually for life at Google.) 3. When important: use anonymous email from a small provider outside the US. (Registration via VPN) 4. Use a good offshore VPN service. I have very good experiences with YourPrivateVPN. Nice side effect: Youtube blocks can be avoided.
    • Naive to say the least.

      What you describe will help with regular law enforcement. Complete bull with respect to NSA. They have access to raw data flows and access to encryption certificates. You're only spreading your ignorance if you think otherwise.
    • Good advice

      In general, except when you consider NARUS. You really would have to setup an an offshore VPN, and then carefully use the internet from there. This would still not shield your APN access from mobiles. And if you are using these technologies to protect your privacy, it's like lighting up a flare to the Watchers, according to several reports published over the years.

      Also, it is likely that NARUS-like software is in use at the other end of the pipe. If there are sharing agreements with those countries, within NATO's footprint this is extremely likely your efforts are moot. You would need to know someone currently working in Intelligence and willing to share with you classified secrets just to make sure you are solid.

      Long way around of saying, that all of this is not likely. IMHO Businesses and individuals are at risk of merciless government mismanagement of the data collected, and there is really not a lot that can be done to stop it without legally calling the officials in front of Congress and charging them to put a stop to it. Narus is the ultimate BackOrifice (I know.. B.O. is ancient, dinosaur ancient, but you get the idea). The really scary bit, is if what NARUS says is true about their software, then, once the hackers get a hold of it, we all have our work cut out for us to keep our servers and users from being effortlessly sideswiped.

      search wiki narus boeing
      [use the wiki references to visit their homepage to view all the product features]
  • Being spied on is my uhmm "data" secure?

    Recent widow of MARCUS PENN martyred, Patty Sears I have been followed spied upon had my phones tapped computer F**ked with for more than a year all while Fighting for Justice For Buddy who was an abused senior in nursing homes where he suffered life threatening injuries. It turns out tough he was Jewish and Christian, his family was trying to De-Christianize him recruiting certain workers at the facilities to break the law and force him to deny JESUS thru torture if necessary and they tortured him to the brink of death several times. The same hospital Holy Cross in Silver Spring, MD kept pulling him out back from death's door so the family got people at that hospital to De-Christianize him but they killed him instead with 2 needles found in the trash can not the medical waste container. The second needle meant for me. Yet the police, hospital administration all cover-up murder and subsequent threats to me and even attacks upon me; two days ago I was poisoned. My phones are still tapped. I'm given a lot of trouble over the computer and been beat up for using it to advocate. even a corrupt Montgomery County Circuit judge looks at my info but no one not even Governor Martin O'Malley who was is in this having promised before tv cameras to protect MARCUS PENN can stop my being under surveillance nor being attacked. What's the point I'm no criminal nor even suspect the info is not meant to catch or stop killers. It will never end I think so I am glad that NSA gets everyone else's info too. With all that other data being processed they may get so bogged down that they will finally leave me the bleep alone. For details IM Yahoo robucks.
  • Speculation, how does it work?

    This article and a number of other ones are all based on material from either the Washington Post, a newspaper that seems to be bent on publishing before verification, and the Guardian, a newspaper that seems to have the desire to make the US look bad at every possible moment, even if the evidence is extremely flimsy and ephemeral. Doesn't ZDNet have any unique sources that they can use?
    • 'Verification' by who? The guilty?

      Your comment is humorous. These stories are never 'verified' by verifiable sources. There is precious little media left that isn't owned by the big boys that own our government and our media. It's like trusting the 'consensus' opinion implicitly.
  • Is Prism Government spying an Oversized Boondoggle?

    Oh My oh MY. I think it was Henry Kissinger that stated during Watergate that Listening to 4 years of taped conversation would take 4 years.

    Take all the face-book, all the blogs, all the tweets all the data on phone calls which include the GPS data of everybody with a Cell Phone turned on, and you have a pile of data that in beyond Comprehension. Now the Feds are planing to build a big gigantic storage facility to store this mass of data. How long will it take to search this pile of data?
    Every day I get several calls from scarmmers who hid their telephone telephone number or spoof it and use auto dialers to call everybody and there siblings. Do you think the terrorists are not using spoofing and throwaway Cell phones.
    This boondoggle will keep several and maybe Thousands of Supercomputers andmany federal employees busy for ever.
    • No boondoggle - it's for use on an 'as-needed' basis

      Their plan isn't to search all the data. Their plan is to store it all and then use it selectively as it becomes 'helpful' on a case-by-case basis. If you take everything anyone has said or done in the whole of their life, you can make a good person a criminal by parsing it a certain way. I'm not saying that's what they're going to do, but the opportunity and the temptation to do so will always be there. Ponder that.
  • you poor slobs...

    So all the bellicose noise coming from Congress, senate, government about China cyber spying is just a joke now that we are damn sure the government is spying on its own people.
    So the emperor is wearing no clothes. What a joke...on you
  • What’s wrong with you people?

    Recently you gave your government a serve for not protecting you from terrorists, including the home grown ones. When they do something constructive you abuse shite out of them. C'mon now: you cannot have it both ways. Your noses are so obese you cannot see past it to work out what is going on. All you're darn well done is scatter them a little to regroup with greater knowledge about the inner workings of your defences. The authority have had one hand tied behind their back trying to do their job which will be nothing compared to two hands tied behind their back after this lot of stupidity. The way you people bang on about freedom and other misnomers you'd think it was a perfect world. HUH.

    Footnote: So what if they collect a little info about yourselves. You are not doing anything illegal are you?? Well are you??
    Another thing: if they expose what they find then jump on the buggr's, not before... isn't that what the lawyers are for.
    • What's wrong?

      That's an very deceptive (hopefully just ignorant) comment.

      When information can be taken out of context and put together erroneously, it can make any good citizen into a criminal, if it is desired to do so. The majority of 'security problems' we have are sponsored acts for the purpose of getting desired reaction or complacency out of the public to more elaborately form the Matrix around us.

      The 'solution' providers are often linked to the initiated 'problem'. Problem->reaction->solution.
  • Imagine

    Suppose an organization (like say, US Govt), had the ability to store the entirety of digital communications of its citizenry for a number of years back as well as forward into the future, and could query that data as needed on any individual or group ... and once the existence of this endeavor became known (PRISM, Boundless Informant), said organization escaped accountability under the guise of ... national security.
    Now suppose that this organization was headed by someone who wished to use the information not only for national security, but for political security. Imagine the implication of that, dear citizens. If this doesn't send a chill up your spine, I don't know what will!
  • Even more Boundless: Narus always watching everything and everyone.

    If it's digital, they are recording it with Narus.. and not just the "metadata". Think about about it.. do you need a US$2b just store 3 billion datapoints of metadata a month? The reason they need this one and the ones they already have.. is to digitally RECORD and apply NARUS analytics to prioritize your risk assessment classification. In other words, they run NARUS on ALL digital connections and record ALL of the transmissions, then apply NARUS to figure out who is going to do what.

    NARUS makes Prism look like a 14.4k modem by comparison.

    Just wait until some other contractor blows up a leak on the NSA’s use of NARUS insight software.

    search wiki narus boeing

    [use the wiki references to visit their homepage to view all the product features]
    “That is it can track individual users, monitor which applications they are using (e.g., web browsers, instant messaging applications, e-mail) and what they are doing with those applications (e.g., which web sites they have visited, what they have written in their emails/IM conversations), and see how users’ activities are connected to each other (e.g., compiling lists of people who visit a certain type of web site or use certain words or phrases in their e-mail messages).”

    “A single NarusInsight machine can monitor traffic equal to the maximum capacity (10 Gbit/s) of around 39,000 256k DSL lines or 195,000 56k telephone modems. But, in practical terms, since individual internet connections are not continually filled to capacity, the 10 Gbit/s capacity of one NarusInsight installation enables it to monitor the combined traffic of several million broadband users.”