Inside the NSA's perspective on the release of certain previously classified documents

Inside the NSA's perspective on the release of certain previously classified documents

Summary: ZDNet's David Gewirtz had the opportunity today to be briefed by and speak with certain senior intelligence officials in order to explore the circumstances of a privacy compliance error and a new document release. This is their story.


Earlier today, the Director of National Intelligence declassified a series of previously (and correctly) classified documents regarding an error in the capture and intelligence analysis of telephony metadata that took place back in 2009.

Also today, I had the opportunity to be briefed by and speak with certain senior intelligence officials in order to explore the circumstances of the original error and this document release. The briefing was authorized, but the officials requested not to be identified publicly by name so that they might "feel free to discuss information more freely."

I've done these "on background" meetings before and the reason for official anonymity has less to do with anything nefarious and more to do with the fact that officially attributed statements need to be approved by many levels of legal review, where these "on background" things are much more like real conversations (and, frankly, much more useful).

The documents being declassified were described as "properly classified," meaning that when they were classified, they met the criteria for documents that could not be disclosed because of national security. The gentlemen I talked with today said that, from a national security perspective, these documents are still sensitive, but the President determined that the need for transparency with the American people was of a greater importance in this instance.

On the other hand, none of the officials I spoke with mentioned another little detail: that the EFF had filed a Freedom of Information Act Lawsuit and a court ordered the documents to be released.

While the documents are being released, there are still some elements that remain redacted, but the belief is that the ability for the public and the press to see the details of the 2009 incident were of particular import in light of the recent scrutiny of the NSA.

The most interesting piece of meat in our discussion was the core of what went wrong back in 2009. The intelligence community has a classification standard called RAS (reasonable articulable suspicion) which is a base-level standard any intelligence suspect must satisfy before investigation is permitted. It's a relatively high bar, because any RAS suspect must be able to be justified by the RAS criteria and that criteria must stand up before a judge.

In 2009, the NSA was scanning telephony metadata and was supposed to select for analysis only those pings that met RAS criteria. This would have allowed a much larger percentage of metadata to pass through the system untouched.

However, that did not happen. The compliance violation was that roughly 17,000+ items were on something called an Alert List (a list of possible terrorism-related items), but only about 1,800 met the RAS criteria. Even so, the NSA subjected incoming traffic analysis against all 17,000 to the intelligence process, not just the ten percent or so that were RAS-worthy.

To make matters worse, once an item that didn't meet RAS criteria was identified, it was sent to a supervisor (which is not something that's supposed to happen) and, on top of that, the non-RAS item was, and I'm quoting here, "mis-described," implying all the selectors had been RAS-approved.

The details of all that are in the documents released by DNI James Clapper.

The officials I spoke to today did their best to convince me that there was no intentional effort to evade the law in these instances, that the NSA discovered the problem, and reported it to the FISA court. I was also told that once this problem had been identified, the NSA subjected itself to (and this is another quote), "a thorough, painstaking, and self-critical scrub." During that process they found other issues (the specific details of which haven't been shared with me), and those issues were reported to the FISA court as well.

The key contention was that the NSA system is both wildly complex and operated by humans, and that combination is bound to result in errors. I tend to side with this view of the NSA data collection operation rather than the Big Brother view, simply because I know how hard many of America's civil servants work, I understand the scale of both the protection challenge and the data management challenge, and -- in my experience -- if something complex can either go wrong because someone is acting in an "evil" way or something can go wrong because something broke, the very high odds favor the something breaking.

Even so, the senior intelligence officials I spoke with today wanted to reinforce that this "transparency exercise" should help reassure the public that policy decisions are being made by our leaders in ways to provide robust protections for privacy, vigorous oversight, and an attention to national security.

That said, they were also sure to reinforce that anything as technologically complex as the NSA's systems was "not going to have perfection," but they do make it a high priority to find and fix problems.

Stay tuned. There's more to come.

By the way, I'm doing more updates on Twitter and Facebook than ever before. Be sure to follow me on Twitter at @DavidGewirtz and on Facebook at

Topics: Security, Government, Government US, Privacy


David Gewirtz, Distinguished Lecturer at CBS Interactive, is an author, U.S. policy advisor, and computer scientist. He is featured in the History Channel special The President's Book of Secrets and is a member of the National Press Club.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Whitewash and Rationalizing

    'On the other hand, none of the officials I spoke with mentioned another little detail...' ; red flag went up for me but not for you it seems. What else have they so conveniently and selectively omitted, if they saw fit to misrepresent that salient not so insignificant morsel?
    These men are specially commissioned to spread disinformation by posturing humility. and you appear to be a chosen dupe. Not that you normally are at all. Normally far from it in fact.
    'Well you know, nobody's perfect, eh?' And we're just supposed to let bygones be bygones? It's just not that simple, not so easily reduced to black and white.
    I'm not buying in. Too much incontrovertible evidence where NSA and Associates have been caught red handed egregiously violating US law and that of other Nations as well.
    As more and more info surfaces, NSA and ilk are in full damage control mode. Somewhat desperate to have to be pretending to be contrite at all.
  • Apology NOT accepted

    So you have interfered in foreign affairs; poisoned the global economy, ceded control to the unethical global corporations and sold privacy down the river ... and you want to apologise for looking at a few emails in error?

    Nobody is worried about a few human errors: we are worried about systematic exploitation.

    Gewirtz's support for the Government has long since descended below the feeble. I tire of it.
    • Hear! Hear!

      'Nobody is worried about a few human errors: we are worried about systematic exploitation.'
      THAT is my final paragraph in my above Post. It IS massively systemic, longstanding so intrusive violation under scrutiny.
      Too many Folk lulled to sleep at the switch. And perhaps I am too kind to Gewirtz's mesmerizing. Mea Culpa.
  • Your faith in Hanlon's Razor is commendable ...

    ... but you shouldn't ignore Lord Action.

    Hanlon - Never attribute to malice that which is adequately explained by stupidity.

    Action - Power corrupts, and absolute power corrupts absolutely.
    • Misspell, should be Acton

      Auto-correction in acton, i mean action.
  • Transparency v. Secrecy

    "from a national security perspective, these documents are still sensitive, but the President determined that the need for transparency with the American people was of a greater importance in this instance ... the EFF had filed a Freedom of Information Act Lawsuit and a court ordered the documents to be released ... there are still some elements that remain redacted, but the belief is that ... the details of the 2009 incident were of particular import in light of the recent scrutiny of the NSA"

    In this case, does "recent scrutiny" mean "the information revealed by Ed Snowden"?
  • I Can Accept the Trust in Professionalism of Civil Servants

    ... in part because I worked for many years in state and local governments. I wholly concur with regards to your statements about how hard most of these folk work on our behalf and their overall professionalism and dedication.

    And, yes, mistakes can and will be made -- especially in any complex undertaking. But ....

    My biggest concern with the ongoing NSA scrutiny -- just like my opposition to the war in Vietnam -- is not the competency and professionalism of the workers. It is the appropriateness of the policy directives they are required to implement.
    • "Odd" NSA behaviour

      If DG's report is accurate, it seems that the NSA tried to give a briefing demonstrating that they were trustworthy guys who didn't intentionally deceive the public on matters that weren't secrets or matters of national security ...

      ... and they blew it by then telling an obvious, pointless lie about the circumstances that led to the document release, even though that clearly WASN'T a secret or a matter of national security.

      It sounds as if the organisation either has such severe cultural problems that casual disinformation has become a reflex (even when it's deeply inappropriate and counter-productive), or that maybe things have gotten so bad inside the NSA that it's developed a "self-destruct" mode, where people within the organisation consciously or subconsciously want it to fail.

      In the "institutional deathwish" scenario, an organisation with apparently unresolvable internal stresses will show behaviours that seem calculated to bring about an "accidental" crisis that results in the organisation's destruction. When this happens to corporations, the worst that happens is that the corporation goes bust or shuts down (or gets taken over and restructured) but if a "deathwish" pathology arose in an organisation entrusted with national security ... well, that would be dangerous.
      Eric Baird
  • NSA knows and therefore must be complicit

    Can't count the times I've heard intelligence services tell us that they are only human but that they can fix it if we just give them more money.

    When the NSA publically reveals what they know about 911 - and they certainly do know - I'll begin to give them my respect. Covering up the truth - as they are - in my opinion is a criminal act.

    Not so happy anniversary for those who know and give a rats...
    • Exactly

      The best way to misdirect scrutiny from your large scale illegal activity is to admit to something smaller, fix it, and apologize. We are talking about people who are masters of manipulation and misdirection. They are the propaganda arm of the U.S. government. Anyone being taken in by this carefully sculpted information release is severely naive at best. Welcome to Sheeple Land, where the government is made up of selfless saints who love everyone, and just want to protect us from bad things.

      *cue the maniacal laughter coming from the back room*
      • Flipside

        Those who think that the government is bent on using its power for malicious purposes are just as naïve as the sheeple you elude to above.
        • Sheeple - like you - are uneducated

          There is a near infinite amount of clear detailed documented history of governments using their power for malicious purposes. What are you in Kindergarten?

          Sheesh. At least take a world history course, or even American history, and do some research in to False Flags. Moron.
          • Most government/intelligence employees are good.

            Sorry for getting exasperated. I just sometimes get tired of arguing that 2+2 really is equal to 4.

            Many people in intelligence have no idea of the greater agenda that they are supporting, and the fact is that many of them are actually doing good. Many others however, know that there is something wrong but have no choice other than to remain silent.

            After Snowden the axe came down hard. It takes a lot of courage to sacrifice your future to reveal the lie. There are true heroes in the world that will be willing to step forward. These people care more for their children and the world that they will live in than what may happen to them - though usually their children have been threatened as well. Anyone who has volunteered to go to war and potentially lose his or her life in the process should be able to relate to this.
    • Rethink 911????

      The first question on the website is "Did you know a third tower fell on 911"?

      Yes, I did know that. In fact I have read quite a bit about it and seen New York City firemen talk about the third tower and heard a number of experts talk about the absolutely ridiculous contentions that many like to say was proof of some kind of conspiracy.

      Why do we need this nonsense in our life?

      If New York City Fire fighters are willing to come on television and talk about exactly what happened with the third building and why it makes all the unfortunate sense in the world why it fell, why is it that's not good enough for others?

      If well known experts literally lay waste to conspiracy theories about secretive planned explosions within the third building, along with other preposterous claims, why is that not good enough?

      I have seen a number of productions and writings about the wonderful and oh so fanciful conspiracy theories about how this or that conspiracy took place on 911, and every single potential theory has been literally shot to bits by people who would know better. What only makes it all the worse is that the theories seemed preposterous and implausible before they were even explained. Not even believable or worthy on the face if it before explanation.

      The whole of 911 conspiracy theories revolve out of some kind of bizarre and convoluted way of thinking about what the motivations for such conspiracies were for and what they may have been trying to accomplish, yet if that were the case much of the things that have been theorized were not only far far too risky to even contemplate for they were apparently trying to accomplish and from most of the weird goals that I have heard about many of them would be a real question of "WHY", simply why would anyone every think this was a good plan?

      Thirdly, the whole thing is lunacy because we know from experience that when it comes to keeping a big secret, the government is hopeless, along with the CIA and practically any other similar service. It seems the only secrets they can probably keep for any significantly long time are those related to things that are not readily known about by the public to begin with, not things like passenger aircraft slamming into the upper floors of the tallest buildings in New York. A government conspiracy that starts off with the government orchestrating such a public spectacle as that, would soon find itself so full of swiss cheese holes it would be a hopeless mess. Instead, despite the best efforts of conspiracy theorists to make it into some kind of conspiracy, far from coming up full of holes, its come up over and over making perfect sense that the entire event was pretty much exactly as explained from the beginning and nothing at all to do with any kind of government conspiracy.

      As I said earlier, why do we need this nonsense in our life?
      • Can you believe Major General Albert N. Stubblebine?‏

        As I said earlier, why do we need this nonsense in our life?

        You need it because it is not nonsense and you need it because the problems of the world will never be solved without first assessing reality truthfully. Without an accurate assessment of a problem you will not get to a solution. Be responsible and look at the data without looking through an emotional cloud.

        Don’t believe me. Will you believe Major General Albert N. Stubblebine?‏

        Over 2,000 accredited and working Architects and Engineers at AE911 must all be crazy then. Of course it doesn’t take much intelligence to know that a 47 story building with simple office fires and no other damage can come down at freefall through the path of greatest resistance in its own footprint. This has never happened, than God, in the history of the world, and never will again outside of controlled demolition. These aren’t theories. These are scientific facts. The only theoretical part of it is exactly who did it and exactly how.

        None of this stuff is crazy or weird. The only people who look at the evidence and do not get it are those who really don’t look at the evidence or are too emotionally involved to process the data accurately.

        Buildings NEVER collapse symmetrically outside of controlled demolition. It would require every single support pillar to fail at exactly the same time which certainly could not have happened in this situation. This isn’t rocket science man. Just use your common sense.
      • They have NOT been able to keep it secret

        "We know from experience that when it comes to keeping a big secret"

        Actually the Manhattan project has as many as 100,000 involved and it was a very well-kept secret.

        And more to the point - They have NOT been able to keep it secret. The secret of 911 has had thousands of leaks. It is not a secret. The only thing that keeps it from being 100% exposed is that the controlled mainstream media still hides the evidence and many people still believe everything that the MSM tells them.

        They haven't kept the secret. There is only a certain sector of the population that refuses to see what is in front of their eyes. Contrary to what you say that theorists, except for the infiltrators, stories hold up while the government's stories do not. You are free to believe what you choose but I would suggest that others not be dissuaded from at least looking at the evidence. Perhaps they will come to the same conclusion as you and that is all right, but at least they need to decide for themselves. As time passes however you will see your position as one of diminishing resonance.

        To your point "why do we need this" other than my obsession with the truth I can certainly respect the commenters on this blog site and stop beating a dead horse. I'm sure that you have better things to do than put up with my obsessions. We will see if I can restrain myself.
  • No, not properly classified

    Crimes, not just this example, are committed, violating the Fourth Amendment of the United States Constitution. The threat that these crimes pose to Democracy should be the ONLY factor in whether or not to classify this. These are lawbreakers who are hiding behind "National Security," nothing more.
  • Doubts

    Fair statements that suggest some of the folks in the NSA might conceivably be honest: but it doesn’t deal with the real problem: this system is self-evidently too complex to manage (see above), too easy to corrupt (see Lovenet); & ultimately & inherently dangerous, not to mention crazy (see Fukushima).
    Kootenay Coyote
  • I'm done reading

    johnfenjackson, I absolutely could not have put it better.
    "Gewirtz's support for the Government has long since descended below the feeble. I tire of it."
    Gerwitz, like Alexander and Obama, are either totally unaware of how naked their arguments are or they are totally cynical. Unfortunately, without mind reading powers nor deep knowledge what a mind is, these two states will always be isometric and indistinguishable when overlayed.
    • I have to agree about Gewirtz.

      Lately, when I see Gewirtz as the story author, I already know I'm going to hear propaganda from a self-important guy who makes money from the security industry. He spends his word count trying to convince us that the government has only good intentions and our best interests at heart. As long as the sheep keep believing these lies, he keeps making cash. I'm almost to the point of not reading anything Gewirtz writes anymore, as he obviously has a vested interest in perpetuating the myth that our police state government has noble intentions, despite the massive piles of evidence to the contrary.

      The reality is, our government is group of heavily armed thugs who break our laws daily and ignore the Constitution when its restrictions don't mesh with their own secret agenda. They feel they are above the law and act accordingly.

      The federal government is so out of control that our only hope would be for all of us to stop paying taxes at the same time and simply let them go bankrupt. Then, we could form a new government that would actually follow the rules set by the Constitution. Sadly, that will never happen because our nation is inhabited by a vast number of undereducated and frightened sheeple who are easily manipulated by fear. So, we're stuck in this situation until they finally declare complete martial law and everyone with a brain flees to Canada, Ecuador, or someplace in Europe like Switzerland. We're no longer "the land of the free and home of the brave." We're the land of the oppressed and the home of the weak-willed.