The real story in the NSA scandal is the collapse of journalism

Updated June 9 to include details of the Guardian's coverage, a link to the Post's correction policy, and a quote from the Huffington Post.

Updated June 10 to include a quote from a follow-up article in the Post directly contradicting its initial claims and another observation after the release of the leaker's identity.

On Thursday, June 6, the Washington Post published a bombshell of a story, alleging that nine giants of the tech industry had "knowingly participated" in a widespread program by the United States National Security Agency (NSA).

One day later, with no acknowledgment except for a change in the timestamp, the Post revised the story, backing down from sensational claims it made originally. But the damage was already done.

The primary author of the story, Barton Gellman, is a Pulitzer Prize winner, and the Washington Post has a history in investigative journalism that goes back to Watergate and All the President's Men. On a roster of journalistic failures, this one has to rank near the very top.

This story was part of a busy week for attention-grabbing stories on the topic of U.S. Government surveillance. The Post was playing catch-up to the Guardian, whose UK and US editions had broken numerous stories, several of them by-lined by Glenn Greenwald. On Wednesday, the Guardian had published details of a Top Secret court order that required Verizon to hand over records disclosing the call data of millions of its customers. On Friday, they published another classified document outlining a U.S. Presidential Policy Directive to draw up a hit list for cyber-attacks.

And on Thursday, shortly after the Post published its story, the Guardian went public with a similar story about NSA surveillance. Based on comments by the author of the Post article, that timing is not coincidental.

Related coverage on ZDNet:

• Verizon records vacuumed up by NSA under 'top secret' Patriot Act order
• U.K. government 'complicit' in NSA's PRISM spy program
• PRISM: Here's how the NSA wiretapped the Internet
• NSA 'top secret' spying order affects millions of Americans: FAQ
• Obama's secret order draws up overseas target list for cyberattacks
• Boundless Informant: US gov't collects 100 billion surveillance records a month

The allegations by the Post are shocking:

The National Security Agency and the FBI are tapping directly into the central servers of nine leading U.S. Internet companies, extracting audio, video, photographs, e-mails, documents and connection logs that enable analysts to track a person's movements and contacts over time.

The story alleges that the NSA is "reaching deep inside the machinery of American companies that host hundreds of millions of American-held accounts on American soil." It specifically names nine companies: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple. And the story alleges, "From inside a company's data stream the NSA is capable of pulling out anything it likes."

Within hours after the story broke, it had been amplified by other news agencies and tech websites and had inspired expressions of outrage over this invasion of privacy. And seven of the nine companies named issued categorical denials that they knew of or participated in any such program.

And then a funny thing happened the next morning. If you followed the link to that story, you found a completely different story, nearly twice as long, with a slightly different headline. The new story wasn't  just expanded; it had been stripped of key details, with no acknowledgment of the changes. That updated version, time-stamped at 8:51 AM on June 7, backed off from key details in the original story.

Crucially, the Post removed the "knowingly participated" language and also scrubbed a reference to the program as being "highly classified." In addition, a detail in the opening graf that claimed the NSA could "track a person's movements and contacts over time" was changed to read simply "track foreign targets."

Here's what a key paragraph in the story originally looked like in a browser window:

And here's the same paragraph, as it appeared in the heavily edited story the next day. Note that the Post's editors had added so much to the story that this paragraph was pushed to the second page in the four-page online version:

I saved a copy of the original story and used the "compare documents" feature in Microsoft Word to show the before and after versions. You can see the differences for yourself in the redlined document saved here.

Declan McCullagh of CNET examined the Washington Post story independently and concluded that the Post story was wrong.

Those reports are incorrect and appear to be based on a misreading of a leaked Powerpoint document, according to a former government official who is intimately familiar with this process of data acquisition and spoke today on condition of anonymity.

"It's not as described in the histrionics in the Washington Post or the Guardian," the person said. "None of it's true. It's a very formalized legal process that companies are obliged to do."

The real story appears to be much less controversial than the original alarming accusations. All of the companies involved have established legal procedures to respond to warrants from a law enforcement agency or a court. None of them appear to be participating with widespread surveillance.

Update June 10 - In a separate story published over the weekend (with Barton Gellman listed as one of three authors), the Post further backed down from the "direct access" claim:

One top-secret document obtained by The Post described it as "Collection directly from the servers of these U.S. Service Providers: Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Skype, YouTube, Apple."

Intelligence community sources said that this description, although inaccurate from a technical perspective, matches the experience of analysts at the NSA. From their workstations anywhere in the world, government employees cleared for PRISM access may "task" the system and receive results from an Internet company without further interaction with the company's staff.

In intelligence parlance, PRISM is the code name for a "signals intelligence address," or SIGAD, in this case US-984XN, according to the NSA's official classified description of PRISM and sources interviewed by The Post. The SIGAD is used to designate a source of electronic information, a point of access for the NSA and a method of extraction. In those terms, PRISM is a not a computer system but a set of technologies and operations for collecting intelligence from Facebook, Google and other large Internet companies.

According to a more precise description contained in a classified NSA inspector general's report, also obtained by The Post, PRISM allows "collection managers [to send] content tasking instructions directly to equipment installed at company-controlled locations," rather than directly to company servers. The companies cannot see the queries that are sent from the NSA to the systems installed on their premises, according to sources familiar with the PRISM process.  [emphasis added]

So what went wrong with the Post?

The biggest problem was that the Post took a leaked PowerPoint presentation from a single anonymous source and leaped to conclusions without supporting evidence. McCullagh quotes one of his named (not anonymous) sources, former general counsel of the NSA Stewart Baker, as saying the slides look "flaky":

"The PowerPoint is suffused with a kind of hype that makes it sound more like a marketing pitch than a briefing -- we don't know what its provenance is and we don't know the full context," Baker said. He added, referring to the Post's coverage: "It looks rushed and it looks wrong."

"Rushed" would indeed be the best way to describe why the Post story changed so dramatically in a 24-hour period. Normally, an investigative piece like this would be reported thoroughly before being published. Instead, it looks like the Post rushed to publish, perhaps fearing that the slide deck had been leaked to another publication that would beat them to the punch.

Update: A Huffington Post story seems to confirm the theory that this post was published prematurely:

Recently, it became clear to Gellman that the scoop might be broken elsewhere and so the Post "decided to push it through." Gellman said he "would have been happier to have had a day or two" more to work on the PRISM story, but it was clear, for competitive reasons, the Post had to move quickly.

Almost no one who reacted to the story initially did so with any skepticism about the Post's sources or its conclusions. Indeed, a common thread among reactions to the denials by those big tech companies was that they were using careful wording and common talking points to avoid responding to the specific allegations. In fact, the wording of those statements was similar because each company was responding to the specific language in the Post and Guardian stories.

That rush to publication set off the Internet echo chamber and the cable news networks at a full-throated roar. The story and its key, now apparently discredited arguments have been spread far and wide.

The Post compounded its error by quietly correcting its story and not publicly acknowledging that there were errors in the original story. It's worth noting that the Post has a published policy for how its writers and editors are supposed to handle online corrections. Here's what that policy, headlined "Digital Publishing Guidelines - Clarifications and corrections," says:

When a correction is made online, the story editor is responsible for alerting universal, home-page and social teams to make the necessary changes to headlines and blurbs. The change should be made within the article and the correction should also be noted at the top of the item.

Articles

Clarifications and corrections should be clear, concise and direct. They must be comprehensible to anyone who reads them, including readers who may have missed the story that is being corrected. Anyone reading the correction should be able to understand how and why the mistake has been corrected.

More than 48 hours after the story was significantly edited, there's no such "clear, concise, and direct" explanation for the substantive changes on the online article.

In fact, the revised story still claims the NSA and the FBI are "tapping directly into the central servers" of those companies when that allegation no longer appears to be true.

Update June 10: And one more thing. In its original story, the Post calls the source of the documents "a career intelligence officer" who provided these materials "in order to expose what he believes to be a gross intrusion on privacy." We now know that the source was Edward Snowden, who was not an intelligence officer but an "infrastructure analyst" who had been in his current position with an external contractor for only three months. The "career intelligence officer" description seems exaggerated. 

In short, one of the great journalistic institutions of the 20th Century is now engaged in outright click-baiting, following the same "publish first, fact-check later" rules as its newer online competitors.

Something tells me the Pulitzer committee won't be considering this story for next year's awards.