Leaked paper reveals Australia's obsessive metadata secrecy

The Australian government has been discussing a detailed data-retention wish list with internet service providers for more than four years — with citizens kept in the dark.
Written by Stilgherrian , Contributor

Last Friday, the Australian Attorney-General's Department sent internet service providers (ISPs) a confidential discussion paper — subsequently leaked to Fairfax Media — that attempts to clarify exactly what metadata they'll be required to store under the government's proposed mandatory data-retention scheme. The detailed requirements are presumably designed to feed into the "statutory specification" of metadata that will be included in legislation to be introduced to parliament in coming weeks.

Until now, the only official government description of metadata we'd seen — apart from that breathtakingly confused TV performance by Australia's favourite Attorney-General Senator George Brandis QC — was the hilariously inadequate one-pager (PDF) that the Attorney-General's Department (AGD) tabled in Senate Estimates on October 15, 2012, after much prodding by Greens Senator Scott Ludlam.

You might therefore think that the description of the government's metadata needs in Friday's document was a recent development.

You'd be wrong.

A confidential document obtained by ZDNet shows that even more detailed descriptions of the government's data-collection ambitions had been discussed with ISPs as far back as early 2010.

The document, Carrier-Carriage Service Provider Data Set Consultation Paper version 1.0 (PDF), is a 16-page PDF file created on March 9, 2010, at 14:49. Its core sections are similar in structure to the nine-page document obtained by Fairfax Media this week, with the addition of tables of "sample data to further illustrate the expected type of data to be retained for each specific retention requirement from the data set", discussion questions for industry to answer, and an introductory background section rather than an executive summary.

The 2010 version of the document was quite specific about the data to be collected. For mobile calls, for example, the data would include the IMSI and IMEI of both the calling party's and called party's devices, whereas the current version simply specifies the "identifier(s)" of the devices. This is in line with the government's intention to make the legislation technology neutral.

References to web-browser sessions and file transfers that were in the 2010 version have vanished, too, in line with such ideas being dropped as the data-retention debate has evolved.

The document made clear that it was a "proposed data set" and a "basis for dialogue".

Industry was asked to comment on eight questions:

  1. Which elements of the proposed data set are presently retained? For those retained, how long are they retained for, and for what reason?
  2. How much storage space is required to store data currently being retained?
  3. Is the majority of your network equipment ETSI LI compliant?
  4. Which requirements of the proposed data set are presently not retained?
  5. Are there major technological changes required to retain any of the requirements of the proposed data set? If so, what are they?
  6. Should data retained under this regime be available to the C-CSP for commercial purposes?
  7. Should a mandatory data-retention regime apply to all telecommunications industry participants?
  8. Are there significant issues associated with a 12- to 18-month lead time for full implementation of a data regime? If so, what are they?

This document had previously been released under Freedom of Information in 2010 — but with 90 percent of the content blacked out, including the entire description of data to be collected. Disclosure of the document uncensored "could be misleading to the public and cause confusion and premature and unnecessary debate," wrote a legal officer in the AGD's FoI and Privacy Section, Claudia Hernandez, at the time. "As the matters are not settled, and proposed recommendations may not necessarily be adopted, release of such documents would not make a valuable contribution to public debate."

It's those redactions that make this leaked, unredacted version most interesting. The description of the data set to be collected, yes, but also certain glossary entries, the question of whether ISPs would be able to make commercial use of the data, the proposed time frame — even the rather obvious points that metadata "can be used to reveal associations between members of criminal organisations" and "that new requirements be introduced to ensure that the telecommunications data currently retained continues to be available for law enforcement and national security purposes".

Even the headings "A. Data Set" and "B. Data Set Explanatory Statements" were redacted, as well as the part of the glossary entry for BRAS, a Broadband Remote Access Server that said it can "provide unique identifiers such as IP address to subscribers" — which is really just a straightforward description of what a BRAS does.

Surely there comes a point where secrecy goes beyond protecting law enforcement and intelligence methods, and beyond preventing "premature and unnecessary debate" — assuming such prevention is a desirable thing in a democracy — and becomes obsessive.

Editorial standards