'The elephant in the room': Data retention's controversial origins

It began with a secret consultation paper in 2009, secret meetings were held with industry, it went public in 2010, and the government finally outed a proposal for requiring internet service providers (ISPs) to retain customer data for up to two years in 2012, leading to criticism from the public, privacy advocates, and industry. But the minutes of the original meetings show that the government knew the proposal would be controversial from the very beginning.

ZDNet first revealed in 2010 that the Attorney-General's Department had been meeting with the telecommunications industry over a potential proposal that would require them to retain customer data for law-enforcement investigations. Industry sources had said at the time that there was a consultation paper that the government had put together explaining the proposal, but this was never made public.

The paper, entitled Carrier-Carriage Service Provider Storage Model, was released yesterday with minutes from the meetings between 2009 and 2012, following a Freedom of Information request filed by Pirate Party Australia secretary Brendan Molloy.

The consultation paper shows that the government considered multiple methods to keep the data, including one where the carriers hand over the data to a government repository, with the law-enforcement agencies then able to access the data when required. The second model proposed the use of a private third-party provider to hold the data, while the third is the one that the government settled on — providers storing their own customers' data.

While heavily censored, the minutes from the 2009 meeting do show early concern about the system from the telcos, with one participant suggesting that two years was too long, one asking who would have access to the data, and another questioning who would bear the cost for implementing the data-retention system.

The minutes for meetings in 2010 (PDF) were almost completely censored, but in the minutes for the 2011 meetings (PDF), one unnamed telco revealed that it "generally keeps consumer stuff for two years for possible [Telecommunications Industry Ombudsman] investigations".

When questioned by one telco representative about how useful any change in law would be if technology adjusts again and makes the new law redundant, Attorney-General's Department first assistant secretary Catherine Smith said that the government couldn't make any guarantees.

"The reality is, government puts forward policies which the industry may or may not agree with. The reality is [the industry] has been reminded by law enforcement of the benefits to society of these particular policies (child safety, protection, etc)," the minutes recorded Smith as saying.

Tensions were mounting in the discussions, with the minutes referring to data retention as "the elephant in the room".

"Need to work more on this, have intense discussions just about data retention. Must discuss more in-house and with industry as well," the minutes said.

In the December 2011 meetings, the government proposed a potential co-regulatory model that would see the industry agree to hold data, but one industry stakeholder expressed concern about whether every internet service provider would participate in such a scheme.

By 2012, shortly before the release of the public discussion paper on telecommunications security legislation reform, the discussion had moved on to how it would be implemented. This included whether new entrants starting up on the National Broadband Network (NBN) would meet the threshold required to retain customer data.

"There may be a completely different set of circumstances as new players enter the industry, so current threshold requirements are likely to change," said an unnamed attendee.

A joint parliamentary committee is currently considering whether implementing a two-year data-retention scheme would be feasible. The committee has yet to report back to parliament with its findings, and it is unlikely that any legislation arising from this report would pass through the parliament before the September federal election.