It should not be a surprise that Australia's metadata enforcement agencies have taken to warrantless access to two years' worth of customers' call records, location information, IP addresses, billing information, and other data stored by the nation's telcos in much the same manner that a squabbling mass of toddlers enjoy endless servings of cake.
In a number of submissions made to the Parliamentary Joint Committee on Intelligence and Security review of the data retention regime that came into being on March 2015, the agencies were generally happy with the regime.
Some agencies, like the Australian Commission for Law Enforcement Integrity, would in an ideal world like to see the two-year period for retention be stretched to a longer period.
"It will be many years before the telecommunications data which is presently still retained by telecommunications providers, outlives its usefulness to law enforcement," it said.
"The dangers of mandating a minimum retention period include the possibility that telecommunications providers, which presently retain more data than is required under the regime, will eventually, and perhaps sooner rather than later, reduce their holdings, and that all providers will treat the minimum as a maximum."
However, a general feeling amongst the agencies was the wish to gain greater transparency for the charges they must pay to telcos to access retained data.
Free PDF: Australia's encryption laws: An insider's guide
Despite a submission from the Department of Home Affairs (DHA) that is not yet available, a submission from the Australian Securities and Investments Commission (ASIC) revealed some of the department's intentions.
"ASIC holds similar concerns to those expressed in the portfolio submission by DHA that the costs charged by some service providers are unclear, inconsistent, and lack transparency," it said.
"It has been ASIC's experience that it is often difficult to understand and reconcile the significant discrepancies between some service providers for access to comparable datasets.
"ASIC supports the recommendation by the DHA for a review of the charging and request frameworks between agencies and providers."
Victoria's Independent Broad-based Anti-corruption Commission (IBAC) further said it had issues with no being able to share retained data with other agencies, despite being able to do so with data gained from the "more intrusive" intercept regime of the Telecommunications (Interception and Access) Act (TIA Act).
IBAC added it too wanted standardised processes and costs, while WA Police added a standardised format for data would be useful.
"There is no requirement under the TIA Act for carriers to provide advice or instruction on how to 'read' or interpret the data," WA Police said.
As a response to the uptake of encrypted over-the-top services, the Australian Criminal Intelligence Commission said it had "substantially increased its access to telecommunications data retained for a period between six and nine months."
"Further, new technologies entering the Australian market, particularly 5G, will likely give consumers more options to achieve digital anonymity, which will create further significant challenges for intelligence and law enforcement agencies," it said.
However, other submissions argued the introduction of 5G would further reduce the privacy of Australians as the data retention regime would keep their location, sometimes down to a metre, for two years.
The New South Wales Law Enforcement Conduct Commission said it makes use of telco data in 90% of investigations, and has helped with physical surveillance.
"Historically it can be said that around 30% of all surveillance deployments were inefficient due to the absence of the target or the inability to locate the target," the ECC said.
"With access to phone mapping, this situation is nullified in that deployments are able to be more targeted with this knowledge of the target's current location."
Meanwhile: Commonwealth Ombudsman singles out Home Affairs over stored communications and metadata handling
On the opposite side of the equation, the Australian Human Rights Commission (AHRC) relied heavily on the 2014 Digital Rights Ireland case in the European Court of Justice that saw the EU data retention directive be tossed aside.
"The Commission considers that the mandatory data retention regime goes beyond what can be reasonably justified," the AHRC said.
The most pressing issue for the AHRC was the need for warrants or a court authorisation body to approve access to retained data, followed by a reduction in the two year window, and only being able to access it in the case of " sufficiently serious crimes".
As an addition to the regime, the Commission called for penalties for inappropriate access or misuse of personal data.
"The Commission is concerned that the operation of the regime in its current 'catch-all' form is not a proportionate restriction of the right to privacy and freedom of expression," it said.
When the metadata laws were passed, access was reduced to 21 enforcement agencies; however, subsequently, 61 agencies that previously had access to metadata looked to be added as declared enforcement agencies.
As reported previously by ZDNet, the Attorney-General's Department (AGD) had been advising agencies and departments to attempt to access metadata through other means.
"On advice from the Attorney-General's Department, the department has considered other methods of obtaining metadata using statutory coercive powers under portfolio legislation, and by engaging the Australian Federal Police (AFP) to obtain metadata," the Department of Agriculture and Water Resources wrote in a letter dated 10 June 2016, and published on RightToKnow.
"The department has received preliminary legal advice as to the merits of using coercive powers, which suggests that the approach is problematic due to the construction of portfolio legislation.
"Advice received from the AFP indicates that it does not have the resourcing, compliance, or risk considerations to obtain metadata on behalf of other agencies, including the department."
In November, the Communications Alliance detailed a list of agencies that tried to access telco metadata following the introduction of Australia's metadata retention regime.
The industry group pointed out that a request for metadata does not mean data was disclosed. It was not possible to accurately compile how many requests and disclosures were made.
Shock, horror: Home Affairs cannot be bothered listing all agencies with access to metadata
"We have seen, for example, one carrier that made 132 disclosures in response to 114 requests over a 12-month period, while some other carriers have experienced smaller volumes over similar periods," it said.
"Determining volumes is further complicated by the fact that while responses to some requests are derived from the mandatory data retention store, some requests can also be met by interrogating business systems or databases that hold similar or identical information for commercial use."
The list contained four local councils, Centrelink, and the Victorian Institute of Teaching.
Comms Alliance added that its list might not be complete.
A month later, the Department of Home Affairs said it would take "considerable time and resources" for it to determine how many agencies across Australia's three tiers of government have accessed metadata held under the nation's data retention laws.
"Section 280(1)(b) of the Telecommunications Act 1997 creates an exemption to the general prohibition against the disclosure of metadata for Commonwealth, state, or territory entities that are not enforcement agencies," Home Affairs said.
"The authorities that can utilise this exemption are not specified."
Agencies that have the power to order the disclosure of information could force the issue with a court order or notice to produce powers, the department said.
"Listing all Commonwealth, State, and Territory agencies with this existing lawful authority would take considerable time and resources. Examples include tax authorities and corrective services," it said.
In March 2017, AGD said it had no issue with the ability of government agencies to make demands on telco data outside of the scope of Australia's data retention laws.
"There have long been provisions in the Telecommunications Act 1997 allowing records, including telecommunications data, to be disclosed where required or authorised by law," a spokesperson for AGD told ZDNet at the time.
"These powers are distinct from the data retention regime set out under the Telecommunications (Interception and Access) Act 1979."
In its response to ZDNet, AGD did not say it would look to prevent agencies from accessing metadata by other means.
Australia's data retention regime came into being after it was supported by both major parties in Parliament.
Speaking in June 2016, then Shadow Communications Minister Jason Clare said Labor helped "fix" the government's data retention legislation.
"The changes we forced the government to make mean tighter rules, and for the first time real oversight over the use and misuse of this data," Clare said.
Encryption laws are creating an exodus of data from Australia: Vault
Detrimental effects are both real and perceived, according to Australian cloud provider.
Commonwealth Ombudsman singles out Home Affairs over stored communications and metadata handling
Continues trend of former Department of Immigration agencies dragging the chain.
Home Affairs cannot be bothered listing all agencies with access to metadata
With a disclosure notice or court order, government agencies otherwise exempted are able to tap Australia's metadata stores.
Amendments to Australia's encryption laws stranded before election
Once again, Labor has been Charlie Brown to the Coalition's Lucy with the football.
Employees not the target of encryption laws: Home Affairs
Australian developers really do need to relax. Cops and spooks are being told very clearly that the Assistance and Access Act isn't for dragooning you into deceiving your bosses.