The Commonwealth Ombudsman report [PDF] into how 20 agencies across federal and state levels government agencies across Australia handle stored communications and metadata over the period of the 2016-17 financial year has been released, with Home Affairs being the only agency that was handed recommendations.
Home Affairs was called upon to ensure it could "accurately account for the number of telecommunications data authorisations it issues in any given period" to comply with its record keeping obligations, and have a central system to store or monitor telecommunications data once it had been handed to investigators.
The recommendations were a result of the former Department of Immigration and Border Protection (DIBP) having 8 record keeping issues identified, as well as a statistical issue, and 42 instances of telecommunications data being accessed outside the parameters of authority. The Ombudsman explained that 41 of those instances were due to an automatic input from DIBP's database which has since been resolved.
Also falling under the Home Affairs banner following its transferral into the Peter Dutton-helmed superministry is the Australian Federal Police, which disclosed that between October 13 to 26, 2015, all authorisations by ACT Police were not authorised, due to the AFP Commissioner failing to authorise any ACT officers for that period.
"In response to this disclosure, our Office suggested the AFP quarantine all telecommunications data obtained under the 116 authorisations made by the unauthorised ACT Policing officer between 13–26 October 2015 from further use and communication," the report said.
"Following the inspection, the AFP accepted this suggestion; however it did not act to quarantine the affected data at that time, which resulted in additional use and communication of the data."
Must read: Why Australia is quickly developing a technology-based human rights problem (TechRepublic)
In February 2018, the data was partially quarantined only after being prodded by the Ombudsman.
"In April 2018 the AFP advised the affected telecommunications data had not yet been fully quarantined and it was seeking legal advice regarding the use of the affected telecommunications data, the report said.
"Due to the scale of this non-compliance, we will continue to monitor this issue closely with the AFP."
The Ombudsman also found one area of NSW Police that routinely used its power without written or electronic approvals.
"The area's process at the time of our inspection was for access to telecommunications data to be verbally approved and a written record of the verbal approval to be made in a log," the report said.
"We do not consider this practice is permitted by the Act and suggested to NSW Police that it review its policies and procedures to ensure all authorisations for telecommunications data are in written or electronic form and signed by the relevant authorised officer."
Overall, the report said agencies were "generally exercising" their powers appropriately.
In the prior installment of the report released in 2017, which covered the 2015-16 financial year, Australian Customs was handed the only three recommendations contained within the report.
"In our view, Customs does not have sufficient processes in place to demonstrate that it is only dealing with lawfully accessed stored communications," the report said.
The Ombudsman said when it conducted its inspection on February 18, 2016, there was no stored communications product that would allow it to determine if Customs was only dealing with lawfully accessed information, and no records were kept to determine whether telcos had stored information when relevant warrants were in force.
"In every instance, we were unable to determine who had received the stored communications from the carrier and, therefore, whether the communications had been properly received in accordance with s135 of the Act," the report said.
"Furthermore, for the assessed destruction of stored communications, there were no records available to demonstrate who within Customs had authorised the destruction to occur or when the approval had been granted.
The Ombudsman said Customs was not equipped with the appropriate procedures to determine whether stored communications were properly received and destroyed.
It was also found that 11 of 14 historic preservation notices covered by the inspection were made by officers who were not authorised to apply for stored communications warrants, but Customs has subsequently adjusted its nominated officers after the Ombudsman visit.
The first two recommendations made by the Ombudsman called for Customs to have processes to demonstrate it was only dealing with lawfully accessed stored communications, and that the agency manage stored communications in accordance with the Telecommunications Act.
The third recommendation called for Customs to have a new record keeping system as the Ombudsman had found a litany of issues.
On the issue of whether the agency was cooperative and frank, the Ombudsman found it was "Non-compliant with the exception of staff from one branch, who were cooperative and attempted to provide our office with access to relevant information."
Customs was not well prepared for its inspection, the Ombudsman added.
Echoing the sort of excuses it would use again in the future, Customs said the period covered was "a time of significant organisational disruption" as it shifted across to the DIBP, before it would be subsumed into Home Affairs.
On the issue of data retention, across three inspections of the DIBP between November 2015 and May 2016, the Ombudsman was told that while authorisation was at the senior executive level, authorisations were mostly made by those who were at the non-executive level.
Although no other agencies were given recommendations at the time, the Australian Competition and Consumer Commission, Australian Federal Police, Crime and Corruption Commission Queensland, Independent Commission Against Corruption South Australia, Tasmania Police, Victoria Police, and Western Australia Police all had to undertake remedial action.
In November 2017, a Commonwealth Ombudsman report into how the Australian Federal Police managed to trip over the one caveat in Australia's metadata retention system -- needing a warrant to access the metadata of a journalist when attempting to identify a source -- found AFP officers did not "fully appreciate their responsibilities" when using metadata powers.
The one recommendation from the report called on the AFP to make all staff that use metadata powers undergo training to have a "thorough understanding" of the laws and their responsibilities.
Australia's data retention laws, passed by both major parties in March 2015, force telecommunications carriers to store customer call records, location information, IP addresses, billing information, and other data for two years, and make this information accessible without a warrant by law-enforcement agencies.
Last week, the Australian Communications and Media Authority said nine telcos did not have such a system in place as of June 30, 2018.
The cost to telcos of developing, installing, and maintaining an interception capability on their networks has slowly decreased, from AU$22.6 million in 2015-16 down to AU$22 million in 2016-17, and then further down to AU$21.5 million in 2017-18.
The ACMA's 2017-18 report also reported almost 2.3 million disclosures of customer information by carriers under legal obligations to assist government agencies and departments.
With a disclosure notice or court order, government agencies otherwise exempted are able to tap Australia's metadata stores.
The Communications Alliance has detailed a list of agencies that tried to access telco metadata following the introduction of Australia's metadata retention regime.
The Department of Home Affairs has been told law enforcement and national security agencies are already using the Act as the department continues to 'support' its implementation.
ASIO will immediately seek to use the legislation when it comes into force.