Census reports highlight government IT incompetence

As details of the train wreck that was Australia's 2016 online Census have emerged, ZDNet has observed how it was a confluence of failure, indeed, an omnishambles of fabulous proportions. It was inexcusable.

On Thursday, though, came two reports that not only provided further evidence of incompetence at the Australian Bureau of Statistics (ABS), but they were also official. Their subtitles said it all.

One, the report of the Senate Standing Committees on Economics inquiry into the Census, was subtitled "Issues of trust".

Two, the Review of the Events Surrounding the 2016 eCensus [PDF] by the Special Advisor to the Prime Minister on Cyber Security Alastair MacGibbon, was subtitled "Improving institutional cyber security culture and practices across the Australian government".

There's a lot of overlap between the two reports. Much of it we already knew, at least in broad terms.

"There appears to have been significant and obvious oversights in the preparation of the eCensus," said the Senate report (section 6.82), a masterpiece of understatement.

We knew, for example, that the eCensus application fell over in the face of tiny distributed denial of service (DDoS) attacks, even though prime contractor IBM had reassured the ABS that all was well.

We now know the timeline in more detail, and how little testing was done on the "Island Australia" strategy for DDoS mitigation -- that is, simply blocking all traffic from outside the country.

"The testing was limited -- IBM simply activated 'Island Australia' for 10 minutes and monitored the system for international traffic while IBM tried to access the system from overseas," the MacGibbon report said.

"IBM and the ABS's documents indicate a mismatch between the risk, the intended mitigation, and the implementation of the 'Island Australia' strategy."

The ABS and IBM are still arguing about whose fault that was. Indeed, it seems to be the key issue that'll keep the lawyers busy.

We knew that while the eCensus application collapsed on Census Night, the ABS communications strategy collapsed along with it. As Twitter was flooded with complaints, for example, the ABS continued to tweet that things were running "smoothly as expected".

We now know that the communications strategy, focused solely on awareness-raising, was doomed from the beginning.

"The ABS failed to adapt its media and communications in response to the public relations storm that was brewing in the weeks prior to the Census regarding privacy and security in both mainstream and social media. Instead, the ABS stuck rigidly to its plans, foregoing crucial opportunities to influence and drive the conversation around the Census," MacGibbon wrote.

The ABS' "social media crisis escalation matrix" had two main flaws. The most critical "red level scenario" for negative conversation was enacted only if someone had 10,000 or more followers, or a post had over 30 engagements. And the response to such a scenario was simply to hold all social media communications.

The ABS had also decided that the "primary vehicle" for engaging with the public about the extended plan to retain people's names and addresses would be the privacy impact assessment (PIA), a bureaucratic document, with public consultation limited to a mere four weeks just before Christmas 2015.

As the Senate report put it (section 4.76), "It is apparent to the committee that level of consultation undertaken by the ABS in the lead-up to this decision [to retain names and addresses] was manifestly inadequate, especially considering the changes affect every Australian household. At a minimum, the PIA should have been conducted by an independent body."

The Senate also noted (section 4.36) that a 2005 PIA foresaw privacy issues with name and address retention, so the ABS decided to link only 5 percent of names and addresses.

The 2015 PIA for the 2016 Census, however, concluded that linking 100 percent of names and addresses would be OK (section 4.34), because the likelihood of the "small number of potential risks to personal privacy and public perception of the ABS" eventuating would be "very low".

"The ABS appeared unable to explain why the results of the 2005 PIA were significantly different from the 2015 PIA," the Senate noted (section 4.36).

Taken together, the reports document a comprehensive failure of ABS management. The ABS failed to understand and manage the IT aspects of the most important project on its agenda. And the ABS failed to notice that the world outside had moved on, both in its understanding of privacy risks, and in the use of social media in opinion formation.

Like so many organisational failures, it all rested on cultural problems.

MacGibbon's report, for example, cited a damning 2014 review by independent consultancy CapDA of the ABS' capacity and capability to run the Census.

Rigorous project management was "not strongly embedded" in ABS culture, and the organisation's approach to agile development meant "security, high performance, and accessibility [were] considered late in the cycle".

"There was no evidence that any application or datacentre performance monitoring is in place," CapDA wrote.

"It was unclear where, and in whom, the responsibility and authority is vested for making key architectural decisions."

MacGibbon observed that preparations for the Census took place during a "complex time" for the ABS. The position of its leader, the Australian Statistician, was vacant for most of 2014. For me, that points to another failure -- one of succession planning.

"However, it is clear that the ABS's culture clearly contributed to the outcomes on Census Night. The ABS's actions since only underscores the importance of culture: It has steadfastly refused to own the issue and acknowledge responsibility for the factors leading to the events and shortcomings in the handling of events on the night," MacGibbon wrote.

The Senate report makes 14 recommendations, most of them obvious from the problems uncovered: Consultation requiring active engagement with the non-government and private sectors, not just government agencies; open tendering proceses, not the vendor lock-in with IBM; more rigorous testing of the eCensus application; PIAs to be conducted externally, not by the ABS itself; and so on.

Recommendation 11 had me slightly baffled, however.

"The committee recommends responsible ministers seek six-monthly briefings on the progress of Census preparations. These briefings should cover issues including, but not limited to, cybersecurity, system redundancy, procurement processes, and the capacity of the ABS to manage risks associated with the Census," it says.

How ministers are meant to understand these briefings when one government minister doesn't even understand how decimals work remains a mystery.

For me, though, the key issue is this entire exercise comes once more from the CapDA review, this time as cited in section 6.70 of the Senate report.

"CapDA's report highlights the professionalism and dedication of the staff at the ABS, but in the end recommends that the ABS did not have the internal capacity to develop and deploy an eCensus. If they did not have the ability to develop a solution themselves, it stands to reason that they would only have a limited capacity to question and challenge a contractor employed to develop such a solution."

If no one in your organisation understands the technology, then you can't manage technology projects, whether they're executed by internal staff or external contractors.