ATO summons expert to review 'unusual and unfortunate' outage

Commissioner of Taxation Chris Jordan has announced an independent review into the "unprecedented failure" of the Australian Taxation Office's (ATO) storage hardware that took its online services offline earlier this week.

The ATO's website, tax agent, and business portals initially crashed on Monday, as a result of a "world first" hardware issue. The outage continued through Tuesday, which is when the ATO called in Hewlett Packard Enterprise (HPE) to help it determine the underlying cause of the problem that the ATO said was encountered for the first time anywhere in the world.

Jordan said on Friday it was the ATO's worst unplanned system outage in recent memory.

"This was an extremely unusual and unfortunate event," he said in a statement. "The issues we have experienced this week do not relate to our overall IT capability or skills."

Almost everything was back up and running on Wednesday morning, when the tax office admitted that it did experience "some" data corruption -- not an entire petabyte, however -- as a result of the hardware-related incident, and noted it was in the process of having the data fully restored from a back-up.

"We are realistic that there may be some intermittent performance issues in the next couple of days as the full restoration process proceeds," the commissioner said Friday.

The storage hardware in question was upgraded in November 2015 by HPE and was seen by the ATO to be "state-of- the-art" at the time, with the ATO noting it is "basically the same" hardware used by other large clients of HPE.

"What compounded the problem beyond the initial failure was the subsequent failure of our back-up arrangements to work as planned," Jordan explained. "The failure of our back-up arrangements meant that restoration and resumption of data and services has been very complex and time consuming."

"We, and HPE, will continue to work on the stability and performance of all of our systems and we will have staff working over the weekend to catch-up on any backlog of work."

Jordan said the review will be conducted by an independent expert who will determine the nature of the failure(s) and their root cause(s), the adequacy of back-up and contingency arrangements, and the likelihood of recurrence.

It will also consider the ATO's response to the failure, including how it managed business resumption processes and the effectiveness of its communication on the outage.

"I will be doing everything I can to learn from what has happened this week and to put in place any necessary changes to minimise the risk of any recurrence," Jordan said. "Please be assured that I am committed to providing a quality, reliable, and contemporary service to the Australian community."