'

IBM staff avoid sacking or any discipline over Census failure

Big Blue has stuck to its claims that geoblocks would provide enough protection for the online Census against DDoS attacks, and that Nextgen was to blame.

IBM Australia and New Zealand managing director Kerry Purcell has told the Senate Economics References Committee investigating the August 9 Census bungle that the company has not sacked, nor disciplined, any of its staff over the issue.

"Directly related to the Census, the answer is no," Purcell told the committee on Tuesday.

Purcell opened the company's appearance by offering an unreserved apology for the events on the night of August 9, which saw the Census website taken down, and said the company was in discussions with Treasury on how to resolve the AU$30 million set to be spent by the Australian Bureau of Statistics (ABS) in total on remediation costs.

Under questioning, Big Blue representatives repeated many of the arguments in its submission to the inquiry last week, and continued to stand behind its geoblocking procedures as the correct way to defend against distributed denial-of-service (DDoS) attacks.

Of the four DDoS attacks suffered on the night, IBM said the first one peaked at 3Gbps, the second one a mere 210Mbps, but could not report the totals for the third or fourth ones. In its submission, Vocus said it saw traffic peak at 563Mbps, and the attack lasted 14 minutes.

By comparison, the attack that took down Brian Krebs' blog last month reached 665Gbps.

IBM maintained that its Island Australia plan for the Census, which geoblocked traffic from outside Australia, was a simple and effective mechanism that the ABS signed off on. Big Blue said it had no reason to doubt whether ABS had the technical expertise to agree to its plan.

"In summary, the geoblocking approach was reviewed with the ABS and signed off by ABS, it was reviewed and discussed with ASD. I'm not aware that ASD ever passed comment saying one way or the other whether they endorsed it or not," Michael Shallcross, IBM distinguished engineer, told the committee.

The Australian Signals Directorate (ASD) was asked by ABS to review the security of the Census site, but it declined to do so, IBM said last week in its submission.

Shallcross reiterated on Tuesday that there was a "qualitative" difference in the nature of the traffic that made up the fourth DDoS attack, which eventually led to IBM's web servers exhausting their resources, particularly the number of threads used on the systems.

Even though it was offered DDoS protections by its upstream partners, IBM said it did not take up those offers for a number of reasons: One being it involved a four week training period to establish a traffic pattern, and the spike on Census night might appear as a DDoS attack; deep packet inspection was dismissed due to privacy constraints; and the NextGen solution may interfere with the load balancing mechanism put in place by IBM.

"We felt that the geoblocking was a very well-adapted solution for the particular characteristics of the traffic, and it's one which we had experience in 2011 with both Telstra and Optus, that they could very effectively and easily implement," Shallcross said.

Once the Census site came back online, both Telstra and NextGen DDoS solutions were in place, IBM said.

When asked what it would do differently, IBM admitted it would do a hard power cycle test on its routers, and said NextGen and Vocus should have improved their internal communications to explain the intent of the Island Australia plan.

The contract for the online Census between the ABS and IBM was initially valued at AU$8.7 million, with AU$1 million of variations made mostly to cater for mobile visitors to the site. For the 2011 Census, the initial value of the contract was AU$6.8 million, with AU$2.4 million of change orders made.