How the ABS prepared for the same-sex marriage survey using the public cloud

Given a go-live date from Prime Minister Malcolm Turnbull of around four weeks, the Australian Bureau of Statistics turned to AWS to run the online and call centre components of the same-sex marriage survey in the public cloud.
Written by Asha Barbaschow, Contributor

It was announced on September 7, 2017, that Australia would be participating in a non-binding postal survey on whether people of the same sex should be extended the right to marry.

Everyone who enrolled to vote in Australia was sent a form via post, with those out of the country or unable to complete the physical form given the option to respond online.

Unlike electoral voting, which is compulsory in Australia, responding to the same-sex marriage survey was voluntary, and the government did not know how many people were expected to provide a response.

Voting in Australia is usually conducted, or at least overseen, by the Australian Electoral Commission (AEC), which has the processes in place to conduct Australia-wide vote casting.

However, Prime Minister Malcolm Turnbull gave the mandate to the Australian Bureau of Statistics (ABS), an agency that came under fire for its last attempt at an Australia-wide technology-based project, the 2016 Census.

On August 9, 2016, the ABS experienced a series of denial-of-service (DDoS) attacks, suffered a hardware router failure, and baulked at a false positive report of data being exfiltrated, which resulted in the Census website being shut down and citizens unable to complete their online submissions.

The Census was run on on-premises infrastructure procured from tech giant IBM.

The ABS previously said that IBM failed to adequately address the risk posed to the Census systems it was under contract to provide, and that IBM should have been able to handle the DDoS attack.

According to Andrew Phillips, public sector country manager for Amazon Web Services (AWS) in Australia and New Zealand, the issues the ABS ran into on Census night were all of the issues private cloud can result in.

So, in order to avoid private cloud problems, and with less than four weeks until the online component needed to be ready for the survey, the ABS turned to AWS to discuss its options in the public cloud.

"This time around, there was no way they could have even gone with an owned and managed system because it would have taken -- three weeks they gave us -- it would have taken that long to raise the purchase order, let alone to build, ship, and install," Phillips told ZDNet at AWS re:Invent in Las Vegas last week.

"It was hundreds of thousands people [potentially] going to be responding ... they didn't know the capacity requirements."

Phillips said the ABS was justifiably concerned about the potential for another DDoS attack to strike, so it wanted a vendor involved that could scale and fend off a potential attack.

With the voting climate making the AEC more aware of the potential for interference in vote-casting situations, the ABS brought the Australian Signals Directorate (ASD) into conversations with AWS.

"The Census isn't politically charged, this debate was politically charged," Phillips added. "They were concerned that somebody might try and do something to impact it."

The ABS was also concerned that there was going to be an overload of people calling in to ask questions.

"The ABS isn't set up for this kind of thing, the AEC is," Phillips said.

So the ABS also turned to AWS for its call centre offering, Amazon Connect.

"The decision was made to run the online components on us and to have the IVR software running on us as well," Phillips explained. "And we had three weeks to get it all done."

AWS stress tested the online response component for over a million people responding at the same time, even though in reality there was only going to be tens of thousands of people at most.

"But the nature of the cloud is we could have tested it for one hundred million people," he explained.

"If you want to test a private cloud environment for anything more than the numbers you'd expect, you have to buy this equipment -- it's not as though the vendors will lend it to you.

"With the cloud, if you've got it slightly wrong, you can adjust the storage type, the instance type, the speed -- you can adjust all of that and try again."

If the ABS had purchased the servers, particularly with the time limit constraints it was under, it wouldn't have been able to modify an on-premises environment without adding more kit.

"On the actual day when it ended up going live, we saw a six-fold increase of the traffic that usually goes to the ABS, but the system had been tested for 100 times, so it worked perfectly," Phillips added.

The survey ended at the end of October, and the results were announced on November 15 on a website also hosted by AWS. 61.6 percent of the population that took part in the survey responded "yes".

"All of the compute systems that we used to run the actual online survey are all turned off now and they've not costing the customer a cent; in the past you would've had to build that system and then find a purpose for it," Phillips said. "99 times out of 100 it just sat there.

"The ABS didn't just use us to save money -- although it cost them a fraction of what it would have cost -- they ended up using us because they wanted it to be secure, they wanted to be protected from DDoS attacks, plus they wanted it to be infinitely scalable. And then they were able to turn it off."

According to Phillips, no one can accuse the ABS as a government agency moving slow in the tech space.

Although tight-lipped on the potential for further government initiatives to be run on the AWS public cloud, Phillips did say that he would be keen on the opportunity to "do some experiments" with the ABS to see if in three years time, the public cloud is the right place for the Census to be run.

Disclaimer: Asha Barbaschow travelled to to AWS re:Invent as a guest of AWS.


CIA to continue cloud push in the name of national security

The intelligence agency's director of digital futures has touted the partnership with AWS as one providing a 'game-changing' environment for the CIA to perform like a Silicon Valley startup while protecting national security.

ATO called out for not tracking costs in digital transformation program

The costs and savings associated with the program undertaken to make the ATO more 'contemporary and innovative' have not been tracked, a report from the Audit Office has found.

RBA wants banks involved in Australian government digital identity solution

Rather than including banks, the DTA has selected a pair of government departments, one responsible for the robo-debt debacle and the other dealing with consistent IT outages, and a postal service that wants voting to occur via the blockchain.

Australian national security COAG says yes to facial biometric database

The group of Australian state and territory leaders has unanimously approved the prime minister's request for a country-wide database of citizens' driver's licence details.

AWS Secret Region manages cloud workloads for classified government data (TechRepublic)

Amazon's new region is targeted at the US Intelligence Community, offering compliance and security for sensitive information.

Editorial standards