Looking to 2017: ABS CIO plans statistics revolution

Australia's former chief statistician Brian Pink warned in the Australian Bureau of Statistic's (ABS) annual report in 2013 that ageing infrastructure and reduced budgets from government had the potential to "seriously compromise" the agency's sustainability. The ABS' CIO Patrick Hadley is tackling the matter head on, but is noticeably calm about the task facing him.

"The application portfolio we have has grown over time, and it would be fair to say, reflects a somewhat old business model which was organised around individual statistical outputs and surveys," he told ZDNet.

As Australia's national statistics agency, the ABS is over 100 years old, seeing its beginnings back in 1905 as the Commonwealth Bureau of Census and Statistics. Its role was traditionally in taking a Census of the Australian population every five years to provide the government with information on the population to better target spending. Its role has expanded over the 100 years hence, with the agency now comprising over 3,000 staff, incorporating new surveys and reporting, and looking to new projects such as the possibilities of big data uses for its statistics.

But many of the older applications still remain, Hadley said.

"We've got quite an extensive portfolio of applications, which have evolved without necessarily having the incremental funding for investment in that portfolio."

He said that there was "always a focus on doing more for less", and he was constantly on the look out for how to reduce cost in the agency.

"Any initiatives that we have to take cost and complexity out of the environment is something we pursue. It's a continual pressure for us, but there's also opportunities for us," he said.

To that end, the ABS has embarked on what it is calling ABS 2017, an initially self-funded business and IT transformation project to remove the risk posed by the legacy business processes and infrastructure.

"Whenever you embark on major organisational transformation you really take on two transformational projects. One is how you're seeking to transform the business and the other is how you transform the teams such as IT to build the capacity to deliver that transformation," Hadley said.

"We're looking at a business transformation program whereby we're looking at coping with the range of challenges for new and improved statistical products and outputs, better data quality, faster, less cost, all that kind of thing."

Hadley said that the Technology Services Division of the ABS was assessing the applications portfolio, looking at the legacy systems, and seeing what can be used in other parts of the organisation, what needs to be upgraded, and what needs to be decommissioned.

"One of the key challenges as we move forward, particularly for the legacy environment, we have to make sure we can continue to support and manage risk in the technology environment," he said.

"We have a discreet applications support group and we divide our work there in relation to planned and unplanned support. We do have some proactive planned support to make sure we're focused on software currency and addressing some areas of risk we've identified."

Virtualisation first

The ABS has maintained a "virtualisation first" policy with VMware's vSphere for its datacentre environment comprising of over 2,000 Windows and Linux virtual servers. Hadley proudly boasts that the ABS is a veteran in virtualisation and one of the leaders in its field.

"It has enabled us to improve productivity, to reduce ongoing IT costs, and support costs, as well as enable a more agile, and speedy set up where we need to respond to business requirements," he said.

The agency was now looking to expand its virtualisation out to the 3,500 Windows 7 desktops and notebooks in the ABS over the next 12 months, Hadley said.

"We'll be looking at as part of that, a virtualisation of the desktop software, and that is coupled with our mobile strategy which is looking to advance the capability of remote-working, tele-working, and potentially [bring your own device]."

BYOD is still in the early stages, and the ABS has recently gone to tender for mobile fleet management. Hadley said that it would require ABS-specific apps to be adapted for Android, which the agency did not support at this stage.

Like most government agencies, ABS was traditionally a BlackBerry-only shop, but as with other government agencies, the ABS had already moved away from offering BlackBerry to now offering iPhones. The agency's executive team had also been equipped with iPads, and Hadley said that in planning for the next Census in 2016, the ABS would be looking to provide iPad Minis to the staff out in the field who distribute and collect Census forms.

"One of the things we're doing is building our capability in terms of mobile applications development. When we look at Census, we'll be looking at the development and deployment of a number of mobile applications for our field force management, and Census management," he said.

In the 2011 Census, 33 percent of all responses from the public came through the agency's eCensus online form. The goal for 2016 is to double that to 66 percent.

"That causes all sort of significant technology issues as you may imagine," Hadley said.

"It has a great many advantages in terms of reduced paper handling, and it gives you an opportunity to tailor and improve the quality of Census returns."

Hadley said that the online forms would be easier to fill out, and would give the ABS the benefit of being able to manage its field workforce better, with a lower workload for staff in getting the forms out to all Australians.

While technology is improving how the Census is conducted, Hadley was wary of the risk that the technology would then take on in meeting the ABS's requirements.

"I'm very conscience that for the 2016 census, the more that we deploy and adopt the technology, the more and more that the risk shifts to the technology and the technologies deployed. We're being very, very cautious about examining every link in the chain, and that we thoroughly load test, performance test, and we've got an alternative plan should we need it," he said.

After the 2011 Census, the ABS conducted a trial run again in August 2012, and then one in August 2013. This year there will be another trial in August covering 100,000 premises in several states, but the real test of the systems would come in August 2015, when the systems are ready to go for 2016.

"We have a full scale dress rehearsal in August 2015. That's about the time frame where all our development really has to be in place for that point," he said.

"We fully test end to end at that point."

The ABS does heavy duty load testing, and real-time management of its field force during that test, he said.

Cloud limitations

Prior to the election, Communications Minister Malcolm Turnbull indicated he would like to see more government agencies adopting cloud services, and said that smaller agencies would be required to use cloud services where "minimum efficient scale hurdles are not met". Hadley said his agency would like to look towards using cloud services, but the privacy requirements in the Census and Statistics Act which binds the agency would potentially limit any cloud options for the ABS.

"A cautionary note for us is that under the specific legislation we operate under, the Census and Statistics Act, there's a number of obligations on us to ensure we maintain the privacy, and confidentiality of any data that we collect under that Act," he said.

"That does, in some instances, prevent us from taking advantage of some opportunities that might be presented in cloud, particularly in areas such as the security and custody of data."

He said that the Act required that the data be stored on site within the ABS and accessible to ABS staff.

"We could not store data in a cloud, government or private."

For now, all data is stored in the agency's Canberra datacentre, but Hadley said that the ABS was looking to expand its state offices to include datacentre capability, particularly in New South Wales.

"It's all within the area of ABS both in terms of property footprint and staff access," he said.

Skills shortage

The ABS' ICT Environment document (PDF) highlights a number of diverse legacy systems that the agency still supports, including a variety of internet service stacks, such as RedHat JBoss, Websphere and Microsoft IIS 7.5. Hadley said that one risk for the agency wasn't just finding staff with the right set of skills, but also retaining that knowledge in the organisation.

"Aside from any of the current constraints in relation to finding resources and recruitment, it is quite difficult to find people in the market who are interested in those kinds of skills and that kind of work in IT," he said.

"Predominantly we're looking to utilise our existing in-house expertise and cross-skill. We pay very close attention to our attrition and particularly retirements, and we find we have specific skill sets and business knowledge go out the door.

"With that in mind what we do is retain a register of staff who may have retired or left the ABS. We have a bench, if you like, we could call on in times [when] we have a particular issue."