ABS prepares for Census data crunch
The Australian Bureau of Statistics (ABS) is on the hunt for a testing firm to ensure that its systems can handle the loads caused by crunching the data obtained from the population Census taken every five years.
In tender documents released today, the ABS said that it is looking for a contractor to handle load and performance tests of its five in-house Census data dissemination systems.
The ABS is looking to certify its system to process large amounts of data, with hundreds of users being tackled at a time. This processing would need to occur both prior to and on the day that the ABS releases the first data from the most recent Census.
The successful contractor will be required to:
- Provide an understanding as to the maximum capacity of each component of the system
- Ensure that the 2011 Census Dissemination solutions are capable of processing the specified maximum load for each system
- Ensure that the 2011 Dissemination solutions are capable of achieving acceptable response times under the predicted maximum peak load for each sub-system
- Ensure that the 2011 Dissemination solutions are capable of handling the predicted maximum peak load on Census First Data Release day.
The ABS estimates that its website will experience 30 million page views on the Census Data Release Day based on 2006 statistics, with 750,000 hits expected in the first hour alone.
The successful contractor will create thousands of virtual users to test the log on capability of its Census Dissemination systems. The ABS expects its Datapack product, for example, to handle a total of 5000 users, with a peak of 2000 logged on at any one time, downloading data packets of up to 1.3GB each.
The main data release day for the 2011 Census data is expected to be in June 2012, according to the ABS.
The initial contract is for two years with two 12-month extensions attached.