Big data may seem like something out of a George Orwell novel, but the use, and possible sale, of big data — personal information held by local, state and federal government and industry — is about to become a big issue for all Australians.
The Abbott government is seeking to find ways to use the wealth of information held by departments and agencies to provide better services — and potentially be sold to the private sector to help balance the budget.
At the 2013 federal election, the coalition laid out a policy for e-government and the digital economy. One of its planks involves agencies, researchers and the private sector working together on projects to make better use of some of the thousands of datasets held by departments.
Governments hold a wide range of information, from tax records to welfare payments, traffic fines to property values.
High-profile cases of privacy breaches have made many wary of personal records being swapped between departments or handed over to private corporations.
The immigration department recently inadvertently allowed access to source data that contained personal information on a large number of asylum seeker detainees, Telstra accidentally exposed the account details of more than 15,000 customers, and American retail giant Target last year released the credit records of 70 million customers.
While cyber attacks are often publicly blamed for information releases, the breaches are just as likely to be due to human error or system glitches.
Much of the data held by government is personal information collected from its day-to-day activities.
Public servants are understandably loathe to release such information and are aware of the potential problems for them and their departments, not to mention embarrassment and legal action from clients, if breaches occur.
Developments in technology also make it harder to strip identifying information from personal data.
A US expert has estimated that over half of government-held personal data that has supposedly been made anonymous could be pieced back together using currently available software.
Australian Information Commissioner John McMillan, who raised some of these issues in a speech last week in Canberra, says there are many challenges in making better use of government data.
Unlike Britain, Canada and the United States, Australia has no overarching framework for how local, state and federal government provide open access to the data they hold.
Australia is also yet to sign up to international partnerships on "open government", meaning it is missing out on ways to make public services more accessible and create jobs through innovative technology.
But steps have been taken in the past month to put in place a system of protections, known as Australian Privacy Principles.
The principles govern the collection, use, storage, disclosure and correction of personal information by government and industry.
They cover such things as agencies telling people their own personal information is being shared, taking reasonable steps to protect the data, and stricter rules for sensitive data such as health, genetic and biometric information.
Prof McMillan's office has also been given tougher powers under the Privacy Act, including the ability to assess how agencies are meeting standards, award compensation following an investigation, force agencies to change their ways and seek civil court penalties for the most grievous breaches.
He says, with such checks and balances in place, there is no reason why big data can't be used to "expand knowledge, enable innovation, boost productivity, and even save lives".
"The value of information can be enhanced when it is openly accessible and reused frequently."
A recent example is the release by transport departments of data to make mobile device apps that provide information on transport routes, timetables and arrival times.
It can also have an educational role, as in the publication of data about endangered or protected species and their habitats, or statistics on homelessness.
Prof McMillan says it's a debate worth having as an increasing amount of information is collected by government and industry.