Compulsory voting changes Australia election data game: Obama's data scientist

Australia's population size, and the fact that voting in Australia is compulsory, makes the data required by the political campaigns for the Australian federal election significantly different to the campaigns run in the US in 2012, according the Obama 2012 campaign's chief data scientist Rayid Ghani.

Ghani's role in the re-election of US President Barack Obama in 2012, as the expert on big data and analytics, has often been portrayed as one where the campaign looked at every facet of a person's life from their shopping habits, to their magazine subscriptions in order to determine how they vote. Ghani, who is in Australia for the annual CeBIT conference in Sydney, told ZDNet that this was more fantasy than reality.

"The reason we didn't use a lot of that information about what magazines you buy or what car you have or what pets you own is that it is not really predictive of your voting behaviour," he said. "It wasn't that we couldn't use that data, we chose not to because it wasn't very useful.

"If you're trying to predict whether someone is going to vote or not, knowing if they've voted in the past five or six elections is a lot more predictive than what car they drive."

Ghani joined the campaign behind US President Barack Obama's re-election almost by chance. Ghani had been working for services giant Accenture as a senior research scientist, and while he was doing research at Accenture, in his spare time would work on social causes that were close to his heart. He decided in 2011 that he wanted to combine the two into one job.

"I got tired of [my work] not having a social impact. I wasn't even looking to join the campaign. I didn't think the campaign was even doing anything like this, so I just decided to stop and look for something that would merge the two things, and then the campaign just came up randomly," he said.

"They were starting up just a few blocks from where I live."

Ghani said he was ultimately brought onto the Obama campaign because he was an outsider to the US political system.

"The idea of bringing me in was to have someone from the outside not from the political world who has a lot of experience not only in the academic but the business worlds on applying lots of analytics tools to real world problems."

His role within the campaign evolved over time, from developing tools to gather and analyse data, to refining algorithms, to looking at how to collect data from email and over social networks.

Ghani said that the tools developed for the Obama campaign could potentially be used by political parties for the upcoming Australian federal election, but he said much of the Obama campaign focused on driving people out to vote, and this is something Australia doesn't need to consider as voting is compulsory for citizens over 18 years of age.

"We in the US, we spend a lot of time getting people out to vote; for you, that's not an issue. We spend a lot of time registering people. Again you don't have to worry about that," he said.

One of the main sources of data that the Obama campaign used was the voter database that has the name, address, date of birth, and ethnicity of every voter, as well as whether they have voted or not in each election. It was not the only source of data, however. Ghani said that 90 percent of the data used by the campaign was obtained by volunteers calling and door-knocking residences across the US and asking them whether they intend to vote, and who they were planning to vote for.

But that required millions of volunteers working on the campaign and the scale of the campaign would be very different to anything Prime Minister Julia Gillard or Opposition leader Tony Abbott could organise in Australia, given the massive population difference between Australia and the US.

"We had millions of people going door to door, making phone calls, and persuading [people] to get out to vote. You don't have that size here."

He said the Australian political parties also had access to vast amounts of information about the voters, including polling data that is out there already, and this could be used to better determine which voters in Australia need to be targeted for the election.

"The basic data exists in Australia today; a lot more than it does in other countries."

Ghani said that the Obama campaign discovered that neighbours, relatives and friends were much more likely to convince a person to vote than if the plea came directly from the campaign, and he said that is something that Australia could use. He said that Australian campaigns could also use some of the tools developed during the course of the campaign around fund raising and volunteer recruitment through social media.

"I think there are a lot of analytics, and a lot of organising and a lot of technology infrastructure that carries over."

Now that Obama has secured a second term in office, Ghani said he was looking to use the tools developed as part of the campaign as part of a new startup that would assist not-for-profits.

"We were able to predict people who were at risk of certain behaviour, whether that was voting for the other side, not voting at all, and then we ran experiments to figure out who are these people and what kind of interventions work to change their behaviour," he said.

"The same thing applies for most social problems you think about. If you think about education and people who are at risk of dropping out of high school, or childhood obesity or any other addition of sort," he said. "That's the sort of problems I'm working on right now, how do we take the things we've learned and applied them to problems around education, healthcare, energy and change people's behaviour to optimise [the] social good."