​Culture and lack of skills holding back Australia's open source adoption: Coles

When it comes to the adoption of open source technologies in Australia, Niall Keating, head of Big Data at supermarket giant Coles, has said that it is both the culture and talent shortage that is holding the country back.

Having worked in the Hadoop space for the past four years, Keating said he has seen a shift in attitude towards open source, watching businesses deploy such technology in earnest to solve real business problems.

While Keating acknowledges that he is experiencing a greater adoption of open source, he concedes that there is still a long way to go, and believes it can be advanced with the help of the education sector.

"We've seen a great rise of data scientists -- there's been a lot of talk about it globally and Australia is the same. I've met a lot of great data scientists here in Australia and I've seen a lot of universities creating new courses or repositioning courses around data science," Keating said at the 2016 Hadoop Summit in Melbourne on Wednesday.

"That's great, but I'm the technology guy that has to enable these data scientists so I need software engineers -- I need to see the rise of the software engineers again."

Keating said Australia needs more talent in the industry that can build highly scalable back-end systems, and people that understand the end usage and can and are willing to share resources.

"I'd love to see that skill set rise again -- see universities invest in open source technologies and teaching that," he said.

"Great engineers will drive confidence, they'll drive out cost, and confidence will engage businesses and that will lead to business cases and we'll get more momentum around the ecosystem, around data and data science.

"I think universities need to get more on board. I'd like to see more graduates coming out with hardcore computer engineering skills or software engineering skills."

When the university has done its part, then comes the need to change attitudes, with Keating noting Australia tends to be a "little bit risk adverse" when trying new things.

"We try not to make mistakes and manage our costs," he said.

"I think within every enterprise there's an innovation agenda, trying to move towards agile and development, trying to get that balance between being risk adverse versus trying something new."

Keating believes it is the lack of opportunity -- rather than lack of brain power -- preventing more individuals from heading down the software engineering path.

"We've seen that in Sydney with Atlassian -- I just don't think there's enough of those companies focused on research and development, software driven product companies. Maybe that's a part of the tax system here in Australia which was historically conducive to startups so a lot of budding entrepreneurial software engineers would go off to Silicon Valley or off to the UK," he said.

"I do see it changing. The tax rules have changed, we see big companies like Telstra and Westpac investing in startups. I think the future is bright."

Keating said this is perhaps indicative that Australia is moving towards a higher level of growth and adoption around open-source technology.

As the first individual to reach out to Hadoop developers Hortonworks from Australia, Keating said that exploring technologies like Hadoop was purely for tests and pilots, but noted there is now a shift for its application in business production

With Keating expecting data to be the core of a lot of products moving forward, he said the use of technologies such as Hadoop will be integral to ensuring success.

"Data will drive great customer experience like what we see with Uber and Airbnb," he said.

"Hadoop will be core to that, the cloud will be core to that, and IoT will be core to that -- real-time will be core to all of that.

"I think we need great software engineers in order to enable real-time systems as I think that's what customers will expect."

Disclaimer: Asha Barbaschow travelled to the 2016 Hadoop Summit as a guest of Hortonworks.