​ANZ bank to move 'quaint' data ponds into single Hadoop-based data lake

Australia and New Zealand Banking Group (ANZ) is embarking on a project to create an enterprise-wide data lake in a bid to capitalise on the most strategic asset the bank believes it has.

According to Darren Abbruzzese, general manager of data at ANZ, as the bank realised the value of its data, it captured everything it could. However, as a result, little "ponds" of data popped up in different departments throughout the organisation, and the information inside remained valuable and accessible only to those working on a particular project.

"We've got these nice little quaint data ponds throughout the organisation that basically sprung up on a project-by-project basis or business unit by business unit basis," Abbruzzese said, speaking at the Gartner 2017 APAC Data & Analytics Summit in Sydney on Tuesday.

As a result, ANZ is aggregating its data into a single Hadoop-based data lake, which will also see the bank move away from the concept of data warehouses.

Abbruzzese expects the process to take a number of years, and the bank will start with transitioning over its high-priority data assets and work backwards from there.

In addition, Abbruzzese wants to see the lake top up in real time.

"It is a challenge, as we've got a lot of legacy technology. We've still got some mainframes running, got huge batch processes -- but it's something we're really keen to prosecute because we think improving the flow of data through our organisation will really help drive an advantage for us," he explained.

According to Abbruzzese, the data lake will be of no value to the organisation unless it is used, which requires it to be accessible in a manner not restricted to data scientists.

"One of the things we're doing is documenting in a non-technical, non-data scientist way the metadata that is in there. As more data moves into the lake, we're adding it into a central registry that anyone in the organisation can go to and search for what they require from inside the lake," he said.

"We want to make sure it gets used and not turned into a data swamp. Make it more consumable to the average punter within the organisation.

"You kind of only get one shot at this -- at least within the same organisation -- then you'll have to move to the next one and take all the lessons learned."

Of key importance to ANZ is securing its data, with Abbruzzese explaining that nothing will destroy an organisation faster than a data breach.

"Thankfully, banks invest probably as an industry each year well over AU$1 billion. I know at my own bank, it is hundreds of millions of dollars we spend on security in general all the way through all the different layers of the organisation," he said.

"I would never shave a single dollar off any security program, because it's our crown jewels."

Having all of the bank's data in the one place will allow greater control when it comes to security, Abbruzzese said, as he is expecting a central asset to be easier to manage than the existing ponds of data.

"We believe, as a bank, that the data on our customers and our transactions that we have is an A-grade strategic asset, and if we don't manage it appropriately and we don't invest into it, then we will fail as an organisation," Abbruzzese added.

Earlier this month, ANZ released its first-quarter market update, reporting an 8 percent increase in statutory net profit to AU$1.6 billion.

For the three months ended December 31, 2016, ANZ said initiatives including Apple Pay and Android Pay drove ongoing net customer acquisition gains in its Australian Retail Transaction Banking segment.

ANZ was the first bank in both Australia and New Zealand to launch Apple Pay, in April last year, and was the only member of Australia's big four to embrace Apple Pay, having left the group of banks requesting collective bargaining rights with Apple and demanding open access to iPhone's near-field communication (NFC) technology.

ANZ is currently the only bank in New Zealand offering the service.

"Customer experience is the future of banking and a great example of customer experience is Apple Pay," Abbruzzese said. "When we released Apple Pay, the growth that we had in credit card applications far exceeded what we were getting out of normal system growth because [customers] wanted that experience."

He said it became very clear to ANZ that customers will associate a brand with good experience, and that delivering a great customer experience requires great data.