Big data: Big hype or big hope?

One quarter of APAC businesses have invested in big data, according to a research report just released by Gartner. Customer experience use cases are top priority, especially in financial services and retail — a trend that's reflected in Australia.
Written by Angus Macaskill, Contributor

A recent Gartner report on big data revealed that while the Asia-Pacific (APAC) region currently trails the US adoption curve, which stands at 37 percent of businesses, APAC organisations think they'll catch up with their US peers within two years. The report said that globally, media and communications and banking sectors are at the forefront of the big data investment, with the results mirrored in Australia.

Well-known Australian brands like the Commonwealth Bank of Australia (CBA) and retailers Coles and Woolworths are among the early movers on big data. They are part of the 64 percent of businesses globally that Gartner says have already invested in big data or plan to do so by June 2015, an increase of 6 percentage points on the 2012 survey.

US leads the pack — for now

Over one third (37.8 percent) of US businesses have already invested in big data, ahead of Europe, the Middle East and Africa (EMEA), which is at 26.8 percent; APAC, which is on 25.6 percent; and Latin America, which is at a distant fourth, with 17.8 percent.


However, Asian businesses show plenty of big data ambition, and a further 44.7 percent will invest in the next two years, lifting the APAC adoption rate to the same proportions as the US, and comfortably ahead of EMEA.

Industries furthest down the investment path are media and communications (35 percent have invested), banking (34 percent), and services (32 percent), while government (16 percent) and utilities (17 percent) are laggards.

This data is congruent with some of the publicised activities of Australian organisations. Take banking, where the initiatives of CBA and UBank, and its parent National Australia Bank (NAB), have all been widely reported. CBA provides an interesting example of an organisation that's utilising Big Data in a number of ways. Firstly, the bank has the goal of delivering personalised and optimised real-time offers to clients at the time they engage with the bank. This desire to enhance customer experience is a common focus for analytics, and is cited by Gartner's research as the primary focus of respondents' big data initiatives (55 percent of businesses are doing so).

Putting data in the shop window

CBA also uses big data in a way that's more obvious to customers — as part of its products and services. The bank's current advertising promotes its big data capabilities as a competitive differentiator. Reminding corporate clients that "40 percent of transactions go through our systems", the banking powerhouse also spruiks the added value of its data modelling in "turning raw data into insights and innovations you'll see on your bottom line".

The proportion of transactions going through CBA gives the bank an attractive competitive weapon. Its ability to turn that data into proprietary customer insights and associated personalised offers raises the barriers to entry for potential new competitors. In a relatively small country like Australia, where some sectors have a handful of large market leaders, that monopoly of data can help entrench the status quo and make it tougher for smaller competitors to grow.

It's true of the banking sector, and also true in the Australian retail sector where Coles and Woolworths (of which more later) represent around 80 percent of the Australian grocery market (according to a Ferrier Hodgson report in May 2011, sourced from the Wesfarmers and Woolworths 2010 annual reports).

CBA also uses analytics in consumer marketing. Its interactive Signals service generates statistics such as average monthly deposits, savings held, and the uptake of financial services like mortgages, credit cards, and personal loans. Plug in details like your age, gender, postcode and so on, for a personalised infographic showing how you compare against your peer group. It's available to share socially or for download, displayed together with a call to action of "talk to us today". Competitor UBank offers a similar service, based on over 1 billion transactions, along with data from the government census and consumer trends data from data analytics business Quantium.

Quantium occupies a strategic position in the Australian data economy. It sources, organises and makes sense of de-identified business transaction information, and provides data and consulting services to clients that need consumer insights. Clients include some of the largest consumer businesses in Australia, such as Qantas, Telstra, IAG, eBay, Suncorp, NAB, David Jones, and Woolworths. Grocery chain Woolworths took a 50 percent stake in Quantium in May this year, a deal with echoes of UK retail giant Tesco's 100 percent purchase of analytics firm Dunnhumby in 2011. The reported AU$20 million price tag for the Quantium stake shows the value that Woolworths places on big data.

Retail detail

When the Quantium deal was announced, Woolworths said it planned to use Quantium's data and services expertise to extend its existing customer analytics capabilities — a key battleground in the intense competition for customers with rival Coles. The relationship gives Quantium access to de-identified data from Woolworths' customer transactions, from its debit and credit cards and the Everyday Rewards loyalty card program. That data will supplement existing data sources and enrich offerings to other Quantium clients.

Woolworths major competitor Coles is also betting big on big data. Chief financial officer Terry Bowen from parent company Wesfarmers was recently reported in The Australian as saying that the whole group is ramping up efforts to use data analytics across its entire supply chain, with benefits to both Wesfarmers and suppliers. Data from Coles' Flybuys customer loyalty program and credit cards provide the means to enrich basic shopping transactional with customer demographic data.

Government gets the big data bug

Gartner's research identified government as the least advanced sector globally for big data investment (just 16 percent had done so), but the Australian government is taking action with the August 2013 release of The Australian Public Service Big Data Strategy (PDF). Australian government CIO Glenn Archer said in his announcement statement that the strategy "provides a whole-of-government approach to big data in order to enhance services, deliver new services, and provide better policy advice, while incorporating best practice privacy protection and leveraging existing ICT investments".

In addition to enhancing services, the strategy highlights the potential of big data to support cost reductions, referencing a seminal report from McKinsey Global Institute, Big data: The next frontier for innovation, competition, and productivity. That report estimates a potential saving of up to 20 percent in admin costs in Europe's public sector, equating to up to €300 million.

The Public Service Big Data Strategy is running in parallel with the recently launched whole-of-government Data Analytics Centre of Excellence (DACoE). The DACoE is being led by the Australian Taxation Office (ATO), a reflection of where some of government's deep analytics skills reside — news that may raise concerns for those who have been economical with the facts in their tax returns. According to the strategy document, the "DACoE will build analytics capability across government by establishing a common capability framework for analytics, sharing technical knowledge, skills, and tools, and building collaborative arrangements with tertiary institutions to shape the development of analytics professionals. The DACoE will work within guidelines established via the Big Data Strategy and other work of the Big Data Working Group".

The next steps are for a multi-agency big data working group appointed in February to start delivering on six action items identified in the strategy. These include a best practice guide for big data, identification of big data barriers, enhancing skills, producing a guide for responsible data analytics and an information asset register, and keeping abreast of big data technologies.

Back to school

One of the action points identified is to "enhance skills and experience in big data analysis", a challenge that's not unique to government, nor to Australia. The dearth of analytics skills is highlighted in Gartner's research report as one of the top three big data challenges, together with determining how to get value from big data and defining big data strategy.


McKinsey estimated a potential shortage by 2018 in the US of "140,000 to 190,000 people with deep analytical skills", and around 1.5 million managers with analytics know-how.

The University of Technology Sydney is one organisation seeking to remedy the analytics skills scarcity. Its Advanced Analytics Institute (AAI) will deliver research on data and analytics science, and aims to help produce graduates with deep analytics skills. It has already collaborated with organisations and government departments such as Microsoft Research, Nokia, AMP, SAS, IBM, Westpac, the ATO, and the Department of Human Services. If Australia is to be part of the accelerated take-up of big data in APAC that Gartner's research predicts, it will have a large contribution to make.

Many rivers to cross

Several other challenges will determine the speed with which Australian organisations embrace data analytics. Overcoming the top two Big Data challenges identified by Gartner's research — determining how to get value from big data and defining big data strategy — will be key.

These challenges are an inescapable barrier for organisations starting out on the big data path. As Gartner's report states, initial efforts start with knowledge gathering, and are followed by strategy setting.

"The investment is small, and mostly consists of time," the report says.

Then businesses move on to proof of concept, and, following a successful pilot, start deploying and using data analytics tools, a point where investments rise. In Gartner's study, 70 percent of the organisations that have made big data investments "have moved past the early knowledge gathering and strategy formation phases and into piloting (44 percent) and deployment (25 percent)".

However, for organisations adopting big data now, the learning curve should keep getting shorter, due to the increasing numbers of successful early adopters they can learn from. For example, the earliest of big data adopters — global online businesses like Google, Amazon, Yahoo, and Facebook — all had to go through an experimentation and pilot process. These trailblazing businesses had to chart their own course, and in many cases, they even had to invent some of the tools that today are considered fundamental to big data; MapReduce and Hadoop, for instance. Big data adopters are now benefiting from the experience of the original pioneers and from leaders in big data in their own countries and industries.

In Australia, organisations like CBA, Woolworths, and Wesfarmers/Coles are examples that others can follow. Businesses in the value chains of early adopters are potentially in a great place to benefit, because early adopters get most value from big data when they have access to data from across their supply chain — as is the plan of Wesfarmers/Coles. That approach has benefits and obligations, and participating organisations need to agree on principles for data ownership, governance, and privacy, as well as how to value big data expertise leveraged in the relationship.

Other developments can help businesses accelerate returns on big data. For instance, valuable data can now be sourced from government data sets, often for no charge, such as the tables of population census data released by the Australian Bureau of Statistics. According to a research study of open data economies by Cap Gemini (PDF), Australia is already one of the "trendsetters" in open data initiatives, together with the US, the UK, Canada, and France.


The growth of data sets in the Australian economy will help increase the rate of big data adoption as increasing numbers of businesses and government agencies see the value in sharing information with non-competitors to fill gaps in their own data sets — helping to increase the overall value of data. Data services businesses, such as Quantium, Dun & Bradstreet, and others not yet created, will also have a part to play. This, together with an increase in deep data analytics skills, will help grow the big data capability in the local Australian economy, at least for larger organisations. It's not so clear how small businesses (over 90 percent of all organisations in Australia) could afford their own big data tool sets and expertise, but it's likely that some analytics-as-a-service offerings will spring up to help the minnows compete with the whales. Recent announcements of such services from HP, EMC, and IBM point to growth in the analytics-as-a-service model.

As Gartner's report states, "for big data, 2013 is the year of experimentation and early deployment". We can expect to see a number of businesses in Australia approaching maturity in big data utilisation from 2014 onwards, and many more following in their footsteps. The big data adoption curve resembles where cloud was 18 or 24 months ago — lots of learning, a little early experimentation, and not much deployment. As cloud has developed as a business capability, expect big data to develop similarly.

Editorial standards