Companies that are able to adapt to a world where innovation is increasingly driven by machine learning, or artificial intelligence more broadly, are the ones that will come out the other end of the tunnel and thrive, according to Oliver Rees, GM of Torque Data at Virgin Australia.
Rees, whose data analytics consultancy firm Torque Data was acquired by Virgin Australia in 2015, told ZDNet that one of its tasks has been "reengineering [Virgin's] analytical capability", ensuring the airline is well-prepared to embrace the opportunities that are offered by machine learning.
While not new to machine learning, Virgin Australia has been seeking better methods of developing, applying, and assessing machine learning algorithms, recently turning to Massachusetts-based company DataRobot, which operates on the belief that automated machine learning will not only increase productivity for data scientists, but also open up the world of data science.
Rees told ZDNet that Torque, as the data analytics arm of Virgin, has been investigating ways to improve customer experience for members of Virgin's Velocity Frequent Flyer loyalty program.
"We want people within our program to be able to redeem points for great experiences, and to do that, we want to be able to better predict when is the best time for particular people to redeem points and what should they be redeeming them against," Rees said.
"For a given individual or a group of people, we want the ability to be able to better understand what it is that they might be really interested in doing at a certain destination -- from both a business and a leisure point of view -- and how can we serve them better at that destination, and then be able to tailor that to their specific requirements and serve that up to them in a meaningful way.
"I think we need to take it upon ourselves in the industry to build the predictive models that understand what the needs and wants of our customers are, and go through the whole curation process, become their concierge."
Using DataRobot's automated machine learning service, Virgin Australia is looking to build models that can predict the types of people that are more likely to travel, the types of travel people are likely to undertake, the prices that travellers are willing to pay, the importance of accommodation relative to travel, and the importance of experience compared to travel.
A lot of information is provided willingly by customers, Rees said, but the company also uses previous purchases to predict future preferences.
"There's a lot we can impute from what they do already, but also by actually looking at events and the type of events that people like to attend -- that's a really powerful piece of information," Rees said.
"It all comes back to using what we understand already about people, and not necessarily as individuals either, but as groups, so we look at tribes and cohorts ... Often people want guidance around what do people like me do and how can I as an individual benefit from the learnings of others.
"We need to not only understand people based on what they tell us themselves, but also what people like them are telling us, what experiences people like them are enjoying ... that's where the competitive battleground is."
Using technology provided by DataRobot, which raised more than $124 million since its inception in 2012, Rees said Torque has been able to build new predictive models at one-tenth of the time it had previously taken, and the models are up to 15 percent more accurate than previous ones.
"We have the ability to run multiple different statistical techniques against the same dataset in a very short space of time and have this competitive element whereby the models compete against each other for the best outcome," Rees said. "[DataRobot's system is] constantly reviewing whether this particular technique can outperform this other technique, and that's running really in real-time before my eyes, whereas an analyst might spend a number of hours running one particular model.
"The ability for us to then deploy that model is also enhanced. It's direct API link and we can start to operationalise the analytics far more quickly because there's a reduced number of steps in doing that."
Rees also noted that analysts can be biased towards specific statistical techniques, the same way artists prefer to use watercolour, oil, or acrylic paints.
"So it removes that element of bias as well," he added. "I think that's very powerful; it makes us stronger as an organisation."
Additionally, using an automated machine learning service means analysts are able to spend more time on uncovering opportunities to use analytics and less time on grunt work like manipulating data, Rees said.
"We're actually moving really smart people into different roles where we're using their intellect in a really powerful way," he said.
"The way to hang on to great analytical teams is to give them leading edge tools to work with and challenging meaningful analytical problems to solve.
"People are very interested in building their understanding around how new technology is going to impact their work. Giving people the opportunity to learn how it works and recognising that we're all on this journey together ... I think it's been a real positive for us."
Rees believes technology, like that offered by DataRobot, will allow Virgin to "democratise analytics" throughout the organisation, enabling all business units to be data-driven.
"Analytics is far too important to be left to the analyst. With these sorts of tools ... we can have the whole business being more data-driven because the power to build really great models lies in many more hands," he said.
"It doesn't mean, by the way, that we're not going to need the really powerful hardcore analytical teams that we have -- of course we're going to need those sort of people -- but it means that a wider number of people within our business are going to be able to rely on analytics to help them with faster decision-making."
According to Rees, the real challenge for large organisations like Virgin is "avoiding falling into the trap of doing nothing and not embracing the new technologies available".
"It can very easily be seen as something that creates extra work, creates extra stress, creates extra pressure. We can't ignore what we need to do on day-to-day basis at the expense of developing new capability," he said.
"Being in an organisation that is prepared to be innovative and move forward is a really important. It's a fine balance."
Rees said the application of machine learning is a "quantum leap forward" in how data can be used to drive better outcomes for both customers and businesses, and that businesses are barely scratching the surface of what's possible.
Technology vendors are scrambling to create an abstraction layer that can clean up data and make it easier to prep for analytics. The problem is humans haven't been the best stewards of organized data. As a result, machine learning is the next magic bullet for data management issues dating back decades.
According to IDC, spending on new technologies will accelerate over the next five years and boost spending on information and communications technology overall.
Artificial Intelligence seems to be the buzzword du jour for organizations, but this is not an obvious or straightforward transition even for those building advanced products and platforms. Many organizations are still struggling with digital transformation to become data-driven, so how should they approach this new challenge?
Video: How machine learning creates demand for human workers (TechRepublic)
Artificial intelligence and IoT are increasing demand for human workers, says Calabrio CEO Tom Goodmanson, and they're more stressed out than ever.
When quantum computers come online, all encryption will fail. James Barratt explains how AI will aid the way for the emergence of quantum computing.