IBM Watson beats humans in Suncorp claims assessments

Suncorp ran Watson silently alongside its consultants for six months, with the AI returning the most accurate insurance claims assessments.
Written by Asha Barbaschow, Contributor on

Suncorp Group around five years ago kicked off a project to overhaul its insurance claims assessment process, implementing a digital application-like form that allowed a customer to make a claim online.

While the process had a decent uptake from customers, Suncorp Digital Service executive manager Vlad Vereshchagin told IBM Think in Sydney on Wednesday that consultants still had to call the customers once they had submitted a claim, mostly to determine liability.

"That task could only be done by humans, our consultants ... because firstly, there was a need to understand circumstances of the claim or event or accident, exactly how it unfolded; and the second reason is that the decision about liability is binding, it has financial consequences ... it's irreversible, we cannot go back if we've made a mistake," he explained.

"All that blocked our automation endeavours and we couldn't really progress beyond that step."

Suncorp identified around 26-30 types of collisions that a vehicle could go through; it also punched in close to half a million motor claims per year with known assessment outcomes.

"So we thought if we had half a million claims here, which are well understood, and only about 30 scenarios, surely we could come up with a methodology which would stitch this together and actually create some sort of automation engine," Vereshchagin explained.

Suncorp had announced in November 2017 it would introduce IBM Watson to its claims assessment process, using its Natural Language Classifier capabilities to help the insurer better understand the circumstances of the claim and determine liability.

SEE: IBM Watson: A cheat sheet (TechRepublic)

"For us, it was a simple assessment of 'at-fault' or 'not at-fault' for each event, and that's in principal what we did," he said.

"When we started, our quality assessment was around 60% which wasn't good enough, the end result is we are above 96% accuracy with our assessments. That journey took us a couple of years to refine and a lot of that came from the quality of our training sets. As the saying goes, rubbish in, rubbish out."


SEE: How IBM Watson is revolutionizing 10 industries (TechRepublic)

The organisation wanted to take this further, but senior executives were worried that handing traditionally human decisions over to a machine was too risky and blocked requests to deploy Watson to all of its online operations, Vereshchagin said.

Vereshchagin said the team decided instead to demonstrate that the system was actually capable of making decisions and was at least as good as humans, if not better.

"What we did was we put it in shadow mode and ran it alongside our claim management backend system, ran it alongside our consultants, who were still doing their work," he explained, noting the consultants did not know Watson was working alongside them.

"We got the results from Watson and our consultants and after six months, we could actually show the comparison and as it turned out, surprising or not surprising, humans also make mistakes, and actually they make mistakes more often than the machine does."

Vereshchagin said the machine was also more disciplined in assessing the claims.

As a result, 50% of all insurance claims are now zero touch.  

Suncorp is now toying with the idea of introducing a similar process to its home insurance claims, with Vereshchagin noting it doesn't necessarily require the use of Watson, but rather, it wants to apply automation into different steps of the process, such as in place of a human performing a home claim assessment.


Editorial standards