Imagine pulling up an AI-powered weather app and seeing clear skies in the forecast for a company picnic that afternoon, only to end up standing in the pouring rain holding a soggy hot dog. Or having your company implement an AI tool for customer support, but which integrates poorly with your CRM and loses valuable customer data.
According to new research, third-party AI tools are responsible for over 55% of AI-related failures in organizations. These failures could result in reputational damage, financial losses, loss of consumer trust, and even litigation. The survey was conducted by MIT Sloan Management Review and Boston Consulting Group and focused on how organizations are addressing responsible AI by highlighting the real-world consequences of not doing so.
"Enterprises have not fully adapted their third-party risk management programs to the AI context or challenges of safely deploying complex systems like generative AI products," Philip Dawson, head of AI policy at Armilla AI, told MIT researchers. "Many do not subject AI vendors or their products to the kinds of assessment undertaken for cybersecurity, leaving them blind to the risks of deploying third-party AI solutions."
As ChatGPT's popularity soared as both a standalone application and as an API, third-party companies began leveraging its power and developing similar AI chatbots to produce generative AI solutions for customer support, content creation, IT help, and checking grammar.
Out of 1,240 respondents to the survey across 87 countries, 78% reported their companies use third-party AI tools by accessing, buying, or licensing them. Of these organizations, 53% use third-party tools exclusively, without any in-house AI tech. While over three-quarters of the surveyed companies use third-party AI tools, 55% of AI-related failures stem from using these tools.
Despite 78% of those surveyed relying on third-party AI tools, 20% failed to evaluate the substantial risks they pose. The study concluded that responsible AI (RAI) is harder to achieve when teams engage vendors without oversight, and a more thorough evaluation of third-party tools is necessary.
"With clients in regulated industries such as financial services, we see strong links between model risk management practices predicated on some sort of external regulation and what we suggest people do from an RAI standpoint," according to Triveni Gandhi, responsible AI lead for AI company Dataiku.
Third-party AI can be an integral part of organizational AI strategies, so the problem can't be wiped away by removing the technology. Instead, the researchers recommend thorough risk assessment strategies, such as vendor audits, internal reviews, and compliance with industry standards.
With how fast the RAI regulatory environment is evolving, the researchers believe organizations should prioritize responsible AI, from regulatory departments up to the CEO. Organizations with a CEO who is hands-on in RAI reported 58% more business benefits than those with a CEO who is not directly involved in RAI.