A growing number of organizations want to take advantage of AI and machine learning, but finding the right tools and staff to do so can be difficult. Among other challenges, there's a shortage of data scientists who can actually build machine learning models.
"There's a huge talent gap for AI right now," Rajen Sheth, director of product management for Cloud AI at Google, said to ZDNet. "It's a very new area, and there aren't a lot of people who know how to use it right now."
To address that challenge, Google Cloud is rolling out a set of new products aimed at maximizing the impact of a data scientist's work. The first product, Kubeflow Pipelines, makes it easier for data scientists to coordinate with the other members of a team needed to actually bring a machine learning model into production. The second product, the AI Hub, serves as a marketplace where anyone from an organization can access machine learning components such as data sets or models.
"The vast majority of models that data scientists create never make it to production," Sheth said. "You're taking that really valuable asset of a data scientist and wasting it."
With Kubeflow Pipelines and AI Hub, data scientists' work will be more accessible, and thus more valuable, to the rest of an organization.
Kubeflow Pipelines are a new component of Kubeflow, a popular open source project that makes it easier to deploy machine learning workflows on Kubernetes. Kubeflow Pipelines provides a workbench to compose, deploy and manage reusable end-to-end machine learning workflows.
It effectively lets anyone "snap in and out" various components of a workflow, Sheth explained. So for instance, a data scientist can build a model, and another person in the organization can snap in the right data, while a developer can snap in an API. With this level of collaboration, people can experiment and put things into production more quickly.
"We realized that AI is a team sport," Sheth said. "it's more than just the data scientist building the model. It might be the data engineer that brings in the data, it might be an engineer that puts the model into production, it might be the developer that puts it into the application -- these are all uncoordinated right now."
Meanwhile, the AI Hub is aimed at making it easier for people in an organization to discover, share, and reuse existing tools and work. This could include pipelines, Jupyter notebooks, TensorFlow modules and more. Data scientists and others can add their own components to the marketplace for others within their organization to use.
In addition to serving as a private, secure hub for businesses, the marketplace will come stocked with machine learning resources developed by Google Cloud AI, Google Research and other teams across Google. In the future, it will also provide access to content from Kaggle, Google's online community of over 2 million data scientists.
Over time, Google plans to allow people to purchase tools from third parties on the marketplace. Currently, Google works with its cloud customers to match them with suitable AI partners. For instance, SparkCognition may be a suitable partner for an aviation company using Google Cloud that's looking for predictive maintenance tools. Eventually, a firm like SparkCognition could offer tools via the AI Hub.