Google unveils 'Pathways', a next-gen AI that can be trained to multitask
Today's AI models, according to Google's AI lead and co-founder of the Google Brain project, Jeff Dean, are at the one-trick pony phase – they are "typically trained to do only one thing". But a new approach called Pathways could provide something akin to a trainable dog that can do multiple tricks.
Dean describes Pathways as a "next-generation AI architecture" that "will enable us to train a single model to do thousands or millions of things."
Artificial Intelligence
Pathways can remove the limits of an AI model's capacity to respond to information from just one sense and allow it to respond to multiple senses, such as text, images and speech.
SEE: Even computer experts think ending human oversight of AI is a very bad idea
"Pathways could enable multimodal models that encompass vision, auditory, and language understanding simultaneously," Dean explains.
The model, for example, could then process the word 'leopard', the sound of someone saying 'leopard', or a video of a leopard running.
In all three cases, the AI recognises the concept of a leopard. Dean argues that such a model would be "more insightful and less prone to mistakes and biases".
Bias is a touchy subject for Google. In December, its top AI ethics researcher, Timnit Gebru, resigned after Dean reportedly disagreed with a paper she co-authored about the energy costs associated with creating new AI models, and the fact that its models ingested everything from the web, including racist, sexist and abusive content, thus potentially influencing how an AI model works. The paper has not been published but was viewed by MIT Technology Review.
"We'd like to train one model that can not only handle many separate tasks, but also draw upon and combine its existing skills to learn new tasks faster and more effectively," Dean says.
What he seems to describe is an AI that can plug in to other AI models and exploit their best parts by using a Pathways model that "dynamically learns which parts of the network are good at which tasks -- it learns how to route tasks through the most relevant parts of the model."
So, Pathways could improve the ability of machine learning to make decisions, and possibly bring it a step closer to an AI that can actually reason through a problem or scenario.
Dean argues that Pathways not only has a larger capacity to learn different tasks, but that it's also more energy efficient because only relevant parts of a network are activated for a given task.
"Pathways will enable a single AI system to generalize across thousands or millions of tasks, to understand different types of data, and to do so with remarkable efficiency -- advancing us from the era of single-purpose models that merely recognize patterns to one in which more general-purpose intelligent systems reflect a deeper understanding of our world and can adapt to new needs," he explains.