Facebook AI researcher urges peers to step out into the real world

Facebook AI researcher urges peers to step out into the real world

The video game Breakout from Atari first burst onto the scene in 1976. The game, a simple matter of moving a paddle horizontally to bounce a ball into walls and chip away at them, has become a mainstay of training artificial intelligence.

But perhaps it is time to break out of Breakout.

That was one notion that emerged Wednesday morning in a keynote talk by Joelle Pineau, a scientist with Facebook's AI research unit, at the prestigious NeurIPS conference on machine learning in Montreal.

Also: IBM, Apple, and Facebook represent new tilt toward business for venerable AI conference

Pineau's immediate concern is that too much of today's work in machine learning, despite many virtues, leaves out aspects of rigor. For example, a lot of the research is done with simulated worlds, things constructed in a pristine fashion within the computer. Those simulations may help make results reproducible, which is good, but they miss a lot of the complexity of the natural world, which may make the work less meaningful, less rigorous, Pineau suggested.

"I love that the simulators help reproducibility," said Pineau. "We want that convenience, but we also want to include some of the real world.

"We have to break out of these simulators and tackle the real world," she urged.

Pineau focused on a flavor of machine learning she's passionate about, called reinforcement learning, which makes extensive use of simulators. Reinforcement learning technology is about enabling a computer "agent" to take sequences of actions in an environment, and to figure out a rule for making good choices between the possible actions based on maximizing a reward that the agent receives.

Also: Facebook aims to be more engaging to humans in latest AI research

Atari's Breakout is one of many video games that are used to simulate the environment in which the agent makes choices. Such simulated training has led to computers increasingly surpassing human performance in the games. And many researchers are exploring promising avenues for use of reinforcement learning, such as for control systems for "neuro-stimulation" devices embedded in the body that could help prevent epileptic seizures.

But Pineau implied that such critical uses of reinforcement learning need to eventually grapple with real-world complexity.

Reflecting on the matter, Pineau and her team wondered, "Could we make this a lot more interesting with some natural-world signals?" They inserted videos of street scenes into the background of Breakout, and re-trained the reinforcement learning systems with the new, more "noisy" environment.

Also: Facebook Oculus research crafts strange mashup of John Oliver and Stephen Colbert

"Videos are an endless source of natural noise," observed Pineau. As she pointed out, when humans are asked to play the game with the video background, "it's a little harder, but they can still play." That implies there are challenges that emerge when adding noise that can result in more impressive results if the computer can succeed against such adversity.

There is "a lot we can do to incorporate natural noise in the environment," said Pineau. "We can set the bar much higher in the realism of our simulators.

Also: Pineau's slides for her talk are available for download on the NeurIPS website

"From static images to video backgrounds, there is a lot further we can go."

That includes the "creation of simulation environments that are photo realistic," she said. She showed some images of simulated house interiors, with real-world imagery grafted into the space, right down to strikingly reflective mirrors.

Must read

• Facebook AI now fixes bugs like spellcheck corrects typos (CNET)
• Four ways machine learning is evolving, according to Facebook AI (TechRepublic)

What is to be gained with all this, said Pineau, is doing "better science" by bringing more rigor to machine learning and AI.

Pineau ended her talk with the exhortation, "I encourage you to step out into the real world!"

36 of the best movies about AI, ranked

Previous and related coverage:

What is AI? Everything you need to know

An executive guide to artificial intelligence, from machine learning and general AI to neural networks.

What is deep learning? Everything you need to know

The lowdown on deep learning: from how it relates to the wider field of machine learning through to how to get started with it.

What is machine learning? Everything you need to know

This guide explains what machine learning is, how it is related to artificial intelligence, how it works and why it matters.

What is cloud computing? Everything you need to know about

An introduction to cloud computing right from the basics up to IaaS and PaaS, hybrid, public, and private cloud.