Dear Zuckerberg: Dream bigger with your AI butler
HERB with his favorite snack.
Of course it made all the tech blogs. When Mark Zuckerberg starts talking about AI butlers you're going to see some coverage.
Robotics
In case you didn't hear, as part of his annual self-improvement goal, Zuckerberg wants to use 2016 to create an AI butler for his home. What will it do? Some of the tasks he mentioned are controlling home lighting and temperature, recognizing friends' faces at the door, and maybe even displaying VR representations of data to help him work.
That's pretty cool stuff, especially when we're talking about a guy tinkering in his (very well-appointed) garage in the few hours of free time he must have. But a lot of the coverage focused on the functionality Zuckerberg was describing, which shouldn't have been much of a story.
What struck many AI researchers and was probably apparent to anyone who's up on the latest smart home gadgets is that just about everything Zuckerberg outlined already exists, much of it off-the-shelf. He's essentially envisioning a souped up Amazon Echo. Huge props for trying to MacGyver that kind of system himself, but the only news here is that we may be looking at the best-funded Instructables project in history.
After the announcement I spoke with Sidd Srinivasa, associate professor at Carnegie Mellon University's Robotics Institute and founder of CMU's Personal Robotics Lab. Srinivasa is also the creator of a decidedly more ambitious robot butler. The butler's name is HERB, which stands for Home Exploring Robot Butler.
"Mark spoke very eloquently, I thought," Srinivasa tells me. "My only critique is that I wish he would dream bigger. What the community is looking for from someone like him is a big idea."
What Srinivasa is talking about is the next frontier of AI personal assistance, an embodied presence that can physically manipulate the environment around it. The distributed sensors and IoT technology in today's smart homes will doubtless soon become the eyes and ears of physical robotic systems. Those systems might come in the form of mobile robots or articulated arms dangling from a kitchen cabinet, like the ones created by Moley Robotics to cook meals.
The big challenge is manipulation, and the reason researchers like Srinivasa are looking for someone to take a big stand is that they've finally arrived on the threshold of several major breakthroughs concerning how robots navigate and manipulate the world around them.
"If you asked me five years ago," says Srinivasa, "I would have said the hardware just isn't there yet. I don't think that's true anymore. There are several robot arms using multimodal sensing that are pretty damn good. We work with a company that makes powered arms for use with wheelchairs, and we're seeing that people with high spinal cord injuries can do amazing things with those arms."
In other words, with humans acting as the master controller, robots are incredibly dexterous manipulators. Sensor technology has also advanced while prices sensor prices plummet. HERB uses color and depth cameras, tactile sensors, strain gauges, and force sensors to collect data from his environment. The work underway now is to develop powerful algorithms that will enable him to manipulate objects with a degree of dexterity similar to that of humans.
We're not there yet, but we're getting closer. HERB, which is the most advanced manipulation robot of its kinds, demonstrated just how close a couple years back with a so-called Oreo cookie challenge. The task? Separate an Oreo cookie and scrape off the cream without any breakage.
"An Oreo was by far the most delicate and sensitive object HERB had worked with," Srinivasa remembers. "If you look at an Oreo, they all look alike, but they have different physical properties, different parameters, different squishiness, and other variations. Manipulation is interesting because the only way you can sense those things is by interacting with them.
HERB completed the challenge completely autonomously (see vid above).
Which brings us back to Zuckerberg. Good on him for wanting to create the kind custom-made smart home most people are content to wait a few more years for. But after his house tells him his friends are at the door he's going to have to get up and open it. And man, that's no job for a billionaire.