There's a huge opportunity in robotics for early career computer scientists and serious software engineers

Maya Cakmak is pioneering ways for non-experts to program robots. Her work is opening a new field you should know about.
Written by Greg Nichols, Contributing Writer

There's a major roadblock to deeper market penetration of enterprise robotics, and a new generation of early career computer scientists and more seasoned software engineers may hold the answer.

I recently had a chance to speak with Maya Cakmak, assistant professor at the University of Washington, Computer Science & Engineering Department, where she directs the Human-Centered Robotics Lab.

Professor Cakmak's research centers on human-machine interaction, and in particular programming by demonstration (PbD).

To understand PbD, consider collaborative robots from companies like ABB and Kuka. The units consist of articulated arms that can be programmed to help workers do a variety of things, such as pick and place objects, test devices and components, and perform simple but precise manufacturing tasks.

So-called "cobots" are relatively inexpensive and operate alongside humans, and many of the use cases involve small- to mid-sized businesses. That should be huge for adoption, but the dam hasn't broken just yet.

The reason is that programming a robot is no easy task, which presents challenges when it comes to deployment. That's particularly true of small- and mid-sized businesses, which are unlikely to have a qualified roboticist onboard. As robots become more complex, capable, and sensor-rich, the problem will only compound.

Professor Cakmak and her students are working with an autonomous mobile robot called Fetch from Fetch Robotics, finding novel ways to allow non-robotics-experts to program the unit to do specific tasks.

One of the big takeaways? She and other researchers are making huge progress, but the day when an average Joe without a computer science backward can program a robot like Fetch flawlessly is still far off. When it comes to complex robots that combine mobility with dexterity to do complex tasks in series, it's even farther on the horizon.

In the interim, a new industry is emerging for computer scientists and software engineers without engineering degrees: robotics deployment. The industry is primed to grow hand-in-hand with the surging robotics market.

Here's what Professor Cakmak has to say about it.

Which companies are really working toward allowing non-experts to program their robots?

Some are specifically targeting the challenge. Baxter and Sawyer [from Rethink Robotics] are marketed as easy to program. They say company floor workers can program it, and they have some videos. So there are some solutions that target ease of programming. But I've met some students who have worked at these companies, and the software is still difficult for most robots. Some of the more accurate robots out there have 300-page user manuals. I've seen some code for these, and you have to know algebra and matrix transformations to still be able to do anything. So we're still far from robots that are easy for non-experts to program. But companies like Rethink in the US and Franka Emika in Europe are working on it.

Unpack Programming by Demonstration for me. On a basic level, what does that mean?

Basically, you demonstrate a task and the robot figures out what the program should be to recreate what you demonstrated. The demonstration could be given in different ways. Right now, the only practical way is to move the robot through the steps physically. But there are challenges. The robot has to relax its arms so a person can put it through the steps of a task, and usually that demonstration alone isn't enough. There needs to be some command in between steps. For example, if you move the robot's arm in one direction until it touches something, the robot won't really know whether it's simply trying to go to that position, or whether you want it to go downward until it hits something.

Tell me about some of your work with the Fetch robot.

Fetch is a mobile manipulator. We made one big system at the high level that allows you to program the robot with a visual language. You're not really typing code, but instead dragging and dropping blocks and combining them in certain ways to define the logic of the program. And the different components correlate to moving the head around, for example, or doing some base navigation.

The robot's arms are where we don't really have block actions. For mobility we can tell it to go to XY on the map. For arms we don't really have a single "pick up" command. I mean, picking up an object is still a whole research field, and the reason is that researchers are trying to figure out how to program a robot to pick up robustly--to pick up a whole range of objects. What we're trying to do is let a person program manipulator actions by demonstration, and then define those. So "pick up a bottle" would be become a block at that higher level.

So there's this gulf between these fabulously sophisticated machines and the end-users, who may not know how to get the most out of them. What are the opportunities there?

There's a spectrum. On one end is software engineers who are not roboticists. These are people who can write serious software. How do we create APIs for them so they can deploy robots to do complex tasks? On the other side, there's a new generation of kids coming out of high school with programming experience. Perhaps robotics companies will soon hire people to deploy robots using those high level APIs. Maybe people with a computer scientist background will be trained for a week on a specific system.

So in that spectrum, right now the opportunity is at the more technical extreme. I see robotics startups hiring more software engineers. Currently, the standard for my students is to use the Robotic Operating System (ROS), which a lot of these companies build on. You could imagine software engineers learning a package version of ROS in shorter duration. So within a month, say, they'd be learning to program robots. That's one big opportunity.

Eventually I also see this robot deployment phase. People who are minimally trained can go in and have robots adapted to a warehouse or a home, say.

Can you give me an example of how these simplified APIs are creating new roles and allowing robots to be used in new ways?

Sure. I collaborate with Savioke quite a bit. Savioke makes a mobile robot for room service delivery in hotels.

One of my students built a tool for end-user delivery. What they did, they had this one really nice, well-defined application of the robots delivering things from the front desk to a hotel room. They programmed it with the software and a robotics team deployed it. So there you have some sophisticated programming.

Then hotels started asking for new features. They wanted the robot to sing happy birthday, for example. It's been doing staff-to-guest delivery, but can it do staff-to-staff delivery?

So these requests inspired having an easy way to repackage and reprogram the robot. Someone had to develop a new API. It uses easy things like blocks, and it can add a loop. That was actually the start of this block-based programming idea. Visual block-based programming for hotels. So now the company has their internal team also using it. It makes development much easier.

And now they've also hired customer representatives, people who go to hotels to discuss their problems, ask about potential features. These are people who are not programmers but can actually program a robot on the spot. So you see all these opportunities both on the technical end and the more entry-level end.

What role do you think PbD will eventually play in the proliferation of robotics?

It's really critical. It will enable use cases that engineers didn't think about. People who see problems can figure out how to use robots to solve them. It will empower those people to program robots for themselves.

Editorial standards