Credit: Uli Benz / Technical University of Munich
A team of researchers at the Institute for Cognitive Systems (ICS) at TU München have collaborated with a group in Japan to develop Mask-bot, a 3D image of a human face projected onto the backside of a plastic mask, creating very realistic features that can be seen from various angles, including the side.
“Mask-bot will influence the way in which we humans communicate with robots in the future,” predicts Prof. Gordon Cheng, head of the ICS team.
One key distinction between comparable approaches to three-dimensional heads, note the researchers, is that rather than using a front projection, Mask-bot uses an on-board rear projection to ensure a seamless face-to-face interaction. The projector uses a high-compression fish-eye lens to spread the beam to a wide angle while a macro adapter shortens the focal distance to the transparent mask roughly 4.7 inches away.
Mask-bot can create facial expressions and simple dialog. For instance, when you say “rainbow”, Mask-bot flutters its eyelids and responds with: “When the sunlight strikes raindrops in the air, they act like a prism and form a rainbow.” And when it talks, it also moves its head a little and raises its eyebrows to create a knowledgeable impression.
Is this the remote boss of the future? (Credit: Jeff Bots)
Mask-bot comes with additional bells and whistles. It is bright enough to function in daylight thanks to the strong projector and coating of luminous paint sprayed on the inside of the plastic mask. This aspect means that it can potentially be used in video conferencing.
“Usually, participants are shown on screen. With Mask-bot, however, you can create a realistic replica of a person that actually sits and speaks with you at the conference table. You can use a generic mask for male and female, or you can provide a custom-made mask for each person,” explains Takaaki Kuratate, creator of the Mask-bot.
Mask-bot doesn’t need a video image of the person speaking in order to work. A program converts a normal two-dimensional photograph into a correctly proportioned projection for a three-dimensional mask. Additional algorithms add in voice and facial expressions based on a motion capture system.
Finally, Mask-bot can realistically reproduce content typed via a keyboard–in English, Japanese and soon German–using a text-to-speech system. It can produce a female or male voice, which can then be set to quiet or loud, happy or sad, all at the touch of a button.
Compared to mechanical faces which require dozens of actuators and compressors to appear lifelike, Mask-bot will be able to combine expression and voice much faster. Mask-bot 2, the next generation of the prototype will see the mask, projector and computer control system all contained inside a mobile robot, say the Munich researchers. The cost for the new version will also drop from about EUR 3,000 ($4,123) to EUR 400 ($550).
Besides video conferencing, “These systems could soon be used as companions for older people who spend a lot of time on their own,” said Kuratate.
