An LCD screen with multitouch and off-screen gestural control

Summary:Researchers at the Massachusetts Institute of Technology have created a working prototype of a bidirectional LCD that allows a viewer to control on-screen objects without the need for any peripheral controllers or even touching the screen. In true Minority Report fashion, interaction is possible with just a wave of the hand.

Researchers at the Massachusetts Institute of Technology have created a working prototype of a bidirectional LCD (captures and displays images) that allows a viewer to control on-screen objects without the need for any peripheral controllers or even touching the screen. In near Minority Report fashion, interaction is possible with just a wave of the hand.

"This is a level of interaction that nobody's ever been able to do before," says Ramesh Raskar at MIT's Media Lab, who created the prototype along with colleagues Matthew Hirsch and Henry Holtzman, as well as Douglas Lanman at Brown University.

The BiDI Screen, as it's called, is based on LCD technology in which arrays of optical sensors are interlaced with a panel's pixels to detect multiple points of contact with the surface. This enables touch screen interaction.  But to get the screen to see the world in front of it, the researchers displaced the sensor layer of photodiodes behind the liquid crystal layer.  The LCD screen works double duty, switching between display mode and capture mode in real time. In display mode, the LCD functions as normal with backlight and liquid crystals modulating to produce an image. When in capture mode the screen serves as a pinhole array to capture the angle and intensity of light passing to the sensor layer and the backlight is disabled. By correlating data from multiple views across the sensor array, the system images objects (such as fingers) that are located beyond the display’s surface and measure their distance from the display. Computer software then enables gestural motion control of the on-screen objects.

The researchers designed the screen to operate within the limits of consumer off-the-shelf technology and managed to keep a thin form factor despite the additional layer of cameras. The approach resulted in an increased angular resolution as compared to other approaches that use a small number of cameras behind, to the side, or in front of the displays. such as Microsoft's Surface or Project Natal.

The research can potentially unlock a wide array of applications, such as in-air gesture control of everything from CE devices like mobile phones to flat-panel TVs, according to the MIT researchers.

Specifically, as higher-resolution video cameras and LCD screens become available, the researchers say their design should scale to provide high-resolution photographic images — enabling demanding videoconferencing, gaze tracking, and foreground/background matting applications.

The BiDi screen was presented at Siggraph Asia last week, in Yokohama, Japan.

Paper: BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Fields (9-pages, PDF)

BiDi Screen, 3D gesture interaction in thin screen device from Matt Hirsch on Vimeo.

Topics: Hardware

About

Christopher Jablonski is a freelance technology writer. Previously, he held research analyst positions in the IT industry and was the manager of marketing editorial at CBS Interactive. He's been contributing to ZDNet since 2003. Christopher received a bachelor's degree in business administration from the University of Illinois at U... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.