Interactive 3D-environments based on MPEG-4
Computer scientists at the Fraunhofer Institute for Digital Media Technology (IDMT) have developed a media player for interactive 3D-environments. 'Our system allows us to actively involve the viewers -- they can walk through rooms and select objects, for instance,' says one researcher. This is possible because each element -- a person, a video clip or a sound -- can be integrated into the display. And if you're a -- filmed -- spectator, your image also can be interactively inserted into the viewer.
You can see above one example of how 3-D displays can be used interactively with the Fraunhofer technology (Credit: Fraunhofer IDMT). Here is a link to a larger version of this image.
For example, here is one scenario imagined by IDMT. "With just one click of the mouse, music-lovers can look over the guitarist's shoulder in a three-dimensional scenario. The joystick enables viewers at home to 'walk' right across the stage and experience the recorded concert from all sides. The surround sound moves with the spectator -- if you turn your back to the stage, then the sound seems to be coming from behind."
Uwe Kühhirt, who heads the development at the IDMT, adds that "Camera perspectives can also be interactively selected. Ambient sensors that determine factors such as the brightness, the temperature or the number of spectators, enable the scene to be dynamically controlled -- for example, the viewer can be integrated into the three-dimensional scene with the aid of a video camera."
And if you don't have a 3D display at home or at work, don't worry. You'll be able to benefit from this technology on any time of monitor -- even your cell phone. "'The player receives the data, such as a scene from a concert, and calculates the optimum image and sound reproduction for the respective playback system,' says Kühhirt."
This development has been based on a previous project named "Interactive Audiovisual Application Systems" (IAVAS) which ended in September 2004. You'll find more information about IAVAS here and there.
The new IDMT display system will be shown during IFA 2007, which will be held in Berlin between August 31 and September 5, 2007. The IDMT's exhibit will be called "Immersive Environments – Interactive 3D-scenarios based on MPEG-4" and be on display on stand 18. You'll find a short introduction on the Interfaces Exhibits page. "The technology allows for immersing and interacting within a three-dimensional mixed reality scene based on the object and scene concept of MPEG-4. Instead of using special shutter glasses one can realize three-dimensional audiovisual applications by using an auto-stereoscopic display and-- with the help of a multi-channel sound system -- implement immersive scenes."
If by chance you're attending IFA 2007, please visit the IDMT booth and tell me what you thought about the technology.
Sources: Fraunhofer-Gesellschaft Research News, August 2007; and various websites
You'll find related stories by following the links below.