X

Photos: Microsoft Kinect pushes natural UI frontier

Computer vision set to change how we interact with all our machines, says Microsoft...
By Natasha Lomas, Contributor
40154148-1-610-microsoft-kinect-research-1.jpg
1 of 9 Microsoft

Computer vision set to change how we interact with all our machines, says Microsoft...

Microsoft has been showcasing how its Kinect computer-vision technology is pushing beyond the boundaries of gaming. Computer vision refers to a research field exploring how computers can extract data from images. Just as our eyes and brain sense and inform us about our immediate environment, a computer equipped with a camera and computer-vision software can identify objects or sense movements in its vicinity.

During a tech demo day at its Seattle offices last week, Microsoft demonstrated how computer vision and Kinect-style natural user interfaces (NUIs) could change how people interact with machines and screens in future. In February, the company announced it will be launching an SDK for Kinect to enable third-party developers to create software for a Kinect motion sensor plugged directly into a Windows PC - potentially bringing NUIs to a monitor near you soon.

"GUI plus NUI is the way people will interact with computers in future," said Craig Mundie, Microsoft's chief research and strategy officer, in a statement.

Pictured above is one of Mircosoft's NUI research projects - called Avatar Kinect - which enables Kinect users' avatars to mimic the head movements, speech and facial expressions of their human owners. Participants can also hang out with other Avatar Kinect users in virtual environments and shoot animated videos to share online, said Microsoft.

40154148-2-610-microsoft-kinect-research-2.jpg
2 of 9 Microsoft

In the second generation of Microsoft's Surface touchscreen technology - Surface 2.0 - the table itself is one large camera, so it can recognise objects such as this paint brush, and also identify the direction of users' hands and finger orientation.

40154148-3-610-microsoft-kinect-research-3.jpg
3 of 9 Microsoft

Here, a Microsoft researcher uses a handheld digital camera to take a series of 2D images of a statue.

40154148-4-610-microsoft-kinect-research-4.jpg
4 of 9 Microsoft

The images of the statue are then automatically converted and rendered into a photo-realistic 3D model of the statue, which can be manipulated on screen. The system does not require a high-end camera to generate the 3D model, even working with a mobile cameraphone.

40154148-5-610-microsoft-kinect-research-6.jpg
5 of 9 Microsoft

Here, in a project known as Mirage Blocks, a stereoscopic projector and the Kinect camera are used to render real objects digitally in a 3D space. The virtual objects could then be used for remote collaboration on projects.

40154148-6-610-microsoft-kinect-research-5.jpg
6 of 9 Microsoft

This photo-realistic digital talking head is the fruit of another Microsoft research project. The head can be animated in real time so it appears to speak the text that is being typed into the dedicated text box on the left. Microsoft envisages using the technique to create more believable avatars in future.

40154148-7-610-microsoft-kinect-research-7.jpg
7 of 9 Microsoft

This demo from Microsoft's Applied Sciences Group shows how touchscreens could evolve. Instead of touching a surface, the user makes gestures above a retro-reflective screen which has a camera and a projector pointing down at it.

Computer-vision algorithms are then used to interpret individual gestures - such as the closed-finger loop pictured above - enabling the user to write, paint or control navigation on top of the screen.

40154148-8-610-microsoft-kinect-research-8.jpg
8 of 9 Microsoft

Microsoft envisages various NUI technologies being used in tandem in future. Here, Microsoft's Mundie shows a conceptual demo of various work and social NUI technologies organised into spherical hubs to illustrate how the various techs could be displayed and accessed together.

40154148-9-610-microsoft-kinect-research-9.jpg
9 of 9 Microsoft

Microsoft expects mobile phones to be used as the remote control to organise the various work and social hubs, as shown above.

At the Mobile World Congress (MWC) trade show earlier this month, Microsoft CEO Steve Ballmer showed a video of a smartphone running its Windows Phone 7 OS being used in conjunction with the Kinect gaming peripheral - so that multiple players can participate in the same game across different devices.

"2011 is shaping up to be a very fast-paced year," said Ballmer at MWC. "It should be a great year for the development of intelligent mobile devices and the value that they add to society."

Related Galleries

Holiday wallpaper for your phone: Christmas, Hanukkah, New Year's, and winter scenes
Holiday lights in Central Park background

Related Galleries

Holiday wallpaper for your phone: Christmas, Hanukkah, New Year's, and winter scenes

21 Photos
Winter backgrounds for your next virtual meeting
Wooden lodge in pine forest with heavy snow reflection on Lake O'hara at Yoho national park

Related Galleries

Winter backgrounds for your next virtual meeting

21 Photos
Holiday backgrounds for Zoom: Christmas cheer, New Year's Eve, Hanukkah and winter scenes
3D Rendering Christmas interior

Related Galleries

Holiday backgrounds for Zoom: Christmas cheer, New Year's Eve, Hanukkah and winter scenes

21 Photos
Hyundai Ioniq 5 and Kia EV6: Electric vehicle extravaganza
img-8825

Related Galleries

Hyundai Ioniq 5 and Kia EV6: Electric vehicle extravaganza

26 Photos
A weekend with Google's Chrome OS Flex
img-9792-2

Related Galleries

A weekend with Google's Chrome OS Flex

22 Photos
Cybersecurity flaws, customer experiences, smartphone losses, and more: ZDNet's research roundup
shutterstock-1024665187.jpg

Related Galleries

Cybersecurity flaws, customer experiences, smartphone losses, and more: ZDNet's research roundup

8 Photos
Inside a fake $20 '16TB external M.2 SSD'
Full of promises!

Related Galleries

Inside a fake $20 '16TB external M.2 SSD'

8 Photos