Microsoft mashes up multiple natural-user-interface inputs

Publicly and privately, Microsoft officials have been making much of the company's myriad multi-touch input projects (especially with Windows 7 and Windows Mobile 7). But Microsoft's view of what the user interface of the future will look like is more complex than that.

Instead of allowing users to interact only with touch or only with speech, Microsoft is working on interfaces that will combine multiple natural-input techniques. At last week's Financial Analyst Meeting, Microsoft officials showed off a demo of an automated front-desk receptionist, which the company plans to deploy internally later this year. The receptionist will make corporate-shuttle reservations, provide campus information and the like.

(The automated receptionist, it turns out, is one of the fruits of a Microsoft Research effort, known as the "Situated Interaction project." Microsoft officials are discussing that project at this week's Research Faculty Summit, according to the agenda for the event, which kicks off on July 28. Other projects upon which the Situated Interaction team is investigating include "multi-participant engagement and dialog models, conversational scene analysis, spatio-temporal trajectory reasoning, and behavioral modeling.")

Craig Mundie, Microsoft's Chief Research and Strategy Officer, outlined some of Microsoft's thinking during his keynote at FAM on July 24. Mundie told Wall Street analysts and press in attendance:

"When people talk about a natural user interface, you know, we talk about handwriting and touch and speech and these things, but this is what a natural user interface is really going to be all about. And it won't be just your receptionist. I mean, you should be able to come to computers and interact with them in a much more natural way, to ask questions, have them respond to you to do tasks that are valuable to you. And I think this is just the tip of the iceberg, but it's the first example built in a completely new way using these robotics technologies that we brought to the market two years ago. And so this isn't really about just programming arms that assemble cars in the factory or making things that run around hospital floors, this is in many ways the beginning of building very complex interactive applications."

Mundie also showed off during FAM a demo I've seen before that combined a variety of natural-interface technologies -- everything from facial recognition (which also is expected to find its way into the next version of Windows Live Photo Gallery, by the way), to more spatial recognition. (Guess that explains, at least in part, Microsoft's recent decision to merge the PhotoSynth photo-stitching team with the Virtual Earth mapping one.)

Mundie referred to the demo as an example of "first life" -- which he described as "a mirror world of 3-D that everybody can participate in constructing and maintaining and which gives us a navigational metaphor that's completely consistent with the world we already live in."

I have to admit, I am old-school when it comes to interacting with my PCs and devices: I like the keyboard. What about you? What kind of input are you hoping for with cell phones, ultra-light-weight PCs and laptops in the future?