Microsoft's voice-enabled assistant technology still in the works

Microsoft's voice-enabled assistant technology still in the works

Summary: Microsoft is still a big believer in voice becoming a key way for users to communicate with and query their devices in a more natural way.

SHARE:

Back in 2011, Microsoft CEO Steve Ballmer was touting heavily the idea that voice-enabled assistants would become key to the way users interacted with their PCs and devices.

nuienvisioning

Since that time, Microsoft execs have gone largely quiet on that front. Even at this week's TechFest Microsoft Research fair (at least on the day that was open to invited external guests), Microsoft had little to show or tell about developments on the voice-control front.

In spite of the silence, Microsoft's work on this front is alive and well. At the company's newly relaunched Envisioning Center, voice-enabled assistant technology was almost as front-and-center as touch.

The Envisioning Center features demos of technologies that Microsoft officials believe are between three and 10 years away from widespread usage. The Envisioning Center replaces Microsoft's separate Microsoft Home demo space and its Office Labs Envisioning work. The single redesiged center is meant to show that home and work are no longer separate.

I got a tour of the Envisioning Center this week while on Microsoft's Redmond campus. (No photos were allowed, but Microsoft did post a few of its own.) The Center included collaborative office, remote "touchdown" spaces, team project rooms, small-huddle brainstorming areas, and kitchen and living-room spaces. Large-screen, multitouch Perceptive Pixel Inc. (PPI) displays were everywhere -- on walls and even embedded in desks. Kinect sensors were built into display surfaces throughout. And voice-assistant technology figured prominently in home and work settings.

clippy2

In these demos, users could ask their displays to pull up information then refine their queries without having to figure out specific keywords or artfully craft and hone their search queries. Users could just ask things like "Is this part in stock? Will it fit in my design? Are there local suppliers who can get this to me today?" and let the system figure out the relevant context and access the necessary metadata attached to any given object. This is not simply Clippy 2.0 (without the avatar) or Apple's Siri; it's more far-reaching than either.

As I blogged back in 2011, there are a number of current and future Microsoft technologies at work behind the scenes in these voice-enabled-assistant demos. Kinect sensors, which include voice recognition, are one piece of the puzzle. But the work coming from the Bing, Microsoft Research and the former TellMe team (which is part of Microsoft's Online Services unit) are all key, too.

The Bing team has been working with Microsoft Research to improve Bing's inherent natural-language-search capabilities. At the same time, Microsoft has been doing work around Bing/Tellme/social-graph integration, officials have said, with the ultimate goal being a speech natural-user-interface (NUI) service to help users accomplish tasks in a more natural and conversational manner. 

There's also an augmented-reality-focused team in Bing that's been working on everything from camera tracking, to visual and audio recognition, to optical character recognition and translation and vision-based natural-user interfaces. The team already has made available some AR deliverables, including the Bing translation app, augmented-reality-enriched Bing Maps, and the Bing Vision and Bing Audio technologies in Windows Phone.

For those continuing to wonder whether Microsoft intends to unload Bing and throw in the towel on search, I'd say these efforts show Microsoft has no such plans. Bing is central to Microsoft's plans for the future, as I've said before and will repeat again.

As I noted earlier, the Envisioning Center tech demos are meant to represent projects that are three to 10 years away from commercial reality. It will be interesting to see whether this speech-enabled assistant technology materializes on the nearer or further end of that scale ....

Topics: Emerging Tech, Big Data, Cloud, Microsoft

About

Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

25 comments
Log in or register to join the discussion
  • Microsofts voice recognition is far superior to apple/google but if their

    personal assistant ai is still 3-10 years away they're going to get killed in the market. It should have come out with WP8. They need to get it out asap and worry about improving it over time in the public sphere.
    Johnny Vegas
    • I think that really depends on how good it is.

      I've played with Siri and I was less than impress. I haven't been able to play with the Google alternative, but I think if MS can come up with a good offering that is as good or better than why Siri was hyped up to be they can definitely be competitive on that front, even if its a couple more years off.
      Sam Wagner
      • Bing is a lost cause

        The only way it will gain any real relevance is if Google fails.

        The catch is, Google is failing. Recently I was in the UK and decided to look up a person I could find easily with Google in the US. Google decided to force me to use its UK version of Google search. And that did not produce a single hit that I would normally get with the US version of Google search. So I the next step was to ensure I was using the Google edition of search. Still no luck. Then I decided to go back an use Yahoo (tried Bing very few hits, Yahoo's implementation of Bing worked like a charm and got me the results I wanted). Because its a name I wont publish it out here. Though if Google or Bing or Yahoo would like details, I can prove it to them.

        My point. Google's action to make searches regional is working against it. Bing currently keeps it Global, that is a big plus for Bing if Google doesn't address the problem now.
        Uralbas
        • Double check your post.. my bad.

          I am using a Win8 machine that fails to type letters. Oh well. Win 8 and its flaws only cause extra work. Though it was my flaw not to review my post properly.

          The second paragraph above should read as follows:

          The catch is, Google is failing! Recently I was in the UK and decided to look up a person that belongs to a group called ASO out of South America. When in the US, I could easly find it using Google. Though because I was in the UK, Google decided to force me to use its UK version search. I did not produce a single hit that I would normally get with the US version. So the next step was to ensure I was using Google US Edition of search. Still no luck. Tried using Bing, with very few results, still better than Google. Then I decided to go back an use Yahoo, it worked like a charm! and retrieved the proper results. Due to its personal nature, I can't publish it here, though if Google or Bing or Yahoo would like details, I can prove it to them.
          Uralbas
          • Well there's your problem

            You're using Windoze 8. It makes you do extra work. You like that.
            CaviarGreen
          • are you sure this did not effect your search?

            "I am using a Win8 machine that fails to type letters."
            coastin
    • Try Maluuba on WP8

      its free
      everss02
      • It's ok

        mainly returns a bunch of wolfram data on screen. Doesn't read data to you.
        frankwick
  • Already late

    They should bring something nice.
    AleMartin
  • I rarely criticize microsoft. I do it for the first time.

    The big white man the brother - Bill Gates!
    The red-skinned Sharp-sighted Falcon the Web designer welcomes you!
    Windows 8 did not sustain the test for reliability. It is known to you.
    But, it is possible that you do not know that the new operating system is absolutely not suitable for use in modern Russia.
    You created a good classical product. The best programmers worked wonderfully well.
    However the new generation of vandals does not read classics. Them does not interest romanticism of programming.
    Nevertheless, they use modern electronics.
    Therefore, quickly and successfully, they crack computers of users.
    Antiviruses are powerless.
    Firewalls do not protect.
    System administrators are helpless.
    Instead of trampling down another's mobile phones, you should think of clients.
    I congratulate employees women of your empire on March 8.

    If you do not read my congratulation on March 7, on March 8 I will publish this text and I will congratulate.
    One does not sharpen the axes after the time they are needed. Russian proverb.
    However to me it is unclear, how I learn that you will read.

    Translated by Google.

    http://answers.microsoft.com/ru-ru/windows/forum/windows_8-windows_install/%D0%B0%D0%BA%D1%82%D0%B8%D0%B2%D0%B0%D1%86%D0%B8/b8282cc0-1c6f-4fa6-ba26-56298e603074
    https://productforums.google.com/forum/?hl=ru#!topic/gmail-ru/N6EWMXmHHuQ
    http://newshe.msk.ru/?p=7399
    Валерий Пионкин
    • I'm at a loss

      Something was lost in translation.
      William Farrel
      • Da kamrade

        Me Tarzan. You Jane.
        CaviarGreen
      • I can help

        He wrote a post to Bill Gates, saying that he usually is not critical of Microsoft but, although many good programmers worked on win8 it just does not work in modern Russia.

        It is like producing a classical novel but the problem is that the young generation in Russia do not read the classics. Referring to the code and old style of programming, he says that the young Russians don't care and they know how to hack into it and will and that they do use modern devices (remember that Linux is very much in use in Russia).

        He also writes: "Instead of trampling down another's mobile phones, you should think of clients."

        Looks like he is talking about Microsoft and Android here.... Yep that is it.

        "I congratulate employees women of your empire on March 8." This has something to do with Microsoft employees being not treated with respect (women of your empire).

        "One does not sharpen the axes after the time they are needed. Russian proverb."

        This basically says Microsoft brings too little too late.

        He gave Bill Gates one day head start be fore he published this, hoping he would read it and reply to his forum post.

        I had many servers at a US data center that was bought by Russians and learned to work on the servers with them remotely. Communication is tricky....
        coastin
        • Thanks, coastin

          I'm not sure how he came to his conclusions, but at least I can follow what he is saying now.

          Nice work on the translantion.
          William Farrel
          • You are welcome

            Russian translation is difficult at best.
            coastin
          • I rarely criticize microsoft. I do it for the first time.

            windows 8 is not protected
            patching holes useless
            microsoft can not perform the safety and service at the same time
            Валерий Пионкин
      • I rarely criticize microsoft. I do it for the first time.

        Related articles in Russian
        I rarely criticize microsoft. I do it for the first time.
        http://newshe.msk.ru/?p=7399
        http://newshe.msk.ru/?p=7426

        Russian gangsters make special electronics.
        Used to break not only the program, but also electronics.
        Валерий Пионкин
  • voice recognition

    In terms of recognition accuracy, WP has got Siri beat by a long shot. However, Siri understands a ton more natural language queries. I'm amazed at the accuracy of TellMe sometimes.
    frankwick
  • Microsoft's voice-enabled assistant technology still in the works

    ibm was working on voice recognition for ages now as are indian scientists, and maybe everybody else. car manufacturers were one of the first adopters of voice recognition/synthesis that people got tired of it and abandon it wholesale (kind of having a nagging better half always on your @#* big time.) the problem with recognition is the way people speak, some mumble words that ordinary human can make sense of, but no machine can. add to it the different regional accent people have, and you have a recipe for chaos. maybe the future is now and m$ may have some hidden tricks up its sleeve to pull a fast one ... but the fact remains, i.e. will consumers be receptive this time?
    kc63092@...
  • Finally news about where tellme is going

    Thanks for the article, I have been waiting eagerly for news about where tellme is going ever since it was created. To hear that it will be integrated with bings natural speech recognition is exciting because there will finally be an intelligent assistant not a keyword based search engine assistant like siri
    soccerfreak_93_9@...