How Microsoft's Kinect could bring gesture recognition to business apps

How Microsoft's Kinect could bring gesture recognition to business apps

Summary: A Microsoft Most Valuable Professional (MVP) shows off how Kinect sensor integration with an ERP application might work

SHARE:
TOPICS: Microsoft
9

Enthusiasts are clamoring for Microsoft to share more about how/when/if the company will include Kinect sensor integration in Windows 8. But in the interim, the Softies and Microsoft community are already thinking about Kinect integration beyond the operating system.

I've blogged before about Microsoft's work on bringing the Kinect and Xbox into the healthcare realm. And the Redmondians have been talking since fall 2010 about connecting up the Kinect to the company's Lync unified communications application/service.

But wait... there's more. What would it look like to bring the Kinect gesture-recognition capability to business applications, like Microsoft's Dynamics GP ERP? Believe it or not, there are folks out there already thinking through this scenario.

Microsoft Dynamics GP Most Valuable Professional (MVP) Jivtesh Singh, who works for Smith and Allen Consulting Inc., blogged on August 24 about his efforts on this front. From his post:

"(E)arlier this year, I saw Business Analyzer (Available with Dynamics GP 2010 R2), and how it let you add beautiful KPIs (key performance indicators), Charts and Graphs to your personal dashboard. What if we let people navigate Business Analyzer with their gestures. That seemed like a good fit – and could be used in meetings!

"Microsoft has done a great job with the Kinect SDK and the Kinect ToolBox. Using the two I was able to track 4 gestures, and then send commands to Business Analyzer to navigate through the reports."

(Note: So far, the only version of the Kinect SDK released by Microsoft is a beta for hobbyists/non-commercial applications. Microsoft execs have said they are working on a commercial version of the SDK, but haven't provided a ship target.)

The four gestures Singh incorporated in his demo:

  • Swipe to Right – Moves to next Graph in Business Analyzer
  • Swipe to Left – Move back to previous Graph
  • Expand Gesture – Zooms into a graph
  • Contract Gesture – Zoom out

He even recorded a two-minute YouTube video demonstrating his work:

I do little on my PC that I believe would be improved by gesture navigation. But as Singh noted in his post, it was an injury causing acute back pain that led him to think this kind of scenario through.

Readers out there have any other business-focused applications that they think would be a good fit with Kinect gesture/voice input?

Update: Reader Johan Broberg (@pontemonti) noted that Microsoft also demonstrated a Kinect-powered Dynamics AX shop-floor manufacturing app at its Convergence conference this year.

Topic: Microsoft

About

Mary Jo has covered the tech industry for 30 years for a variety of publications and Web sites, and is a frequent guest on radio, TV and podcasts, speaking about all things Microsoft-related. She is the author of Microsoft 2.0: How Microsoft plans to stay relevant in the post-Gates era (John Wiley & Sons, 2008).

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

9 comments
Log in or register to join the discussion
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    <i>I do little on my PC that I believe would be improved by gesture navigation.</i>

    Obviously. After all, most business tasks are already done on a computer we've been improving that experience for 30 years.

    I think there will be simple improvements, like locking your computer automatically when you leave and waking it when you sit down, but I think the vast majority of Kinect-like functionality will be in enabling situations that haven't already been taken over by a computer. Take, for example, rebuilding an engine. It would be nice to have the schematics on a big screen, but be able to navigate through them without having to put down your wrench.
    Rich Miles
    • RE: How Microsoft's Kinect could bring gesture recognition to business apps

      @Rich Miles That's exactly where it's going to be useful: places where you can't readily use a mouse/keyboard.
      Aerowind
    • Yes, I understand those kind of use cases

      like those you describe... but as far as controlling business apps, like he demonstrated in the video... I am still not sold. We'll see, I guess. MJ
      Mary Jo Foley
    • RE: How Microsoft's Kinect could bring gesture recognition to business apps

      @Rich Miles

      What I want from Kinect: on a multi-monitor system, I want the focus to go to the monitor I'm looking at, not the one where I last typed or moused. For example, if I'm reading this article and hit the space bar, don't scroll down my Outlook window just because Outlook is the last place I clicked or typed.

      It would be nice to be able to use gestures for some simple commands, but I'm not sure how practical that would end up being other than flipping through open windows - I'm thinking of things like Mute and Sleep, but that's probably a lot to ask.
      1DaveN
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    Thanks for the great write up Mary Jo. I already see some great ideas in the comments by Dave and Rich.

    I recently also saw a new product using Kinect, which looks pretty good that lets you prepare presentations optimized for use with Gestures.
    http://www.so-touch.com/?id=software&content=air-presenter

    Jivtesh
    jivtesh
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    Kinect is great, truly! As a tool for the blind, or a sensor for robotics...

    ...But even stupid things can be done with great tools. Kinect and ERP? Perhaps one gesture is missing from the demo... giving the universal raised middle finger gesture to close the application? You know what your momma said about this kind of thing, you'll go blind doing this with ERP.
    P Newton
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    I think the biggest value is for presentations and meetings. It is a replacement for the wireless gyro mouse, laser pointer and clicker. It would make the transition from one presenter to another much easier and let the presenter drive without having to hold a navigation device.
    Gestures are only one part of what Kinect does. There is also voice commands. We use the voice recognition on the xbox quite a bit for movies and chat. There are also games in the pipeline that will use voice commends as an integral part of the game.
    neillfam
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    Reminds me of my former employer:
    http://t.co/H0DqxVh
    jezza101
  • RE: How Microsoft's Kinect could bring gesture recognition to business apps

    My family [url=http://www.ny-nflshop.com/]nflshop[/url] and [url=http://www.ny-nflshop.com/cheap-pittsburgh-steelers-jerseys/troy-polamalu-jersey-kids-womens-youth-authentic]troy polamalu jersey[/url] i see somebody put forward . that is definitely most certainly instructive and moreover very helpful.. I would like to consider much more in regard to connected endorsing.. [url=http://www.ny-nflshop.com/cheap-green-bay-packers-jerseys/aaron-rodgers-jersey-kids-womens-youth-authentic]aaron rodgers jersey[/url] Many thanks for revealing an individual's know-how on this ..
    makrekdw38-24353606033503686940876738725179