Man hacks Kinect to assist stroke victim

Summary:Chad Ruble has hacked Kinect software to help his disabled mother, who can't use a keyboard after suffering from a stroke 12 years ago.

Chad Ruble has hacked Kinect software to help his disabled mother, who can't use a keyboard after suffering from a stroke 12 years ago.

Ruble's mother suffered a stroke which caused her to develop Aphasia -- a condition that irrevocably damages the language functions in the brain and makes it difficult to communicate both thoughts and emotion.

Worried for his mom and pondering ways that technology may improve her quality of life, Ruble wanted to create a new type of keyboard that she could use to express how she was feeling without needing to read individual, standard keys.

The first stage was to design a keyboard with visual icons -- representing emotions including happiness, sadness, and feeling tired. In order to hone this further, Ruble included a series of bars that his mother could use to track how intensely she felt, such as 'very tired' or 'very happy'.

He then hacked his $109.99 Kinect using the SimpleOpenNI library to process the input from the Kinect through gesture recognition code to track the position of his mom's hand. To complete the hack, Ruble used a "sample processing sketch" provided by Daniel Shiffman to generate and send a complete message via email by 'clicking' the green button. Once complete, by waving her hand the message can be reset by crossing the screen's red "X" mark.

Ruble plans to add more refined features which will allow more variety in messages but maintain "a super simple interface" for his mum. He writes on his blog:

"It would also be fun to snap and attach a jpeg from the RGB Kinect camera to the email. Still, its clear that mom is happy with the result. I might also send the emails to a Posterous account so that she can have her own blog."

Someone else has to launch the program for her at the moment, but as Ruble writes, it's certainly a great start.

Image credit: Chad Ruble

This post was originally published on

Topics: Innovation


Charlie Osborne, a medical anthropologist who studied at the University of Kent, UK, is a journalist, freelance photographer and former teacher. She has spent years travelling and working across Europe and the Middle East as a teacher, and has been involved in the running of businesses ranging from media and events to B2B sales. Charli... Full Bio

zdnet_core.socialButton.googleLabel Contact Disclosure

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Related Stories

The best of ZDNet, delivered

You have been successfully signed up. To sign up for more newsletters or to manage your account, visit the Newsletter Subscription Center.
Subscription failed.