Turns Out Kinect Can Read Sign Language

Kinect's patent application said the device could recognize American Sign Language. Microsoft then backed off the claim, at least for the initial generation of the device, saying that sort of application would be supported in later models. Then people started developing open-source Kinect drivers and doing all sorts of cool things with them. And lo and behold, the commercially available Kinect recognizes sign language.


This is a proof of concept by some French researchers showing Kinect recognizing gestures for "Hello" and "Sorry." It's not clear how detailed Kinect can get in reading gestures - I'm guessing ones differentiated by finger positions or numbers held up might be a little tough to read.

It's still a solid demonstration of the device's capabilities, and if you're wondering about its practical application - a person who cannot speak can use some other input already available - it's possible this could be used in an educational program teaching sign language, to the hearing and non-hearing alike.

Illustration for article titled Turns Out Kinect Can Read Sign Language

Kinect Used to Translate Sign Language [Ubergizmo]


I'm a sign language interpreter, and I'm wondering why no one bothers to ask people who actually speaks sign language about this?

Just to throw a few facts out there;

-like the article (thankfully) says; you also need fingers, not only hands. That's the least of the problems though.

-to understand sign language, you need to be able to see the face just as much (if not more) than the hands. (Though most beginners focus on the hands.)

-Sign language isn't international. (they're languages, just like the spoken ones.-which means that they've developed naturally in their respective countries.-and yes, with accents.)

-Sing language uses different grammar than whichever language it's intended to be translated to. Kinect can't translate let's say American sing language to English any more than it can translate French to English.

Yes, kinect can recognize gestures; but even if those gestures happen to resemble signs from sign language it doesn't mean that kinect can understand sign language any more than it can understand English based on it understanding commands like "kinect; on!".