On Wednesday, Microsoft showed in a very lengthy blog post how the Kinect is being used to help deaf and hearing impaired users how to interact. The Microsoft Research Asia and Chinese Academy of Sciences has put together a prototype that should help right away.
The real time tool is built to help translate sign language into spoken language, and spoken language into sign language in real time for users. With an estimate 20 million people in China who are hard of hearing and hearing impaired, this project should be a hit soon.
How It Works
The latest tools use the Kinect in all new ways to capture the conversation from both sides, from each person. The written and spoken translation is rendered in real-time and the person on the other end translates that into understandable signs for the other party. This is all done with the Kinect, and the words appears on the screen for the user.
The Kinect is the key part of this tool, since it is able to interpret the signs from one person and the other party is able to translate those words for the hearing impaired user. By gesturing to a Kinect device to the prototype, words appear within a second for them, and this could help the 20 million in China, along with hearing impaired users around the globe.
The Kinect Role
The Kinect part of this experiment is the main key of making this work. It captures the information via the Kinect sensor and that connected to the Kinect for Windows software developer kit helps users translate gestures, voice commands, movements, and other natural expressions for hearing impaired and deaf users. Demos have been shown and it has proven very promising.
Although promising, it still remains a prototype since the Chinese language in this example has many words. They have done 300 sign language words out of 4,000 and have a lot of work to do. The Kinect though puts all of the people together so that they can work quickly to get this done for users, and hopefully bring this type of hearing impaired assistance to users everywhere.
I love this type of Microsoft labs stuff. It shows that the Kinect isn’t just for games, and that it makes a difference everywhere and for everyone.