Sign language conversion and training by machine learning with an IOT approach

Abstract

For the people, who are living without sound due to disabilities, sign language can be a handy tool to make their life easy and comfortable. Sign language is actually a medium of communication for the people who are either deaf or deaf and mute. Besides, normal people can also use sign language to communicate with these people whose viewpoints always remain unspoken just because of their ill-fate. Generally, mute or deaf people use manual communication and body language to communicate with others. However, there is no such device by which a person can learn sign languages and express their thoughts easily. To make the things easier and simple we decided to create a device that could help to learn sign language via “Learning Application” and a mechanism to translate sign language into text. For translating “Sign Languages” into text we have developed one special glove attached with necessary sensors to take the relative hand position of the user. We have used one Machine-Learning classifier to classify the raw data of the sensors to have better accuracy. Then through the help of one IOT device we have send the data to mobile application where the actual meaning of sign is expressed into text to the receiver. As a part of the whole process, we have developed two different mobile applications. Of these two applications one can be used to train sign languages and other one simply carries the information that the user want to provide to the receiver. However, in some cases deaf and mute people might know the Sign-Language but the normal person might not. Therefore, along with helping the deaf and mute people this training application will also help normal people to understand the language to communicate disabled personnel. We have used American Sign Languages (ASL) as our standard medium for both of the application. The other application has a great usability just after the training session. When a mute and deaf person is already trained how to use sign language with our first application then he/she may simply use our system to express their words with the second application. We have chosen the smartphone because of the availability and functionality of the device. As Majority number of people owns smartphone it will be easier for anyone to learn and use the sign languages through our device. On the other hand we have used IOT devices for sending words to a longer distance. Nowadays almost every smartphone user is using internet in their handheld device. So it can be easily said that the device is user friendly as well. Overall, this device can help to eliminate the obstacles and communication barrier faced by a deaf or mute person when it comes to communicate with others.