Bangla sign language recognition and sentence building using deep learning

Abstract

Modern age being the era of Information technology, it would not have come this far without the piled up data or information. Whereas communication is the basis of collecting or gathering data or information, almost 5% of the world's population is not blessed with the ability of verbal communication [1]. Sign language varies from the verbal language in every form and rule. This creates a gap between people conversing in verbal language and those communicating in sign language. Verbal languages are easy to interpret for having a common rule-following but sign language differs from region to region. This hampers the communication between normal people and those interacting in sign languages. Human to human interpretation is tough because of the enriched word wise signs and vocabs. To eradicate this issue, we are proposing a machine-based approach for training and detecting the Bangla Sign Language. Our aim is to create a multi modal system to for recognising Bangla signs. In addition, we hope to train the system with enough samples containing different signs used in Bangla Sign Language. In this research, we are using the Convolutional Neural Network (CNN) for training each individual sign. In addition to working as a medium of communication between the deaf and mute with the remaining society, this approach would also serve as a tool for the hearing deprived to learn and use the sign language properly.