Two dimensional convolutional neural network CNN approach for detection of Bangla sign language

Abstract

Sign language is known as the primary communication medium for deaf and mute people. But the lack of available resources and a steep learning curve deter the average person from learning it making communication with the mute and deaf difficult. This problem creates an opportune place for the application of machine learning which has given rise to our emerging field. A large number of papers with high accuracy have already been published for English, French, and other languages. But the number of papers on its application for Bangla Sign language is few. Most of the researchers use SVM, ANN or KNN as classifiers. We chose CNN because it is excellent at high accuracy image classification. In this paper we use a large dataset consisting of 30 classes with 500 images each totalling to about 15000 images of bangla sign alphabets. Previous works were done only on 10 classes. We began work on those 10 bangla alphabets and later increased the number of classes to 30. We tested the accuracy’s of pre trained CNN models such as DenseNet201,VGG16, InceptionV3, Resnet50, MobileNetV2, InceptionResnet, EfficientnetB2 along with our custom CNN model and were able to achieve 97.97%, 96%, 96.22%, 56.44%, 90%, 94%, 4%,98.3 % train accuracy and 86.43%, 88%, 88.33%, 54.50%, 60%, 53%, 4.2%,87% validation accuracy respectively. Our custom CNN model has consistently given better training and validation accuracy than any pre-trained model with lesser layers which in turn require less computations making for a lighter and faster model while maintaining high accuracy.