Optimizing American sign language recognition with binarized neural networks: a comparative study with traditional models

Abstract

Sign language is essential for effective communication among individuals with hearing or speech impairments. Automated recognition systems for sign language are crucial for facilitating learning and translation across different sign language variants. However, existing systems often struggle with high computational demands and large memory footprints, limiting their applicability in real-time and resourceconstrained environments. This research aims to develop an optimized pipeline for American Sign Language (ASL) recognition, focusing on the comparison between Binarized Neural Networks (BNNs) and traditional full-precision neural networks. Using Larq, an open-source Python library for training binarized models, we exploit BNNs’ advantages of reduced memory and computational needs, making them ideal for embedded systems and edge devices. The study utilizes a dataset of ASL alphabet images, encompassing a variety of hand configurations and movements. Data augmentation techniques are applied to address challenges like data imbalance and occlusions. Both binarized and traditional models are trained and optimized, and their performance is evaluated using metrics such as accuracy, precision, recall, F1-score, memory footprints and inference times. The results indicate that BNNs offer competitive performance with significantly lower computational requirements, paving the way for more efficient and accessible ASL recognition systems and demonstrating the potential of binarized models in this domain.