Privacy preserving federated learning approach for speech emotion recognition

Abstract

Emotions are a critical factor in intrapersonal communication and significantly influence how we convey our intentions and feelings. Recognizing emotion through speech not only enhances our understanding of interpersonal dynamics but also holds immense potential across diverse sectors such as healthcare, human-machine interaction, automated customer service and more. However, the majority of the existing speech recognition systems are highly centralized, raising concerns over potential data leakage. To address this issue, we have introduced a privacy preserving system to recognize emotion from audio data using federated learning. Our approach leveraged the distributed model training and aggregation strategy, ensuring data privacy while eliminating the need for data sharing to a centralized system. Moreover, we explored the potential of CNN and LSTM models, both as a distributed and centralized formats, with MFCC as features in the federated learning setting. Experimental evaluation of our approach on the IEMOCAP and CREMA-D datasets achieved a maximum accuracy of 68.65% and 68.82%, surpassing the existing federated learning techniques and rivaling centralized benchmarks.