Speech emotion recognition: clustering and deep learning approach to detect conflicting emotions through vocal expressions

Abstract

Deep models have improved speech emotion recognition, yet overlapping emotions and speaker variability still limit practical deployment. The guiding hypothesis of this paper is that a useful solution must clearly tackle overlapping and blended emotions, along with enhancing classification strength. These issues are addressed with a multi-branch ensemble approach and enhanced with a fuzzy post-processing phase aimed at revealing secondary emotion memberships and emphasizing unclear, overlapping statements; speaker-adversarial training was added as a supportive strategy to minimize confounding factors. evaluation on augmented benchmark corpora demonstrates competitive accuracy and improved handling of label overlap. This dual emphasis on accuracy and interpretability provides a principled way to surface uncertainty in emotional input and aims to provide a foundation for SER systems to be more robust for real-world use.