Cross-domain emotion recognition using SAM-based region extraction: a comparative study on FER and emotic datasets

Abstract

In the digital age, understanding facial expressions and recognizing emotions will be crucial for enhancing human-computer interactions. This research will help to develop a framework for better facial analysis and emotion recognition, integrating traditional computer vision with advanced deep learning models. To refine facial analysis and focus precisely on expressive regions, we will explore the application of advanced segmentation techniques, particularly the Segment Anything Model (SAM), to accurately delineate facial features before emotion classification. The detected facial regions, and potentially their segmented components, will be preprocessed with resizing, grayscale conversion, and normalization, ensuring consistency for emotion analysis. These preprocessed images will then be fed into a CNN model, which will be trained using the rich and contextually diverse EMOTIC dataset, subsequently mapped to predict one of seven fundamental emotions: Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral. Real-time emotion recognition will also be implemented using webcam video frames. Moreover, considering the significant role of facial expressions in communication, this research will contribute to mental health monitoring and improved human-computer interaction. By advancing the integration of OpenCV, advanced segmentation, and deep learning with a focus on comprehensive training data, we aim to create a robust and accurate system for facial analysis and emotion recognition. Instead, we want to create a model pipeline that is both objective and based in reality, capable of performing well in ordinary emotional settings. Rather than attempting to artificially balance the dataset, we accepted its inherent unevenness because emotions do not occur in equal amount in real life. We hoped to achieve more than just a high-scoring algorithm by allowing the model to learn from data that reflects how people actually express their feelings. We set out to create a model that understands emotions with the delicacy, depth, and diversity of human experience, detecting feelings not only in clean, controlled situations, but also in the chaotic, beautiful complexity of everyday life.