Multi-modal emotion recognition for determining employee satisfaction
Abstract
Emotion Detection has been very popular in the field of research for a couple of
years. In the past, emotion recognition has been studied and applied in order to
detect the overall emotional state of a person using individual modalities such as
facial recognition from images. However, in order to ensure the authenticity of the
real time emotional state detected from the data that is received, it is required
to use multiple modes. In our research, we have classified emotional states into
six specific entities which are: Happiness, Sadness, Neutral, Disgust, Anger and
Surprise. The real time emotional state of the candidate is classified into one of
these entities according to the candidate’s response. We have used two important
modes to detect the real time emotion of the candidate, Emotion recognition from
facial expression as well as emotion recognition from sentimental analysis. For our
research, we have mainly used the Convolutional Neural Network(CNN). We have
trained and tested both the facial recognition and sentimental analysis datasets with
all the six entities. Therefore, results can be obtained from both the modes in order
to justify the candidate’s emotional state in real time. The two separate results from
the individual independent artificial neural networks are then fed into a machine
learning algorithm called Support Vector Machine (SVM) so that the final emotional
state can be achieved. Our goal is to apply this multimodal emotion detection
technique on employees in various offices and workplaces by asking them questions
regarding their work so that their genuine emotions can be obtained from their
answers. This is important because in this way, employee satisfaction in workplaces
can be recognized which is vital for mental health as well as productivity. In fact,
the mental health of the employees not only affects their individual well-being but
it affects the overall productivity and environment of the workplace. In order to
improve certain aspects of a workplace for better performance along with employee
satisfaction and productivity, determining the emotional condition of each employee
is vital