Leveraging sequential deep learning models for detecting multitude of human action categories

Abstract

In today’s world, where science and technology are constantly evolving day by day, people are drawn to tangible experiences and visual representations. There’s a growing effort to teach machines about human movements and postures to enable smart decision-making. This has led to increased interest in the field of human action recognition (HAR) among researchers globally. Our research focuses on implementing advanced technologies to address criminal activities, specifically emphasizing Human Activity Recognition (HAR). Moreover, our dataset includes 1275 videos, covering 20 different actions involving both violent and non-violent behaviors. In addition, we have developed a pipeline that utilizes YOLO-v8 to extract background, followed by models for accurate video classification. two models,conv-lstm and lrcn, were incorporated into our deep learning pipeline. Through our observations, we found that the LRCN model outperformed the other model, achieving an accuracy of 62% and an F1 score of 60% for the 20 classes, for 17 classes an accuracy of 63% and an F1 score of 66%. for binary classification LRCN got accuracy of 88% and an F1 score of 87%Our research focusses the potential of advanced technologies to significantly improve Human Activity Recognition (HAR) in addressing various aspects of criminal activities in real-time scenario. This marks a substantial step forward in intelligent decision-making and public safety.