Advanced personnel security system using behaviour and activity analysis

Abstract

Safety is of utmost importance in workplaces, where human error or malicious intent can jeopardize employee safety. This thesis explores the development of an advanced personal security system, leveraging machine learning to enhance security protocols. The proposed system integrates body language detection, keystroke analysis, social media monitoring, and weapon detection within the facility to establish a robust security framework. Body language detection, powered by a Temporal Segment Network with a ResNet backbone and context fusion, precisely recognizes personnel behavior and emotional states by predicting 26 categorical emotions and 3 continuous dimensions, an GloVe-centered embedding loss enhances semantic consistency to help prevent risky actions. Social media monitoring, facilitated by web scraping with Auto-archiver, provides insights into potential security risks by analyzing the online behavior and communications of individuals with access to sensitive areas. This multi-faceted approach aims to proactively identify and mitigate threats, safeguarding the integrity of workplaces. The system aligns with IAEA standards, ensuring applicability in nuclear power plants and other high risk workplaces. By integrating machine learning with traditional security measures, this research sets new standards for security protocols in critical infrastructure, addressing evolving threats in an increasingly digital environment. The proposed personnel surveillance AI focuses on four core components: analyzing employee body language for warning signs in their emotions, scanning CCTV feed to detect weapons in the hands of employees, surveilling employee keystroke patterns to detect abnormalities in their typing and conducting sentiment analysis of social media activities. This innovative approach enhances the efficiency of security surveillance, offering a proactive solution to ensure the safety and security of workplaces.