Patch-based deepfake localization: unveiling manipulated regions in images through visual artifact analysis and deep learning

Abstract

Artificial Intelligence (AI) generative technologies such as DALL·E 3, DALL·E 4, Stable Diffusion, MidJourney, and Generative Adversarial Networks are developing quickly, making it more difficult to distinguish between real, synthetic, and AI-generated (AIG) images. As these technologies progress, traditional detection methods and datasets have become outdated, leading to poor detection accuracy and inference time, particularly for video content. Our research introduces an efficient pipeline for real-time video analysis (RTVAS) to overcome these issues in an AI-generated content detection (AIG-CD) system. We created our own dataset with the help of newly developed generative models for robust training. Our multi-stage processing pipeline includes preloading the detection model (SAAT-ResNet50-3BD) at startup, real-time frame capture, adaptive preprocessing methods to speed up inference time, and face detection to focus on relevant regions. Our proposed model uses pre trained weight and includes enhanced feature extraction, attention mechanism to find any artifacts and irregularities with proper regularization techniques to prevent overfitting that distinguishes between AI-generated (AIG), synthetic media (SM) photos altered by real people. The classification decision is obtained by combining each frame’s frequency and statistical average forecasts. As a scalable and effective detection system, our proposed model outperforms Xception (93.6%), EfficientNetV2 (96.5%), MobileNetV3 (94.3%), and VGG16 (65.9%) models in terms of performance matrices. With a detection accuracy of 98.5%, our proposed model offers a superior balance between detection accuracy and compute efficiency, making it highly suitable for practical applications in media forensics, content regulation, and deepfake detection in video frames (VDF).