Detection of deepfake videos using computer vision and deep learning

Abstract

DeepFakes are one of the most alarming concepts in this era of Metaverse and technological advancement. DeepFakes are artificially-generated manipulated photos or videos using Deep learning, Generated Adversarial Network (GAN), autoencoder-decoder pairing structure etc. There are several other Deepfaking tools such as; FaceSwap, DeepFace-Lab, DFaker, DeepFake-tensorflow etc. DeepFakes can become concerning if it is used for political purpose, committing fraud, spreading misinformation, pornography, defamation on social media etc. As a result, it is obvious that DeepFakes can be very distressing on the wrong hand if not detected properly. To address this issue, our research aims to develop effective methods for DeepFake video detection, focusing on deep learning approaches, and computer vision techniques. We deployed a dataset consisting of both real and fake videos, obtained from DeepFake Detection Challenge (DFDC) and FaceForensics++. To detect the fake videos, we followed the method of employing temporal feature and exploring visual artifacts within frames. Employing temporal feature uses LSTM and CNN whereas visual artifacts within frames mostly employs deep learning method to detect DeepFakes. We ensembled LSTM and CNN to detect DeepFakes successfully. ResNeXt101-32x8d have been used to extract features and a custom CNN model is added with LSTM for better accuracy for detecting DeepFake. Our ensemble model, which combines LSTM and CNN, successfully detects Deepfakes with an accuracy of 94.05%. Through further improvements and the implementation of learning rate schedulers, such as CosineAnnealingLR, CyclicLR, MultiStepLR, and ReduceLRonPlateau, we achieved even higher accuracy. Among these schedulers, MultiStepLR demonstrated the highest accuracy of 95.33%.