Abstract
Deepfakes are one of the most advanced technological frauds that can be seen in the outer world, and it has been classified as one of the significant adverse impacts of deep learning. Deepfakes are synthetic media created by superimposing a targeted person’s visual characteristics into a source video. This results in a video which contains content that the targeted person has never done. This kind of digital fraud can cause many social relevant problems like damaging the image and dignity of famous public figures, hate campaigns, blackmailing etc. Because of these reasons, it is high time to find some methods to detect these deep fakes even before they are published. For that, a deepfake detection method is proposed using a deep neural network. A combination of a temporal model-based and a deep model-based deepfake detection is used. For the temporal based model, a combination of ResNext and LSTM architectures and for the deep model based deepfake detection, a triplet model architecture is used. The datasets used for training this model are DFDC, Celeb-DF, and Faceforensics++, composed of different deepfake creation techniques. The extensive experiments show that the temporal model obtained the highest testing accuracy of 92.42% accuracy, at a frame rate of 100, and the triplet model obtained an accuracy of 91.88%. The final pipeline of these models obtain a testing accuracy of 94.31%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, Y., Chang, M.C., Lyu, S.: Exposing AI created fake videos by detecting eye blinking. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7. IEEE, December 2018
Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, pp. 8261–8265 (2019)
Li, L., et al.: Face X-ray for more general face forgery detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5001–5010 (2020)
Li, Y., Lyu, S.: Exposing deepfake videos by detecting face warping artifacts. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 46–52 (2019)
Afchar, D., Nozick, V., Yamagishi, J., Echizen, I.: MesoNet: a compact facial video forgery detection network. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7. IEEE, December 2018
Chintha, A., et al.: Recurrent convolutional structures for audio spoof and video deepfake detection. IEEE J. Sel. Top. Signal Process. 14(5), 1024–1037 (2020)
Guera, D., Delp, E.J.: Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE, November 2018
Sabir, E., Cheng, J., Jaiswal, A., AbdAlmageed, W., Masi, I., Natarajan, P.: Recurrent convolutional strategies for face manipulation detection in videos. In: Applications of Computer Vision and Pattern Recognition to Media Forensics at CVPR 2019 (2019)
Montserrat, D.M., et al.: Deepfakes detection with automatic face weighting. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2851–2859 (2020)
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nieaner, M.: FaceForensics++: learning to detect manipulated facial images. In: ICCV (2019)
Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-DF: a large-scale challenging dataset for DeepFake forensics. In: IEEE Conference on Computer Vision and Patten Recognition (CVPR) (2020)
Dolhansky, B., Howes, R., Pflaum, B., Baram, N., Ferrer, C.C.: The deepfake detection challenge (DFDC) preview dataset. arXiv preprint arXiv:1910.08854 (2019)
Singh, A., Saimbhi, A.S., Singh, N., Mittal, M.: DeepFake video detection: a time-distributed approach. SN Comput. Sci. 1(4), 1–8 (2020). https://doi.org/10.1007/s42979-020-00225-9
Lyu, S.: DeepFake detection. In: Sencar, H.T., Verdoliva, L., Memon, N. (eds.) Multimedia Forensics. Advances in Computer Vision and Pattern Recognition, Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-7621-5_12
Hao, H., et al.: Deepfake detection using multiple data modalities. In: Rathgeb, C., Tolosana, R., Vera-Rodriguez, R., Busch, C. (eds.) Handbook of Digital Face Manipulation and Detection. Advances in Computer Vision and Pattern Recognition, Springer, Cham (2022). https://doi.org/10.1007/978-3-030-87664-7_11
Jiang, L., Wu, W., Qian, C., Loy, C.C.: DeepFakes detection: the DeeperForensics dataset and challenge. In: Rathgeb, C., Tolosana, R., Vera-Rodriguez, R., Busch, C. (eds.) Handbook of Digital Face Manipulation and Detection. Advances in Computer Vision and Pattern Recognition, Springer, Cham (2022). https://doi.org/10.1007/978-3-030-87664-7_14
Suratkar, S., Sharma, P.: A simple and effective way to detect DeepFakes: using 2D and 3D CNN. In: Rao, U.P., Patel, S.J., Raj, P., Visconti, A. (eds.) Security, Privacy and Data Analytics. Lecture Notes in Electrical Engineering, vol. 848, pp. 227–238. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9089-1_19
Kaliyar, R.K., Goswami, A., Narang, P.: DeepFakE: improving fake news detection using tensor decomposition-based deep neural network. J. Supercomput. 77(2), 1015–1037 (2020). https://doi.org/10.1007/s11227-020-03294-y
Chen, J.I.Z., Smys, S.: Social multimedia security and suspicious activity detection in SDN using hybrid deep learning technique. J. Inf. Technol. 2(02), 108–115 (2020)
Kumar, T.S.: Construction of hybrid deep learning model for predicting children behavior based on their emotional reaction. J. Inf. Technol. 3(01), 29–43 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
John, J., Sherif, B.V. (2022). Multi-model DeepFake Detection Using Deep and Temporal Features. In: Chen, J.IZ., Tavares, J.M.R.S., Shi, F. (eds) Third International Conference on Image Processing and Capsule Networks. ICIPCN 2022. Lecture Notes in Networks and Systems, vol 514. Springer, Cham. https://doi.org/10.1007/978-3-031-12413-6_53
Download citation
DOI: https://doi.org/10.1007/978-3-031-12413-6_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12412-9
Online ISBN: 978-3-031-12413-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)