Abstract
Deepfake image detection has emerged as an important area of research due to its wide-ranging implications for various security systems. In particular, in the field of deep learning, the task of detecting fake images has traditionally been challenging due to its complicated and abstract nature, especially in the field of computer vision where accurate analysis and understanding of facial landmarks play a crucial role. This study introduces a rational-augmented convolutional neural network (RACNN) for deepfake image detection. The RACNN combines a convolutional neural network (CNN) with a reasoning generator, which generates binary masks to highlight the crucial regions that contribute to the CNN’s decision-making process. To improve the accuracy and efficiency of the reasoning generator, a reinforcement learning technique is used to train it to generate accurate and compact masks. Through extensive experiments conducted on a large dataset of deepfake images, the effectiveness of the RACNN method is demonstrated, achieving an impressive accuracy rate of 94.87% on an open-source dataset, namely FaceForensics++. The comparative analysis shows the superiority of the RACNN model over existing approaches, especially in terms of accuracy. This robustly demonstrates the effectiveness of the RACNN in accurately distinguishing between real and fake images. The AUC of 95.69% on the dataset serves as a strong indication of the effectiveness of our proposed method in accurately detecting fake facial images generated by various deepfake techniques. Our model proves to be a promising way to advance the field of deepfake image detection, providing potential improvements to the capabilities of such systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and material
Enquiries about data availability should be directed to the authors.
References
Afchar D, Nozick V, Yamagishi J et al (2018) MesoNet: a compact facial video forgery detection network. In: 2018 IEEE international workshop on information forensics and security (WIFS). IEEE, pp 1–7
Agarwal S, Varshney LR (2019) Limits of deepfake detection: a robust estimation viewpoint. arXiv preprint arXiv:1905.03493
Ahmed SRA, Sonuç E (2023) Deepfake detection using rationale-augmented convolutional neural network. Appl Nanosci 13(2):1485–1493
Al-Dhabi Y, Zhang S (2021) Deepfake video detection by combining convolutional neural network (CNN) and recurrent neural network (RNN). In: 2021 IEEE international conference on computer science. Artificial intelligence and electronic engineering (CSAIEE). IEEE, pp 236–241
Albelwi S, Mahmood A (2017) A framework for designing the architectures of deep convolutional neural networks. Entropy 19(6):242
Altuncu E, Franqueira VN, Li S (2022) Deepfake: definitions, performance metrics and standards, datasets and benchmarks, and a meta-review. arXiv preprint arXiv:2208.10913
Amerini I, Galteri L, Caldelli R et al (2019) Deepfake video detection through optical flow based CNN. In: Proceedings of the IEEE/CVF international conference on computer vision workshops
Aneja S, Nießner M (2020) Generalized zero and few-shot transfer for facial forgery detection. arXiv preprint arXiv:2006.11863
Awotunde JB, Jimoh RG, Imoize AL et al (2022) An enhanced deep learning-based deepfake video detection and classification system. Electronics 12(1):87
Chen J, Lu Y, Yu Q et al (2021) TransuNet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306
Chin CS, Si J, Clare AS et al (2017) Intelligent image recognition system for marine fouling using SoftMax transfer learning and deep convolutional neural networks. Complexity 2017
Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258
Dolhansky B, Howes R, Pflaum B et al (2019) The deepfake detection challenge (DFDC) preview dataset. arXiv preprint arXiv:1910.08854
Dong F, Zou X, Wang J et al (2023) Contrastive learning-based general deepfake detection with multi-scale RGB frequency clues. J King Saud Univ Comput Inf Sci 35(4):90–99
Guarnera L, Giudice O, Guarnera F et al (2022) The face deepfake detection challenge. J Imaging 8(10):263
Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6
Guo Z, Yang G, Chen J et al (2021) Fake face detection via adaptive manipulation traces extraction network. Comput Vis Image Underst 204:103170
Hsu CC, Hung TY, Lin CW et al (2008) Video forgery detection using correlation of noise residue. In: 2008 IEEE 10th workshop on multimedia signal processing. IEEE, pp 170–174
Huang J, Rathod V, Sun C et al (2017) Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7310–7311
Ilyas H, Javed A, Malik KM et al (2023) E-cap net: an efficient-capsule network for shallow and deepfakes forgery detection. Multimed Syst 29(4):2165–2180
Jameel WJ, Kadhem SM, Abbas AR (2022) Detecting deepfakes with deep learning and gabor filters. ARO Sci J Koya Univ 10(1):18–22
Jin X, He Z, Xu J et al (2022) Video splicing detection and localization based on multi-level deep feature fusion and reinforcement learning. Multimed Tools Appl 81(28):40993–41011
Kaur G, Sinha R, Tiwari PK et al (2022) Face mask recognition system using CNN model. Neurosci Inform 2(3):100035
Khan IR, Aisha S, Kumar D et al (2023) A systematic review on deepfake technology. Proc Data Anal Manag ICDAM 2022:669–685
Khormali A, Yuan JS (2022) Dfdt: an end-to-end deepfake detection framework using vision transformer. Appl Sci 12(6):2953
Kim Y, Chen H, Alghowinem S et al (2022) Joint engagement classification using video augmentation techniques for multi-person human–robot interaction. arXiv preprint arXiv:2212.14128
Lewis JK, Toubal IE, Chen H et al (2020) Deepfake video detection based on spatial, spectral, and temporal inconsistencies using multimodal deep learning. In: 2020 IEEE applied imagery pattern recognition workshop (AIPR). IEEE, pp 1–9
Li Y, Lyu S (2018) Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656
Li Y, Yang X, Sun P et al (2019) A large-scale challenging dataset for deepfake forensics. 35:36. arXiv:1909.12962
Li Y, Yang X, Sun P et al (2020) Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3207–3216
Lin D, Tondi B, Li B et al (2022) Exploiting temporal information to prevent the transferability of adversarial examples against deep fake detectors. In: 2022 IEEE international joint conference on biometrics (IJCB). IEEE, pp 1–8
Mao X, Li Q, Xie H et al (2017) Least squares generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2794–2802
Nguyen TT, Nguyen QVH, Nguyen DT et al (2022) Deep learning for deepfakes creation and detection: a survey. Comput Vis Image Underst 223:103525
Passos LA, Jodas D, da Costa KA et al (2022) A review of deep learning-based approaches for deepfake content detection. arXiv preprint arXiv:2202.06095
Rana MS, Nobi MN, Murali B et al (2022) Deepfake detection: a systematic literature review. IEEE Access 10:25494–25513
Rathgeb C, Tolosana R, Vera-Rodriguez R et al (2022) Handbook of digital face manipulation and detection: from deepfakes to morphing attacks. Springer Nature, Berlin
Rossler A, Cozzolino D, Verdoliva L et al (2019) Faceforensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1–11
Saikia P, Dholaria D, Yadav P et al (2022) A hybrid CNN-LSTM model for video deepfake detection by leveraging optical flow features. In: 2022 international joint conference on neural networks (IJCNN). IEEE, pp 1–7
Sun N, Tao J, Liu J et al (2022) 3d facial feature reconstruction and learning network for facial expression recognition in the wild. IEEE Trans Cognit Dev Syst 15(1):298–309
Suratkar S, Kazi F, Sakhalkar M et al (2020) Exposing deepfakes using convolutional neural networks and transfer learning approaches. In: 2020 IEEE 17th India council international conference (INDICON). IEEE, pp 1–8
Suratkar S, Bhiungade S, Pitale J et al (2022) Deep-fake video detection approaches using convolutional-recurrent neural networks. J Control Decis 1–17
Tak H, Jung JW, Patino J et al (2021) End-to-end spectro-temporal graph attention networks for speaker verification anti-spoofing and speech deepfake detection. arXiv preprint arXiv:2107.12710
Tiwari A, Dave R, Vanamala M (2023) Leveraging deep learning approaches for deepfake detection: a review. arXiv preprint arXiv:2304.01908
Yang C, Ding L, Chen Y et al (2021) Defending against GAN-based deepfake attacks via transformation-aware adversarial faces. In: 2021 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
Zhang Y, Zheng L, Thing VL (2017) Automated face swapping and its detection. In: 2017 IEEE 2nd international conference on signal and image processing (ICSIP). IEEE, pp 15–19
Zhao H, Zhou W, Chen D et al (2021) Multi-attentional deepfake detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2185–2194
Acknowledgements
The authors did not receive any specific grant from funding agencies in the public, commercial, or non-profit sectors.
Funding
No fund received for this study.
Author information
Authors and Affiliations
Contributions
SRAA: Conceptualization, Methodology, Software, Visualization, Writing—original draft. Emrullah Sonuç: Validation, Supervision, Writing—review and editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
No ethics approval is required.
Human/animals participants
The authors that there is no research involving human participants and/or animals in the contained of this paper.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ahmed, S.R., Sonuç, E. Evaluating the effectiveness of rationale-augmented convolutional neural networks for deepfake detection. Soft Comput (2023). https://doi.org/10.1007/s00500-023-09245-y
Accepted:
Published:
DOI: https://doi.org/10.1007/s00500-023-09245-y