Evaluation of deepfake detection using YOLO with local binary pattern histogram

Štěpán Hubálovský; Pavel Trojovský; Nebojsa Bacanin; Venkatachalam K

doi:10.7717/peerj-cs.1086

Evaluation of deepfake detection using YOLO with local binary pattern histogram

PeerJ Comput Sci. 2022 Sep 13:8:e1086. doi: 10.7717/peerj-cs.1086. eCollection 2022.

Authors

Štěpán Hubálovský¹, Pavel Trojovský², Nebojsa Bacanin³, Venkatachalam K¹

Affiliations

¹ Department of Applied Cybernetics, University of Hradec Králové, Hradec Králové, Czech Republic.
² Department of Mathematics, Faculty of Science, University of Hradec Králové, Hradec Kralove, Czech Republic.
³ Department of Computing, Singidunum University, Belgrade, Serbia.

Abstract

Recently, deepfake technology has become a popularly used technique for swapping faces in images or videos that create forged data to mislead society. Detecting the originality of the video is a critical process due to the negative pattern of the image. In the detection of forged images or videos, various image processing techniques were implemented. Existing methods are ineffective in detecting new threats or false images. This article has proposed You Only Look Once-Local Binary Pattern Histogram (YOLO-LBPH) to detect fake videos. YOLO is used to detect the face in an image or a frame of a video. The spatial features are extracted from the face image using a EfficientNet-B5 method. Spatial feature extractions are fed as input in the Local Binary Pattern Histogram to extract temporal features. The proposed YOLO-LBPH is implemented using the large scale deepfake forensics (DF) dataset known as CelebDF-FaceForensics++(c23), which is a combination of FaceForensics++(c23) and Celeb-DF. As a result, the precision score is 86.88% in the CelebDF-FaceForensics++(c23) dataset, 88.9% in the DFFD dataset, 91.35% in the CASIA-WebFace data. Similarly, the recall is 92.45% in the Celeb-DF-Face Forensics ++(c23) dataset, 93.76% in the DFFD dataset, and 94.35% in the CASIA-Web Face dataset.

Keywords: Celeb DF-Face Forensics++; Celeb-DF; Deepfake; FaceForencies++; LBPH; YOLO.

Grants and funding

This work was supported by the Project of Excellence Faculty of Science, University of Hradec Kralove 2214/2022-2023. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.