- Generalizable Face Forgery Detection with Self-Blended Consistency Learning
Awards
Author(s):
Category:
Institution:
Region:
Winner Category:
Year:
Abstract:
- Recent work on generalizable face forgery detection introduced two novel concepts: self-blended images and consistency representation learning. Self-blended images work on the principle of creating ‘hard’ forgeries by using the same face to create a forgery instead of using a source and target face, while consistency representation learning encourages the model to produce consistent representations for differently augmented images. Both induce the model to learn more generalized and consistent representations and clues rather than overfitting to individual forgery techniques. In this study, we combine the two concepts together into a single framework – self-blended consistency learning (SBCL). After extensive experiments to find optimal parameters and settings, we compare SBCL with existing work on the FF++, CDF, DFD, DFDC, DFDCP, and FFIW datasets in the commonly-used cross-dataset and cross-manipulation evaluation scenarios. We demonstrate that SBCL is able to achieve better generalizability to unseen datasets containing unseen forgery techniques and domains. Specifically, on cross-dataset evaluation of CDF, DFDCP, and FFIW, SBCL improves upon the current state-of-the-art by 3.44%, 0.88% and 2.97% points respectively. We also create a web application demo to showcase the framework, which is able to perform image and video inference, and visualize the self-blending consistency learning process.