Source-linked AI summary
Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization
Shichao Dong, Jin Wang, Renhe Ji, Jiajun Liang, Haoqiang Fan, Zheng Ge
TL;DR
Binary deepfake detectors generalize poorly to unseen manipulation methods, and the paper attributes this stumbling block to unexpectedly learned identity representations. It proposes an ID-unaware model that focuses on local artifact areas; extensive experiments report generalized artifact features and state-of-the-art performance.
Problem
Binary deepfake detectors can perform well on seen attacks yet show significant performance drops on cross-dataset evaluation, while the reasons for this failure lack thorough analysis.
Method
The ID-unaware Deepfake Detection Model uses an Artifact Detection Module to focus on local artifact areas and reduce the influence of identity representations.
Results
Extensive experiments show that the model learns generalized artifact features and outperforms state-of-the-art methods.
Takeaways & Limitations
The study provides a new perspective for understanding deepfake-detector generalization by reducing the influence of Implicit Identity Leakage.
Takeaways & Limitations
Hand-crafted artifact detectors can fail when unseen forgeries lack artifacts introduced during training, and identity capture is considered without explicit supervision.
Abstract
from arXiv · showhide
In this paper, we analyse the generalization ability of binary classifiers for the task of deepfake detection. We find that the stumbling block to their generalization is caused by the unexpected learned identity representation on images. Termed as the Implicit Identity Leakage, this phenomenon has been qualitatively and quantitatively verified among various DNNs. Furthermore, based on such understanding, we propose a simple yet effective method named the ID-unaware Deepfake Detection Model to reduce the influence of this phenomenon. Extensive experimental results demonstrate that our method outperforms the state-of-the-art in both in-dataset and cross-dataset evaluation. The code is available at https://github.com/megvii-research/CADDM.
1. Introduction
The paper identifies Implicit Identity Leakage as a cause of poor cross-dataset generalization in binary deepfake detectors and proposes an ID-unaware model that focuses on local artifacts. Experiments report generalized artifact learning and state-of-the-art performance.
- Motivation: Binary deepfake detectors often perform well on seen attacks but suffer significant performance drops on media generated by newly proposed methods.This motivates a closer analysis of why their cross-dataset generalization remains weak.
- Implicit Identity Leakage: Fake-image synthesis can lose identity information, creating an identity gap between genuine and fake images that classifiers may encode as a decision boundary.This bias can cause false judgments when models encounter unseen face-swap methods.
- Proposed Method: The ID-unaware Deepfake Detection Model uses an anchor-based Artifact Detection Module to localize multi-scale artifact areas and classify them with binary labels.Focusing on local artifact and genuine areas is intended to reduce reliance on global identity information.
- Results: Extensive experiments show that the model accurately predicts artifact locations, learns generalized artifact features, and outperforms state-of-the-art methods.The experiments also demonstrate the effectiveness of the proposed approach.
- Implicit Identity Leakage: Implicit Identity Leakage is the unintended capture of identity information by binary classifiers supervised only with binary labels.The paper verifies this phenomenon qualitatively and quantitatively across classifiers with various backbones.
2. Related Work
Related work includes binary-classification detectors and methods designed to improve generalization by modeling artifacts or other signals. Hand-crafted detectors are limited when unseen forgeries lack the artifacts present during training.
- Generalization Methods: Researchers have pursued generalization through varied loss functions, richer features, temporal continuity, and hand-crafted artifact modeling.Examples include eye blinking and non-3D projection generation signals.
- Binary Classification: Many deepfake detectors treat detection as binary classification, using a backbone encoder and classifier to determine whether an image was manipulated.Other approaches analyze frequency-domain information alongside RGB or high-level semantic features.
- Limitations: Hand-crafted detectors often fail when new forgeries do not contain the specific artifacts introduced in the training phase.Their effectiveness therefore depends on the persistence of those artifacts across manipulation methods.
3. Implicit Identity Leakage
The paper identifies Implicit Identity Leakage: binary deepfake detectors accidentally capture image identity without identity supervision. This identity representation helps in-dataset performance but misguides cross-dataset detection.
- 3. Implicit Identity Leakage: Implicit Identity Leakage is the accidental capture of identity representation by binary classifiers during training without explicit identity supervision.The phenomenon is defined as identity information captured from the deepfake dataset by binary classifiers.
- 3.1. Verifying the Existence of ID Representation: Frozen-feature linear classification across multiple backbones verifies that binary classifiers retain substantial identity information, especially with strong backbones such as Efficient-b3.The experiment evaluates ResNet-18/34/50, Xception, and Efficient-b3 features on FF++, Celeb-DF, and LFW.
- 3.2. Quantifying the Influence of ID Representation: The study quantifies identity influence with a multivariate interaction metric that captures joint contributions among image regions.The input is sampled into 16 × 16 grids, and the resulting image-level identity score is evaluated with frame-level AUC.
- 3.2. Quantifying the Influence of ID Representation: Identity representation boosts FF++ in-dataset evaluation but hinders Celeb-DF cross-dataset evaluation.Table 1 describes this opposite effect across the two evaluation settings.
- 3.2. Quantifying the Influence of ID Representation: All tested classifiers produced AUC > 0.5 in-dataset and AUC < 0.5 cross-dataset for identity-based influence, confirming enhancement versus misguidance.The classifiers used ResNet-18/34/50, Xception, and Efficient-b3 backbones and were trained on FF++.
4. ID-unaware Deepfake Detection Model
The ID-unaware Deepfake Detection Model reduces identity leakage by directing detection toward local artifact areas. It uses an Artifact Detection Module and Multi-scale Facial Swap to support artifact localization and training.
- 4. ID-unaware Deepfake Detection Model: The ID-unaware model reduces Implicit Identity Leakage by focusing on local artifact areas instead of global identity information.Its Artifact Detection Module is designed to guide attention toward local representations that indicate face forgeries.
- 4.1. Artifact Detection Module: The Artifact Detection Module detects artifact positions using multi-scale anchors on backbone features.Four extra feature layers have scales 7×7, 5×5, 3×3, and 1×1; detection is applied after the first three layers during training.
- 4.2. Multi-scale Facial Swap: Multi-scale Facial Swap creates fake images with ground-truth artifact positions when public datasets lack fine local artifact annotations.It uses multi-scale sliding windows and different blending methods, while enriching artifact features in training.
- 4.2. Multi-scale Facial Swap: MFS selects artifact regions with a sliding window guided by summed structural dissimilarity between fake and source images.Larger DSSIM values indicate areas more likely to contain artifacts; the selected region supplies the artifact-position annotation.
- 4.2. Multi-scale Facial Swap: MFS generates new fake images by blending the fake and source images through a mask, using methods such as Poisson or alpha blending.For alpha blending, the generated image is IF ∗ M + IS ∗ (1 − M), with the artifact position given by the selected window.
- 4.3. Loss Function: The training objective combines global classification and artifact-detection losses, with β controlling their trade-off.Detection loss includes anchor confidence and localization terms, while classification measures fake-versus-genuine prediction accuracy.
5. Experiment
Experiments evaluate the model’s identity leakage, component contributions, robustness, and generalization across datasets and manipulation methods. The proposed approach reduces identity reliance while improving cross-dataset and in-dataset performance.
- Experimental setting: The study trains on FF++ and evaluates on FF++, Celeb-DF, and DFDC-V2, with additional cross-method and robustness experiments.Models use EfficientNet-b3 for the identity-leakage comparison, and video-level AUC is reported during inference.
- Implicit Identity Leakage: Binary-classifier features separated identities, and this identity information hindered performance on Celeb-DF.The comparison uses t-SNE feature visualization and evaluates whether identity information is present in learned representations.
- Implicit Identity Leakage: Our model produced overlapping identity features and detected fake images with less influence from identity information by focusing on artifact areas.Linear identity classification was easier and more accurate on binary-classifier features than on the proposed model’s features.
- Component analysis: 64.05% and 48.73% AUC were achieved by the baseline on Celeb-DF and DFDC-V2, respectively, while augmentations improved these results by 16.02% and 13.73%.The baseline performed best in-dataset, but its cross-dataset performance was substantially lower.
- Component analysis: Beyond data augmentation, MFS and ADM further improved cross-dataset AUC by 11.08% on Celeb-DF and 9.03% on DFDC-V2.ADM guides learning toward local artifact representations, while MFS aligns fake-image identity information with source images.
- Generalization and applicability: The method improved both in-dataset and cross-dataset performance, achieved 97.4% average video-level AUC, and improved cross-method generalization over binary classifiers.It also showed potential for plug-and-play use with other backbones and improved cross-dataset results when combined with SBI.
6. Conclusion
The paper identifies Implicit Identity Leakage as a cause of reduced generalization on unseen datasets and proposes an ID-unaware model to alleviate it. Experiments show improved generalized artifact learning and state-of-the-art performance.
- Experimental verification identifies Implicit Identity Leakage as deepfake detectors' sensitivity to data identity information, which reduces generalization on unseen datasets.
- The proposed ID-unaware Deepfake Detection Model is designed to alleviate Implicit Identity Leakage.
- Reducing Implicit Identity Leakage enables the model to learn generalized artifact features and outperform state-of-the-art methods.
- The research provides a new perspective for understanding deepfake detection model generalization and informs future field development.