Source-linked AI summary
Adversarial Threats to DeepFake Detection: A Practical Perspective
Paarth Neekhara, Brian Dolhansky, Joanna Bitton, Cristian Canton Ferrer
TL;DR
DeepFake detectors face the practical question of whether adversarial attacks can evade systems with different preprocessing and face-detection methods. The paper studies transferable black-box attacks and universal perturbations, reporting strong transfer of universal attacks across detectors.
Problem
DeepFake detectors can be vulnerable to adversarial examples, while differences in preprocessing and face detection make attacks harder to transfer across models.
Source: §1 · p. 1Method
The paper studies black-box transfer attacks, develops perturbations robust to detector differences, and trains universal perturbations for target labels.
Source: §1 · p. 2, §2.3 · p. 3, §3.5 · p. 5Results
Universal perturbations achieved 100% attack success on the targeted test model and at least 75% across unseen detectors when attacking EN-B7 NLab.
Source: §4.4 · p. 7Takeaways & Limitations
A single universal perturbation can be added across video frames, and the reported cross-model transfer poses a practical threat to detectors in production.
Source: §4.4 · p. 7, §5 · p. 8The white-box attack setting assumes complete access to the victim detector, including its face-extraction pipeline and classifier architecture and parameters.
Source: §3.3 · p. 4
Abstract
from arXiv · showhide
Facially manipulated images and videos or DeepFakes can be used maliciously to fuel misinformation or defame individuals. Therefore, detecting DeepFakes is crucial to increase the credibility of social media platforms and other media sharing web sites. State-of-the art DeepFake detection techniques rely on neural network based classification models which are known to be vulnerable to adversarial examples. In this work, we study the vulnerabilities of state-of-the-art DeepFake detection methods from a practical stand point. We perform adversarial attacks on DeepFake detectors in a black box setting where the adversary does not have complete knowledge of the classification models. We study the extent to which adversarial perturbations transfer across different models and propose techniques to improve the transferability of adversarial examples. We also create more accessible attacks using Universal Adversarial Perturbations which pose a very feasible attack scenario since they can be easily shared amongst attackers. We perform our evaluations on the winning entries of the DeepFake Detection Challenge (DFDC) and demonstrate that they can be easily bypassed in a practical attack scenario by designing transferable and accessible adversarial attacks.
1. Introduction
DeepFake detectors face a practical threat from adversarial examples, especially when attacks can transfer across models with different preprocessing and face-detection pipelines. The paper studies black-box transfer attacks and universal perturbations against these systems.
1. Introduction
- DeepFake evasion can cause outsized harm, while the rarity of DeepFakes already makes their detection difficult.The introduction also describes DeepFakes being used to fuel misinformation or defame individuals.Source: §1 · p. 1
- Practical black-box attacks matter because earlier approaches required model access or many queries and per-frame prediction scores.The paper notes that query limits and withholding raw scores can thwart such attacks.Source: §1 · p. 1
- Different preprocessing steps and face-detection methods make adversarial perturbations harder to transfer across DeepFake detectors.The paper identifies these differences as challenges to cross-model transfer.Source: §1 · p. 1
- The study targets practical attacks on state-of-the-art DeepFake detectors, developing transferable perturbations and universal attacks that can be applied across videos.The universal perturbations are designed for use across all frames of all videos.Source: §1 · p. 2
2. Background
The background describes the DFDC dataset and the frame-based CNN pipelines used by leading DeepFake detectors. It then motivates studying saliency and adversarial transfer, which prior attacks often addressed only with model access or extensive queries.
2.1. DeepFakes and DFDC
- The DFDC dataset contains over one million training clips made with varied face-swap techniques, and its competition winners released their architectures and training strategies.The dataset includes eight manipulation techniques, including GAN-based methods.Source: §2.1 · p. 2
2.2. DeepFake Detection
- Leading DeepFake detectors classify frames with CNNs, often after tracking and cropping faces, then aggregate selected frame labels into a video prediction.The best-performing DFDC methods operated frame by frame rather than using temporal sequence models.Source: §2.2 · p. 2
- Guided backpropagation saliency maps helped compare detector attention, and initial observations found shared focus on face edges, eyes, lips, and teeth.The paper uses these shared input regions to motivate testing whether adversarial changes can fool multiple detectors.Source: §2.2 · pp. 2–3
2.3. Prior Work on Fooling DeepFake Detectors
- Prior DeepFake attacks either assumed full access to the victim model or required repeated queries and access to per-frame classifier scores.The paper notes that query limits and withheld scores can thwart the latter black-box approach.Source: §2.3 · p. 3
- Transferable attacks are practically relevant, but detector differences in preprocessing and face detection hamper cross-model transfer.The paper proposes techniques to address these obstacles and universal perturbations that can be shared without adversarial-machine-learning expertise.Source: §2.3 · p. 3
3. Methodology
The methodology frames attacks as targeted Real/Fake label flips under per-frame distortion constraints. It evaluates white-box attacks, transformation-robust transfer attacks, and universal perturbations against detectors with differing pipelines.
3.1. Threat Model
- The threat model seeks to flip each video’s predicted label while keeping per-frame L∞ distortion bounded; target labels swap Real and Fake.Source: §3.1 · pp. 3–4
3.2. Victim Models
- Victim systems classify each frame after face tracking, cropping, and preprocessing, then aggregate scores from all or some frames into a video label.Source: §3.2 · p. 4
3.3. White-box attacks
- With complete access to the detector and its face-extraction pipeline, iterative gradient-sign attacks fool all studied methods, but transfer across methods is limited.The attack is optimized per frame and stops at success or a maximum iteration count.Source: §3.3 · p. 4
3.4. Black-box: Transfer attacks
- Transformation-robust attacks optimize expected target-class success across sampled inputs, and the authors report that this robustness significantly improves transferability.The transform distribution includes translations, down/up-sampling, and Gaussian noise.Source: §3.4 · pp. 4–5
3.5. Universal attacks
- A single targeted perturbation is trained on videos labeled opposite the target, with transformation robustness and L2 regularization to limit distortion.It is added across all frames; the authors also find that targeting certain detectors can yield input- and model-agnostic perturbations.Source: §3.5 · p. 5
4. Experiments
The experiments evaluate attacks against the top three DFDC challenge-winning detectors using public DFDC videos and success rate and L∞ distortion metrics. White-box attacks achieve perfect same-model success, while transfer and universal attacks extend evasion to unseen detectors, with universal perturbations offering a shared attack across videos.
4.1. Experimental Setup
- Evaluations use the first 100 Fake and 100 Real videos from the public DFDC validation set, with attack success rate and mean L∞ distortion as metrics.The evaluation set comprises 30,300 frames; success rate counts flips of originally correct predictions.Source: §4.1 · p. 6
4.2. White-box attacks
- White-box attacks achieve 100% success on the same test model as the victim model.The attack uses an L∞ bound of 16/255; the resulting perturbations are described as imperceptible.Source: §4.2 · p. 6, Table 3 · p. 7
4.3. Transfer attacks
- Transfer attacks significantly improve cross-detector success over the simple white-box attack; attacking EN-B7 NLab achieves at least 72% success across other detectors.Perturbations transfer most readily between models with the same architecture.Source: §4.3 · pp. 6–7, Table 4 · p. 7
4.4. Universal attacks
- At ϵ = 40/255, universal perturbations trained on EN-B7 NLab achieve at least 75% success across all unseen detectors.The same single perturbation achieves 100% success on its victim model across frames and videos of the same label.Source: Table 5 · p. 7, §4.4 · p. 7
- For universal attacks, perturbations below ϵ = 0.1 are fairly imperceptible while still achieving high success rates on various test models.This magnitude experiment uses EN-B7 NLab as the victim model.Source: §4.4 · p. 8
5. Conclusion
The work presents transferable black-box attacks and universal adversarial perturbations as practical threats to DeepFake detection. It reports bypassing state-of-the-art detection networks and emphasizes the need for detectors robust to adversaries.
- Universal adversarial perturbations make attacks accessible: one perturbation can be added to every frame of any video to bypass multiple detection systems.Source: §5 · p. 8
- Figure 6 covers attack success rates on unseen detectors for universal perturbations trained on EN-B7 NLab at different L∞ norms.Source: Figure 6 · p. 8
- Figure 7 visualizes images perturbed at different magnitudes of universal perturbations trained on EN-B7 NLab.Source: Figure 7 · p. 8
- The proposed attacks bypass recently proposed state-of-the-art DeepFake detection networks, motivating stronger detectors robust to adversaries.Source: §5 · p. 8