Source-linked AI summary
Explainable Deepfake Detection with Feature-robust Augmentation and Evidence-grounded Explanation Optimization
Zhu Xu, Jiaqi Tang, Pokai Chen, Yuxin Peng, Yang Liu
TL;DR
Explainable deepfake detectors must remain reliable under image-quality degradation while grounding explanations in genuine manipulation evidence. The paper combines feature-robust augmentation and evidence-grounded preference optimization, then achieves first place in the ACM Multimedia 2026 challenge.
Problem
Existing explainable deepfake systems are vulnerable to image-quality degradation and may omit manipulation evidence or include irrelevant information.
Method
The framework combines degradation-aware augmentation, supervised contrastive learning, mean-teacher stabilization, and DPO trained on evidence-omitted or irrelevant-information rejected explanations.
Results
The method ranks first among 138 participating teams in the ACM Multimedia 2026 Explainable Deepfake Detection Challenge and achieves the highest reported explanation metrics.
Takeaways & Limitations
Evidence-grounded optimization improves explanation faithfulness by prioritizing genuine manipulation traces, while the framework supports both detailed forensic analysis and concise fact-checking.
Abstract
from arXiv · showhide
Explainable deepfake detection extends binary classification by requiring models to not only predict authenticity but also provide interpretable justifications. This expanded scope is critical in practice, where users like forensic analysts need insight into the rationale behind the detection. Despite advancements, current approaches suffer from two critical deficiencies: (1)vulnerability to image quality degradation: detection accuracy plummets on low-quality samples, while naive augmentation strategies may induce feature drift and impair performance as diversity expands. (2) factually flawed explanations: explanation models may omit manipulation evidence or hallucinate irrelevant details, undermining interpretability. To address it, we propose a framework with two innovations. For robust deepfake detection, we introduce Feature-robust Augmentation, which comprises diversified degradation-aware augmentation strategies, and a supervised contrastive learning pattern paired with a mean-teacher architecture that stabilizes features against augmentations through consistency constraints. For explanation, we devise an evidence-grounded preference optimization process that guides model to prioritize genuine manipulation traces by learning from chosen-rejected explanation pairs, where rejected samples are constructed via evidence omission or irrelevant information injection. The proposed approach wins the first place in ACM Multimedia 2026 Explainable Deepfake Detection Challenge.The code is available at https://github.com/oceanflowlab/EDD.git.
1 Introduction
Explainable deepfake detection must combine accurate authenticity decisions with faithful visual evidence, yet existing systems remain fragile under quality degradation and prone to incomplete or irrelevant explanations. The proposed framework addresses both vulnerabilities through feature-robust detection, evidence-grounded optimization, and concise explanation generation, achieving top challenge performance.
- Explainable detection requires both accurate predictions and visual evidence that forensic users can understand.
- Image quality degradation exposes detection brittleness, while increasingly diverse augmentations can blur classification boundaries through feature drift.
- Evidence-grounded preference optimization teaches explanation models to prioritize genuine manipulation traces rather than omit evidence or inject irrelevant information.
- The framework also provides a concise explanation model optimized for semantic fidelity and brevity-sensitive use cases.
- The approach ranks first in the ACM Multimedia 2026 Explainable Deepfake Detection Challenge and reports state-of-the-art overall performance.
2 Related Work
Prior work addresses degradation robustness and explanation generation, but naive augmentation can cause feature drift and VLM post-training often prioritizes reasoning format over evidence fidelity. This work instead uses mean-teacher consistency and evidence-grounded preference optimization.
- Naive degradation augmentation may cause feature drift and blur classification boundaries as augmentation diversity expands.
- Existing VLM post-training often emphasizes multi-step reasoning or response length while neglecting evidence accuracy and completeness.
- Evidence-grounded preference optimization constructs chosen-rejected explanation pairs to favor genuine manipulation evidence over superficial reasoning patterns.
3.1 Overview
The framework unifies robust binary deepfake detection with faithful explanation generation, targeting quality variation in images and evidence quality in rationales.
- A visual backbone and classifier perform binary authenticity prediction, while a second component generates explanations from the image and deepfake label.
- Degradation-aware augmentation, supervised contrastive learning, and mean-teacher consistency regularization support robustness against quality variation and feature drift.
3.2 Robust Deepfake Detection via Feature-robust Augmentation
Feature-robust augmentation combines degradation diversity with authenticity-aware feature clustering and mean-teacher stabilization. Consistency constraints align degraded views with a stable clean-view reference, preserving augmentation benefits while limiting feature drift.
- Detector architecture: DINOv3-7B aggregates CLS, REG, and average patch tokens before an MLP classification head supervised by binary cross-entropy.
- Degradation-aware augmentation: The augmentation pipeline applies diverse distortions, distractors, random probabilities, and random ordering to expose the detector to varied quality conditions.
- Supervised contrastive learning: Supervised contrastive learning pulls same-authenticity features together across degradation conditions and pushes different classes apart.
- Supervised contrastive learning: Class-level contrastive learning alone leaves intra-instance feature drift possible because different views may scatter within the same class cluster.
- Mean-teacher stabilization: Mean-teacher consistency aligns student features from degraded views with teacher features from clean views, using an EMA teacher as a stable feature anchor.
3.3 Evidence-Grounded Explanation Optimization
The framework uses evidence-grounded preference optimization to make explanations prioritize genuine manipulation evidence over omissions and irrelevant details, while also supporting concise outputs for practical scenarios.
- Evidence-grounded preference optimization: Evidence-grounded preference optimization addresses evidence omission and irrelevant information intrusion in generated deepfake explanations.The method explicitly prioritizes genuine manipulation evidence rather than superficial reasoning patterns.
- Preference dataset construction: The preference dataset pairs accurate chosen explanations with rejected explanations that omit critical manipulation traces or contain irrelevant information.Rejected explanations are constructed from ground-truth explanations to represent specific factual errors.
- Preference optimization: Direct Preference Optimization fine-tunes the SFT model to favor chosen explanations over rejected ones.The policy is optimized relative to an SFT reference model, with the objective penalizing evidence omission and hallucinated content.
- Concise explanation model: A concise explanation model uses GRPO with semantic-fidelity and conciseness rewards to produce compact statements of critical manipulation cues.The model targets social-media fact-checking and real-time alert scenarios where brief outputs are preferred.
4 Dataset Overview.
XPlainVerse is a large explainable deepfake detection dataset containing authentic and manipulated images paired with reference explanations, with designated challenge training, validation, and test subsets.
- Dataset composition: XPlainVerse contains 1 million authentic and manipulated images paired with reference explanations for evaluating generated justifications.The challenge uses subsets of 450k training, 110k validation, and 200k test samples.
5 Experiments
Experiments evaluate detection robustness, explanation faithfulness, challenge performance, and component ablations. The framework maintains predictions under degradation, produces evidence-grounded explanations, and ranks first overall among 138 participating teams.
- Experimental setup: Detection is evaluated with accuracy and macro F1, while complex explanations use BERTScore, Entity F1, and Claim F1; simple explanations use BERTScore and SLE.The overall score is a weighted sum of these metrics.
- Challenge results: 138 participating teams: the method ranks first overall and achieves the highest score for all explanation metrics.The authors attribute complex-explanation performance to evidence-grounded learning and simple-explanation performance to rewards balancing semantic fidelity and conciseness.
- Qualitative results: Under varied image degradations, the baseline misclassifies samples whereas the proposed model maintains accurate predictions across degradation types.Degradation-aware contrastive learning exposes the model to diverse patterns, while mean-teacher alignment prevents feature drift.
- Qualitative results: The baseline explanation contains irrelevant information, SFT-only captures partial evidence with irrelevant details, and the proposed model gives complete, accurate explanations without spurious information.The proposed explanation enumerates detectable manipulation traces while avoiding fabricated details.
- Ablation studies: Combining irrelevant-induced, evidence-omitted, and combined rejected samples yields the best complex-explanation ablation result.Evidence omission and irrelevant hallucination are treated as complementary failure modes.
6 Conclusion
The framework combines robust detection and faithful explanation methods for explainable deepfake detection. It ranks first in the ACM Multimedia 2026 challenge and is presented as a promising direction for transparent forensic AI.
- Conclusion: The framework uses degradation-aware contrastive learning with a mean-teacher architecture to prevent feature drift while preserving augmentation benefits.
- Conclusion: Evidence-grounded preference optimization via DPO teaches the model to prioritize genuine manipulation evidence, complemented by GRPO-optimized concise explanations.
- Conclusion: The method ranks first in the ACM Multimedia 2026 Explainable Deepfake Detection Challenge.