PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection

Xuechao Zou, Shun Zhang, Kai Li, Yi Zhou, Xinyu Sun, Yuhui Chen, Zhe Wu, Congyan Lang, Junliang Xing

arXiv:2608.06865v1cs.CVcs.AIcs.MA

TL;DR

Deepfake detection is challenged by emerging generators, limited benchmark coverage, coarse labels, and unreliable single-model reasoning. The paper introduces FaceVid-Forensics-100K and a four-agent framework whose judge reconciles texture, lighting, motion, and physics evidence. On the reported benchmark, the framework achieves 69.87% accuracy, 81.82% recall, and 53.28% F1, outperforming the compared methods.

  • Problem

    Emerging generators and existing benchmarks limit generalization because coverage is narrow, annotations are often binary, and single MLLMs may miss subtle cues.

  • Method

    The paper builds FaceVid-Forensics-100K and uses four specialized forensic agents plus a judge agent to produce evidence-grounded predictions and explanations.

  • Results

    69.87% accuracy, 81.82% recall, and 53.28% F1 are achieved on the reported benchmark, with the framework outperforming the compared methods.

  • Takeaways & Limitations

    The results support multi-perspective collaborative reasoning as a basis for generalizable deepfake video detection.

Abstract

from arXiv · show

The malicious use of generative artificial intelligence to create highly realistic deepfake videos raises serious ethical concerns and poses substantial challenges to AI safety. However, existing deepfake video benchmarks provide limited coverage of recent synthesis methods and generally lack reliable fine-grained textual annotations. Meanwhile, conventional detectors and multimodal large language models (MLLMs), whether operating as a single model or relying on a single analytical perspective, often fail to capture subtle forgery artifacts, limiting their generalization to emerging AI-generated methods. To address these limitations, we introduce FaceVid-Forensics-100K, a large-scale deepfake video dataset comprising 100,000 videos and spanning 33 synthesis methods across face swapping, face reenactment, and entire-face synthesis, including recent generators such as Seedance 2.0. The dataset provides fine-grained textual annotations of visual observations and verdict-consistent forensic explanations, automatically synthesized through a multi-model aggregation and conflict-resolution pipeline powered by advanced MLLMs. Building on this benchmark, we propose a multi-agent forensic reasoning framework that employs four specialized domain-expert agents to independently analyze forgery cues from four perspectives: texture, lighting, motion, and physics. A judge agent then reconciles their reports to produce a final prediction together with an explanation. Extensive evaluations on out-of-domain test sets show that, despite being composed entirely of small open-source MLLMs, our framework outperforms all methods including closed-source GPT and Gemini models and ranks first across all reported metrics on this benchmark. The project page is available at https://xavierjiezou.github.io/ARGUS/.

Introduction

Recent generators produce realistic, temporally coherent face videos that challenge detectors trained on earlier methods and benchmarks with coarse supervision. The paper addresses these gaps with a broad, evidence-annotated dataset and coordinated multi-agent forensic reasoning.

  • New video generators synthesize entire faces with stable identity, appearance, and plausible motion, weakening detectors trained on earlier forgery techniques.
  • Existing benchmarks cover limited synthesis methods and often provide only binary labels, making fine-grained forensic evidence unavailable.
  • FaceVid-Forensics-100K contains 100,000 videos spanning 33 synthesis methods and supplies textual observations across texture, lighting, motion, and physics.
  • Four specialized agents independently analyze forensic dimensions, while a judge reconciles supporting or conflicting reports into an authenticity prediction and explanation.
  • 69.87% accuracy, 81.82% recall, and 53.28% F1 are achieved on the reported benchmark, with F1 improving 5.83 percentage points over the strongest single-model baseline.

Related Work

Prior deepfake research established benchmarks and detectors for earlier manipulations, while newer work uses MLLMs and multi-agent systems for evidence-grounded reasoning. The paper positions its approach against limitations of narrow coverage, binary supervision, and single-model inference.

  • Early benchmarks progressed from limited face-swapping and reenactment datasets to standardized, higher-quality, larger-scale, and perturbation-aware evaluations.
  • Conventional detectors learn from binary labels and target cues including blending boundaries, gaze, lip motion, temporal coherence, and spatiotemporal inconsistency.
  • MLLM-based detectors recast deepfake detection as evidence-grounded visual reasoning, but single-model inference may overlook weak cues or let one artifact bias verdicts.
  • Table 1 compares representative datasets using video coverage and whether observation and explanation annotations are available.
  • Multi-agent systems coordinate specialized decision makers through communication, critique, debate, or iterative collaboration, including in multimodal tasks.

FaceVid-Forensics-100K

FaceVid-Forensics-100K is a broad face-centric deepfake dataset designed for evidence-driven detection, combining many synthesis methods with fine-grained observations and verdict-consistent explanations. Its splits emphasize evaluation on unseen generators.

  • FaceVid-Forensics-100K contains 100,000 face-centric videos across 33 synthesis methods and face swapping, face reenactment, and entire-face synthesis.
  • The dataset adds fine-grained textual observations across texture, lighting, motion, and physics, together with verdict-consistent explanations.
  • Videos originate from existing forgery datasets, internet collections, and recent generative models, with sources including AIGVDBench, GenVidBench, ViF-Bench, and FaceForensics++.
  • The dataset contains 21,075 real and 78,925 fake videos after real-video deduplication and collection balancing.
  • 20 held-out EFS generators are assigned exclusively to the OOD test set, producing approximately 68.9K training, 23.5K in-domain test, and 7.6K OOD videos.
  • Five MLLMs independently annotate observations and verdict–explanation pairs, while DeepSeek-V4 Pro aggregates observations by dimension and resolves conflicts using the ground-truth label.

Multi-Agent Forensic Reasoning

The proposed system decomposes video forensics across four specialized observation agents and a judge agent that integrates their reports, optionally with video frames. Training first uses supervised fine-tuning, then GRPO refines judge decisions while preserving specialized evidence extraction.

  • Framework: The framework samples video frames and sends them to four specialized observation agents plus a central judge agent.
  • Observation Agents: Texture, lighting, motion, and physics agents independently inspect skin and blending, illumination and shadows, temporal stability, and anatomical or geometric plausibility.
  • Judge Agent: The judge receives expert observations and optionally video frames, weighs corroborating or conflicting evidence, and outputs a binary verdict with a concise explanation.
  • Training Pipeline: All agents undergo supervised fine-tuning, after which GRPO refines the judge agent while the four observation agents remain frozen.
  • Training Pipeline: During supervised training, the judge consumes concatenated observation reports and optional visual input to generate the target explanation and verdict.
  • Training Pipeline: GRPO uses grouped binary-accuracy rewards and a KL penalty against the frozen SFT policy to refine classification without changing specialized evidence extraction.

Experiments

Experiments evaluate the framework on held-out generators and compare it with vision detectors, general-purpose MLLMs, forensics-tuned MLLMs, agent combinations, training strategies, and alternative reasoning designs. The complete multi-agent system achieves the strongest reported out-of-domain performance, while complementary observations, video access, joint training, and GRPO each contribute to results.

  • Evaluation Protocols: The OOD split contains 5,716 real and 1,920 fake videos from 20 generators excluded from training.Small vision models are retrained, while MLLMs use released checkpoints or official APIs.
  • Observation-Agent Contribution: Combining all four observation agents gives the best F1 under both judge configurations, with a clearer advantage when the judge lacks direct video input.When frames are available, the performance gap narrows because the judge can recover some missing visual information directly.
  • Training Ablations: Jointly training observation agents and the judge is more effective than training either alone, while GRPO provides a further gain.The ablations attribute contributions to specialized observation learning, evidence reconciliation, and policy optimization.
  • Reasoning Strategies: The multi-agent design performs best across both backbones, exceeding direct prediction, chain-of-thought prompting, and three multi-turn variants.The comparison indicates that independent evidence collection and judge-based reconciliation, rather than longer prompts or repeated video access, account for the improvement.

Conclusion

The paper presents a broad, finely annotated benchmark and a collaborative four-perspective forensic framework for deepfake detection. Experiments show consistent out-of-domain gains over existing vision-based detectors and MLLMs.

  • Conclusion: FaceVid-Forensics-100K combines broad coverage of recent synthesis methods with fine-grained forensic annotations.The framework analyzes four complementary forensic perspectives and produces both authenticity predictions and explanations.
  • Conclusion: The approach consistently outperforms existing vision-based detectors and MLLMs on out-of-domain benchmarks.The conclusion highlights multi-perspective collaborative reasoning as effective for generalizable deepfake video detection.

A Additional Details of FaceVid-Forensics-100K

FaceVid-Forensics-100K combines real videos from four sources with entire-face synthesis videos from multiple generators, using source- and method-level split composition. Its OOD evaluation holds out generators from training.

  • Dataset Composition: The real subset contains 21,075 videos from AIGVDBench, GenVidBench, FF++, and ViF-Bench.After preprocessing and deduplication, 11,626 real videos are assigned to training, 3,733 to in-domain testing, and 5,716 to OOD testing.
  • Dataset Composition: The entire-face synthesis subset includes 74,509 videos from eight seen generators, with 55,134 for training and 19,375 for in-domain testing.Generator counts range from 456 for Runway Gen-2 to 32,690 for VideoCrafter2.

B.1 Detailed Out-of-Distribution Results

The framework generalizes across unseen generators while remaining competitive on known manipulations. Qualitative cases show that independent forensic analyses and judge reconciliation recover subtle cues missed by single-agent baselines.

  • Out-of-distribution performance: 68.73% macro-average accuracy without video and 69.47% with sampled frames rank first and second across evaluated methods.With sampled frames, the framework surpasses TALL++ by 5.25 and Skyra by 4.24 macro-average points.
  • Out-of-distribution performance: The framework ranks first or second on 11 of 20 held-out generators, while both GRPO settings achieve the best result on six generators.This distribution indicates that gains extend across several generator families rather than one source.
  • In-domain performance: On known manipulations, the framework reaches 87.34% accuracy and 96.35% F1 without video input after GRPO, while specialized vision models remain strongest overall.TFCU reaches 98.66% accuracy and 99.46% F1 on the in-domain test set.
  • Qualitative comparison: In qualitative comparisons, single-agent systems incorrectly predict Real, whereas independent texture, lighting, motion, and physics analyses support the judge’s correct Fake verdict.The judge emphasizes persistent foreground–background motion mismatch and uses other anomalies as corroborating evidence.
  • Explanation evaluation: Explanation quality is evaluated by two independent judges using a shared rubric covering accuracy, grounding, clarity, and verdict consistency.Scores are reported on 100 real and 100 fake OOD videos.
  • Explanation evaluation: Video input improves explanation-quality scores across framework configurations, but the real-versus-fake score difference is specific to the sampled subset.DeepSeek-V4 Pro gives the highest overall score, 6.37, to the SFT configuration with video input; GPT-5-mini assigns 6.25.

C.1 Efficiency Analysis

The framework offers a middle ground between fast single-agent strategies and slower multi-turn dialogue. Its qualitative advantage comes from independently collecting and reconciling complementary forensic evidence.

  • Efficiency: 4.781–5.486 seconds per video is the framework’s latency on InternVL3.5-8B, versus 14.150 seconds for Multi-turn.The framework is faster than Multi-turn while its latency lies between Single/CoT and Multi-turn baselines.
  • Efficiency: 151.293–181.88 tokens per second is the framework’s throughput across the two evaluated MLLMs.Latency and throughput are measured on 100 randomly sampled OOD videos using one NVIDIA RTX 5090 GPU.
  • Efficiency: Future work could use prompt compression to optimize inference efficiency because the framework introduces more prompts than single-agent methods.BEAVER is identified as one possible prompt-compression technique.
  • Qualitative comparison: Single, CoT, and Multi-turn baselines incorrectly predict Real on a difficult fake video, while MAS identifies complementary anomalies and predicts Fake.The MAS judge highlights frozen gaze and absent biological micro-motion alongside texture, lighting, motion, and physics evidence.
  • Model combinations: Heterogeneous observation–judge MLLM combinations remain competitive with homogeneous configurations on the OOD test set.With reports only, InternVL3.5 observations judged by Qwen2.5-VL achieve the best F1 of 47.54%.

E Impact of Model Parameter Scale

Scaling Qwen2.5-VL from 3B to 32B parameters consistently improves OOD accuracy and F1 under SFT. The evaluation isolates model capacity across two judge-input settings.

  • Evaluation setup: All parameter variants use SFT and are compared when the judge receives either observation reports alone or reports plus sampled frames.This design isolates the effect of model capacity across 3B, 7B, and 32B Qwen2.5-VL variants.
  • Scaling results: Scaling from 3B to 32B raises report-only accuracy from 61.36% to 66.40% and F1 from 45.81% to 49.92%.The intermediate 7B model reaches 63.61% accuracy.
  • Scaling results: With sampled video frames supplied to the judge, F1 reaches 50.81% at 32B parameters.This setting exhibits the same upward scaling trend as the report-only configuration.
  • Framework configuration: Observation agents receive sampled frames and dimension-specific prompts, while the judge reconciles their reports with or without direct frame access.The four dimensions are texture, lighting, motion, and physical plausibility.

G Training Dynamics

Observation-agent SFT converges rapidly and similarly across forensic dimensions, while judge-agent GRPO reaches a high reward plateau in both video-input settings. Video input reduces early policy drift and late reward variability.

  • SFT dynamics: Observation-agent SFT loss drops steeply within roughly the first 10% of training, while token accuracy plateaus over the second half.The four agents converge similarly, with small stable differences across dimensions.
  • GRPO dynamics: Judge-agent GRPO reward rises sharply early and fluctuates around a high plateau in both video-input settings.Because reward is parsed-verdict accuracy, the dynamics indicate refinement of an already correct decision policy.
  • GRPO dynamics: With video input, KL divergence peaks near 0.010 versus 0.015 without video input, then both configurations decay below 0.006.The video-input judge reaches comparable reward while drifting less from its frozen SFT initialization.
  • GRPO dynamics: Reward standard deviation decreases in both settings and approaches zero with video input during the final third of training.Without video input, reward variability remains higher late in training.
Loading 2608.06865v1…