PaperlayerView exact revision on arXiv

Source-linked AI summary

MSU Team at the Explainable Deepfake Detection Challenge 2026: Grounded Artifact Evidence for Deepfake Detection

Artem Filippov, Aleksandr Gushchin, Kirill Koltsov, Dmitriy Vatolin, Anastasia Antsiferova

arXiv:2610.09952v1cs.CV

TL;DR

Realistic manipulated images make authenticity harder to verify, motivating detectors that provide visible evidence as well as a real/fake label. The paper combines forensic detection with explanation-derived artifact supervision and class-conditional explanation generation; on the final test split, it reports 0.9349 detection accuracy, a 0.5571 explanation score, and a 0.7456 final challenge score.

  • Problem

    As manipulated images become more realistic, detectors need to provide visible evidence alongside binary authenticity labels so decisions and failures can be audited.

    Source: §1 · p. 1

  • Method

    The system combines a multi-backbone forensic detector, explanation-grounded pseudo-mask supervision, patch-level contrastive learning, and class-conditional Qwen3-VL explanation generation.

    Source: §1 · pp. 1–2, §4.3 · p. 4

  • Results

    0.9349 detection accuracy, a 0.5571 explanation score, and a 0.7456 final challenge score were achieved on the final test split.

    Source: §7 · p. 6

  • Takeaways & Limitations

    Validation ablations indicate that explanation-derived local supervision improves detector generalization, local contrastive learning adds a further gain, and GRPO simplification improves the official simple-explanation score.

    Source: §7 · p. 6

    The VLM generates explanations only after the detector predicts a class; it does not participate in authenticity decisions, compare hypotheses, or verify detector evidence.

    Source: §6 · p. 6

Abstract

from arXiv · show

Recent advances in generative image models have made many manipulated images highly realistic, raising the need for detectors that are not only accurate but also able to provide visual evidence for their decisions. In this paper, we present our solution to the Explainable Deepfake Detection Challenge [2] on the XPlainVerse dataset [1], where systems are required to predict whether an image is real or fake and generate both complex and simple explanations grounded in visible forensic cues. Our method follows a modular detection-and-explanation design. For the real/fake decision, we build a multi-backbone detector that combines several DINOv3 models with Mesorch manipulation-localization features, bringing together pretrained visual representations, DCT-aware cues, and multi-scale forensic information. To inject explanation evidence into the detector, we use a Grounding-DINO-based pseudo-mask generation pipeline that converts local artifact descriptions from training explanations into weak patch- level supervision for an Artifact Evidence Map. We further introduce a local patch-level contrastive objective that separates artifact and authenticity evidence in the detector feature space without requiring paired images or pixel-level manipulation masks. For language output, we use class-conditional Qwen3-VL models to generate complex explanations for fake and real predictions, followed by a GRPO-optimized text simplification model. The proposed methods were trained and evaluated on the challenge subset of XPlainVerse. On the full test split, our submission achieves 0.9349 detection accuracy, 0.5571 explanation score, and a 0.7456 final challenge score.

1 Introduction

The challenge requires real/fake predictions alongside complex and simple explanations grounded in visible forensic evidence. The proposed system combines a forensic detector with explanation-derived artifact supervision, patch-level contrastive learning, and class-conditional language generation.

1 Introduction

  • The challenge evaluates real/fake classification alongside semantic similarity, simplicity, and grounding of explanations to entities and visual evidence.The setting uses XPlainVerse, which pairs real and manipulated images with image-level labels and technical and general-user explanations [1].Source: §1 · p. 1
  • The detector combines several DINOv3 backbones [4] with Mesorch's DCT-aware, multi-scale forensic representations.The design separates authenticity classification from language generation.Source: §1 · pp. 1–2
  • A Grounding-DINO-based pipeline converts local artifact cues in fake-image explanations into weak patch-level pseudo-masks for detector supervision.The masks are intended to encourage use of visible, semantically grounded manipulation evidence when available.Source: §1 · pp. 1–2
  • The patch-level contrastive objective separates local fake and authentic evidence without paired images or pixel-level manipulation masks.It is motivated by image-level contrastive work DRCT and local-prediction work HiDA-Net.Source: §1 · pp. 1–2
  • Class-conditional Qwen3-VL models [10] generate explanations selected by the detector prediction, and a text-only model simplifies them using GRPO [11].Classification remains in the specialized detector, while VLMs produce the challenge's language outputs.Source: §1 · p. 2

2 Related Work

Prior datasets and methods address synthetic-image detection, manipulation localization, contrastive learning, and explainable detection, but differ in coverage and design. The paper draws on these approaches while noting that existing dataset formats and annotations do not fully match the challenge data.

2.1 Datasets

  • Image-level synthetic-image benchmarks mainly target real/fake classification and support cross-generator generalization, but mostly contain fully synthetic images and lack the explanation annotations required here.GenImage pairs ImageNet real images with GAN- and diffusion-generated counterparts. WildFake contains over 3.5M real and fake images, while Community Forensics collects 2.7M images from 4,803 text-to-image models.Source: §2.1 · p. 2
  • Explainable datasets include face-forgery VQA, natural-language artifact explanations, and multimodal abnormality questions, but their formats, protocols, and artifact distributions differ from challenge data.The cited examples are DD-VQA, FakeClue, and LOKI.Source: §2.1 · p. 2

2.2 Methods

  • Detection methods use reconstruction cues such as AEROBLADE, CLIP adaptation such as FatFormer, or frequency information such as SPAI.Recent NTIRE challenge results motivate the paper's multi-backbone design, including DINOv3 models [4].Source: §2.2 · p. 2
  • DRCT separates real, generated, and diffusion-reconstructed samples with image-level contrastive learning, while HiDA-Net emphasizes spatial sensitivity to local edits.These works motivate dense features and a patch-level objective in addition to global classification.Source: §2.2 · p. 2
  • Localization methods combine RGB and noise-sensitive cues in TruFor, RGB-DCT cues in CAT-Net, and DCT-aware multi-scale features in Mesorch.The paper uses Mesorch for its frequency-aware, localization-oriented representation alongside DINOv3.Source: §2.2 · p. 2
  • Prior VLM systems adapt visual components for forensic detection or explanation, whereas this paper separates a specialized detector from class-conditional explanation generators.Examples include FakeVLM, ForenDeX, and FakeXplainer.Source: §2.2 · p. 2

3 XPlainVerse Dataset and Challenge Subset

The challenge uses a curated 760,000-image subset of XPlainVerse, with specified public splits and labeled real/fake counts.

3 XPlainVerse Dataset and Challenge Subset

  • The challenge subset contains 760,000 images: 270,000 real and 490,000 fake.XPlainVerse [1] contains one million real and manipulated images paired with natural-language explanations; a substantial portion derives from MultiFakeVerse [15].Source: §3 · p. 2
  • The public train, validation, and test splits contain 450,000, 110,000, and 200,000 images, respectively.The labeled train–validation data comprise 180,000 real and 380,000 fake images.Source: §3 · p. 2
  • The test split contains 90,000 real and 110,000 fake images.Source: §3 · p. 2

4.1 Pseudo-mask Generation Pipeline

The pipeline derives weak patch-level artifact supervision from fake-image explanations through artifact vocabulary construction, phrase grounding, and conversion of grounded regions into patch targets.

4.1 Pseudo-mask Generation Pipeline

  • Fake-image explanations provide weak artifact supervision: pseudo-masks mark regions where textual evidence and open-vocabulary grounding agree, rather than pixel-accurate manipulation annotations.The pipeline runs offline on training and validation splits; real images receive zero artifact targets.Source: §4.1 · p. 2

4.1.1 Artifact Vocabulary Construction.

  • Qwen3-VL-32B-Instruct [10] extracts artifact phrases from explanations, merges near-duplicates, and selects representatives to form a recurring artifact vocabulary.The vocabulary is divided into local and global attributes, and only local attributes are grounded to avoid noisy box supervision for global observations.Source: §4.1.1 · p. 2

4.1.2 Image-level Grounding Phrase Extraction.

  • For each fake image, Qwen3-VL selects locally visible attributes and turns them into concrete visual phrases for Grounding DINO.Grounding DINO returns candidate boxes; low-confidence boxes are discarded and retained boxes are merged because artifact categories can overlap.Source: §4.1.2 · p. 3, §4.1.3 · p. 3

4.1.3 Grounding and Patch Targets.

  • The patch target records the fraction of each detector-grid patch covered by the union of grounded boxes, with overlapping boxes counted once.Image-level labels remain the primary supervision; the patch target only regularizes the artifact branch.Source: §4.1.3 · p. 3

4.2 Detector

The detector fuses DINOv3 and Mesorch spatial features into a shared patch representation for image-level classification and artifact localization. Patch-level contrastive learning further organizes local evidence, and the training objective combines classification, artifact-map supervision, and contrastive learning.

4.2 Detector

  • The detector outputs an image-level fake probability, an Artifact Evidence Map, and patch embeddings for the local contrastive loss.Source: §4.2 · p. 3

4.2.1 Multi-backbone Features.

  • The detector combines DINOv3 backbones with Mesorch features, aligning and fusing their spatial representations into a Unified Forensic Feature Map.Mesorch is used as a feature extractor, and its spatial representation is aligned to the DINO patch grid.Source: §4.2.2 · p. 3, §4.2.1 · p. 3

4.2.3 Image-level Classification Branch.

  • The global branch combines max-pooled patch features with fused DINO global embeddings to predict whether an image is fake.The branch is trained with focal loss using α=0.5 and γ=2.0.Source: §4.2.3 · pp. 3–4

4.2.4 Artifact Evidence Map.

  • A shared patch head produces the Artifact Evidence Map, whose patch probabilities estimate artifact evidence and are supervised with BCE targets.Real-image targets are zero.Source: §4.2.4 · p. 4

4.2.5 Local Patch-level Contrastive Objective.

  • A local contrastive objective organizes patch embeddings to separate evidence consistent with fake images from authentic evidence while treating uncertain regions conservatively.The objective uses matched patches from fake–fake, real–real, and fake–real image pairs; ambiguous or contradictory patches can be ignored or handled through stop-gradient anchors.Source: §4.2.5 · p. 4

4.2.6 Training Objective.

  • The final detector objective combines image classification, artifact-map supervision, and local contrastive learning.These terms supervise image classification, weak artifact localization, and patch-space organization.Source: §4.2.6 · p. 4

4.3 Class-Conditional Vision-Language Explanation Generation

The explanation module uses class-conditional Qwen3-VL models to generate evidence-bearing complex explanations and a text-only model to simplify them. At inference, the detector's prediction selects a complex-explanation generator, whose output is then simplified.

4.3.1 Class-conditional Complex Explanation Models.

  • Separate Qwen3-VL-8B-Instruct generators produce complex explanations for fake and real images, with class separation intended to reduce leakage such as hallucinated artifacts for real images.At inference, the detector prediction selects which generator is used.Source: §4.3.1 · p. 4, §4.3 · p. 4

4.3 Class-Conditional Vision-Language Explanation Generation

  • The complex explanation is the primary evidence-bearing text output, while the simple explanation is derived afterward for general users.Source: §4.3.1 · p. 4, §4.3 · p. 4

4.3.1 Class-conditional Complex Explanation Models.

  • Both complex-explanation generators are LoRA-adapted from Qwen3-VL-8B-Instruct, with the vision tower frozen.Real images are oversampled when training the real-explanation branch because they are less frequent.Source: §4.3.2 · pp. 4–5

4.3.3 Complex-to-simple Explanation Model.

  • A text-only simplifier is trained after supervised fine-tuning and optimized with GRPO using the official simple-explanation score.Fake-image explanation pairs provide most of the simplification signal because real-image complex and simple explanations are often nearly identical.Source: §4.3.3 · p. 5

4.3.4 Inference Pipeline.

  • At inference, the detector's predicted label routes each image to exactly one complex-explanation model, and the selected explanation is passed to the simplifier.The final output combines the detector label, complex explanation, and simple explanation.Source: §4.3.4 · p. 5

5 Experiments

Experiments show gains from explanation-grounded detector supervision and explanation optimization, followed by strong full-test challenge performance.

5.1 Detector Ablation

  • 0.9417 validation AUC followed the Artifact Evidence Map gain from 0.8975 to 0.9139 over the image-level detector.Ablations used the public validation split; the final submission retained both pseudo-mask and local contrastive losses.Source: §5.1 · p. 5, Table 4 · p. 5

5.2 Explanation Generation Evaluation

  • 0.9703 normalized SLE versus 0.4264 for the organizer baseline accompanied Entity F1 of 0.6235 versus 0.5536 and Evidence F1 of 0.5447 versus 0.4620.The internal held-out evaluation used generations with the correct final answer to isolate explanation quality from detector errors.Source: §5.2 · pp. 5–6, Table 5 · p. 6

5.3 Final Submission Results

  • 0.9349 detection accuracy, 0.5571 explanation score, and 0.7456 final challenge score were achieved on the full final test split.The final detector was trained on all available public labeled data with both pseudo-mask and local contrastive losses retained.Source: §5.1 · p. 5, §5.3 · p. 6, Table 6 · p. 6

6 Discussion

The discussion identifies computational cost and the detector-first role of the language model as practical limitations, while cautioning that metric gains do not guarantee user accessibility.

6 Discussion

  • Several DINOv3 backbones, Mesorch features, and multiple Qwen3-VL models make the system computationally heavy and less attractive for practical deployment.The authors consider this complexity acceptable for a challenge submission.Source: §6 · p. 6
  • Because the detector decides authenticity first, the class-conditional VLM does not participate in that decision, compare hypotheses, or verify detector evidence.It generates an explanation for the detector-predicted class.Source: §6 · p. 6
  • Improved official simple-explanation metrics, especially normalized SLE, do not necessarily make explanations substantially more useful or accessible to ordinary users.Source: §6 · p. 6

7 Conclusion

The conclusion presents a modular explainable deepfake detector evaluated on the XPlainVerse challenge subset, reporting final test results and validation evidence for its supervision and simplification components.

7 Conclusion

  • 0.9349 detection accuracy, 0.5571 explanation score, and 0.7456 final challenge score were achieved on the final test split.The system was trained and evaluated on the XPlainVerse challenge subset.Source: §7 · p. 6
  • Validation ablations indicate that explanation-derived local supervision improves detector generalization, with a further gain from local contrastive learning; GRPO simplification improves the official simple-explanation score.The simplification contribution also feeds into the final result.Source: §7 · p. 6
Loading 2610.09952v1…