PaperlayerView exact revision on arXiv

Source-linked AI summary

Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection

Zhiya Tan, Jing Huang, Changtao Miao, Lin Tan, Xin Zhang, Weiwei Feng, Jianshu Li, Joey Tianyi Zhou

arXiv:2609.39066v1cs.CV

TL;DR

Existing forgery detectors lack interpretable evidence or reason from evidence, while MLLM explanations can verbalize predetermined decisions. ATAR combines semantic inspection, forensic tools, and multi-turn reasoning with curriculum-based training; it reports strong results across forgery tasks, including 78.5% image-level F1 on six zero-shot IMDL benchmarks.

  • Problem

    Conventional detectors lack interpretable evidence, while MLLM approaches generate post-hoc explanations of predetermined decisions rather than reasoning from evidence.

    Source: §1 · p. 2

  • Method

    ATAR combines semantic inspection and forensic-tool evidence in multi-turn reasoning, trained with trajectory synthesis, a tool-selection curriculum, and structured evidence rewards.

    Source: §1 · p. 2

  • Results

    ATAR reports 78.5% image-level F1 on six zero-shot IMDL benchmarks, 99.9% F1 on Deepfake detection, 87.0% F1 on AIGC detection, and state-of-the-art detection across IMDL, Deepfake, DMDL, and AIGC benchmarks.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    ATAR produces detection and localization decisions with traceable evidence, while reducing true-positive reasoning hallucination to 7.2% versus 34.8% in general-purpose MLLMs.

    Source: §5 · p. 8

Abstract

from arXiv · show

Conventional image forgery detection methods produce binary scores or pixel-level masks without interpretable evidence, while recent multimodal large language model (MLLM)-based approaches generate post-hoc explanations of predetermined classification results rather than reasoning from evidence. Inspired by the forensic workflow of human judicial experts, we propose Agentic Tool-Augmented Reasoning (ATAR), a framework integrating 22 specialized forensic tools across seven complementary domains to autonomously detect, localize, and explain image forgeries through multi-turn reasoning. A Dual-Stream Forensic Reasoning paradigm combines a high-level semantic anomaly path, which magnifies suspicious regions for fine-grained inspection, with a low-level forgery artifact path, which invokes forensic tools to extract objective evidence. We further introduce Forensics Curriculum Learning: during General Experience SFT, an automated teacher-student mentoring pipeline synthesizes multi-turn tool-usage reasoning trajectories; during Forensic Scene RL, a Tool Prior Curriculum guides early tool exploration and progressively transfers control to the agent, while a Structured Evidence Reward provides fine-grained process-level supervision. Experiments across IMDL, Deepfake detection, DMDL, and AIGC detection show that ATAR achieves 78.5% average image-level F1 on six zero-shot IMDL benchmarks, surpassing the strongest MLLM baseline by 11.8 percentage points, and remains competitive with specialized detectors on other tasks while producing substantially more faithful and grounded explanations.

1 Introduction

Existing forgery detectors either lack interpretable reasoning or explain predetermined decisions rather than reasoning from evidence. ATAR combines semantic inspection, forensic tools, multi-turn reasoning, and curriculum-based training to detect, localize, and explain forgeries.

Motivation

  • Conventional detectors lack traceable reasoning, while MLLM approaches verbalize module decisions rather than reasoning from underlying evidence.The introduction frames the shared limitation as decoupled analysis and reasoning.Source: §1 · p. 2, Figure 1 · p. 2

ATAR

  • ATAR follows a forensic workflow: it magnifies suspicious regions, invokes 22 tools for low-level evidence, and reasons across turns to produce a verdict, localization, and supporting evidence.The semantic and artifact paths are complementary and the agent can decide whether further analysis is needed.Source: §1 · p. 2

Training

  • Forensics Curriculum Learning uses teacher–student trajectory synthesis for General Experience SFT, then guides tool selection early in Forensic Scene RL before transferring control to the agent.Structured Evidence Reward separately supervises classification, localization, and reasoning quality.Source: §1 · pp. 2–3

Experiments

  • 78.5% image-level F1 and 56.4% pixel-level F1 on six zero-shot IMDL benchmarks, with 99.9% F1 on Deepfake detection and 87.0% F1 on AIGC detection without AIGC training data.The IMDL results improve on TruFor by 5.6 pp in image-level F1 and 6.0 pp in pixel-level F1.Source: §1 · p. 2

2 Related Work

Prior forgery-detection methods offer either low-level detection without interpretable evidence or MLLM explanations that can lack grounded low-level evidence. Tool-augmented reasoning and training also face challenges in tool exploration and disentangled reward supervision.

2.1 Image Forgery Detection and MLLMs

  • Deep-learning detectors exploit low-level traces but provide no interpretable evidence, while MLLM approaches can hallucinate explanations when their visual encoders miss low-level artifacts.The related work notes that no existing method bridges this gap with external forensic tools.Source: §2.1 · p. 3

2.2 Tool-Augmented Reasoning

  • Tool-augmented reasoning includes multi-turn visual inspection and external-tool methods, but forgery detection faces inefficient exploration in large tool spaces and rewards that conflate classification, localization, and evidence quality.Existing curricula are described as operating at the prompt level rather than guiding tool selection.Source: §2.2 · p. 3

3 Method

ATAR operationalizes the forensic workflow through multi-turn reasoning, with the model choosing semantic inspection or forensic tools, interpreting observations, and producing a verdict, localization, and traceable evidence. Its training has General Experience SFT followed by Forensic Scene RL.

Method overview

  • ATAR reproduces the three-stage forensic workflow through multi-turn reasoning and returns a classification verdict, forged-region localization, and a traceable evidence chain.At each turn, it can magnify a region or invoke a tool, then decide whether to analyze further or form a judgment.Source: §3 · p. 3
  • Forensics Curriculum Learning comprises General Experience SFT with synthesized reasoning trajectories and Forensic Scene RL with a Tool Prior Curriculum and Structured Evidence Reward.The overview figure illustrates a cropped suspicious region, two JPEG tools, and a conclusion based on converging evidence.Source: §3 · p. 4

3.1 Dual-Stream Forensic Reasoning

Dual-Stream Forensic Reasoning pairs semantic inspection of suspicious regions with tool-based analysis of low-level artifacts. The multi-turn loop lets the model combine these observations into a verdict and traceable evidence.

3.1 Dual-Stream Forensic Reasoning

  • High-level semantic anomalies are inspected by cropping and magnifying suspicious regions, while semantically flawless low-level artifacts require forensic-tool verification.The paths are not mutually exclusive; the model may switch when one direction yields insufficient evidence.Source: §3.1 · p. 4
  • The tool library contains 22 unsupervised tools across seven domains, including compression, sensor noise, frequency patterns, structural inconsistencies, copy-move, AI-generation patterns, and foundation-model anomaly detection.The cited example uses DINOv3 Layer 4 features with PCA-reduced Mahalanobis distance.Source: §3.1 · p. 4
  • Each reasoning turn uses grounding to specify a region, a tool call to invoke analysis, or an answer to deliver the verdict and evidence chain.The environment returns tool results as input for the next turn, and the accumulated trajectory supports the final answer.Source: §3.1 · p. 4
  • The two-stage training overview illustrates a suspicious tiger crop, double_jpeg_dq and jpeg_ghost tool calls, and a splice conclusion based on converging evidence.The training stages are General Experience SFT and Forensic Scene RL.Source: §3.1 · p. 4

3.2 Forensic Reasoning Trajectory Synthesis

ATAR addresses the lack of multi-turn reasoning annotations with an automated pipeline that turns raw forgery data into tool-augmented trajectories. The pipeline annotates instances, evaluates tools, guides student exploration, and trains the model for multi-turn reasoning.

3.2 Forensic Reasoning Trajectory Synthesis

  • An automated four-stage pipeline addresses the lack of multi-turn reasoning annotations by converting raw forgery datasets into tool-augmented reasoning trajectories.Manual annotation of these trajectories is prohibitively costly.Source: §3.2 · p. 4

3.2.1 Instance Annotation.

  • An MLLM-assisted iterative process decomposes binary masks into per-object masks with semantic labels, bounding boxes, and tampering types, using SAM for instance separation when needed..Source: §3.2.1 · p. 4

3.2.2 Per-Sample Tool Evaluation.

  • 75.7% of AutoSplice samples favor compression tools, while adversarial composites favor noise tools at 29.7%; every tool domain leads on at least one dataset.These results show that optimal tool domains vary across training datasets.Source: §3.2.2 · pp. 4–5, Table 1 · p. 5

3.2.3 Teacher-Guided Trajectory Generation.

  • A teacher guides a student exploring without ground truth by checking crops against uncovered masks, giving corrective feedback, and tracking whether tampered instances have been found.Crop feedback distinguishes hits, relevant crops, and irrelevant crops using mask coverage and overlap; repeated failures can trigger ground-truth location hints.Source: §3.2.3 · p. 5

3.2.4 General Experience SFT.

  • LoRA adapts Qwen3-VL-8B for domain-specific visual-textual inputs while freezing the vision encoder and training only on model-generated tokens.The multi-modal projector remains unfrozen, and environment-returned observations are excluded from the loss.Source: §3.2.4 · p. 5

3.3 Tool Prior Curriculum Learning

The Tool Prior Curriculum addresses inefficient exploration in a 22-tool action space by providing optimal tool guidance early and progressively returning selection to the agent. Its tool replacements are tiered by curriculum intensity, while high-level semantic trajectories bypass the curriculum.

3.3 Tool Prior Curriculum Learning

  • Suboptimal choices among 22 tools can waste turns, mislead the model, and leave RL with sparse signals; the curriculum injects optimal priors early and progressively removes them.GRPO uses within-group normalization and asymmetric clipping, giving larger updates to positive-advantage trajectories.Source: §3.3 · p. 5
  • At high intensity, selections are replaced with Top-K tools; at medium intensity, with second-tier tools; at low intensity, the model keeps its selection.High-level semantic trajectories bypass the curriculum, and the intensity undergoes global decay over training.Source: §3.3 · p. 5
  • ToolAcc rises from 41.8% to 72.0% as curriculum decay transfers tool selection toward the model.Rewards from replaced tools provide gradient signals that guide the model toward selecting those tools independently.Source: §3.3 · p. 5

3.4 Structured Evidence Reward

Structured Evidence Reward avoids conflating a correct verdict with sound reasoning by supervising classification, localization, reasoning quality, and output format as separate components. Localization is judged against textual region predictions, while reasoning quality is assessed through per-turn and trajectory-level criteria.

3.4 Structured Evidence Reward

  • The total reward combines four independently supervised components for classification, localization, reasoning quality, and valid formatting.Classification rewards a correct verdict, while the format component penalizes malformed action tags or invalid tool calls.Source: §3.4 · pp. 5–6
  • Because the model predicts textual regions rather than masks, an LLM judge matches predictions to overlaid ground-truth masks and identifies missed or hallucinated regions.Precision and recall are computed from matched, predicted, and missed instances.Source: §3.4 · pp. 5–6
  • Reasoning quality averages three criteria: strategy selection, evidence quality, and consistency between the final verdict and accumulated evidence.The criteria assess tool choice or semantic observations, grounding in tool outputs or the image, and whether contradictions are acknowledged.Source: §3.4 · p. 6

4 Experiments

ATAR is evaluated across forgery-detection tasks, with results on zero-shot and in-domain test sets, ablations, explanation quality, tool selection, and case studies. The experiments report strong cross-task performance and examine how training and reward choices affect tool use and reasoning.

4.1 Experimental Setup

  • ATAR is tested on in-domain IMDL, Deepfake, and DMDL data and out-of-domain IMDL, document-forgery, and AIGC sets, using image-level classification and pixel-level localization metrics where applicable.Textual regions are converted to pixel masks with Grounded-SAM; AIGC testing covers classification only, while the listed document-forgery sets evaluate localization only.Source: §4.1 · p. 6

4.2 Main Results

  • 78.5% average iF1 and 56.4% pF1 on six zero-shot IMDL benchmarks put ATAR first on iF1, ACC, pF1, and IoU; its base model reaches only 29.0%.ATAR surpasses TruFor by 5.6 pp in iF1 and 6.0 pp in pF1; GRPO adds 4.2 pp over SFT-only.Source: §4.2 · pp. 6–7
  • 36.5% pF1 on DMDL leads all methods; Deepfake detection reaches 99.9% iF1 and +19.3 pp pF1 over TruFor; zero-shot AIGC reaches 87.0% average F1.The AIGC result is across five generators without AIGC training data; the Deepfake result matches TruFor in classification while outperforming it in localization.Source: §4.2 · p. 7, Table 3 · p. 7, Table 4 · p. 7

4.3 Ablation Study

  • Random tools underperform no tools, while progressive curriculum decay improves tool selection; separate reward components address classification, localization, and reasoning failures.Random Tool scores .726 versus .751 without tools and .785 for Full; removing the tool prior lowers ToolAcc to 38.6%, while removing r3 triples TP hallucination to 21.6%.Source: §4.3 · p. 7

4.4.1 Reasoning Quality.

  • ATAR leads explanation-quality dimensions, averaging 4.12 versus 3.33 for the runner-up, and removing r3 raises true-positive hallucination from 7.2% to 21.6%.Human–judge agreement is κ=.54–.72 and ρ=.74–.86, with highest reliability on Faithfulness (ρ=.86).Source: §4.4.1 · pp. 7–8, Table 7 · p. 8

4.4.2 Tool Specialization.

  • RL sharpens diffuse SFT tool use into dataset-specific specialization, raising optimal-tool hit rates from 56–65% to 77–92%; mixed-forgery Korus has the lowest RL hit rate at 76.8%.The selected top domain matches dominant forgery types, while uninformative tools are suppressed, such as Compression on uncompressed Columbia images.Source: §4.4.2 · p. 8, Table 8 · p. 8

4.4.3 Case Study.

  • Case-study traces show ATAR revising tool-refuted hypotheses, combining semantic and signal-level evidence, and accumulating evidence across regions rather than following a fixed pipeline.Source: §4.4.3 · p. 8

5 Conclusion

ATAR operationalizes the judicial forensic workflow for explainable image-forgery detection, combining semantic inspection and forensic tools. Its training raises tool accuracy, while structured evidence supervision accompanies lower true-positive hallucination and strong results across four task families.

5 Conclusion

  • ATAR operationalizes experimental analysis, logical reasoning, and scientific evidence for explainable image-forgery detection.Source: §5 · p. 8
  • Forensics Curriculum Learning raises tool accuracy from 41.8% to 72.0%.Source: §5 · p. 8
  • Structured Evidence Reward accompanies 7.2% true-positive hallucination versus 34.8% in general-purpose MLLMs, alongside state-of-the-art detection across IMDL, Deepfake, DMDL, and AIGC benchmarks.Source: §5 · p. 8

Related papers

Loading 2609.39066v1…