PaperlayerView exact revision on arXiv

Source-linked AI summary

FACT: A Forensic Agent with Compiled Tool-Use Trajectories for AI-Generated Image Detection

Jiaoyang Chen, Bin Hu, Jingyu Hu, Kun Zhou, Qin Zhang, Zhengzhe Liu

arXiv:2609.05876v1cs.CVcs.AI

TL;DR

Fixed forensic cues can be unreliable across diverse, increasingly realistic image generators. FACT learns an image-conditioned tool-use investigation policy through an Evolve–Distill–Refine pipeline. It achieves the best performance among compared methods across six benchmarks, including on held-out generators and broader forgery settings.

  • Problem

    As generators become more realistic and diverse, a forensic cue that works for one family may not remain reliable for another.

    Source: Introduction · p. 1

  • Method

    FACT evolves a forensic skill, compiles it into action–observation trajectories, distills a compact agent, and refines its tool selection and stopping with cost-aware GRPO.

    Source: Introduction · p. 2

  • Results

    FACT achieves the best performance among compared methods across six benchmarks, including on recent held-out generators and broader forgery settings.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    The reported results support FACT's investigation-policy framing across held-out generators and settings that include deepfakes and manipulated images.

    Source: Introduction · p. 2, Conclusion · p. 7

Abstract

from arXiv · show

AI-generated image detection is increasingly open-world: new image generators produce highly realistic images that make visual artifacts harder to identify. Existing detectors usually rely on a fixed set of forensic cues, so a detector that works well for one generator family may fail on another. We introduce FACT (Forensic Agent with Compiled Tool-use Trajectories), which learns an image-conditioned tool-use policy for forensic analysis. Instead of applying a fixed detector, FACT decides which forensic tools to call, interprets the returned evidence, and stops when sufficient evidence has been collected. FACT follows an Evolve--Distill--Refine pipeline: it evolves an execution-verified forensic skill, compiles the skill into action--observation tool-use trajectories, distills them into a compact agent, and refines the policy with cost-aware GRPO. Across two internal and four public benchmarks, FACT achieves the best performance among all compared methods, including on recent unseen generators, deepfakes, and manipulated images.

Introduction

As generators become more realistic and diverse, fixed forensic cues may not transfer across generator families. FACT instead learns an image-conditioned investigation policy that acquires and interprets evidence, then stops when sufficient evidence is collected.

Introduction

  • Existing detectors rely on different forensic priors, and their strengths vary across generator families, making evidence selection for each image a central challenge.The cited balanced evaluation reports that Veritas, AIDE, Community Forensics, and SDAIE each lead on different generator subsets.Source: Introduction · p. 1
  • On STD3K, the final refined FACT checkpoint achieves 91.40%; on NewGen-900, it obtains 82.44% without generator-specific fine-tuning.NewGen-900 contains nine recent generator families held out from all FACT training stages.Source: Introduction · p. 2
  • Across all six benchmarks, FACT achieves the best performance among compared methods, covering recent held-out generators and broader forgery settings including deepfakes and manipulated images.The comparisons include constituent tools and majority voting.Source: Introduction · p. 2
  • FACT frames detection as image-conditioned tool use: the agent selects forensic evidence, interprets observations, and stops when evidence is sufficient.Its Evolve–Distill–Refine pipeline evolves an executable skill, compiles it into action–observation trajectories, distills a specialized policy, and refines tool selection and stopping with cost-aware GRPO.Source: Introduction · p. 2

Related Work

Image generation has advanced toward realistic, controllable synthesis, while detection has shifted from generator-specific artifacts toward more general and heterogeneous forensic cues. FACT differs from prior agentic forensics by learning a full tool-use investigation procedure rather than routing each image to one detector.

Related Work

  • Image synthesis has progressed from GANs through diffusion and latent diffusion to transformer- and flow-based systems, with newer systems supporting conditioning, editing, and text rendering.These capabilities make synthetic images more realistic, controllable, and easy to revise.Source: Related Work · p. 2
  • Detection has expanded from generator artifacts to pretrained representations, reconstruction discrepancies, broader training diversity, and heterogeneous forensic priors, whose strengths remain generator- and evidence-dependent.FACT treats existing priors as frozen callable experts and learns which evidence to acquire for each image.Source: Related Work · pp. 2–3
  • Unlike UniShield (Huang et al. 2025), which routes by forgery category and detector type, FACT compiles a verified forensic skill into trajectories and refines a full investigation policy for cost-aware stopping.The trajectories are distilled into a compact agent.Source: Related Work · p. 3

Method

FACT frames detection as image-conditioned evidence acquisition: it combines complementary forensic tools and learns when to call them or stop. Its three-stage pipeline verifies an investigation skill, distills tool-use trajectories, and refines the policy for cost-aware decisions.

Overview and Problem Formulation

Forensic Toolbox Construction

  • The toolbox pairs five frozen model-based experts with eight lightweight probes covering spectral, texture, noise, compression, gradient, and patch-consistency evidence.The experts capture differing forensic cues, while probes provide interpretable measurements distinct from neural-detector predictions.Source: Forensic Toolbox Construction · p. 3

Stage I: Tool-Use Skill Learning

  • FACT evolves a textual forensic skill covering tool selection, evidence interpretation, conflict handling, and stopping, retaining revisions only when rerun performance improves on a separate verification batch.The verified skill guides trajectory generation in Stage II.Source: Stage I: Tool-Use Skill Learning · p. 4

Stage II: Distilling Tool-Use Trajectories

  • FACT uses the verified skill to generate image-specific tool-call and observation sequences, then distills the agent-generated actions and final answers into a compact policy by supervised fine-tuning.Tool outputs provide context, while the policy learns the actions, stopping decisions, and final answer; this stage imitates investigation behavior rather than optimizing evidence cost directly.Source: Stage II: Distilling Tool-Use… · pp. 4–5

Stage III: Cost-Aware Tool-Use Refinement

  • Cost-aware GRPO refines the distilled policy by rewarding correct decisions and penalizing tool costs, so additional evidence is favored only when it justifies its cost.The cost coefficient encourages avoiding unnecessary calls, while task reward guards against stopping too early.Source: Stage III: Cost-Aware Tool-Use… · p. 5

Experiments

FACT is evaluated on two internal and four public benchmarks, then examined through training-stage, toolbox, and reward ablations. It outperforms the compared methods across the six benchmarks, while the ablations report gains from skill learning, trajectory distillation, and cost-aware refinement.

Experimental Setup

  • Evaluation spans two balanced internal benchmarks and four public benchmarks, with comparisons against five toolbox experts, their majority vote, and published baselines.Accuracy is reported for four benchmarks, balanced accuracy for LOKI, average split accuracy for HydraFake, and normalized acquisition cost for efficiency.Source: Experimental Setup · pp. 5–6

Comparison with Existing Works

Ablation Studies

  • Across Direct, Skill-guided, Distilled, and Refined variants, accuracy rises from 87.90%/50.80% to 91.40%/82.44% on STD3K/NewGen-900, with intermediate gains after skill learning and distillation.Refinement decreases STD3K accuracy by 0.13 points while increasing NewGen-900 accuracy by 0.55 points.Source: Ablation Studies · pp. 6–7, Table 7 · p. 7
  • Signal probes and model-based experts reach 82.90% and 82.50% alone on STD3K + NewGen-900, while their combination reaches 90.70%.The paper characterizes low-level measurements and high-level detector outputs as complementary evidence.Source: Ablation Studies · p. 7, Table 7 · p. 7
  • Format and valid-tool penalties raise NewGen-900 accuracy from 81.78% to 82.56%; adding tool cost yields 82.44% accuracy and lowers acquisition cost from 1.5500 to 1.3835.Source: Ablation Studies · p. 7, Table 8 · p. 7

Conclusion

FACT reframes AI-generated image detection as learning an investigation procedure, improving over individual experts and static voting while generalizing to recent generators held out from training. Its refined policy jointly optimizes evidence acquisition and stopping, preserving detection performance while reducing normalized acquisition cost.

Conclusion

  • FACT improves over individual experts and static voting across benchmarks, while generalizing to recent generators held out from all training stages.It learns an investigation procedure rather than training another fixed detector.Source: Conclusion · p. 7
  • The refined policy preserves detection performance while reducing normalized acquisition cost, avoiding redundant tool calls without abandoning difficult cases.The results indicate that evidence acquisition and stopping should be optimized jointly.Source: Conclusion · pp. 7–8
  • FACT suggests robustness may depend less on a universal detector and more on learning which evidence each image requires.The authors state that additional qualitative analyses and limitations are in the supplementary material.Source: Conclusion · p. 8
  • Upon publication, the authors will release the code, trained model, and benchmark data and metadata, subject to source licenses.Source: Conclusion · p. 8
Loading 2609.05876v1…