GIFTBench: Diagnosing Generalization in Image Forgery Localization and Informing Model Design

1 Introduction

• Existing benchmarks often entangle manipulation factors, limiting evidence about which variation is associated with cross-dataset localization failures.The paper argues for shared factorized annotations, axis-specific evaluation, and a diverse training distribution.Source: §1 · p. 1
• GIFTBench contains 115,013 manipulated ADE20K images with pixel-level masks and annotations for source, semantic target, operation, and composition.Its unified framework supports axis-specific transfer analysis and training; generalization is also evaluated on twelve external datasets.Source: §1 · p. 2
• GIFTBench reveals asymmetric cross-source transfer, recall-dominated failures, and heterogeneous degradation, while training on it improves aggregate transfer to external datasets.ForenScope also improves cross-dataset localization while retaining image-level detection.Source: §1 · p. 2
• ForenScope combines a classification-adapted DINOv2 representation with multi-depth patch features, learned cross-layer fusion, bounded modulation, and coarse-to-fine decoding.The design rationale and empirical tests are presented later in the paper.Source: §1 · p. 2

Image manipulation localization benchmarks

• Early IFL benchmarks emphasized splicing, copy-move, and limited post-processing [Dong et al., 2013, Hsu et al., 2006, Wen et al., 2016, Guan et al., 2019].Later datasets broadened sources and real-world imagery [Novozamsky et al., 2020, Tralic et al., 2013], then increased scale and manipulation diversity [Mahfoudi et al., 2019, Kadam et al., 2021].Source: §2 · pp. 2–3

IFL models

• Recent benchmark work increasingly addresses generative and instruction-driven editing [Jia et al., 2023, Mareen et al., 2024, Chen et al., 2025, Cai et al., 2026].IFL systems have used constrained convolutions, noise or frequency cues, patch reasoning, transformer encoders, segmentation decoders, and foundation-model representations [Bayar and Stamm, 2018, Wu et al., 2019, Chen et al., 2021, Hu et al., 2020, Wang et al., 2022, Ma et al., 2023, Guillaro et al., 2023, Su et al., 2025, Zhu et al., 2025, Ma et al., 2024].Source: §2 · p. 3

Foundation models for dense prediction

• General segmentation systems show transformer representations can support accurate masks with task-specific decoders [Kirillov et al., 2023, Ravi et al., 2025].In image forensics, semantically invariant representations may suppress weak manipulation-specific evidence.Source: §2 · p. 3

3.1 Benchmark Design

• GIFTBench labels source, semantic target, operation, and composition separately, distinguishing editing pipelines from intended transformations and enabling axis-specific diagnosis.Its construction uses ADE20K images and semantic annotations, guided manipulation planning, multiple editing pipelines, explicit masks, provenance, and automatic and manual checks.Source: §3.1 · pp. 3–4

3.2 Evaluation Protocols

• The protocols test transfer across source, semantic-target, and operation pairings, composition settings, and external datasets, using pixel-level F1 and IoU.Axis-specific results are associated transfer patterns, not strict one-variable-at-a-time interventions; external tests use no target-domain fine-tuning.Source: §3.2 · pp. 4–5

3.3 Benchmark Diagnosis

• Transfer is stronger on Splicing and Copy–Move than Removal and AI–Edit, and off-diagonal source failures are dominated by false negatives and sharp recall drops.Semantic-target groups differ in region size, boundary gradients, and high-frequency energy; operation shifts affect boundary and core recall differently, while composition can redirect attention and yield incomplete or expanded masks.Source: §3.3 · p. 5

3.4 External Generalization of GIFTBench-trained Models

• SparseViT trained on GIFTBench achieves the strongest Avg.-All F1 of 0.6029 among evaluated benchmark-trained baselines, while gains are target-dependent.GIFTBench training improves aggregate external performance for all evaluated architectures relative to their reference training distributions; no training distribution uniformly dominates across external datasets.Source: §3.4 · pp. 5–6, Table 2 · p. 6

4.1 Overview

• ForenScope keeps image-level detection and pixel-level localization separately predicted, using the CLS representation for detection and intermediate patch features for spatial evidence.The localization branch applies bounded CLS conditioning and coarse-to-fine decoding without making the mask depend on a hard detection decision.Source: §4.1 · pp. 6–7

4.2 Multi-Depth Layer–Scale Fusion

• At each of three scales, independent LayerMixers combine projected features from four encoder depths using learned, scale-specific depth mixtures.Separate normalization lets the decoder use different depth mixtures for coarse and fine spatial evidence.Source: §4.2 · p. 7

4.3 Global-Conditioned Coarse-to-Fine Localization

• Bounded residual FiLM injects CLS context into the two coarser scales, while leaving the finest pathway unconditioned; the decoder then fuses scales coarse-to-fine to predict a manipulation map.This limits global conditioning to broader context while preserving a high-resolution path for fine-grained evidence.Source: §4.3 · pp. 7–8

4.4 Training and Inference

• The classification-adapted encoder and classification head are frozen during localization training, while localization modules learn from pixel-level supervision and auxiliary heads supervise all three spatial pathways.An optional inference-only prototype refinement adjusts direct predictions using confident manipulated and real regions, with no target-domain labels or parameter updates.Source: §4.4 · p. 8

5.1 Experimental Setup

• The GIFTBench and CASIAv2 settings use separate training data and adapted checkpoints; OOD datasets are reserved for evaluation, and checkpoints are selected on corresponding-domain validation data.Unless otherwise stated, ablations use the GIFTBench setting.Source: §5.1 · p. 9

5.2 Overall Performance

• ForenScope achieves the highest average F1 among compared direct-output models across nine OOD datasets under GIFTBench and Protocol-MVSS; prototype refinement is strongest overall in Protocol-MVSS.Its relative advantage varies across individual OOD datasets.Source: §5.2 · p. 9
• 0.935 average ROC AUC and 0.871 F1 on eight external datasets containing real and manipulated images for the GIFTBench-adapted detector.The detector retains the image-level classifier inherited from the classification-adapted encoder.Source: §5.2 · p. 9

5.3 Ablation Study

• F17-only causes the largest structural drop; single-scale variants underperform the full model, while removing CLS-FiLM has a smaller effect and fixed prototype refinement improves average without retraining.The w/o-ADE result supports using classification-adapted representations.Source: §5.3 · p. 10, Table 4 · p. 10

• GIFTBench exposes distinct transfer behavior across manipulation source, semantic target, operation, and composition while supporting twelve-dataset external evaluation.Source: §6 · p. 10
• ForenScope uses learned depth–scale fusion and bounded global conditioning in a detection–localization model.Source: §6 · p. 10
• ForenScope improves cross-dataset localization while retaining image-level detection capability.Source: §6 · p. 10

• Generative AI assisted with polishing manuscript writing, identifying relevant literature, and drafting portions of the paper.Source: AI Use Statement · p. 10
• The authors say all AI-assisted content, including factual claims and references, was reviewed, revised, and verified.Source: AI Use Statement · p. 10
• The authors take full responsibility for the final content.Source: AI Use Statement · p. 10

• The appendix specifies what GIFTBench contains and how manipulated samples are constructed.Source: A Benchmark Construction and Data… · p. 11
• It describes how split membership, masks, and provenance are recorded.Source: A Benchmark Construction and Data… · p. 11
• Its details support reconstructing protocol memberships.Source: A Benchmark Construction and Data… · p. 11

A.1 Scope, taxonomy, and task definition

• GIFTBench uses ADE20K images and pixel masks, with source, semantic-target, operation, and composition labels that support multiple protocol views without changing supervision.Source: §A.1 · p. 11

A.2 Construction pipeline

• Construction combines expert-defined constraints, structured VLM manipulation plans, family-specific editing engines, and checks of image, mask, and metadata.Source: §A.2 · pp. 11–12

A.3 Manipulation source instantiation

• The source families include splicing, Copy–Move, removal, and AI-Edit, with editing methods selected or varied by family and target.Source: §A.3 · p. 12

A.4 Composition construction

• Fusion refines a manipulation boundary, whereas Multi-Source combines two to four components; the paper’s Protocol IV baselines train on GIFTBench-Core and measure zero-composition generalization.Source: §A.4 · p. 13

A.5 Quality assurance, masks, and provenance

• Quality controls filter small or unsuitable edits, use automated and manual review, and release binary masks and construction records; twelve invalid candidates were excluded, leaving 115,013 samples.Source: §A.5 · p. 13

A.6 Dataset composition and split integrity

• Primary, donor, and component sources remain within the same train/test pool; the release has 103,319 training and 11,694 test samples, but does not distribute original ADE20K images.Source: §A.6 · p. 14, Table 6 · p. 14

B.1 Protocol definitions; B.2 Metric and aggregation definitions

• The evaluation reports per-image pixel F1 and IoU averaged within groups, macro-averages external datasets equally, and keeps image-level detection AUC separate from localization masks.Source: §B.2 · p. 15, §B.1 · p. 15

B.3 Diagnosis-to-design mapping; B.4 Supplemental diagnostic visualizations

• The diagnostic matrices expose transfer variation but do not isolate semantic shift from source variation; the design mapping is testable motivation, and supplemental radar profiles are descriptive rather than causal.Source: §B.3 · p. 16, §B.4 · p. 16, Figure 6 · p. 16

C.5 Controlled composition/reference results

• Protocol IV compares Fusion and Multi-Source composition scores with matched single-operation references, not independently sampled test distributions.This is a benchmark-protocol reference evaluation, not a ForenScope ablation or external-transfer result.Source: §C.5 · p. 19

C.6 Robustness under image degradation

• Existing stress tests evaluate single-operation images after JPEG compression, Gaussian blur, resize–JPEG, and double-JPEG processing.Table 14 defines the perturbations; Figure 7 shows benchmark-level curves for JPEG and Gaussian blur, not new exp021 Full-Late measurements.Source: §C.6 · p. 19, Table 14 · p. 19, Figure 7 · p. 19

D.1 Classification adaptation with LoRA

• DINOv2 is first adapted with LoRA and an image-level classification head; the adapted encoder and head are then frozen during localization training.The standard configuration uses ADE-domain classification data; the controlled Protocol-MVSS configuration uses CASIAv2.Source: §D.1 · p. 20, §D.2 · p. 20

D.2 Initialization and trainable modules

• The no-CLS/no-ADE-adaptation variant jointly changes encoder initialization and CLS conditioning, so it does not isolate either factor.The localization branch removes the CLS and four register tokens, using only patch tokens as spatial features.Source: §D.2 · p. 20, §D.3 · p. 21

D.4 Multi-scale projection and learnable fusion

• Four tapped encoder maps are projected into three spatial resolutions, creating twelve layer–scale paths for localization.At each scale, a learned softmax-weighted mixture combines the four depths; its coefficients are shared across images at inference.Source: §D.4 · p. 21

D.5 CLS-conditioned localization and decoder

• CLS-derived conditioning modulates the H19 and H37 features, while the unconditioned H74 path preserves access to the highest-resolution spatial evidence.The decoder fuses coarse conditioned features before combining them with H74 and progressively upsamples to the image resolution.Source: §D.5 · p. 21

D.6 Objective, sampling, augmentation, optimization, and selection

• Localization training uses binary cross-entropy plus soft Dice, weighted sampling of manipulated and real images, and in-domain pixel F1 for checkpoint selection.The protocol uses five epochs and does not use external-dataset labels for optimization, validation, or checkpoint selection.Source: §D.6 · pp. 21–22

D.7 Prototype-guided inference

• Prototype refinement is an optional inference-only correction using each test image’s predictions and H37 features, with no target-domain labels or parameter updates.It forms manipulated and real support prototypes, applies a cosine-similarity margin correction, and thresholds the refined output at 0.5.Source: §D.7 · p. 22

E.1 Evaluation protocol for ForenScope

• The external localization comparison reports direct ForenScope output separately from the same checkpoint with fixed prototype refinement.In-domain evaluation uses thirteen masked groups organized by source, semantic target, operation, and composition.Source: §E.2 · p. 23, Table 16 · p. 23, §E.1 · p. 23

E. Complete ForenScope Evaluation and Ablations

• Evaluation covers twelve external localization datasets and image-level detection on eight external datasets containing both real and manipulated images; detection is documented as retained capability, not a standalone leaderboard.In-domain results are also organized by manipulation source, semantic target, operation, and composition.Source: §E.3 · p. 23, §E.1 · p. 23, §E.2 · p. 23

E.4 Ablations and mechanism analysis

• Retrained structural results indicate that ADE adaptation and multi-scale evidence improve transfer, while removing CLS-FiLM has a smaller effect; the joint no-CLS/no-adaptation control isolates neither factor.The pure no-adaptation variant is reported on All-12 but has no thirteen-group in-domain result.Source: §E.4 · p. 24, Table 19 · p. 24
• Fixed-checkpoint diagnostics test learned fusion and CLS conditioning; the Original Full-Late reference is not the Full baseline for the final Static LayerMixer table.A matched FiLM intervention on the final Static LayerMixer checkpoint would require a separate inference run.Source: §E.4 · p. 24, Table 20 · p. 24
• Prototype refinement improves the external All-12 average, but its in-domain gain is not uniform across benchmark groups.It is an inference intervention, distinct from the independently retrained structural variants.Source: §E.4 · pp. 24–25, Table 20 · p. 24

E.5 Reproducibility checklist

• The reproducibility record specifies the model, resolution, feature taps, scales, seed, checkpoint policy, and evaluation manifests; prototype inference uses fixed settings without target labels or parameter updates.Checkpoint selection uses in-domain data, with no target-domain fine-tuning.Source: §E.5 · p. 25

F.1 Dataset-level transfer and representation views

• Figure 8 gives a descriptive two-dimensional projection of TruFor decoder features under different training distributions.The passage describes the visualization but states no outcome to infer from it.Source: §F.1 · p. 25, Figure 8 · p. 25

F.2 External qualitative comparisons

• Figure 9 compares representative external-transfer examples from Protocol-MVSS, Protocol-CAT, and GIFTBench training; the examples do not guide model selection.The caption identifies the qualitative comparison, not a quantitative winner.Source: §F.2 · p. 25, Figure 9 · p. 26

F.3 Good and bad cases on GIFTBench

• Figure 10 contrasts high-agreement cases with examples where every displayed baseline has F1 below 0.15.The figure presents both shared successes and difficult benchmark cases.Source: §F.3 · p. 26, Figure 10 · p. 26