PaperlayerView exact revision on arXiv

Source-linked AI summary

Relative Patch Response Learning for Generalizable AI-Generated Image Detection

Tianyu Wang, Ouxiang Li, Yanbin Hao, Zhenhua Tang, Shuo Wang

arXiv:2610.11876v1cs.CV

TL;DR

AI-generated image detection must generalize to unseen generators and withstand post-processing, while contextual shift makes source-only labels imprecise for mixed-view patches. PRL compares patch-score responses across two mixed views of an aligned pair, using same-source patches as references. It surpasses prior methods across eight standard and three in-the-wild benchmarks.

  • Problem

    Detection must generalize to unseen generators and remain effective after post-processing, while contextual shift makes mixed-view per-patch source labels imprecise.

    Source: §1 · pp. 1–2

  • Method

    PRL compares each patch's score across two mixed views of an aligned pair, measuring source-changed patches against same-source references and adding coherence and area-ranking objectives.

    Source: §1 · pp. 2–3

  • Results

    PRL surpasses the best previous methods by 4.3% and 5.9% in average balanced accuracy across eight standard and three in-the-wild benchmarks, respectively.

    Source: §1 · p. 3

  • Takeaways & Limitations

    PRL also transfers to locally manipulated image detection with no additional training data.

    Source: §1 · p. 3

Abstract

from arXiv · show

Generative models can now synthesize highly realistic images, simultaneously increasing the risks of misinformation and visual forgery. Therefore, detecting AI-generated images becomes more essential, and a reliable detector must generalize to unseen generators and stay robust to unseen perturbations in the wild. Existing detectors are typically trained on either independently collected real and generated images or aligned real-generated pairs designed to mitigate content bias. Building on aligned pairs, recent methods form a mixed view by replacing some patches of the real image with their generated counterparts. However, we find that self-attention lets real and generated patches interact, so the feature of each patch no longer reflects its own source alone. This contextual shift makes a per-patch source label an imprecise target. To this end, we propose Relative Patch Response Learning (PRL). Instead of labeling each patch, PRL compares the same patch across two mixed views of an aligned pair and learns from its patch response, the change of its score between the views. (i) To give precise supervision under the contextual shift, a relative response objective measures the responses of source-changed patches against those of source-unchanged patches, which respond to the shift alone. (ii) To provide a reliable reference for the shift, a reference coherence objective keeps each group of source-unchanged patches moving as a whole. (iii) Since the two views contain different amounts of generated content, an area ranking objective asks the view with the larger generated area to have a higher mean patch score. Extensive experiments demonstrate the superior performance of PRL, which surpasses the best prior methods by 4.3% and 5.9% in average balanced accuracy across eight standard and three in-the-wild benchmarks, respectively.

1 INTRODUCTION

AI-generated image detectors must generalize to unseen generators and remain robust to post-processing, but mixed-view patch labels do not account for contextual shifts in patch features. PRL instead compares patch-score responses across two mixed views of an aligned pair, and outperforms prior methods across standard and in-the-wild benchmarks.

1 INTRODUCTION

  • Detectors must generalize to unseen generators and remain effective after images are resized, compressed, and reshared online.These challenges matter amid risks of misinformation and visual forgery from increasingly realistic generated images.Source: §1 · p. 1
  • Self-attention lets generation traces spread into real patches, so a per-patch source label cannot separate a patch's own-source trace from contextual shift.Figure 2 shows DINOv3 and PPL responding outside the replaced region almost as much as inside it.Source: §1 · p. 2, Figure 2 · p. 3
  • PRL builds two mixed views of an aligned pair and supervises changes in each patch's score rather than assigning each patch a source label.Source-unchanged patches provide a reference for the contextual shift affecting source-changed patches.Source: §1 · p. 2
  • Relative response, reference coherence, and area ranking respectively compare changed patches with references, stabilize reference groups, and rank views by generated area.The view with the larger generated area is asked to score higher.Source: §1 · pp. 2–3
  • PRL surpasses the best previous methods by 4.3% and 5.9% in average balanced accuracy across eight standard and three in-the-wild benchmarks, respectively.PRL also transfers to locally manipulated image detection with no additional training data.Source: §1 · p. 3

2 RELATED WORK

Prior detectors use generation traces or pretrained representations, while alignment-based methods pair real images with matched reconstructions to mitigate content bias. Mixed-view methods add patch-level supervision, but PRL instead compares patch responses across views to account for contextual shift.

2 RELATED WORK

  • Detectors use forensic traces or pretrained vision-foundation-model representations, and are typically trained on separately collected real and generated images.Aligned methods pair matched-content images to reduce content shortcuts, but still train the pair as independent samples with image-level labels.Source: §2 · pp. 3–4
  • Aligned-pair methods improve generalization across generators over separately collected training images, yet do not use pair correspondence as supervision.DRCT, B-Free, AlignedForensics, and DDA construct aligned samples through reconstruction, regeneration, spatial correspondence, or pixel interpolation.Source: §2 · p. 4
  • Mixed-view methods include image-level supervision and PPL's per-patch source labels, whereas PRL measures each patch's response against same-source patches across two views.PRL's overview identifies relative response, reference coherence, and area ranking as its training objectives.Source: §2 · p. 4, Figure 3 · p. 4

3 METHOD

PRL trains a ViT detector on how aligned patches change score between two mixed views, rather than assigning each patch a source label. Its objectives use unchanged-source patches as references, keep those references coherent, and rank views by generated area.

3 METHOD

  • PRL supervises a ViT detector using patch-score changes between two mixed views of an aligned real–synthetic pair.A linear head scores each patch, and its change between views is the patch response.Source: §3 · p. 4, §3.2 · p. 5

3.1 ALIGNED REGION MIXING

  • Random binary masks select aligned real or synthetic patches to create mixed views, and two sampled masks are chosen with different synthetic-patch counts.Patches are partitioned into source-changed sets and same-source reference sets across the views.Source: §3.1 · p. 5

3.2 RESPONSE LEARNING

  • For unchanged-source patches, score responses reflect contextual shift; PRL compares changed patches with same-source references exposed to the same contextual change.The reference response estimates the shift, which is subtracted from changed-patch responses; the objective requires synthetic-in-the-large-view patches to respond more than real-in-the-large-view patches.Source: §3.2 · pp. 5–6
  • Reference coherence penalizes deviations from each unchanged-source set’s mean response, allowing a reference to shift with context only as a whole.This makes the set mean a more reliable estimate of contextual shift.Source: §3.2 · p. 6
  • Area ranking asks the view with more synthetic patches to have a higher mean patch score.The loss evaluates score gain per added synthetic patch and bounds the gradient for any area gap.Source: §3.2 · p. 6
  • Training combines image-level binary cross-entropy with relative-response, reference-coherence, and area-ranking losses; inference scores a single image.The image-level head weights and pools tokens to predict whether an image is synthetic.Source: §3.2 · p. 6

4 EXPERIMENTS

PRL achieves the best reported results across eleven standard and in-the-wild benchmarks, with additional evaluations on locally manipulated images, post-processing robustness, and ablations.

4.1 EXPERIMENTAL SETUP

  • Training uses aligned MSCOCO images and Stable Diffusion 2.1 VAE reconstructions, with a frozen DINOv3 ViT-L/16 backbone adapted by LoRA and JPEG quality 96 debiasing where needed.Evaluation spans eight standard and three in-the-wild benchmarks; balanced accuracy averages accuracy on real and synthetic images.Source: §4.1 · pp. 6–7

4.2 CROSS-BENCHMARK COMPARISON

  • 95.67% average balanced accuracy, 6.5% above DDA (89.16%) and 6.7% above GAPL (88.93%), with PRL best on all eleven benchmarks.On Chameleon, PRL reaches 90.01% versus 82.41% for the strongest prior method.Source: Table 1 · p. 7, §4.2 · p. 7
  • On ForenSynths, PRL reaches 92.57% versus 81.41% for DDA and 51.47% for AlignedForensics; on DRCT-2M it scores 99.27% versus DDA’s 98.47%.PRL is below 90% on five subsets, including DeepFake (55.26%), SITD (84.72%), SAN (88.81%), and SD3.5 and Flux reconstructions (85.17% and 85.19%).Source: §4.2 · pp. 7–8
  • PRL exceeds 90% on all three in-the-wild benchmarks and every subset, with its largest prior-method gap on Chameleon.The benchmarks contain web-collected images with unknown generators and post-processing.Source: §4.1 · p. 7, Table 3 · p. 8, §4.2 · p. 8

4.3 COMPARISON ON LOCALLY MANIPULATED IMAGES

  • On AutoSplice, PRL reaches 86.45% at JPEG quality 100 and remains best at qualities 90 and 75; on DailyBench’s ManipulationBench subset, its average is 68.32% versus DDA’s 66.88%.These evaluations test detection of regions where generated content replaces part of a real image.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 EVALUATION ON ROBUSTNESS

  • Across fourteen degraded settings, PRL averages 92.1% balanced accuracy versus 91.7% for GAPL and 90.6% for DDA, but trails both under the heaviest downscaling and JPEG degradation.It stays above 95% through 2× upscaling and above 90% through blur σ = 1.5 and JPEG quality 80.Source: Figure 4 · p. 8, §4.4 · p. 9

4.5 ABLATION STUDIES

  • On four ablation benchmarks, PRL averages 95.55%, 2.9% above image-level labels and 3.7% above per-patch labels; removing an objective costs 1.4–2.5% on average and 3.2–6.8% on Chameleon.Removing relative response loss Lrel costs the most; on the same DINOv3 backbone, PRL reaches 95.67% versus adapted DDA’s 93.20%.Source: Figure 5 · p. 9, §4.5 · p. 9

5 CONCLUSION

Mixed-view self-attention shifts patch features with context, making per-patch source labels imprecise. PRL instead learns from patch-score changes across views of aligned pairs and outperforms prior methods across standard and in-the-wild benchmarks.

5 CONCLUSION

  • Self-attention lets patches of different sources interact, so each patch feature carries information from the other source and per-patch labels become imprecise.A patch’s feature reflects both its own source and contextual shift.Source: §5 · pp. 9–10
  • PRL compares each patch’s score across two mixed views of an aligned pair, learning from response changes rather than per-patch source labels.Source-unchanged patches respond to contextual shift alone; PRL uses their responses as a reference for source-changed patches.Source: §5 · p. 10
  • PRL surpasses the best prior methods by 4.3% and 5.9% in average balanced accuracy on eight standard and three in-the-wild benchmarks, respectively.Source: §5 · p. 10

A IMPLEMENTATION DETAILS

Implementation uses a frozen DINOv3 ViT-L/16 backbone adapted with LoRA, a patch scoring head, and a token-gating detection head. Paired augmentations preserve alignment, and training samples mixed views using rectangular patch masks.

A IMPLEMENTATION DETAILS

  • A frozen DINOv3 ViT-L/16 backbone uses rank-8 LoRA in every transformer block, with linear patch scoring and a gating-MLP detection head.LoRA is applied to attention and MLP projections; the detection head weights and aggregates tokens before mapping them to an image logit.Source: A IMPLEMENTATION DETAILS · p. 14
  • Training and inference center-crop images to 336 × 336 pixels, resizing instead when both dimensions exceed 1,024 pixels, and apply common augmentations identically within each pair.Augmentations include random JPEG compression, Gaussian blur, and colour jitter.Source: A IMPLEMENTATION DETAILS · p. 14
  • Training samples two rectangular-mask views on a 21 × 21 patch grid, using either the real image or synthetic image as their shared base.The synthetic fraction is drawn uniformly from [0.1, 0.9], and the two masks are resampled until their fractions differ.Source: A IMPLEMENTATION DETAILS · p. 14

B BENCHMARKS

The benchmarks cover standard generator collections and in-the-wild images, including datasets with unknown generators, varied image transformations, and edited real photographs. Some aligned benchmarks pair generated images with shared real sets or frequency-aligned reconstructions.

B BENCHMARKS

  • GenImage pairs ImageNet photographs with images from eight generators, including ADM, BigGAN, Glide, Midjourney, Stable Diffusion, VQDM, and Wukong.Source: B BENCHMARKS · p. 14
  • AIGCDetectBenchmark combines seventeen GAN and diffusion subsets with real images from LSUN, ImageNet, MSCOCO, CelebA, and FFHQ.Source: B BENCHMARKS · p. 14
  • DRCT-2M pairs a shared MSCOCO real set with images from sixteen diffusion models spanning text-to-image, ControlNet, and inpainting generation.Source: B BENCHMARKS · p. 14
  • DDA-COCO frequency-aligns MSCOCO validation images with reconstructions from six VAEs, while EvalGEN contains 2,765 synthetic images from five generators without real counterparts.EvalGEN reports accuracy on its synthetic images.Source: B BENCHMARKS · p. 14
  • Synthbuster provides 1,000 images from each of nine generators.Source: B BENCHMARKS · p. 14
  • ForenSynths contains thirteen CNN-based generator subsets, while UnivFD extends it with eight diffusion and autoregressive subsets for nineteen total.The listed generators include ProGAN, StyleGAN, BigGAN, CycleGAN, Guided, LDM, Glide, and DALL-E.Source: B BENCHMARKS · p. 14
  • The in-the-wild sets include unknown-generator collections and manipulated or edited images, spanning varied platforms, formats, compressions, and editing transformations.WildRF draws from Reddit, Facebook, and X; Chameleon has web-collected images that passed a human perception test; SynthWildX attributes images to three named generators. AutoSplice replaces an object region, and DailyBench’s used subset contains edits from six editors.Source: B BENCHMARKS · p. 15

C COMPARED METHODS

Compared methods use varied detection strategies, including pretrained feature classifiers, frequency cues, aligned reconstructions, and generator-aware representations. The supplied table captions identify balanced-accuracy comparisons on GenImage and AIGCDetectBenchmark.

C COMPARED METHODS

  • UnivFD freezes a CLIP image encoder and trains a linear classifier; FatFormer adds forgery-aware adapters and a frequency branch aligned with text prompts.Source: C COMPARED METHODS · p. 15
  • NPR classifies neighboring-pixel relations associated with generator up-sampling, while SAFE uses wavelet high-frequency components with trace-preserving image transformations.Source: C COMPARED METHODS · p. 15
  • DRCT reconstructs real and generated images with diffusion to create same-content hard samples, then trains with classification and contrastive losses.Source: C COMPARED METHODS · p. 15
  • C2P-CLIP injects category-common prompts into CLIP, while AIDE combines DCT-selected frequency patches processed by a CNN with CLIP semantic features.Source: C COMPARED METHODS · p. 15
  • AlignedForensics pairs each real image with its VAE reconstruction; DDA aligns generated and real images in pixel and frequency domains.Source: C COMPARED METHODS · p. 15
  • Effort separates CLIP weights into orthogonal subspaces, freezing the principal subspace and training the residual; GAPL maps image embeddings to generator-aware prototypes.Source: C COMPARED METHODS · pp. 15–16
  • The supplied captions specify balanced-accuracy comparisons on GenImage and AIGCDetectBenchmark, without reporting table values in these passages.Source: Table 5 · p. 16, Table 6 · p. 16

D DETAILED BENCHMARK RESULTS

PRL scores highly across detailed benchmark subsets, including generators and image types that remain difficult for competing methods. On the ManipulationBench subset of DailyBench, it has the best average, though other detectors lead on some editors.

D DETAILED BENCHMARK RESULTS

  • PRL exceeds 98.8% on every DRCT-2M subset and 99% on every EvalGEN generator; its two lowest DDA-COCO subsets are SD3.5 and Flux reconstructions.The supplied table captions identify Tables 8 and 9 as covering DRCT-2M and DDA-COCO/EvalGEN, respectively.Source: D DETAILED BENCHMARK RESULTS · p. 16, Table 8 · p. 17, Table 9 · p. 18
  • On ForenSynths, PRL’s margin over alignment-based methods widens on every GAN generator, while DeepFake, SITD, and SAN remain its hardest subsets.Table 10 reports balanced accuracy on ForenSynths.Source: D DETAILED BENCHMARK RESULTS · p. 16, Table 10 · p. 18
  • 68.32% balanced accuracy is PRL’s best average on DailyBench ManipulationBench, ahead of DDA at 66.88% and FPD at 63.41%.PRL also leads on the three FLUX-based editors; DDA is stronger on Qwen-Image-Edit and Step1X-Edit, and SAFE on GPT-Image 2.Source: D DETAILED BENCHMARK RESULTS · p. 16, Table 11 · p. 18

E MORE ABLATION STUDIES

The ablations examine variants of aligned-pair training and compare DDA and PRL under matched backbone and adaptation settings. PRL’s default loss weights perform best on each tested axis, with nearby nonzero settings remaining close.

E MORE ABLATION STUDIES

  • Table 13 compares released DDA on DINOv2 with DDA adapted to PRL’s DINOv3-and-LoRA setup and PRL on the same eleven benchmarks.The caption specifies that the compared methods use the same aligned pairs for the matched-backbone comparison.Source: E MORE ABLATION STUDIES · p. 18, Table 13 · p. 19
  • Table 12 covers three ways of using aligned pairs and PRL variants with one objective removed, using the same DINOv3 backbone, LoRA, and training pairs.The supplied passage identifies the table’s scope but does not state the individual ablation values.Source: E MORE ABLATION STUDIES · p. 18, Table 12 · p. 19
  • The default loss-weight setting is best on every tested axis; nonzero weights from 0.1 to 0.5 stay within 1.5% of it, and every setting beats the image-level label.Figure 6 varies one weight at a time over four benchmarks while holding the other two at defaults (λrel, λcoh, λrank) = (0.3, 0.3, 0.1).Source: E MORE ABLATION STUDIES · pp. 18–19, Figure 6 · p. 19
Loading 2610.11876v1…