PaperlayerView exact revision on arXiv

Source-linked AI summary

Look Closer: Patch-wise Supervision for AI-Generated Image Detection

Zhida Zhang, Tao Wu, Siyu Liu, Jie Cao

arXiv:2609.37937v2cs.CV

TL;DR

The paper asks how much image context a detector needs to see, then studies explicit RGB crops trained with patch-wise supervision. PWS reports higher GenImage means than whole-image counterparts across four tested backbones, though evaluation selection histories limit uniform comparison.

  • Problem

    Whole-image detection combines local appearance with object identity and scene composition, while resizing may alter fine-scale evidence; the study asks how much of an image a detector needs to see.

    Source: §1 · p. 1

  • Method

    PWS trains a shared classifier on explicit crops with a separate loss for each patch, then averages crop probabilities at inference.

    Source: §7 · p. 9

  • Results

    PWS reports higher GenImage means than whole-image baselines across four tested backbones, with historical selection limits on common-protocol conclusions.

    Source: §1 · p. 1, §7 · p. 9

  • Takeaways & Limitations

    The study supports treating spatial input and supervision granularity as useful design choices, without assuming every architecture or distribution will benefit.

    Source: §6 · p. 8

    Historical checkpoint and run selection was not uniform or fully recovered, and some results used target evaluation performance for selection.

    Source: §5.1 · p. 5

Abstract

from arXiv · show

How much of an image does a detector need to see? Small RGB regions can retain useful evidence of image synthesis even when they reveal little of the full scene. Motivated by single-patch detection, we study patch-wise supervision: a shared backbone classifies explicit crops, each crop receives its own loss, and patch probabilities are averaged only at inference. The procedure requires neither handcrafted residual filtering nor a learned image-level fusion module. Experiments span single-patch selection, multiple generator collections, and four CNN and Transformer backbones. On GenImage, the reported patch-wise variants improve average accuracy over their whole-image counterparts across all four backbones. Comparisons of supervision granularity, source resolution, crop size, and inference coverage further characterize the approach, while post-processing tests and difficult-image evaluation reveal its limitations. The historical experiments include evaluation-based model selection, so their scores are not presented as a uniformly selected leaderboard comparison. Overall, the study identifies explicit local input and patch-level supervision as a simple, useful combination for investigating generalizable AI-generated image detection.

1 Introduction

The paper asks how much image context a detector needs and studies explicit RGB crops as a way to learn local evidence. It extends single-patch observations to patch-wise supervision and evaluates that approach across backbones and datasets.

1 Introduction

  • Whole-image inputs combine local appearance with object identity and scene composition, while resizing may alter fine-scale detection evidence.The motivating question is how much of an image a detector should see at once.Source: §1 · p. 1
  • The study extends single-patch detection by asking whether other regions can be informative and whether multiple regions can be used without a fusion network.Its motivation draws on earlier EIB-Net [17], which selects a low-complexity patch and uses an information bottleneck.Source: §1 · p. 1
  • PWS gives each explicit RGB crop its own classification loss and averages crop probabilities only at inference.The approach uses a shared classifier without learned within-image feature fusion or a required residual filter.Source: §1 · p. 1
  • The experiments cover four backbones, GenImage and AIGCDetectBenchmark, and studies of supervision, resolution, crop size, coverage, and failure cases.Historical model selection was not uniform, so the reported scores are not a newly standardized leaderboard evaluation.Source: §1 · pp. 1–2
  • PWS has a higher reported mean than whole-image baselines for each of four GenImage backbones, though not every generator subset.The introduction points to the model-selection qualifications for these results.Source: Figure 1 · p. 2

2 Related Work

Prior work explores local classification, patch selection, texture cues, and other representations for generated-image detection. This paper distinguishes its implementation and empirical scope while avoiding a claim that local supervision is new.

2 Related Work

  • Related detection approaches use CNN transfer [13], pretrained representation spaces [10], diffusion reconstruction error [14], or neighboring-pixel relationships [12].PWS instead leaves representation to a conventional RGB classifier while constraining spatial input and supervision.Source: §2 · p. 2
  • Patch Forensics [1] restricts CNN receptive fields and uses local losses with inference-time probability averaging, including facial-image experiments.This paper instead imposes locality through explicit crops while retaining backbone feature-extraction stages, and evaluates natural-image generator collections and a windowed Transformer.Source: §2 · p. 2
  • PatchCraft [18] exploits rich- and poor-texture regions, while SSP [2] selects a low-complexity patch and uses SRM filtering.SSP ablations also include direct RGB-patch classification without SRM; Panoptic Patch Learning [16] studies patch interventions with a contrastive objective.Source: §2 · p. 2
  • EIB-Net [17] uses entropy-guided single-patch selection with a variational information bottleneck; this paper moves toward several regions with one classifier.The present single-patch settings are not the complete EIB-Net system, and cross-paper scores are not a controlled ablation of patch count.Source: §2 · p. 2
  • GenImage [20] and AIGCDetectBenchmark [18] test across generator families, while Chameleon [15] emphasizes high-quality images and reveals settings where local-input detection remains unreliable.DIFF [3] and DiffusionForensics [14] provide supporting facial-image and cross-content experiments.Source: §2 · p. 2

3 From Single-Patch Evidence to Multiple Observations

The section motivates local inputs as a spatial constraint, tests single-patch selection strategies, and explains why multiple crops with separate losses address dependence on one selected region.

3.1 Local input as an inductive bias

  • Limiting spatial context may make local evidence easier to use while reducing access to full-image composition, but does not guarantee semantic-free features.Nearby-pixel relationships motivate local inspection but are not a universal signature; photographs also have spatial dependencies and processing effects.Source: §3.1 · p. 2

3.2 What the single-patch study tests

  • The single-patch protocol selects one 64 × 64 RGB crop from a 4 × 4 candidate grid, comparing minimum-, middle-, maximum-complexity, and random selection.The complexity score aggregates directional filter responses; the classifier receives the RGB patch, not those responses.Source: §3.2 · pp. 2–3
  • Random selection is competitive in the full-data study, while maximum-complexity selection is less accurate but still informative.Useful evidence is not confined to the minimum-complexity region, though this does not imply all patches are equally useful.Source: §3.2 · p. 3

3.3 Why multiple patches and separate losses?

  • Multiple crops reduce dependence on one potentially ambiguous region, while patch-level training makes every local prediction a training unit and the image the evaluation unit.Coverage and supervision are distinct: a multi-patch model could instead train on a pooled image representation.Source: §3.3 · p. 3

4 Patch-wise Supervision

Patch-wise supervision samples explicit crops, shares a classifier across them, and applies loss per patch before image-level prediction. The section distinguishes this training objective from pooled-logit supervision and notes computational and visualization constraints.

4.1 Sampling and input representation

  • Candidate count, training-patch count, and inference-patch count are distinct; selecting one of sixteen candidates still yields one training example.Evaluating one crop from a multi-patch-trained detector is a different experiment.Source: §4.1 · p. 3
  • The input path separates image-level processing, crop extraction, and patch conversion, normalization, or resizing.Cropping before global downsampling differs from cropping after it; the nominal crop side is 64 pixels, stride 32, with a cap of 64 selected patches.Source: §4.1 · p. 3

4.2 A shared classifier, one loss per patch

  • Every patch inherits its parent image’s label, and the shared classifier is trained with focal loss [7].If patch counts vary, the stated objective weights images in proportion to their patch counts; inherited labels are not local manipulation annotations.Source: §4.2 · p. 3

4.3 Where supervision differs from pooling

  • Pooled-logit supervision can let correct, confident patch predictions compensate for poor patches, whereas per-patch losses directly penalize poor local predictions.The nonlinear loss on pooled logits generally differs from averaging individual losses.Source: §4.3 · p. 3

4.4 Image-level prediction and score maps

  • At inference, patch probabilities—not logits—are averaged, and mapped predictions visualize classifier confidence rather than ground-truth artifact locations.Fixed normalization statistics permit patch-wise chunking and score accumulation.Source: §4.4 · p. 4

4.5 Architecture and computation

  • Explicit crops constrain visual field without truncating feature extraction, but multiple crops are not automatically faster and more crops need not improve accuracy monotonically.Backbones require valid input-size configurations, and cost depends on input size and patch count.Source: §4.5 · p. 4

5 Experiments

The experiments test local evidence, patch-wise supervision, and their behavior across generators, backbones, input settings, and difficult evaluations. Reported results favor PWS on GenImage, while selection-history and robustness limitations constrain interpretation.

5.1 Evaluation settings

  • GenImage and AIGCD means include the training source; checkpoint selection was uneven, and some reported scores used target-evaluation scans, so differences do not establish gains under a common protocol.For the Xception Chameleon run, the saved-best-checkpoint evaluation is 68.03 versus 71.29 from the Chameleon checkpoint sweep.Source: §5.1 · pp. 4–5

5.2 From one local observation to multiple patches

  • Random 64×64 crops reached 98.78% accuracy versus 99.01% for whole-image input, while maximum-complexity crops retained 94.91% accuracy.The single-patch study selects one crop from sixteen non-overlapping candidates; that crop covers 6.25% of the resized image canvas.Source: §5.2 · p. 5, Table 1 · p. 5
  • Whole-image training showed a larger train–validation gap than patch-based runs, whose trajectories tracked validation performance more closely.The curves describe observed training behavior and do not identify which visual cues the models use.Source: §5.2 · p. 5, Figure 5 · p. 6

5.3 Performance across generators and backbones

  • Reported GenImage mean accuracies favor PWS over whole-image input for all four backbones: ResNet-18 93.97 vs 72.63, ResNet-50 95.40 vs 75.06, Xception 95.21 vs 79.42, and Swin-T 93.58 vs 77.84.On AIGCD, PWS mean accuracy ranges from 88.41 to 93.43 across backbones; the smaller ResNets do not exceed all external baselines.Source: §5.3 · p. 5, Table 2 · p. 6

5.4 What changes when the supervision unit changes?

  • On GenImage, patch-level supervision outperformed image-level supervision on all three compared backbones: ResNet50 95.40 versus 72.40, Xception 95.21 versus 80.19, and Swin-T 93.58 versus 77.70.Both objectives process crops; PWS applies loss to each patch, whereas image-level supervision averages patch logits before the loss. The comparison does not isolate semantic bias as the cause.Source: §5.4 · p. 6, Table 4 · p. 7

5.5 Resolution, patch size, and inference coverage

  • In the aligned-resolution comparison, PWS means exceeded whole-image means by 10.15 percentage points for ResNet-50 and 6.01 for Xception; 64×64 had the highest Xception mean on both benchmarks.These historical differences do not isolate source resolution or supervision effects; 64×64 is a useful setting in these experiments, not a universal optimum.Source: §5.5 · pp. 6–7, Table 5 · p. 7, Table 6 · p. 7
  • On AIGCD, mean accuracies for one, sixteen, and sixty-four inference crops were 87.33, 92.45, and 93.02, respectively, but did not rise monotonically at larger counts.The result suggests a coverage–cost trade-off rather than a general rule that more crops improve accuracy.Source: §5.5 · p. 7, Table 7 · p. 7

5.6 Processing sensitivity and difficult images

  • Processing reduced PWS Swin-T accuracy from 93.43 clean to 80.71 with JPEG, 86.83 with downsampling, and 87.18 with blur; Chameleon results still missed most synthetic images at the fixed threshold.On Chameleon, SD v1.4-trained Xception reached 37.58 synthetic-class accuracy and 96.63 real-class accuracy, with 67.11 balanced accuracy; its checkpoint was target-selected. The JPEG quality factor is unverified.Source: §5.6 · pp. 7–8, Table 9 · p. 8

6 Discussion and Limitations

The reported comparisons support spatial input and supervision granularity as useful design choices, but do not isolate why patch-wise behavior occurs or establish reliability across settings. The study also identifies limits in patch labeling, evaluation, and practical interpretation.

Supported conclusions

  • The study supports treating spatial input and supervision granularity as useful design choices without assuming every architecture or distribution will benefit.A standard RGB classifier can learn from small regions, and multiple decisions provide broader coverage.Source: §6 · p. 8

Mechanism

  • Patch-wise comparisons do not isolate whether behavior reflects restricted context, supervision granularity, broader crop coverage, local-view augmentation, or averaging correlated predictions.Crops may still contain semantic cues, compression differences, and provenance shortcuts; stronger causal claims require additional controls.Source: §6 · p. 8

Coverage and patch labels

  • Uniform averaging weights clear, ambiguous, and corrupted regions equally, while partial edits and mixed real/synthetic images fall outside the demonstrated labeling scope.Overlap can add redundancy; adaptive selection and confidence-weighted pooling are proposed extensions, but their benefit is not established.Source: §6 · p. 8

Evaluation boundaries

  • Evaluation-selected results may be optimistic, selection budgets are unmatched, and the study provides no systematic multi-seed estimates.The comparisons characterize reported behavior rather than establish an unbiased state-of-the-art ranking; benchmark averages alone do not establish deployment reliability.Source: §6 · p. 8

Practical use and release

  • A detector score is supporting evidence, not proof of image origin; false positives can cause harmful accusations, and high real-image accuracy can coexist with low synthetic-image recall.The reference implementation covers multi-patch sampling, patch-level training, and image-level evaluation, but no study-trained checkpoints are provided and the release does not guarantee every score is reproducible.Source: §6 · pp. 8–9

7 Conclusion

The conclusion presents multi-patch detection as explicit RGB cropping, separate patch losses, and inference-time probability averaging, reporting higher GenImage means than whole-image baselines across four tested backbones. The accompanying implementation and historical records qualify how those results and input settings should be interpreted.

7 Conclusion

  • Reported GenImage means are higher for patch-wise supervision than whole-image baselines across four tested backbones, though selection histories limit conclusions under a common protocol.Supervision, resolution, and coverage studies characterize the design space; processing sensitivity and difficult-image failures define limits.Source: §7 · p. 9

A.1 Patch extraction, model, and optimization

  • The multi-patch implementation repeats image labels for extracted patches, averages patch-level focal losses in training, and averages synthetic-class probabilities by source image at evaluation.Evaluation uses a 0.5 decision threshold; the saved path groups patches by their source-image counts.Source: §A.1 · p. 10

A.2 Input resolution and historical variants

  • The saved multi-patch source feeds 64×64 patches directly to ResNet, Xception, or Swin after normalization, while historical input-resizing variants are not fully recovered.The release defaults to native-resolution cropping with direct-64 input, but this is not asserted to reproduce the full historical pipeline.Source: §A.2 · p. 10

A.3 Checkpoint counters and selection example

  • For the Xception example, GenImage selection chose checkpoint 43 by mean accuracy across eight generators, while a Chameleon sweep selected checkpoint 16 at 71.29.The log-level association does not prove prospective freezing before Chameleon inspection or isolate test-selection optimism from genuine checkpoint differences.Source: §A.3 · p. 10

A.4 Single-patch experiments and relation to EIB-Net

The appendix distinguishes a single-patch baseline from the complete EIB-Net system and records numerical conventions and unresolved table discrepancies. These qualifications limit cross-paper interpretation and caution against ranking models using small numerical differences.

A.4 Single-patch experiments and relation to EIB-Net

  • SPD selects one region per image, using a documented default of resizing to 256×256, forming a 4×4 grid, and choosing one 64×64 crop.Its training setup and backbone-specific transforms are distinct from the multi-patch implementation.Source: §A.4 · p. 11
  • SPD is not the complete EIB-Net architecture: EIB-Net adds a variational information bottleneck, so their scores are not a controlled bottleneck ablation or independent replications.Minimum-, middle-, and maximum-complexity selection uses a local variation statistic, not Shannon entropy; the statistic selects the RGB crop rather than serving as detector input.Source: §A.4 · p. 11

A.5 Numerical conventions and corrections

  • Accuracy and average precision retain percentage scale, while balanced accuracy is the arithmetic mean of the two class accuracies; low-data and timing rows do not support precise advantage claims.Those rows have conflicting subset labels or lack a complete measurement protocol.Source: §A.5 · p. 11
  • Several reported means differ from displayed component rows, and the historical summaries are retained because the tables cannot identify whether a component or mean was transcribed incorrectly.The discrepancies do not reverse the reported whole-image versus patch-wise comparison, but raw results should be checked before using small differences to rank models.Source: §A.5 · p. 11

B Full Per-Generator Results

The appendix expands per-generator benchmark and perturbation comparisons and distinguishes training-domain coverage, perturbation settings, and supervision or resolution controls. The supplied passages describe table scope and qualifications but do not provide cell values sufficient to report winners.

B Full Per-Generator Results

  • AIGCDetectBenchmark tables cover 16 generator subsets, including ProGAN, the PWS training source, so their means are not averages over 16 unseen generators.External detector scores are retained from original comparison tables rather than newly reproduced, and method coverage differs between accuracy and AP.Source: B Full Per-Generator Results · p. 11

B.1 Unperturbed images

  • The unperturbed AIGCDetectBenchmark appendix includes per-generator accuracy and average-precision tables, with reported-mean discrepancies retained pending reconciliation.The ProGAN entry is corrected to 52.75; displayed UnivFD and PWS Xception entries do not match their reported means.Source: Table 11 · p. 12, §B.1 · pp. 12–13, Table 12 · p. 13

B.2 JPEG compression

  • JPEG tables report accuracy and average precision, and the text notes that JPEG affects models differently across generators.The retained tables do not identify the quality factor unambiguously, so the entries are not assigned to QF=90 or QF=95.Source: §B.2 · p. 14, Table 13 · p. 14, Table 14 · p. 15

B.3 Downsampling

B.4 Gaussian blur

  • The Gaussian-blur tables report accuracy and average precision at σ = 1, while the displayed PWS mean and reported mean remain unreconciled.The reported PWS mean is 91.35; the 16 displayed values average to 91.6775.Source: Table 17 · p. 17, Table 18 · p. 17, §B.4 · p. 17

C.1 Supervision and source-image resolution

  • The supervision comparison contrasts patch-level losses with image-level loss after logit pooling, while resolution-aligned variants crop sources resized to fixed canvases.The GenImage accuracy tables identify SD v1.4 training and mark documented target-selected results; other rows have incompletely recovered selection histories.Source: §C.1 · p. 18, Table 19 · p. 18, Table 20 · p. 18

C.2 Patch size

The crop-size and inference-coverage studies examine how local detail and spatial coverage affect performance, while the single-patch experiments probe selection and transfer. Their scope includes unresolved data-label and model-selection qualifications.

C.2 Patch size

  • Crop-size changes can help on one generator and hurt on another, so the detailed results accompany the two benchmark means.Table 21 reports per-generator accuracy for Xception on GenImage and AIGCDetectBenchmark; the 128-pixel means are 91.45 on GenImage and 91.54 on AIGCDetectBenchmark.Source: §C.2 · p. 19, Table 21 · p. 19

C.3 Inference coverage

  • Increasing inference patches chiefly helps from one crop to a modest number; beyond 64, changes are small and need not benefit every generator.The accompanying timing values lack a specified measurement unit, batching, and timing boundaries, so they do not support a standardized latency comparison.Source: §C.3 · p. 20, Table 22 · p. 20, Table 23 · p. 20

D Single-Patch Selection and Transfer

  • Single-patch inputs do not improve every full-data result, and these SPD experiments use separate preprocessing and optimization settings from the multi-patch benchmarks.SPD denotes one selected RGB patch and does not include EIB-Net’s variational bottleneck [17].Source: D Single-Patch Selection and Transfer · p. 21

D.1 Selection, grid size and seed

  • The single-patch study varies selection strategy, candidate-grid size, training fraction, and seed, but conflicting reduced-data labels prevent matched-data-size comparisons.The detailed table retains duplicate min/G2 = 4/seed-3407/full-data rows without specifying their distinguishing configuration.Source: §D.1 · p. 21, Table 24 · p. 21

D.2 Cross-scene and cross-generator transfer

  • The transfer setting trains and validates on LSUN-Bedroom, then tests on ImageNet scenes generated by ADM or SD v1.The retained RMS-S label is also called RMT-S in the surrounding study, and fallback checkpoint selection is unspecified for some rows.Source: §D.2 · p. 22, Table 25 · p. 22

D.3 Training-data fraction, mixed sources and model capacity

The single-patch experiments include data-fraction and mixed-source settings, plus a cross-scene transfer test. A separate exploratory analysis measures local gradient co-occurrence statistics, which are diagnostic rather than detector inputs.

D.3 Training-data fraction, mixed sources and model capacity

  • Single-patch inputs do not improve every full-data result, and the DIFF study reports intra-dataset accuracy across four training-data fractions.The mixed-source GenImage table reports accuracy with 1% of each generator’s training data and with 20% of that subset; the lightweight-network study reports DIFF accuracy.Source: §D.3 · p. 23, Table 26 · p. 23, Table 27 · p. 23

E Single-Patch Transfer Dynamics

  • With ResNet-50 and 20% training data, SPD remains substantially above the whole-image baseline on both ImageNet target curves despite similarly high ADM validation accuracy.The test curves differ between ADM and SD v1, while the smoothed trajectories are not independent repeated trials and do not identify the feature responsible.Source: E Single-Patch Transfer Dynamics · p. 24, Figure 6 · p. 24

F Exploratory Analysis of Local Gradient Statistics

  • The exploratory gradient co-occurrence matrix analysis examines local gradient relationships, but pools its statistics over each resized image rather than detector training patches.The scripts convert images to grayscale, resize to 256 × 256, and compute horizontal and vertical Sobel gradients; the descriptor is not part of SPD or PWS.Source: F Exploratory Analysis of Local… · p. 24, §F.1 · p. 24

F.1 Descriptor and quantization

  • The analysis compares linear-range quantization with a stratified quantizer based on each image’s median, 90th percentile, and maximum gradient magnitude.The implementation clips quantized values to [0,15], and quantization itself can change visible structure, so concentrations around bins 8 and 12 need not be generator-specific.Source: §F.1 · pp. 24–25

F.2 Observed distribution differences

Exploratory GCM entropy differences vary by subset, and real and generated distributions overlap. Patch-score visualizations also show mixed local responses and examples of errors and resizing sensitivity, without identifying artifact locations or causes.

F.2 Observed distribution differences

  • Mean GCM entropy ratios vary by source, with approximately 1.35 for DALL-E 2 and GLIDE but 0.92 for Wukong, contradicting a universal lower-entropy rule for generated images.Real and generated entropy distributions substantially overlap, and their relative positions vary by subset; the plots are exploratory diagnostics, not a detection benchmark or significance test.Source: §F.2 · p. 25, Figure 7 · p. 25, Figure 9 · p. 27

G Patch Scores and Failure Cases

  • Patch responses can disagree within one image, while resizing real examples to 256×256 changes scores from 0.1034 to 0.5818 and from 0.4123 to 0.9216, crossing 0.5.The maps show model predictions rather than verified artifact locations; the examples do not isolate resizing as caused by interpolation, distortion, or changed crop coverage.Source: G Patch Scores and Failure Cases · pp. 28–29, Figure 10 · p. 28, Figure 11 · p. 29

Related papers

Loading 2609.37937v2…