PaperlayerView exact revision on arXiv

Source-linked AI summary

Forensic Twins: Self-Supervised Residual Learning for AI-Generated Image Forensics

Javier Muñoz-Haro, Ruben Tolosana, Ruben Vera-Rodriguez, Aythami Morales, Julian Fierrez

arXiv:2609.31514v1cs.CV

TL;DR

Generator-specific fingerprints shift with each new architecture, while realistic synthetic images can replicate authentic layouts and contexts. Forensic Twins learns residual representations from real images using disjoint crops. Its representations support zero-shot detection and source clustering, with an offline GMM reaching 97.99% mean AUC across 27 generative architectures.

  • Problem

    Generator-specific fingerprints shift with each new architecture, while realistic synthetic images can replicate authentic layouts and contexts.

    Source: §1 · pp. 1–2

  • Method

    Forensic Twins learns residual representations from real images using disjoint crops, steering its objective toward stationary image-acquisition fingerprints rather than semantic content.

    Source: §1 · p. 2, §5 · p. 9

  • Results

    Forensic Twins supports zero-shot detection and source clustering; an offline GMM reaches 97.99% mean AUC across 27 generative architectures.

    Source: §1 · p. 3, §5 · p. 9

  • Takeaways & Limitations

    Modeling only real images makes departures from that class, including generators absent during training, OOD by construction.

    Source: §1 · p. 2

    For GMM detection, robustness to re-compression degrades performance, though a handful of labeled examples largely restores it.

    Source: §5 · p. 9

Abstract

from arXiv · show

Detectors of AI-generated images are typically trained using samples from all Generative AI architectures they must catch, and struggle as soon as a new architecture emerges. Recent approaches have explored self-supervised pre-training as an alternative solution, yet standard frameworks work against the forensic task, e.g., their augmentations overwrite the micro-statistics of image formation. This paper introduces Forensic Twins, a Self-Supervised Residual Learning (SSRL) framework whose pretext task suppresses macroscopic content availability. Each image is mapped through a frozen, off-the-shelf forensic residual extractor, from which two spatially disjoint crops are drawn. Sharing no pixel, the two views retain minimal semantic structure to align, leaving a redundancy-reduction objective with a predominant common signal: the stationary fingerprint of the image acquisition pipeline. Additionally, Forensic Twins is trained exclusively on real images; no AI-generated image is observed at any stage. Experiments show that Forensic Twins attributes AI generator sources with 56.61% accuracy, i.e., 6.13% above the previous state-of-the-art zero-shot method at 375x lower latency. We also demonstrate that fitting a Gaussian Mixture Model (GMM) offline using only the real image embeddings extracted from Forensic Twins turns it into a state-of-the-art zero-shot detector, reaching 97.99% AUC across 27 unseen AI generators, including GANs, diffusion models and commercial systems. Code, weights and exact splits will be made publicly available

1 INTRODUCTION

Highly realistic, coherent synthetic images make detection and attribution increasingly important, while supervised detectors struggle with unseen generators and standard self-supervision can discard forensic traces. Forensic Twins learns from real-image residuals and disjoint views, achieving reported gains in both source attribution and zero-shot detection.

Motivation

  • Generative models produce increasingly detailed and coherent images, and conditioning can guide their layouts, styles, or insertion of entities into authentic contexts.Source: §1 · p. 1
  • Detectors trained on known generators can fail on unseen architectures, while standard self-supervised frameworks may discard low-amplitude forensic traces through semantic shortcuts.Supervised detectors may overfit to source-specific artifacts, and common SSL augmentations can overwrite micro-statistical noise.Source: §1 · p. 1

Contributions

  • Forensic Twins uses a frozen forensic residual extractor and disjoint crops so the shared signal is the stationary fingerprint of image formation, rather than structural semantics.A redundancy-reduction objective aligns the two views; the framework uses a lightweight 3.0M-parameter encoder.Source: §1 · p. 2
  • Forensic Twins is trained exclusively on real images; the paper frames generators absent from training as out-of-distribution and evaluates detection and source clustering.The stated contributions also include releasing code, weights, and data splits.Source: §1 · p. 2
  • 97.99% mean AUC across 27 generative architectures, while image-source attribution improves on the previous state-of-the-art zero-shot method by 6.13% in accuracy.The detection model is a single GMM fitted offline using only real-image embeddings.Source: §1 · p. 3

2 RELATED WORK

Prior work addresses AI-image detection through supervised artifact classifiers, adapted vision-language models, reconstruction errors, or natural-image manifolds, while source attribution remains challenging. Forensic Twins instead steers representations toward image-formation fingerprints and removes test-time optimization from a prior clustering approach.

AI-Generated Image Detection

  • Supervised detectors can degrade on unseen generators; later approaches adapt vision-language backbones, use diffusion reconstruction errors, or measure distance to a natural-image manifold.Forensic Twins shares the real-data-only premise of Zhang et al. (2025), but targets image-formation statistics rather than a representation optimized for semantics.Source: §2 · p. 3

Unsupervised Clustering of Image Sources

  • Nguyen et al. (2025) use forensic self-descriptions for zero-shot detection and unsupervised clustering, refining them per image with test-time optimization at approximately 0.11 images per second.Forensic Twins adopts their residual extractor and removes that bottleneck.Source: §2 · p. 3

Self-Supervised Representation Learning

  • Standard SSL augmentations preserve semantics but can overwrite forensic micro-statistics, while overlapping views leave spatial structure available as an alignment shortcut.Forensic Twins uses residual inputs and spatially disjoint views to steer learning toward forensic representations without contrastive negative samples.Source: §2 · p. 3

3 PROPOSED METHOD: Forensic Twins

Forensic Twins learns label-free representations from real images by combining residual filtering, non-destructive transformations, disjoint crops, and a redundancy-reduction objective. Its frozen encoder representations are then used for real-only density modeling and unsupervised source clustering.

3.2 FORENSIC TWINS: A FEED-FORWARD ARCHITECTURE FOR SEMANTIC DECOUPLING

  • Forensic Twins applies a frozen forensic residual extractor to suppress semantic content, then trains on disjoint crops whose shared signal is intended to be the stationary image-formation fingerprint.The method is introduced as a self-supervised residual-learning framework for label-free representations from real images.Source: §3 · p. 3, §3.2 · pp. 4–5
  • The residual representation models images as semantic content plus high-frequency fingerprint, suppressing low-frequency semantics while potentially retaining structural edges.The frozen FRE produces eight complementary residual channels.Source: §3.1 · p. 4, §3.2 · p. 4

3.1 BACKGROUND

  • The Barlow Twins objective drives the cross-correlation matrix toward identity, making diagonal elements encode invariance and off-diagonal elements reduce redundancy.The embeddings are produced by an encoder and projector from two views.Source: §3.1 · p. 4

3.2 FORENSIC TWINS: A FEED-FORWARD ARCHITECTURE FOR SEMANTIC DECOUPLING

  • Training transformations are limited to shared per-image orthogonal rotations and flips, chosen to preserve forensic information and leave crop location as the views’ distinguishing factor.Transformed images are converted to grayscale and normalized before residual filtering.Source: §3.2 · p. 4
  • Disjoint crops restrict shared semantic structure while retaining correlated stationary fingerprints; their sizes can vary independently across views and training iterations.The encoder uses 2D rotary positional embeddings to handle arbitrary crop sizes, and a projector maps its representations to embeddings.Source: §3.2 · p. 5

3.3 DOWNSTREAM FORENSIC APPLICATIONS

  • For detection, a GMM is fitted to real-image embeddings and unseen images are scored by negative log-likelihood under that modeled distribution.The encoder is frozen for downstream use, and the method treats generated images as potential low-density anomalies.Source: §3.3 · p. 5
  • For source clustering, k-Means is applied directly to encoder representations from an unlabeled mixture of real and AI-generated images.The proposed use relies on learned latent-space structure associated with source-specific generation artifacts.Source: §3.3 · p. 5

4 EXPERIMENTAL RESULTS

Forensic Twins is evaluated for AI-image detection, source clustering, and design choices, with strong results from real-only training. Its ablations examine residual extraction, view construction, architecture, and regularization.

4.1 AI-GENERATED IMAGE DETECTION

  • 97.99% AUC across 27 generators and two real-image distributions places Forensic Twins third overall, 21% above ConV (77.13 vs. 97.99% AUC) using real-only training.FSD reaches 98.52% AUC but takes 2,415.27 ms per decision, versus 6.44 ms for Forensic Twins.Source: §4.1 · p. 6, Table 1 · p. 7
  • 96.10% AUC on commercial generators is Forensic Twins’ lowest category score, while its performance gap across the three generator families is 2.46%.For comparison, FSD falls from 99.88% AUC on GANs to 93.72% on commercial generators.Source: §4.1 · p. 6

4.2 UNSUPERVISED CLUSTERING OF IMAGE SOURCES

  • Forensic Twins outperforms FSD by 6.13, 17.12, and 10.76 accuracy points at k=N, 2N, and 4N, respectively, with N=8 sources.The evaluation compares unsupervised source clustering methods on real and generated images.Source: §4.2 · p. 7, Table 2 · p. 7
  • The t-SNE representation is organized by generator identity but shows no clear semantic-class structure, consistent with forensic rather than content-dominant features.Figure 2 compares seven semantic superclasses and seven generators using the same number of images per class and identical t-SNE settings.Source: §4.2 · p. 7, Figure 2 · p. 8

4.3 ABLATION STUDIES

  • 60.95% AUC with raw RGB rises to 84.14% with fixed SRM filters and 93.77% with the learned FRE, supporting content suppression rather than a specific filter.GAN detection with raw RGB is at 51.76% AUC.Source: §4.3 · p. 8
  • 89.49% AUC with overlapping crops and 57.84% with destructive DINOv2-style augmentations fall below the disjoint-crop base configuration.The authors attribute the larger loss to destructive augmentations overwriting the residual signal shared by the views.Source: §4.3 · p. 8
  • Patch size is the dominant architectural factor; pooling alternatives do not surpass the [CLS] token, and the convolutional stem alone costs 1.37% AUC.Global Average Pooling loses 2.67%; concatenated per-token STD recovers 1.07 points but remains 1.60 points below the base.Source: §4.3 · p. 8
  • The final configuration improves AUC by 4.22% over the 93.77% FRE base, while patch-size and regularization gains are not independent.Patch 8 with λ = 0.001 gains 2.92%, below the 3.13% predicted from separate gains; patch 8 with the convolutional stem reaches 93.66%.Source: §4.3 · p. 9, Table 3 · p. 9

5 CONCLUSION

Forensic Twins learns real-image formation statistics through residual filtering, disjoint crops, and removal of destructive augmentations. It is evaluated on AI-image detection and source clustering, with compression robustness and commercial generators remaining limitations.

5 CONCLUSION

  • The framework suppresses semantic shortcuts with residual filtering, disjoint crops, and removal of destructive augmentations that would erase forensic signal.It is trained without observing AI-generated images.Source: §5 · p. 9
  • The learned representations are tested for AI-generated image detection across 27 architectures and for image-source clustering without generated-image training examples or source labels.The tested generator families include GANs, diffusion models, and commercial systems.Source: §5 · p. 9
  • Commercial architectures remain the weakest family for almost every method, and one-class GMM detection loses robustness under re-compression; a handful of labeled examples largely restores it.The authors identify few-shot calibration as a possible complement to the zero-shot setting.Source: §5 · p. 9

A COMPRESSION ROBUSTNESS ANALYSIS

The one-class detector is vulnerable to re-encoding because its GMM models only post-processing represented in real reference images. Adding compressed real examples recovers some performance, while calibration beyond its tested range remains unreliable.

A COMPRESSION ROBUSTNESS ANALYSIS

  • 97.99 AUC falls to 61–71 under unseen re-encoding; adding real images re-encoded at QF ∼U[50, 100] recovers performance to 64–81.The reported global AUC is over 25 generators not used for calibration.Source: A COMPRESSION ROBUSTNESS ANALYSIS · p. 15
  • 99.27 AUC on native images falls to 57–79 under JPEG for a real/fake-calibrated k-NN, but calibration across native and QF95, 80, 65, and 50 yields 92.2–95.3 in-range.The calibrated k-NN reaches 99.03 on native images; with N = 10, re-encoding AUC is 81.9–90.3, and with N = 25 it is 84.1–91.6.Source: A COMPRESSION ROBUSTNESS ANALYSIS · p. 15
  • QF35 remains an extrapolation boundary: calibration at larger samples gives 58.95 ± 15.46 for QF35, while N = 25 reaches 84.11 ± 4.16.The authors attribute the lower QF35 result at N = 100 (72.95 ± 13.1) to a bias-variance trade-off in k-NN overfitting.Source: A COMPRESSION ROBUSTNESS ANALYSIS · p. 15

B DETAILED IMAGE SOURCE CLUSTERING ANALYSIS

The analysis compares semantic-class recognition with generator-source attribution to assess what image representations encode. Forensic Twins has high source accuracy with much lower semantic accuracy, though its representation retains some content signal.

B DETAILED IMAGE SOURCE CLUSTERING ANALYSIS

  • The probe uses seven balanced semantic classes and seven generators, with 480 images per class; cosine k-NN is evaluated across 10 random fitting/evaluation splits.The semantic images come from the held-out ImageNet-1k validation split, and the source classes span GAN, diffusion, and commercial generators.Source: B DETAILED IMAGE SOURCE CLUSTERING… · p. 16, Table 5 · p. 16
  • EFFORT and OmniAID are highly semantic (99.21% and 99.29% semantic 1-NN), while their source accuracies are 97.27% and 95.15%, respectively.ConV’s semantic accuracy is 99.35% and its source accuracy is 78.35%, showing that real-image-only training alone does not ensure source discrimination.Source: B DETAILED IMAGE SOURCE CLUSTERING… · pp. 16–17
  • NPR and FSD have semantic accuracies of 24.32% and 34.17% but source accuracies near 80%, indicating that their source attribution is unlikely to be a by-product of content.Their source accuracies are 80.60% and 80.65%, respectively.Source: B DETAILED IMAGE SOURCE CLUSTERING… · p. 17
  • 95.57% source 1-NN accuracy and 39.03% semantic 1-NN accuracy give Forensic Twins a +56.54-point gap, with a 192-dimensional representation.Its source accuracy is on par with OmniAID at 95.15%, while its semantic accuracy is 60.26 points lower; the representation is 2.7 to 5.3 times smaller than state-of-the-art representations.Source: B DETAILED IMAGE SOURCE CLUSTERING… · p. 17
  • 39.03% semantic accuracy exceeds the 14.29% chance level, so Forensic Twins is not fully content-invariant; low-level image statistics can covary with depicted category.Changing from 1-NN to 5-NN changes source accuracies by less than 2.4 points and preserves method ordering, apart from the near-tie between FSD and NPR.Source: B DETAILED IMAGE SOURCE CLUSTERING… · p. 17
  • The 95.57% 1-NN result exceeds the lower k-Means accuracy because some sources are multi-modal, whereas k-Means assigns one centroid per source.Figure 3 compares semantic-class and generator representations across methods; its t-SNE distances should be read qualitatively, not as meaningful cluster distances.Source: B DETAILED IMAGE SOURCE CLUSTERING… · p. 17, Figure 3 · p. 18

C COMPUTATIONAL COST

Forensic Twins is among the three cheapest detectors across all four reported cost measures, while remaining substantially smaller and faster than foundation-backbone detectors.

C COMPUTATIONAL COST

  • ForensicTwins has 3.0M parameters, 6.44 ms latency, 10.7 GFLOPs, and 516 MB GPU memory, and is the only detector ranking among the three cheapest on all four cost metrics.Source: C COMPUTATIONAL COST · p. 19

D DETAILED RESULTS ON ZERO-SHOT DETECTION

Forensic Twins’ benchmark performance depends on the real-image reference distribution, and it compares favorably with the other zero-shot detector while retaining weaker generator families. The evaluation also shows substantial differences in worst-case behavior across detectors.

Protocol

  • Each detector’s AUC is measured on a balanced set pairing real images from one stated distribution with images from a single generator family.AUC values are not folded: scores below 50 indicate systematic ranking of generated images as more real than real ones.Source: D DETAILED RESULTS ON ZERO-SHOT… · p. 19

Real-image distribution

  • Forensic Twins ranks fourth on ImageNet-1k (96.94 AUC) but first on MS-COCO (99.04 AUC), so the strongest-method ordering depends on the real-image distribution.The authors therefore average both distributions in the main table; Forensic Twins gains 2.10 points from ImageNet-1k to MS-COCO, while several detectors lose points.Source: D DETAILED RESULTS ON ZERO-SHOT… · p. 19

Comparison with the other zero-shot method

  • Forensic Twins beats FSD by 14.46 and 19.73 points on Firefly and 3.51 and 3.89 points on Flux, and has the higher commercial average on both real distributions.Its commercial average is 94.41 vs. 93.31 on ImageNet-1k and 97.78 vs. 94.13 on MS-COCO; it also uses 3.0M vs. 6.2M parameters at about 375 times lower latency (6.44 ms vs. 2415.27 ms per image).Source: Table 7 · p. 20, D DETAILED RESULTS ON ZERO-SHOT… · p. 20, Table 8 · p. 21

Large-scale supervised detectors

  • Among supervised detectors, EFFORT is the most uniform, while OmniAID has weak results on Guided Diffusion and Firefly and SPAI avoids catastrophic failures but rarely approaches saturation.EFFORT’s worst family is DiT (95.61 on ImageNet-1k, 93.31 on MS-COCO); SPAI’s minima are 76.65 and 84.37, both on Guided Diffusion.Source: D DETAILED RESULTS ON ZERO-SHOT… · p. 20

Worst-case behaviour

  • On MS-COCO, Forensic Twins has the benchmark’s highest minimum AUC at 93.68 and is the only detector besides EFFORT with no family below 90.Its per-family standard deviation there is 1.64, the lowest of the nine detectors; on ImageNet-1k its minimum is 86.53.Source: D DETAILED RESULTS ON ZERO-SHOT… · p. 21

Limitations of our method

  • Forensic Twins’ weakest families include Midjourney and Firefly, and on ImageNet-1k EFFORT, FSD, and OmniAID score higher on 24, 24, and 19 families, respectively.Closing this gap while continuing to train only on real images is identified as the main open direction.Source: D DETAILED RESULTS ON ZERO-SHOT… · p. 21

E ROBUSTNESS AGAINST UNSEEN REAL IMAGES

Detection with a GMM fitted on a single real-image source retains performance on an entirely unseen real domain in both evaluation directions, but the cost is asymmetric.

Single-source real-domain robustness

  • 97.51% global mean AUC is achieved when fitting on MS-COCO alone, compared with 94.40% when fitting on ImageNet-1k alone.The MS-COCO-only model reaches 95.55% on the unseen ImageNet axis and 99.47% on its own axis; the ImageNet-only model reaches 93.44% on unseen MS-COCO and 95.36% on its own axis.Source: E ROBUSTNESS AGAINST UNSEEN REAL IMAGES · p. 22

Asymmetric cost

  • Removing MS-COCO costs 3.59% globally, whereas removing ImageNet costs 0.48%, despite the MS-COCO-only fit using less than half the training images.The single-source fits use 500k ImageNet images or 242k MS-COCO images, and the other real source is unseen at every stage.Source: E ROBUSTNESS AGAINST UNSEEN REAL IMAGES · p. 22

F DETAILED BREAKDOWN ON GENERATORS

The detection benchmark groups images by generator family so its per-family AUC reflects generalization beyond a particular checkpoint. Its coverage spans diverse generation paradigms, conditioning schemes, content domains, resolutions, and formats.

Overview

  • The benchmark covers 27 generator families, grouping checkpoints and configurations of the same generator so each per-family AUC measures generalization beyond one checkpoint.Synthetic images come from DMID, SynthBuster, and OSSIA; for example, StyleGAN3 includes multiple configurations, training datasets, and resolutions.Source: F DETAILED BREAKDOWN ON GENERATORS · p. 22

Image selection

  • For each family, evaluation uses the first 1,000 images or all available when fewer exist, pairs them with the same number of real images, and uses identical generated-image lists for every detector.ImageNet-1k validation and MS-COCO 2017 test images provide the real sets; images are not resized, cropped, or re-encoded before detector preprocessing.Source: F DETAILED BREAKDOWN ON GENERATORS · p. 22

Diversity of the benchmark

  • Families span adversarial, diffusion, and autoregressive generation, varied conditioning and content, and resolutions from 256×256 to 2048 pixels on the shorter side.For families prompted or conditioned on MS-COCO, generated content resembles the MS-COCO real images, while the ImageNet-1k comparison also differs in content; the authors therefore report both real distributions.Source: F DETAILED BREAKDOWN ON GENERATORS · p. 22

Related papers

Loading 2609.31514v1…