PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Universal Fake Image Detectors that Generalize Across Generative Models

Utkarsh Ojha, Yuheng Li, Yong Jae Lee

arXiv:2302.10174v2cs.CVcs.LG

TL;DR

The paper addresses the failure of deep real-vs-fake classifiers to detect images from newer generative-model families. It instead uses nearest-neighbor and linear-probing classification in a feature space not trained for real-vs-fake separation, achieving substantially better generalization, including +15.05mAP and +25.90% acc for nearest neighbor in one unseen-model setting.

  • Problem

    Deep classifiers trained on GAN fakes fail to detect fake images from newer generative-model families and instead treat many unseen images as real.

  • Method

    The paper performs real-vs-fake classification with nearest neighbor and linear probing in a feature space entirely untrained for that task, using pretrained CLIP-ViT features.

  • Results

    +15.05mAP and +25.90% acc over the SoTA are obtained with nearest neighbor when training on ProGAN images and evaluating on unseen diffusion and autoregressive models.

  • Takeaways & Limitations

    Nearest-neighbor and linear-probing baselines provide strong generalization across generative-model breeds, including between GAN and diffusion data.

  • Takeaways & Limitations

    The similarity between GAN- and diffusion-generated fake images that supports generalization remains an open question.

Abstract

from arXiv · show

With generative models proliferating at a rapid rate, there is a growing need for general purpose fake image detectors. In this work, we first show that the existing paradigm, which consists of training a deep network for real-vs-fake classification, fails to detect fake images from newer breeds of generative models when trained to detect GAN fake images. Upon analysis, we find that the resulting classifier is asymmetrically tuned to detect patterns that make an image fake. The real class becomes a sink class holding anything that is not fake, including generated images from models not accessible during training. Building upon this discovery, we propose to perform real-vs-fake classification without learning; i.e., using a feature space not explicitly trained to distinguish real from fake images. We use nearest neighbor and linear probing as instantiations of this idea. When given access to the feature space of a large pretrained vision-language model, the very simple baseline of nearest neighbor classification has surprisingly good generalization ability in detecting fake images from a wide variety of generative models; e.g., it improves upon the SoTA by +15.07 mAP and +25.90% acc when tested on unseen diffusion and autoregressive models.

1. Introduction

The paper targets fake-image detection that generalizes across increasingly diverse generative models, showing that conventional classifiers fail because they focus asymmetrically on fake-image patterns. It proposes classification in a feature space not trained for real-vs-fake discrimination and reports substantially improved generalization.

  • Fake images increasingly come from diverse sources, motivating detectors that generalize beyond the single generative model available during training.
  • Existing real-vs-fake classifiers detect low-level fingerprints associated with the training generator, while treating other images as real.The resulting real class acts as a sink for generated images from unseen models.
  • The proposed solution uses features not trained to separate real from fake, instantiated with nearest-neighbor and linear-probing classification.The method uses a fixed feature space from a pretrained CLIP-ViT model.
  • +15.05mAP and +25.90% acc are achieved by nearest neighbor over the SoTA when ProGAN training images are tested against unseen diffusion and autoregressive models.Linear probing improves over the SoTA by +19.49mAP and +23.39% acc in the same setting.
  • The study contributes an analysis of unseen-model failure, two simple baselines, and experiments on the ingredients needed for generalizable detection.The paper also reports that its baselines achieve state-of-the-art generalization performance.

2. Related work

Prior work detects manipulated and synthetic images using handcrafted cues, learned classifiers, and efforts to generalize across generative models. Earlier results established that classifiers often fail when the target generator differs from the training model.

  • Synthetic-image research includes partial image editing, attribute changes, face replacement, and object insertion into real scenes.
  • Traditional manipulation detectors use cues such as compression artifacts, resampling, and irregular reflections.
  • Learning-based detectors progressed from same-generator detection toward methods intended to generalize across different generative models.

3. Preliminaries

The preliminary analysis evaluates a ProGAN-trained real-vs-fake classifier on unseen generator families and finds severe cross-family failures. Feature visualizations and frequency spectra suggest that the classifier detects generator-specific artifacts rather than modeling real images symmetrically.

  • The baseline trains a binary classifier on 720k ProGAN-associated images, balanced between real and fake classes.It uses ResNet-50 pretrained on ImageNet as the classification network.
  • 79.25% accuracy is reported for GauGAN images, an unseen GAN variant within the training generator family.
  • For many unseen models, including LDM variants, accuracy drops to nearly chance performance because generated images are mostly classified as real.The same-class prediction behavior explains chance-level accuracy on balanced test sets.
  • The learned feature space separates ProGAN fakes from a combined cluster containing training real images, diffusion fakes, and diffusion real images.This organization indicates that the learned real class lacks a distinct region of its own.
  • Frequency spectra show model-specific patterns, supporting the hypothesis that the classifier latches onto GAN artifacts and classifies images lacking them as real.

4. Approach

The approach performs real-vs-fake classification in a feature space entirely untrained for that task, using nearest neighbors or linear probing over frozen CLIP-ViT representations. The feature space is chosen for broad visual exposure and sensitivity to low-level details.

  • The feature space should cover many image types and preserve low-level details relevant to real-fake differences.
  • CLIP:ViT is selected because it was trained on 400M image-text pairs and provides broad visual exposure.
  • The method uses nearest-neighbor and linear-probing classifiers in a feature space entirely untrained for real-vs-fake classification.
  • Nearest-neighbor classification maps images into 768-dimensional CLIP features and assigns the label of the closer real or fake feature bank under cosine distance.
  • Linear classification adds a sigmoid linear layer while keeping the CLIP:ViT encoder frozen, training only the new classifier.The linear layer uses only a few hundred parameters, such as 768.

5. Experiments

The experiments evaluate fake-image detectors trained with access to one generative model and tested across GAN, diffusion, and autoregressive models. Comparisons include trained classification baselines and the paper’s feature-based approaches using average precision and accuracy.

  • Experimental protocol: Evaluation follows a one-source training protocol, using ProGAN or another available generative model while testing on unseen generative models.The study evaluates ProGAN-trained methods across multiple GANs, diffusion models, and DALL-E, and separately repeats the setup with LDM as the source.
  • Evaluation domains: The evaluation compares methods across established GAN variants and newer diffusion and autoregressive generators.The text-to-image evaluation includes LDM, Glide, and DALL-E, with LAION images as real examples and corresponding text-conditioned generations as fake examples.
  • Compared methods: Baselines include image-level, patch-level, co-occurrence-matrix, and frequency-spectrum classification networks.The primary trained-network baseline fine-tunes an ImageNet-pretrained ResNet-50 on ProGAN real/fake images.
  • Metrics: The benchmark reports average precision and classification accuracy for real/fake detection.Accuracy is averaged over real and fake classes, while thresholds are tuned on held-out training validation data.

6. Results

Trained real-vs-fake classifiers generalize poorly beyond their training family, whereas nearest-neighbor and linear probing in frozen CLIP:ViT features maintain substantially stronger performance on unseen generators. Feature-space architecture and pretraining data also affect detection quality.

  • Trained-classifier failure: Trained classifiers often classify unseen-model fakes as real, with performance dropping to nearly chance on models such as LDM variants.Changing to CLIP:ViT, patch-level inputs, co-occurrence matrices, or frequency space does not resolve the inconsistent generalization.
  • Cross-model generalization: +19.49 mAP is the gain of the best fixed CLIP:ViT feature method over the best baseline on unseen diffusion and autoregressive models.Across all settings, the corresponding gain is +9.8 mAP.
  • Cross-model generalization: +25-30% accuracy is achieved by fixed-feature methods over trained deep-network baselines on unseen generative model families.Nearest-neighbor variants reach 82-84% average accuracy and linear probing about 82%, versus 53-58% for trained deep networks on unseen diffusion and autoregressive models.
  • Feature-based methods: Linear classification in CLIP:ViT space preserves and sometimes improves nearest-neighbor generalization, while nearest-neighbor performance is not highly sensitive to voting pool sizes k=1 to k=9.These results support using a frozen pretrained feature space rather than features trained for real-vs-fake classification.
  • Oracle comparison: 84.25% average accuracy is reached by nearest-neighbor k = 9, exceeding the oracle baseline’s 76.26% by 7.99%.The oracle baseline calibrates its threshold directly on each test set, so the comparison indicates that threshold adjustment alone does not explain the gap.
  • Feature-space analysis: CLIP:ViT features separate real and fake representations better than the evaluated ImageNet-pretrained feature spaces.The ablation attributes effectiveness to both encoder architecture and pretraining dataset; CLIP-pretrained visual encoders outperform ImageNet-pretrained ones.
  • Training-source transfer: Using LDM data for training yields 97.32 mAP on unseen GAN images, compared with 60.17 mAP for the trained deep network baseline.The reported improvement for the unseen GAN domain is +37.16 mAP, and the authors observe reciprocal cross-domain generalization between ProGAN and LDM training sources.
  • Feature-space analysis: Distances in CLIP:ViT feature space correlate with visual quality among LDM fakes: closer ProGAN neighbors tend to be less realistic.The observation supports an additional use of the feature space beyond binary detection.

7. Conclusion and Discussion

The paper concludes that frozen, informative feature spaces provide a simple fix for poor cross-generator generalization in learned fake-image detectors. It also identifies the common structure across GAN and diffusion fakes as an open question.

  • Conclusion: Nearest-neighbor and linear probing in a feature space not trained for real-vs-fake classification substantially improve detection generalization, especially for diffusion and autoregressive models.The conclusion presents these methods as strong baselines for fake-image detection.
  • Open question: A shared but unidentified link appears to connect fake images generated by GANs and diffusion models.The paper states that understanding this similarity could help design better detectors, but leaves its nature unresolved.
  • Supplementary material: The supplementary material provides additional baseline details, training information, nearest-neighbor ablations, and performance breakdowns.It is described as complementary information to the main paper.

A. Details about the frequency spectrum classification

The frequency-spectrum baseline detects artifacts associated with its training generator family but fails to generalize broadly. The analysis links this behavior to shared frequency patterns in CycleGAN and StarGAN.

  • Method rationale: The frequency baseline represents GAN upsampling artifacts in frequency space for real/fake classification.Its motivation is that upsampling layers can introduce checkerboard artifacts that are more visible in this representation.
  • Frequency-spectrum baseline: The frequency-spectrum classifier performs perfectly on held-out CycleGAN images and retains this ability on StarGAN.For other evaluated generative models, accuracy falls to almost chance performance.
  • Frequency-space analysis: CycleGAN and StarGAN fake images share a similar 3x3 frequency-space pattern, unlike the patterns of most other generators.The authors interpret the classifier’s behavior as detecting this particular pattern and labeling other images as real.

B. Training details

The appendix describes baseline training and threshold selection, then examines how nearest-neighbor generalization changes across feature-space layers.

  • Training details: The image-level classifier baseline uses the authors’ official code repository, while ViT:CLIP training applies Blur + JPEG augmentation with 0.5 probability.The passage also specifies a batch size of 256, but the supplied text truncates the remaining training details.
  • Threshold tuning: For non-oracle baselines, the ProGAN validation set supplies candidate sigmoid scores used to select the real/fake decision threshold.The threshold is chosen by evaluating real and fake validation images and iterating over their resulting scores.
  • Layer choices: Nearest-neighbor classification evaluates CLIP:ViT layers L0, L8, L16, and L24 with k = 1, using L24 as the default feature space.L24 is the final layer, while L0 is close to pixel-space nearest-neighbor search.
  • Layer choices: Generalization remains consistent across layers except the first, indicating that nearest-neighbor performance is not highly sensitive to the chosen noninitial CLIP layer.This adds robustness beyond the previously observed stability across voting-pool sizes.

C.2. Effect of dataset diversity

The study tests nearest-neighbor detection with training data spanning 2, 4, 8, or 20 object classes and finds generally decent performance across these diversity levels.

  • Dataset variants: The default training source contains real/fake images from 20 LSUN object classes generated by 20 different ProGANs.The comparison reduces this source diversity to datasets containing 2, 4, 8, or 20 classes.
  • Generalization: Nearest-neighbor detection performs decently for most generative models even when trained with real/fake images from only 2 classes.The result holds across all four tested dataset variants for most generative models.
  • Comparison with trained classifiers: This differs from the analogous classifier analysis in [49], where reduced dataset diversity, especially 2 classes, harms detector generalization.The comparison concerns the effect of training-set diversity on the resulting detector.

C.3. Using data from multiple generative models

The multi-domain experiment gives both methods real and fake data from ProGAN and LDM, then compares their behavior on seen and unseen generative-model domains.

  • Experimental setup: The experiment combines real data from RLSUN and RLAION with fake data from ProGAN and LDM.Nearest neighbor and the image-level classifier receive access to both domains.
  • Results: The trained classifier performs well on seen ProGAN and LDM variants but remains poor on unseen domains such as DALL-E.Expanding training access to two generative-model domains does not change this qualitative pattern.
  • Results: Nearest neighbor behaves more consistently across different generative models than the trained classifier.The passage presents this as evidence that the generalization gap cannot simply be resolved by adding access to multiple training domains.

D. Accuracy breakdown of real and fake classes

The class-wise breakdown shows that trained image-level classifiers can misclassify unseen generated images as real, whereas nearest-neighbor predictions remain more balanced between real and fake inputs.

  • Breakdown design: The breakdown separates detection accuracy on real images and fake images for each generative model.Tables 5 and 6 report classifier rows against model-specific real and fake image columns.
  • Image-level classifier: The image-level classifier detects real/fake images effectively within the GAN domain but begins classifying latent-diffusion fake images as real.This asymmetric behavior causes fake-image classification accuracy to fall to almost 0%.
  • Supporting comparisons: The figures additionally examine CLIP layer choice, training-class count, classifier-versus-nearest-neighbor behavior, and precision-recall curves across generative-model families.These comparisons include ProGAN, LDM, and other model families, with Figure 14 explicitly highlighting weak classifier generalization to unseen domains.
  • Nearest neighbor: Nearest-neighbor classification avoids the same large discrepancy between real-image and fake-image predictions across models.Its outputs remain similar whether the tested image is real or fake.
Loading 2302.10174v2…