Are GAN generated images easy to detect? A critical analysis of the state-of-the-art

Diego Gragnaniello; Davide Cozzolino; Francesco Marra; Giovanni Poggi; Luisa Verdoliva

Source-linked AI summary

Are GAN generated images easy to detect? A critical analysis of the state-of-the-art

Diego Gragnaniello, Davide Cozzolino, Francesco Marra, Giovanni Poggi, Luisa Verdoliva

arXiv:2104.02617v1Apr 6, 2021cs.CVcs.AI

TL;DR

Photorealistic GAN images challenge reliable automated detection, especially when generators are unseen or images are compressed and resized. The paper systematically compares detectors in these realistic settings and finds that generalization and low-false-alarm robustness remain limited, while augmentation and avoiding early down-sampling improve performance.

Problem
Photorealistic synthetic images can fool observers, while unseen generators and routine web compression or resizing remove evidence needed by detectors.
Source: §1 · pp. 1–2
Method
The paper experimentally compares several GAN-image detectors across unseen architectures, resolutions, image distortions, and training variations using screening-oriented metrics.
Source: §1 · p. 2, §4 · p. 3
Results
Avoiding down-sampling in the first block yields average gains of about 15% in accuracy and 14% in Pd@5%, while accuracy exceeds 90% across architectures and 97% with StyleGAN2 training.
Source: §4 · pp. 3–4
Takeaways & Limitations
Reliable GAN-image detection remains far from achieved, and augmentation, training diversity, and full-resolution processing provide promising directions for more robust detectors.
Source: §4 · p. 4, §5 · p. 4

Abstract

from arXiv · show

The advent of deep learning has brought a significant improvement in the quality of generated media. However, with the increased level of photorealism, synthetic media are becoming hardly distinguishable from real ones, raising serious concerns about the spread of fake or manipulated information over the Internet. In this context, it is important to develop automated tools to reliably and timely detect synthetic media. In this work, we analyze the state-of-the-art methods for the detection of synthetic images, highlighting the key ingredients of the most successful approaches, and comparing their performance over existing generative architectures. We will devote special attention to realistic and challenging scenarios, like media uploaded on social networks or generated by new and unseen architectures, analyzing the impact of suitable augmentation and training strategies on the detectors' generalization ability.

1. INTRODUCTION

GANs have made synthetic images difficult to distinguish from real content, creating a need for reliable automated detection. The paper argues that existing detectors are less dependable in realistic web conditions involving unseen generators and image distortions.

GANs have substantially improved generated-image quality, enabling both image synthesis and attribute manipulation but also malicious uses such as fake profiles and fake news.Source: §1 · p. 1
Synthetic images contain visible artifacts, including color anomalies and missing symmetries, but these imperfections may disappear as generation technology advances.Source: §1 · p. 1
Architecture-linked fingerprints and Fourier-domain peaks provide more persistent detection evidence tied to GAN up-sampling operations.Source: §1 · p. 1
Existing detectors can appear effective, but their performance drops sharply when training data are poorly aligned or images are compressed and resized.Source: §1 · pp. 1–2
The paper systematically compares promising detectors across challenging datasets, realistic distortions, and screening-oriented metrics to identify ingredients of successful solutions.Source: §1 · p. 2

2. STATE-OF-THE-ART METHODS

State-of-the-art GAN-image detectors exploit generator fingerprints, intrinsic image constraints, learned deep features, or frequency-domain artifacts. Fully supervised methods work well on seen generators, whereas transfer to unseen models motivates adaptation and augmentation strategies.

Early forensic methods identify image provenance through architecture- and training-dataset-dependent fingerprints analogous to device signatures from real cameras.Source: §2.1 · p. 2
Other approaches use limited intensity ranges or disrupted color-band correlations, extracting high-pass chrominance co-occurrences as discriminative features.Source: §2.1 · p. 2
Pre-trained Xception, Inception, and DenseNet models outperform tailored CNNs trained from scratch, particularly in challenging scenarios.Source: §2.1 · p. 2
Frequency-based detectors classify images using spectral peaks and other artifacts associated with GAN up-sampling across architectures, datasets, and resolutions.Source: §2.2 · p. 2
Fully supervised detectors are effective on generators represented during training but fail to generalize to unseen models.Source: §2.3 · p. 2
Few-shot and incremental-learning methods adapt to new manipulations, while blur, compression, and strong augmentation can improve transfer across unseen architectures and datasets.Source: §2.3 · p. 2

3. DATASETS

The experiments train detectors on ProGAN images and test transferability using images from unseen GAN architectures at multiple resolutions, alongside real images. The testing collection spans several modern generators and approximately 39K synthetic images.

Training uses 362K real LSUN images and 362K generated images from 20 ProGAN models covering different LSUN object categories.Source: §3 · pp. 2–3
All training images have 256×256-pixel resolution, with a 4K-image validation subset.Source: §3 · p. 3
Testing targets transferability by using GAN architectures never seen during training, including StyleGAN, StyleGAN2, BigGAN, CycleGAN, StarGAN, RelGAN, and GauGAN.Source: §3 · p. 3
The test datasets contain both 256×256 and 1024×1024 synthetic images, approximately 39K synthetic images overall, and 11.1K low-resolution real images.Source: §3 · p. 3

4. EXPERIMENTAL RESULTS

Across unseen-generator tests, detectors often achieve strong AUC but much weaker low-false-alarm detection, with distortions causing major degradation. Augmentation improves stability, and avoiding early down-sampling yields the strongest reported gains.

Detector comparison

The comparison includes Xception, SRNet, Spec, M-Gb, Co-Net, Wang2020, and PatchForensics, including SRNet because it preserves noise-residual features without early down-sampling.Source: §4 · p. 3

Generalization

AUC often exceeds 0.9 on low-resolution unseen-generator images, but fixed-threshold accuracy and Pd@5% or Pd@1% FAR are substantially less encouraging.Source: §4 · p. 3
High-resolution results are somewhat better, while retaining the same pattern across AUC, fixed-threshold accuracy, and low-FAR detection.Source: §4 · p. 3

Robustness

Compression and resizing cause dramatic performance impairments, whereas augmentation-trained methods remain relatively stable; 2x downsampling has catastrophic effects.Source: §4 · p. 3

Ablations

The ablations test ImageNet pre-training, residual extraction, early down-sampling, stronger augmentation, and alternative backbones.Source: §4 · p. 3

Best variant

Avoiding down-sampling in the first block provides average gains of about 15% in accuracy and 14% in Pd@5%.Source: §4 · p. 3
Overall accuracy exceeds 90% across architectures and rises above 97% when training uses StyleGAN2.Source: §4 · p. 4

Implications

The limited preliminary experiments highlight diversity and full-resolution processing as promising directions, while robustness to compression and resizing remains underdeveloped.Source: §4 · p. 4

5. CONCLUSIONS

The analysis finds that reliable GAN-image detection remains unresolved: training–test misalignment, compression, and resizing seriously impair detectors in realistic settings. It nevertheless identifies ingredients of successful solutions and offers directions for future research.

Reliable GAN-image detection remains far from achieved because realistic mismatches and image processing cause serious performance impairments.Source: §5 · p. 4
The comparative analysis identifies key ingredients of successful detection solutions and provides hints for future research.Source: §5 · p. 4