Source-linked AI summary
Intriguing properties of synthetic images: from generative adversarial networks to diffusion models
Riccardo Corvi, Davide Cozzolino, Giovanni Poggi, Koki Nagano, Luisa Verdoliva
TL;DR
The paper asks which image features best distinguish increasingly realistic synthetic images from real ones. It systematically analyzes many generator families with spatial and frequency-domain second-order statistics, finding persistent artifacts, training-set bias transfer, and mid-high-frequency spectral discrepancies. These findings identify forensic characteristics in autocorrelations and radial and angular spectra, although post-processing can obscure the traces.
Problem
Increasingly realistic synthetic images make it important to identify features that discriminate fake images from real ones.
Method
The paper systematically analyzes many GAN-, VQ-based, and diffusion-model generators using autocorrelation functions, power spectra, and radial and angular spectra.
Results
The study finds that generators retain artifacts and anomalous autocorrelation patterns, may transfer training-set biases, and differ from real images in mid-high-frequency spectral content.
Takeaways & Limitations
Autocorrelations and radial and angular spectra provide additional characteristics for forensic detection beyond visible artifacts and high-frequency traces.
Takeaways & Limitations
The analyzed artifacts are measured on unprocessed images, whereas compression and resizing may modify or hide them in real-world use.
Abstract
from arXiv · showhide
Detecting fake images is becoming a major goal of computer vision. This need is becoming more and more pressing with the continuous improvement of synthesis methods based on Generative Adversarial Networks (GAN), and even more with the appearance of powerful methods based on Diffusion Models (DM). Towards this end, it is important to gain insight into which image features better discriminate fake images from real ones. In this paper we report on our systematic study of a large number of image generators of different families, aimed at discovering the most forensically relevant characteristics of real and generated images. Our experiments provide a number of interesting observations and shed light on some intriguing properties of synthetic images: (1) not only the GAN models but also the DM and VQ-GAN (Vector Quantized Generative Adversarial Networks) models give rise to visible artifacts in the Fourier domain and exhibit anomalous regular patterns in the autocorrelation; (2) when the dataset used to train the model lacks sufficient variety, its biases can be transferred to the generated images; (3) synthetic and real images exhibit significant differences in the mid-high frequency signal content, observable in their radial and angular spectral power distributions.
1. Introduction
The paper studies forensic traces across established and newer image generators using second-order statistics in spatial and frequency domains. It finds persistent artifacts, training-data bias transfer, and spectral differences between synthetic and real images.
- Synthetic-image detection matters because increasingly photorealistic generators create security and ethical risks when misused.
- Improving generators reduce visible synthesis artifacts, motivating analysis of low-level traces tied to their processing pipelines.
- The study systematically examines GAN-, VQ-based, and diffusion-model generators using second-order statistics in spatial and frequency domains.
- Average autocorrelation, average power spectra, and derived radial and angular spectra provide the paper’s main analysis tools.
- No generator appears artifact-free: artifacts may form strong residual power-spectrum peaks, while newer methods often retain unnatural autocorrelation patterns.
- Training-set compression artifacts and biases can transfer to generated images, especially when the dataset has limited variety and strong biases.
- Synthetic and real images also differ in statistical features of high-level content, visible through radial and angular spectra derived from average power spectra.
2. Related Work
Prior work identifies synthetic-image fingerprints and spectral anomalies, especially high-frequency discrepancies associated with generator architectures. This paper extends that perspective to middle frequencies and directional behavior.
- Synthetic images can contain architecture-specific fingerprints that detectors use both to expose them and to trace their generating architecture.
- GAN images exhibit Fourier-domain peaks linked in prior work to decoder up-sampling and aliasing.
- Existing GAN detectors often rely on high-frequency discrepancies, but exploiting these traces remains difficult in real-world conditions.
- The paper reports that middle frequencies also aid discrimination because generators show anisotropic behavior and struggle with diagonal details.
3. Generative Models
The paper analyzes GAN, vector-quantized, and diffusion-based generator families and summarizes their principal architectural properties. These families differ in how they represent data and produce images.
- The study analyzes a list of generative models whose main properties are reported in Table 1.
- Generative Adversarial Networks: GANs train a generator and discriminator in a min-max game, with the generator producing samples intended to fool the discriminator.
- Vector Quantized Generative Networks: VQ-VAEs compress data into a low-dimensional latent space and reconstruct it, using a discrete codebook to structure latent vectors.
- Diffusion Models: Diffusion models generate images by progressively reversing an additive noise process from noisy inputs toward noiseless outputs.
- Latent Diffusion Models: Latent diffusion models combine VAEs with diffusion in a low-dimensional latent space to reduce computational cost while generating high-dimensional images.
4. Forensic Analysis
The paper analyzes second-order spatial and frequency statistics to expose architecture traces, post-processing effects, training-data bias, and spectral differences between real and synthetic images.
- Analysis setup: The analysis averages autocorrelations and power spectra over 1000 controlled real or generated images, examining spatial correlations and frequency content.Noise residuals are used when generation traces are obscured by high-level image semantics.
- Architecture related traces: Diffusion-model fingerprints show spectral peaks and periodic autocorrelation patterns linked to decoder upsampling factors of 4 or 8.Upsampling by 4 produces peaks at multiples of 1/4 and periodicity 4; factor 8 produces denser spectral peaks and sparser autocorrelation patterns.
- Architecture related traces: Post-processing can substantially alter autocorrelations, with strong JPEG compression completely hiding generation artifacts beneath compression artifacts.The tested operations include blurring, sharpening, resizing, and JPEG compression at different settings.
- Training dataset bias: Training-data artifacts can transfer into generated images, as JPEG traces appear in outputs when the training set contains strongly biased compression patterns.The cited example used a dataset composed entirely of JPEG-compressed images at quality factor Q=75.
- Fidelity to natural images: Synthetic radial spectra generally deviate from the real-image power law, with real images often containing richer intermediate-frequency content.FFHQ is an exception because interpolation low-pass filters its images, making their spectral content resemble some synthetic sources.
- Fidelity to natural images: Angular spectra also show significant deviations between synthetic and real sources after low-pass content is filtered.The angular analysis evaluates directional power density and Fisher’s discriminant ratio across 16 angular steps.
5. Conclusions
The paper argues that increasingly sophisticated generators still leave exploitable forensic traces, while failing to reproduce real-image spectral statistics and potentially transferring training-set biases.
- Even sophisticated architectures continue to generate artifacts that can support synthetic-image detection.Artifacts may be more visible in the spatial domain or spectral domain, so both should be examined.
- Generators do not seem able to mimic real-image spectral distributions at mid-high frequencies.Radial and angular spectra may help expose these discrepancies.
- Strongly biased training sets may transfer the same biases into generated images, creating a potential problem for generators and detectors.