PaperlayerView exact revision on arXiv

Source-linked AI summary

A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detection

Keshigeyan Chandrasegaran, Ngoc-Trung Tran, Ngai-Man Cheung

arXiv:2103.17195v1cs.CVeess.IV

TL;DR

Prior work reports that CNN-generated images fail to match real images’ high-frequency Fourier decay, and uses the discrepancy for synthetic-image detection. This paper tests whether the feature is inherent by changing the final upsampling operation, finding counterexamples that avoid the discrepancy and bypass a detector.

  • Problem

    Prior work reports high-frequency Fourier decay discrepancies in CNN-generated images and uses them for detection, while the paper investigates whether those discrepancies are inherent.

    Source: §1 · p. 1, §11 · p. 8

  • Method

    The authors construct counterexamples across GAN models by replacing the last feature-map scaling operation with nearest or bilinear interpolation.

    Source: §1.3 · p. 2, §4.1 · p. 3

  • Results

    The authors report that high-frequency decay discrepancies can be avoided by changing the last scaling step, and that the resulting images bypass the proposed detector.

    Source: §8.2 · p. 5, §11 · p. 8

  • Takeaways & Limitations

    The authors conclude that high-frequency Fourier decay discrepancies are not inherent to CNN-generated images and are not robust features for synthetic-image detection.

    Source: §1.3 · p. 3, §11 · p. 8

Abstract

from arXiv · show

CNN-based generative modelling has evolved to produce synthetic images indistinguishable from real images in the RGB pixel space. Recent works have observed that CNN-generated images share a systematic shortcoming in replicating high frequency Fourier spectrum decay attributes. Furthermore, these works have successfully exploited this systematic shortcoming to detect CNN-generated images reporting up to 99% accuracy across multiple state-of-the-art GAN models. In this work, we investigate the validity of assertions claiming that CNN-generated images are unable to achieve high frequency spectral decay consistency. We meticulously construct a counterexample space of high frequency spectral decay consistent CNN-generated images emerging from our handcrafted experiments using DCGAN, LSGAN, WGAN-GP and StarGAN, where we empirically show that this frequency discrepancy can be avoided by a minor architecture change in the last upsampling operation. We subsequently use images from this counterexample space to successfully bypass the recently proposed forensics detector which leverages on high frequency Fourier spectrum decay attributes for CNN-generated image detection. Through this study, we show that high frequency Fourier spectrum decay discrepancies are not inherent characteristics for existing CNN-based generative models--contrary to the belief of some existing work--, and such features are not robust to perform synthetic image detection. Our results prompt re-thinking of using high frequency Fourier spectrum decay attributes for CNN-generated image detection. Code and models are available at https://keshik6.github.io/Fourier-Discrepancies-CNN-Detection/

1. Introduction

As CNN-generated images become difficult to distinguish from real images in RGB space, prior work reports high-frequency Fourier discrepancies and uses them for detection. This paper challenges the claim that these discrepancies are inherent by changing the generator’s last feature-map scaling operation.

1. Introduction

  • CNN-generated images are increasingly difficult to distinguish from real images in RGB space, motivating research into synthetic-image detection.The paper frames detection as important amid concerns about malicious uses of deepfakes.Source: §1 · p. 1

1.1. Fourier spectrum discrepancies in CNN-generated images

  • Prior work reports that CNN-generated images fail to reproduce real-image high-frequency decay, with some studies treating this as an intrinsic limitation [12] [10].Dzanic et al. [12] proposed detecting generated images from these frequency discrepancies, reporting up to 99.2% accuracy across GAN and VAE models.Source: §1.1 · p. 1

1.2. CNN-generated Image Detectors

  • Earlier detectors used Fourier representations or reduced spectra, including frequency-domain classifiers and methods based on high-frequency attributes [44] [13] [12].The paper notes that these detectors rely on the belief that high-frequency decay discrepancies are intrinsic to CNN-generated images.Source: §1.2 · p. 2

1.3. Our contributions

  • The paper focuses on the generators’ last upsampling layer and reports avoiding spectral discrepancies by changing its feature-map scaling while retaining standard training algorithms and objectives.The approach also bypasses the detector proposed by Dzanic et al. [12].Source: §1.3 · pp. 2–3
  • The study argues that high-frequency spectral decay discrepancies are not intrinsic to CNN-generated images and urges reconsidering their use for detection.The authors report bypassing a detector that exploits these discrepancies.Source: §1.3 · p. 3

2. Related Work

Related work attributes spectral discrepancies to several possible properties of CNN generators and proposes different remedies. These include spectral regularization and frequency-shifted generators.

2. Related Work

  • Dzanic et al. [12] fit a decay function to reduced spectra and use its parameters in a kNN classifier to detect synthetic images.Their account describes generated-image Fourier modes at the highest frequencies as remaining approximately constant rather than decaying like real-image modes.Source: §2 · p. 3
  • Durall et al. [10] attribute GAN spectral mismatch to transpose convolutions and add spectral regularization to the generator loss to penalize distorted samples.They report the effect across DCGAN, LSGAN [31], WGAN-GP [17], and DRAGAN.Source: §2 · p. 3
  • Khayatkhoei and Elgammal argue that increasing resolution raises kernel-spectrum dependencies, inhibiting high-frequency learning, and propose frequency-shifted generators.Their proposed generators shift frequencies toward selected high frequencies.Source: §2 · p. 3

3. Background

The background defines the Fourier representation and the reduced spectrum used to describe image content across spatial frequencies. It also states that sampling resolution bounds represented frequency content.

3. Background

  • The two-dimensional discrete Fourier transform represents an image as Fourier coefficients indexed by spatial-frequency coordinates.For a bandlimited image, recovery without error requires sampling rates above the Nyquist rates for its bandwidths.Source: §3 · p. 3
  • The reduced spectrum is the normalized azimuthal average of Fourier-coefficient magnitudes and indicates signal strength across spatial frequencies.The study focuses on the highest 25% of normalized spatial frequencies, from 0.75 to 1.0.Source: §3 · p. 3
  • The Sampling Theorem states that a bandlimited image can be recovered without error when sampling rates exceed the Nyquist rates set by its bandwidths.The image’s frequency content is bounded to a region of the frequency plane.Source: §3 · p. 3

4. Last Upsampling Operation and Fourier Discrepancies

The paper locates the high-frequency discrepancy at the generator’s final upsampling step, which it separates into feature-map scaling and convolution. It compares scaling methods and describes how convolution can modify their frequency content.

4. Last Upsampling Operation and Fourier Discrepancies

  • Because discrete-signal maximum frequency is constrained by spatial resolution and reported discrepancies occur at the highest frequencies, the paper focuses on the generator’s final upsampling step.It divides that step into feature-map scaling followed by convolution.Source: §4 · p. 3

4.1. Feature Map Scaling

  • Feature-map scaling enlarges both input dimensions and may use zero insertion, nearest-neighbor interpolation, or bilinear interpolation.Zero insertion places zeros between rows and columns; nearest interpolation inserts neighboring values, while bilinear interpolation uses weighted averages of adjacent values.Source: §4.1 · p. 3
  • Zero insertion introduces the most high-frequency content, followed by bilinear and nearest interpolation, while the paper emphasizes spectral trends over exact frequency values.The accompanying schematic illustrates normalized reduced spectra after upsampling a 128x128 CelebA reference image.Source: §4.1 · p. 3, Figure 2 · p. 4

4.2. Convolution

  • The convolution following scaling learns kernels that can suppress or amplify high frequencies to satisfy the optimization objective.The paper gives Gaussian-kernel suppression and bilateral-kernel amplification as examples.Source: §4.2 · p. 4

5. Experiments

The experiments isolate last-layer upsampling choices in a controlled test bed, varying feature-map scaling, kernel size, and convolutional-block count across GANs.

5. Experiments

  • The test bed uses CelebA at 128x128 with DCGAN, LSGAN, and WGAN-GP models to vary the two operations independently.The baseline models use 4x4 transpose convolutions, as do many out-of-the-box GAN architectures.Source: §5 · p. 4
  • Experiment codes identify feature-map scaling, convolutional-block count, and kernel size for the last upsampling step.The study focuses on upscaling by a factor of 2; Baseline denotes the original transpose-convolution setup.Source: §5 · p. 4, Table 1 · p. 4
  • Scaling comparisons use zero insertion, nearest interpolation, and bilinear interpolation with a single 5x5 final convolution.The odd-sized kernel maintains spatial size after scaling.Source: §5 · p. 4
  • Kernel-size sweeps compare 3, 5, and 7, while block-count sweeps compare one and three convolutional blocks.Both sweeps use the test bed's scaling variants.Source: §5 · p. 4

6. Metrics

The paper assesses spectral consistency qualitatively and uses FID to check that sample quality remains comparable to baseline experiments.

6. Metrics

  • Spectral consistency means that synthetic images show power-spectrum decay behavior similar to their training data.Because defining a consistent metric is nontrivial, the experiments compare spectral distributions qualitatively using curves from 4,000 real and GAN images.Source: §6 · p. 4
  • FID scores check that the experimental setups were trained properly and remain consistent with baseline sample-quality scores.The passage does not provide a numerical FID threshold.Source: §6 · p. 5

7. Results

Across the experiments, interpolation-based last-layer scaling usually avoids the spectral discrepancies seen with zero insertion, while increasing kernel count alone does not ensure consistency.

7.1. Effect of Feature Map Scaling Methods

  • Nearest and bilinear interpolation reasonably reproduced real-data spectral distributions across all three GANs, unlike zero insertion and Baseline.The only model change was the last layer's feature-map scaling method.Source: §7.1 · p. 5

7.2. Effect of Kernel Size

  • Smaller kernels produced more turbulent spectral behavior in N.1.3 and B.1.3 LSGAN, while zero-insertion methods remained distorted even with larger kernels.Nearest- and bilinear-interpolation experiments generally reproduced real-data spectral distributions across kernel sizes.Source: §7.2 · p. 5

7.3. Effect of Number of Kernels

  • Increasing kernel count did not ensure consistency; apart from N.3.5 DCGAN, nearest- and bilinear-interpolation experiments approximated real-data spectral behavior.The authors note that GAN objectives impose no spectral requirements, so kernels have no direct incentive to impose spectral consistency.Source: §7.3 · p. 5
  • 22/24 experiments using bilinear or nearest interpolation avoided high-frequency discrepancies, while zero-insertion methods, including Baseline, consistently showed them across 39 experiments.FID scores for all experiments were comparable with baseline FID scores.Source: §7.3 · p. 5

8. Fourier Synthetic Image Detector

The experiments test whether Dzanic et al.'s [12] KNN detector withstands images generated with alternative last-layer scaling. Those interpolation-based setups bypassed the classifier, including when its training set was expanded.

8. Fourier Synthetic Image Detector

  • The study evaluates the robustness of the Fourier-attribute KNN detector proposed by Dzanic et al. [12] against spectral-consistent GAN samples.The cited detector uses high-frequency Fourier attributes to distinguish synthetic images.Source: §8 · p. 5, §8.1 · p. 5

8.1. Experiment Setup

  • The detector is trained separately for DCGAN, LSGAN, and WGAN-GP using Baseline GAN images, then tested on images from the test-bed setups.The study also repeats training with 50% rather than the original work's 10% of the data.Source: §8.1 · p. 5

8.2. Detection Results

  • All nearest- and bilinear-interpolation setups bypassed the KNN classifier, even when it used 50% training data.The authors conclude that the detector fails on images from otherwise identical GAN models with these last feature-map scaling changes.Source: §8.2 · p. 5

9. Extended experiments

The extended experiments test whether last-layer feature-map scaling, rather than kernel changes, can yield spectral consistency across datasets and GAN settings. Nearest and bilinear scaling produce consistent results in several setups, and bilinear-scaled StarGAN samples bypass the detector.

9. Extended experiments

  • Nearest and bilinear interpolation produce spectral consistency in the reported scaling experiments, while smaller kernels mostly preserve it and larger kernels do not substantially alter discrepancies in Z.1.7.The detection table reports successful detection rates for CelebA images tested against GAN-specific detectors; its caption says nearest- or bilinear-scaled images are consistently detected at rates below 10%.Source: Figure 3 · p. 6, Figure 4 · p. 6, Table 2 · p. 6

9.1. LSUN Bedrooms Dataset

  • On LSUN Bedrooms at 128x128, N.1.5 and B.1.5 produce spectrally consistent images and can bypass the detector.The experiments use a subset of LSUN Bedrooms to train DCGAN, LSGAN, and WGAN-GP; the detector uses 10% training data.Source: §9.1 · p. 6, Table 3 · p. 7

9.2. Image-to-Image Translation

  • For StarGAN on CelebA at 256x256, B.1.5 produces spectral consistency and bypasses the detector, whereas N.1.5 retains high-frequency discrepancies.The authors present bilinear scaling as further evidence that discrepancies are not inherent to GANs.Source: §9.2 · p. 7, Table 4 · p. 7

10. Spectral Regularization

The paper contrasts its last-layer scaling change with spectral regularization, which penalizes spectral distortions through the generator loss. It reports achieving spectral consistency without that regularizer or changes to the objective functions.

10. Spectral Regularization

  • Modifying the last feature-map scaling operation achieves spectral consistency without spectral regularization or changes to the objective functions.Spectral regularization adds a generator-loss term to penalize spectral distortions and entails computational overhead for calculating reduced image spectra during training.Source: §10 · p. 7
  • Spectral regularization adds a generator-loss penalty for spectral distortions and incurs computational overhead during training.The overhead comes from calculating reduced spectra for images.Source: §10 · p. 7
  • The reported scaling-based approach achieves spectral consistency for DCGAN, LSGAN, and WGAN-GP without changing their objective functions.The comparison is made against spectral regularization results reported for those models and DRAGAN on CelebA at 128x128.Source: §10 · p. 7

11. Discussion

The discussion presents counterexamples showing that high-frequency spectral discrepancies are not inherent to CNN-generated images, while qualifying that last-layer scaling will not fix them in every situation. It also reports evidence from visual-quality comparisons, standard deviations, spectral-regularization experiments, and higher-resolution reconstruction.

11. Discussion

  • WGAN-GP samples retain comparable visual quality when the last transposed convolutions are replaced with nearest or bilinear methods.The displayed samples compare the baseline with N.1.5 and B.1.5 on CelebA.Source: Figure 7 · p. 8
  • Across multiple GAN architectures, loss functions, datasets, and resolutions, nearest or bilinear scaling at the last upsampling step can avoid high-frequency discrepancies, but is not claimed to work in every situation.The authors use these counterexamples to argue that the discrepancies are not inherent to CNN-generated images.Source: §11 · pp. 7–8

A. Standard Deviation of Experiments

  • Spectral-distribution standard deviations for the reported CelebA, LSUN, StarGAN, and spectral-regularization experiments are described as within an acceptable range.The appendix includes standard deviations omitted from the paper's earlier graphs for readability.Source: A. Standard Deviation of Experiments · p. 11

B. Spectral Regularization

  • The authors leave analysis of spectral regularization out of scope, noting that its source code uses the N.3.5 setup together with SR3.They report that N.3.5 alone is sufficient to achieve spectral consistency in identical setups.Source: B. Spectral Regularization · p. 11

C. Higher Resolution Experiments

  • At 512x512, autoencoder reconstructions from N.1.5 and B.1.5 are spectrally consistent, unlike Baseline and Z.1.5 reconstructions.The experiments use a subset of CelebA-HQ for image reconstruction.Source: C. Higher Resolution Experiments · p. 11

D. Samples

Supplementary experiments examine detector evasion, full-spectrum classification, and spectral behavior across game-engine imagery and additional GANs. The appendices also provide FID comparisons, generated examples, detector procedures, and dataset details.

D. Samples; J. Dataset Details

  • The appendix supplies Baseline, Z.1.5, N.1.5, and B.1.5 samples for three GANs, StarGAN translations, and CelebA-HQ reconstructions, alongside dataset and implementation details.CelebA uses its official 162,770-image training subset; LSUN uses a random 200,000-image subset, and autoencoder experiments use 1,000 CelebA-HQ images.Source: D. Samples · p. 11, J. Dataset Details · p. 18

E. FID scores

  • Nearest and bilinear interpolation produce FID scores comparable to or better than the baseline across all GAN setups.Each score uses 50,000 real and 50,000 GAN images.Source: E. FID scores · p. 11, Table 5 · p. 11

F.2. Additional Results

  • Additional detector evaluations use 50% training data for CelebA, LSUN, StarGAN, and reconstructions; the captions report bypass for StarGAN B.1.5 and reconstruction N.1.5 and B.1.5.The additional CelebA, LSUN, and StarGAN detection results are reported in Tables 6–8, with reconstruction results in Table 9.Source: §F.2 · p. 12, Table 8 · p. 13, Table 9 · p. 17

G.1. SVM and MLP classifiers

  • SVM and MLP detectors can be bypassed by replacing zero insertion with nearest or bilinear scaling in the final layer, with similar detection rates at 50% and 80% training data.Results were averaged over 10 runs; Tables 10 and 11 report the classifier results.Source: §G.1 · p. 12, Table 10 · p. 17, Table 11 · p. 17

G.2. Using entire spectrum as features

  • Nearest and bilinear final-layer scaling leave real and generated images non-separable using the entire 88-dimensional spectrum for 128x128 images.Table 12 reports the full-spectrum classifier results.Source: §G.2 · p. 12, Table 12 · p. 17

I. CRN/ IMLE

H. Virtual KITTI

  • Virtual KITTI images synthesized in Unity show no high-frequency spectral discrepancies relative to real KITTI images.The comparison uses center-cropped 370x370 images.Source: H. Virtual KITTI · p. 13, Figure 22 · p. 18

Related papers

Loading 2103.17195v1…