PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards the Detection of Diffusion Model Deepfakes

Jonas Ricker, Simon Damm, Thorsten Holz, Asja Fischer

arXiv:2210.14571v4cs.CV

TL;DR

The paper addresses the limited evidence on detecting diffusion-model deepfakes, a problem with implications for disinformation and digital trust. It evaluates GAN-based detectors and retraining across GANs and diffusion models, finding that pretrained detectors degrade on diffusion images, while retraining enables strong detection and generalizes asymmetrically to GANs.

  • Problem

    Detection of diffusion-model-generated images is underexplored despite the societal risks posed by highly realistic deepfakes and the stronger prior focus on GANs.

  • Method

    The paper experimentally evaluates existing GAN detectors and retrained detectors across five GANs and five diffusion models, using AUROC, Pd@FAR, feature-space analysis, and frequency analysis.

  • Results

    Existing GAN detectors lose 15.2% average AUROC on diffusion-model images, whereas retraining on diffusion images enables strong detection and reaches 94.26% average Pd@1%FAR on GAN images.

  • Takeaways & Limitations

    Diffusion-model images contain detectable features but fewer artifacts than GAN images, helping explain why diffusion-trained detectors generalize to GANs while GAN-trained detectors do not generalize as well to diffusion images.

Abstract

from arXiv · show

In the course of the past few years, diffusion models (DMs) have reached an unprecedented level of visual quality. However, relatively little attention has been paid to the detection of DM-generated images, which is critical to prevent adverse impacts on our society. In contrast, generative adversarial networks (GANs), have been extensively studied from a forensic perspective. In this work, we therefore take the natural next step to evaluate whether previous methods can be used to detect images generated by DMs. Our experiments yield two key findings: (1) state-of-the-art GAN detectors are unable to reliably distinguish real from DM-generated images, but (2) re-training them on DM-generated images allows for almost perfect detection, which remarkably even generalizes to GANs. Together with a feature space analysis, our results lead to the hypothesis that DMs produce fewer detectable artifacts and are thus more difficult to detect compared to GANs. One possible reason for this is the absence of grid-like frequency artifacts in DM-generated images, which are a known weakness of GANs. However, we make the interesting observation that diffusion models tend to underestimate high frequencies, which we attribute to the learning objective.

1 INTRODUCTION

Diffusion models produce highly realistic images that create societal risks, while their detection remains underexplored. This study evaluates GAN-based detectors on diffusion-model images and finds severe cross-family degradation before retraining, followed by strong detection and asymmetric generalization.

  • Motivation: Diffusion models generate highly realistic images that humans can hardly distinguish from real photographs, creating risks for disinformation and public trust.DM-based text-to-image systems can support targeted narratives with little technical knowledge.
  • Study objective: The paper conducts an extensive experimental study of whether effective GAN-image detectors can detect diffusion-model-generated images.The evaluation covers five state-of-the-art GANs and five diffusion models.
  • Main findings: 15.2% average AUROC degradation occurs when existing GAN detectors are applied to diffusion-model images rather than GAN images.This result indicates substantial cross-family performance loss for pretrained detectors.
  • Main findings: Retraining detectors on diffusion-model images drastically improves detection accuracy and enables detection of GAN-generated images, whereas the reverse direction does not hold.The paper reports this asymmetric generalization alongside the retraining result.
  • Interpretation: Feature-space analysis suggests diffusion models are harder to detect because they produce fewer artifacts, particularly in the frequency domain.The study also observes a mismatch toward higher frequencies and attributes it to the training objective favoring perceptual quality over high-frequency reproduction.

2 RELATED WORK

Prior deepfake detection methods exploit semantic, spatial, or frequency artifacts and learned neural features, but diffusion-model forensics remains comparatively limited. Existing studies examine physical inconsistencies, reconstruction errors, prompts, and frequency-domain behavior, including possible differences from GAN artifacts.

  • Fake Image Detection: Deepfake detectors use semantic inconsistencies, spatial or frequency artifacts, and neural networks that learn forensic feature representations.These approaches form two broad groups: artifact-based methods and learned-feature methods.
  • Diffusion-model forensics: Early diffusion-model forensics identified physically implausible lighting and perspective in images generated by DALL·E 2.These studies provide initial analyses of DM-generated image properties.
  • Diffusion-model forensics: Other methods classify diffusion-model images using reconstruction differences or by incorporating the generation prompt or a generated caption.Prompt information improves classification when available or captioned for real images without prompts.
  • Frequency artifacts: GAN-generated images exhibit periodic, grid-like spectrum artifacts associated with transposed convolutions and observable in frequency representations.These artifacts were reproduced and extended from Fourier spectra to the discrete cosine transform.
  • Frequency artifacts: Whether diffusion-model images contain grid-like frequency patterns depends strongly on the specific model.This contrasts with the established frequency-artifact findings for GANs.
  • Frequency artifacts: Diffusion models add higher frequencies to existing lower frequencies during reverse sampling, motivating work on Fourier features for high-frequency learning.The cited literature reports improved likelihoods from adding Fourier features in one case.

3 BACKGROUND ON DMs

Diffusion models generate images by reversing a noise-adding process with a neural-network approximation of the intractable reverse conditional. Common training predicts the noise to remove using a weighted mean-squared-error objective.

  • Model formulation: Diffusion models are probabilistic generative models commonly formulated through DDPM or score-based modeling perspectives.Numerous modifications improve perceptual quality and sampling speed.
  • Forward and reverse processes: The forward process is designed so that the terminal state approximates Gaussian noise, x_T ∼ N(0,I), from which denoising begins.The reverse process iteratively samples toward a clean image.
  • Reverse process: Because the true reverse conditional is intractable, a deep neural network approximates q(x_t−1|x_t) during denoising.The approximation is represented by a parameterized mean and covariance.
  • Reverse process: The reverse transition is modeled as a Gaussian with mean µ_θ(x_t,t) and covariance Σ_θ(x_t,t).The covariance may instead be fixed as in the original DDPM formulation.
  • Training objective: Training commonly predicts the noise ε_θ(x_t,t) and minimizes the squared error between predicted and true noise.This objective can also be interpreted as a weighted ELBO with data augmentation.

4 DATASET

The study controls comparisons by evaluating models with available checkpoints or samples from LSUN Bedroom at 256×256 resolution. Its dataset spans five GANs and five diffusion models, with model quality contextualized by reported FID values.

  • Dataset selection: The evaluation uses LSUN Bedroom images at 256×256 resolution to reduce biases from mismatched datasets, checkpoints, or samples.The authors require compatible pretrained resources for technical comparability.
  • Models: The dataset contains ten generative models: five GANs and five diffusion models.Samples are downloaded or generated using original-publication code and pretrained models.
  • Models: The evaluated GANs include ProGAN, StyleGAN, and ProjectedGAN, alongside diffusion-related variants.The passage identifies both seminal and more recent GAN models.
  • Dataset characterization: Table 1 reports Fréchet inception distances for the evaluated models, where lower FID indicates higher image quality.The values come from the original publications and a cited source for IDDPM.
  • Models: The diffusion-model set includes DPM, ADM, PNDM, and LDM, with ADM reported at an FID of 1.90 on LSUN Bedroom.PNDM accelerates sampling, while LDM operates through a latent-space autoencoder.

5 DETECTION ANALYSIS

The analysis evaluates existing and re-trained detectors on diffusion-model images, using AUROC and Pd@1%FAR to assess performance and cross-generator generalization. Re-training substantially improves detection and reveals that diffusion-generated images contain detectable, partly shared features, although their learned representations overlap more with real images than GAN-generated representations.

  • Pre-trained detectors: Existing GAN detectors lose 15.2% AUROC on average when applied to diffusion-model images, and the best model detects only 25.7% of fakes at 1% false positives.The evaluation uses 20,000 balanced real and generated samples; Gragnaniello2021 achieves 91.4% AUROC but only 25.7% Pd@1%FAR in the realistic low-false-positive setting.
  • Re-trained detectors: Re-trained detectors achieve near-perfect scores on their training datasets, showing that diffusion-generated images contain features that detectors can learn.Figure 1 reports AUROC and Pd@1%FAR using 20,000 held-out test samples per generator, with diagonal values representing matched training and test datasets.
  • Re-trained detectors: A detector trained solely on ADM images exceeds 90% Pd@1%FAR on every other tested diffusion model, indicating shared features across architectures and training procedures.This compares cross-diffusion-model generalization for a detector trained on one diffusion model with detectors trained on single GANs.
  • Cross-family generalization: Diffusion-trained detectors detect 94.26% of GAN samples on average, whereas GAN-trained detectors detect only 26.34% of diffusion samples at 1% false alarm rate.The comparison uses detectors trained on all GANs versus all diffusion models and evaluates cross-family generalization.
  • Feature-space analysis: t-SNE features show clearer separation between real and GAN-generated images but greater overlap between real and diffusion-generated images.The visualization uses representations before Wang2020’s final fully connected layer; the analysis links this overlap to fewer diffusion-specific artifacts.
  • Feature-space analysis: Diffusion-generated images appear to contain fewer family-specific artifacts, while detectors trained on them learn features shared by GANs and diffusion models.The paper argues that GAN-trained detectors focus on GAN-specific artifacts, whereas diffusion-trained detectors learn artifacts present across both generator families.

6 FREQUENCY ANALYSIS

Frequency analysis finds that traditional diffusion models largely lack GAN-like grid artifacts but systematically underestimate high-frequency content. This mismatch emerges during late denoising steps and is linked to the training objective.

  • Frequency artifacts: GANs show regular grid artifacts in their DFT spectra, whereas most diffusion models show few or no visible frequency artifacts.LDM is an exception, exhibiting a weaker grid that may result from its adversarially trained autoencoder.
  • Spectrum discrepancies: Traditional diffusion models do not reproduce the elevated high frequencies characteristic of most GANs.All examined diffusion models instead tend to underestimate spectral density toward the high-frequency end.
  • Detectability: DM-generated images are less accurately classified than GAN-generated images in both frequency and pixel space, despite frequency features improving DM classification.This supports the hypothesis that fewer detectable artifacts make diffusion outputs harder to detect.
  • Denoising process: The relative spectral error decreases during denoising but reaches its optimum around t ≈ 10 rather than t = 0.The final denoising steps move the spectrum away from real-image statistics, even though images at the optimum still contain visible noise.
  • Sampling steps: Increasing the number of denoising steps reduces high-frequency underestimation.This agrees with prior findings that more steps improve reproduction of higher frequencies.
  • Training objective: Late denoising steps near t = 0 have the highest MSE and govern high-frequency content.The authors attribute the mismatch to objectives that down-weight these steps relative to the variational lower bound, trading spectral fidelity for perceptual quality.

7 CONCLUSION

The paper concludes that diffusion deepfake detection is feasible after adapting existing detectors to diffusion-generated images. Its analysis suggests that diffusion models are harder to detect because they produce fewer artifacts, while retaining a systematic high-frequency mismatch.

  • Conclusion: Re-training state-of-the-art detectors enables successful discrimination between real and diffusion-generated images.The conclusion presents this as a step toward detecting diffusion-model deepfakes.
  • Conclusion: Detectors trained on diffusion images generalize to GAN images, whereas detectors trained on GAN images do not generalize in the opposite direction.The authors connect this asymmetric transfer to the relative abundance of detectable artifacts.
  • Conclusion: Diffusion models predominantly lack the frequency-domain weakness characteristic of GAN-generated images.The paper nevertheless identifies a systematic underestimation of spectral density and attributes it to the diffusion-model loss function.

A.2 Additional Datasets

The additional-dataset analysis broadens evaluation to facial and text-to-image data while documenting dataset preparation and detector fine-tuning. It also compares established GAN-detection approaches and their reported generalization behavior.

  • A.2 Additional Datasets: P2 and ADM’ are additional diffusion models trained on FFHQ and included because facial images are relevant to deepfakes.ADM’ has 93 million parameters, while P2 uses a modified weighting scheme emphasizing perceptually rich diffusion steps.
  • A.2 Additional Datasets: The evaluation collects 10,000 real and 10,000 generated images for LSUN, ImageNet, and FFHQ after model-matched resizing and cropping.Stable Diffusion samples use DiffusionDB prompts, while Midjourney contributes 300 collected images and LAION-Aesthetics supplies real images.
  • B.1 Descriptions of Detectors: Wang2020 trains a ResNet-50 on real and ProGAN images and reports 92.6% average precision with Blur+JPEG (0.1).The work argues that common generator fingerprints support generalization to unseen generators and datasets.
  • B.1 Descriptions of Detectors: Gragnaniello2021 raises average accuracy from 80.71% to 94.42% by removing downsampling from the first layer.The improvement comes at the cost of more trainable parameters.
  • B.1 Descriptions of Detectors: Mandelli2022 ensembles five orthogonal EfficientNet-B4 classifiers whose patch scores are aggregated to detect unseen fake images.Its training data include different content, processing, and generators, including a score-based model.
  • A.2 Additional Datasets: Fine-tuning generally produces results similar to re-training, suggesting that existing knowledge is mostly overwritten during fine-tuning.The comparison is reported across most training and test-data constellations.

B.3 Extended Feature Space Analysis

Feature-space analyses show that detectors trained on diffusion-model images separate GAN-generated images more effectively than the reverse direction. The section also examines perturbation robustness, visual cues, and frequency representations.

  • Feature-space analysis: The feature representations are extracted before the final fully connected layer and compared using MMD and t-SNE.The MMD analysis uses 2048-dimensional detector features, while t-SNE visualizes representative samples in two dimensions.
  • Feature-space analysis: Detectors fine-tuned or trained on diffusion-model images achieve higher MMDs for GAN-generated images than detectors trained on GANs achieve for diffusion-model images.When trained on both model classes, feature-space MMDs are roughly comparable.
  • Image perturbations: For GAN-generated images, blurring and cropping have little effect on detection, whereas compression decreases average AUROC by 1% and added noise by 3.4%.These results use the pre-trained Gragnaniello2021 detector trained on ProGAN images.
  • Image perturbations: For diffusion-model images, blurring and cropping are almost negligible, while JPEG compression causes a 2.42% average AUROC decrease after fine-tuning.Fine-tuning improves robustness to these perturbations but not to Gaussian noise.
  • Fakeness rating: A ProGAN-trained detector assigns higher fakeness scores to many diffusion images containing purely white or black pixels, but this pattern disappears after fine-tuning.Across rankings, detector predictions show no strong correlation with visual quality.
  • Frequency analysis: Diffusion-model images generally exhibit fewer frequency artifacts than GAN images, except that LDM shows artifacts in its DCT spectrum.The section introduces DFT, DCT, and reduced-spectrum representations for analyzing these differences.

C.3 Logistic Regression Experiments

Logistic regression tests whether pixel and frequency representations distinguish real from generated images. Frequency transforms improve accuracy for both model classes, but diffusion-model images remain substantially harder to classify.

  • Classification accuracy: Frequency-domain features improve classification accuracy for both GANs and diffusion models, with average gains of 5.72% and 10.6%, respectively.The best transform is compared against pixel-based classification.
  • Classification accuracy: Average maximum accuracy reaches 90.9% for GANs but only 63.1% for diffusion models.The larger frequency-domain gain for diffusion models does not make them easier to detect overall.
  • Denoising analysis: Relative spectral error stays near zero for much of denoising, but the model fails to reproduce higher frequencies near t = 0.The error is measured relative to the spectrum of noised images at the corresponding diffusion step.
  • Denoising analysis: Logistic-regression accuracy does not decrease around t = 10; images at that stage are noisier, which probably explains their increased detectability.The experiment evaluates pixels and frequency transforms across progressively denoised samples.

C.5 Extended Analysis of Spectrum Discrepancies

The extended analysis links diffusion models’ high-frequency mismatch to reconstruction performance at low noise levels and to the weighting used by their training objectives. More denoising steps improve spectral fidelity and make samples harder to detect.

  • Training objective: Small-noise reconstruction requires modeling fine image details, whereas large-noise reconstruction primarily recovers coarse features.The analysis transfers this denoising-autoencoder observation to diffusion-model training.
  • Training objective: The training objective controls the relative importance of noise levels through a weighting scheme w(t) over reconstruction losses.Prominent diffusion-model objectives are expressed as weighted sums of mean squared error terms.
  • Training objective: Down-weighting low-noise levels in Lsimple is hypothesized to reduce accurate high-frequency modeling, whereas Lvlb gives greater influence to those difficult steps.The authors connect high-frequency reconstruction to successful error prediction near t = 0.
  • Training objective: IDDPM and ADM, trained with Lhybrid, reduce the spectral gap relative to DDPM trained with Lsimple.The authors caution that weighting is not the only factor; successful prediction capacity also matters.
  • Conclusion: The authors conclude that perceptual-quality objectives can leave high-frequency content insufficiently modeled, which likelihood-based Lvlb may capture better.This conclusion contrasts perceptual quality or FID optimization with accurate frequency modeling.
  • Sampling steps: More sampling steps lower spectral error, with DDIM improving faster than normal sampling.This agrees with the observation that DDIM produces better samples at fewer sampling steps.
  • Sampling steps: Images become harder to detect as the number of sampling steps increases and image quality rises.The result is reported from logistic-regression experiments using different timestep counts.

C.7 Frequency Analysis on Additional Datasets

Additional datasets largely reproduce the frequency findings from LSUN Bedroom, while revealing dataset- and training-dependent exceptions. LDM retains grid-like artifacts, whereas ADM and PNDM generally do not.

  • Frequency artifacts: On additional datasets, LDM exhibits grid-like artifacts, while ADM and PNDM generally do not.ADM shows irregular vertical structures on LSUN Cat and LSUN Horse, but they are less pronounced than GAN artifacts.
  • Analysis protocol: The additional-dataset analysis compares DFT, DCT, and reduced spectra to assess artifacts and spectral discrepancies.The corresponding results are presented in Figures 23–25.
  • Dataset variation: ImageNet shows overestimation toward higher frequencies, opposite to ADM’s pattern on other datasets.The authors suggest that different sampling-step counts may contribute and state that spectral discrepancies are highly training dependent.
  • Stable Diffusion: Stable Diffusion has very subtle grid-like artifacts and higher spectral density than real images across the spectrum.The deviations may reflect different real and generated data distributions.
Loading 2210.14571v4…