PaperlayerView exact revision on arXiv

Source-linked AI summary

FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbations

Yonghyun Jeong, Doyeon Kim, Youngmin Ro, Jongwon Choi

arXiv:2202.03347v1cs.CVcs.LGeess.IV

TL;DR

Deepfake detectors can overfit to frequency-level artifacts, limiting detection of images from unknown categories or GAN models. FrePGAN generates perturbation maps and jointly updates them with a classifier, achieving state-of-the-art performance across known and unseen settings.

  • Problem

    Frequency-level artifacts vary across GAN models and categories, causing detectors to overfit training settings and limiting generalization to unseen deepfakes.

    Source: Introduction · p. 1

  • Method

    FrePGAN generates frequency-level perturbation maps for real and fake images while alternating updates with a deepfake classifier.

    Source: Introduction · p. 2, Frequency Perturbation GAN · p. 3

  • Results

    The model achieves state-of-the-art performance in both known and unseen settings.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    Reducing domain-specific frequency artifacts while retaining general image-level irregularities supports robust detection across unknown categories, GAN models, manipulations, and resizing.

    Source: Introduction · p. 2, Conclusion · p. 7

Abstract

from arXiv · show

Various deepfake detectors have been proposed, but challenges still exist to detect images of unknown categories or GAN models outside of the training settings. Such issues arise from the overfitting issue, which we discover from our own analysis and the previous studies to originate from the frequency-level artifacts in generated images. We find that ignoring the frequency-level artifacts can improve the detector's generalization across various GAN models, but it can reduce the model's performance for the trained GAN models. Thus, we design a framework to generalize the deepfake detector for both the known and unseen GAN models. Our framework generates the frequency-level perturbation maps to make the generated images indistinguishable from the real images. By updating the deepfake detector along with the training of the perturbation generator, our model is trained to detect the frequency-level artifacts at the initial iterations and consider the image-level irregularities at the last iterations. For experiments, we design new test scenarios varying from the training settings in GAN models, color manipulations, and object categories. Numerous experiments validate the state-of-the-art performance of our deepfake detector.

Introduction

Deepfake detectors can overfit to frequency artifacts that vary across GAN models and object categories. FrePGAN addresses this by perturbing frequency-level artifacts while retaining image-level irregularities, achieving strong performance in known and unseen settings.

  • Frequency artifacts are easily detected but vary by GAN model and object category, encouraging detectors to overfit training settings.Source: Figure 1 · p. 1, Introduction · p. 1
  • The proposed framework combines FrePGAN with a deepfake classifier to reduce domain-specific frequency artifacts and detect general image-level irregularities.Source: Introduction · p. 2
  • The framework is evaluated under known settings and unseen categories, models, and manipulations.Source: Introduction · p. 2
  • The model achieves state-of-the-art performance in both known and unseen settings.Source: Introduction · p. 2
  • FrePGAN generates perturbation maps that are added to real and fake images to reduce the effect of domain-specific artifacts.Source: Introduction · p. 2

Related Work

Prior deepfake-detection research uses physiological, image-based, and frequency-based cues. These approaches address artifacts, lighting, prediction errors, and spectral traces, but reported limitations include sensitivity to non-facial domains, resizing, and post-processing.

  • Existing detectors are categorized into physiological feature-based, image-based, and frequency-based approaches.Source: Related Work · p. 2
  • Face-focused methods use facial features, head poses, and eye-blinking cues but can be ineffective in non-facial domains.Source: Physiological Feature-based Detection · p. 2
  • Image-based methods analyze compression, demosaicing, lighting, prediction errors, and RGB inconsistencies across manipulations.Source: Image-based Detection · p. 2
  • Frequency-based studies use spectral traces, prediction-residue variance, Fourier transforms, singular-value decomposition, and GAN-specific fingerprints.Source: Frequency-based Detection · p. 2

Deepfake Detection Framework

The deepfake detection framework combines FrePGAN with a classifier. FrePGAN generates frequency-level perturbation maps for both real and fake images, while the classifier distinguishes them after artifact reduction.

Training of Deepfake Detection Framework

The framework trains FrePGAN and the deepfake classifier together with alternating updates. Early training can exploit easy frequency artifacts, while later perturbations encourage features that generalize across GAN models.

Frequency Perturbation GAN

FrePGAN uses frequency-domain perturbations and adversarial training to suppress GAN-specific artifacts before classification. Its generator transforms images into frequency maps, produces perturbations, and is trained with a discriminator and compression objective.

Architecture

  • FrePGAN combines a perturbation map generator with a perturbation discriminator and deepfake classifier.The generator produces maps for both real and fake images, while the classifier distinguishes them using pixel-level information.Source: Frequency Perturbation GAN · p. 3

Frequency Perturbation

  • Frequency-domain processing targets high-frequency artifacts that distinguish real and fake images and can cause detector overfitting.The generator receives FFT-transformed images because the artifacts mainly occur in high-frequency components.Source: Perturbation Map Generator · p. 3, Figure 3 · p. 3
  • The generator applies FFT, frequency-level generation, and inverse FFT to produce a pixel-level perturbation map matching the input image size.The frequency map doubles the channel count by separating real and imaginary components, then the generated map is transformed back to pixel space.Source: Perturbation Map Generator · pp. 3–4

Adversarial Training

  • Adversarial training makes perturbed images difficult for the perturbation discriminator to distinguish from real images.For real inputs, the ideal perturbation is empty; for fake inputs, perturbations should make their distribution resemble real images.Source: Perturbation Discriminator · p. 4, Training of FrePGAN · p. 4
  • The perturbation generator minimizes adversarial and compression losses while alternating updates with the perturbation discriminator.The compression term limits perturbation magnitude, and the adversarial and discriminator losses are optimized through alternating training steps.Source: Training of FrePGAN · p. 4

Deepfake Classifier

The deepfake classifier distinguishes generated images from real images using perturbed inputs and is trained with cross-entropy loss. Its training procedure minimizes the classifier loss over paired real and fake labels.

Deepfake Image Prediction

At inference, the trained perturbation generator modifies each new image before the trained classifier produces its real-or-fake prediction. The final decision is therefore based on the classifier's output for the perturbed image.

Implementation Details

The implementation uses established neural architectures for perturbation generation, discrimination, and classification, with Adam optimization and a 20-epoch training schedule.

Experimental Results

Experiments evaluate FrePGAN across known and unseen categories, models, manipulations, and resolutions. The model consistently demonstrates robust detection, including strong cross-model and resizing performance.

Experimental Setup

  • The evaluation covers known and unseen domains, including manipulated faces, resized faces, unknown categories, and unknown generative models.Average precision and accuracy are used for comparison with image-based and frequency-based state-of-the-art detectors.Source: Experimental Results · p. 5, Deepfake Detection Performance · pp. 5–6
  • The experiments use ProGAN training data and test across multiple datasets, GAN architectures, object categories, and image resolutions.Unknown-category tests use horse images, while manipulation and resizing experiments use 1,024 × 1,024 face data.Source: Dataset · p. 5

Manipulation Robustness

  • FrePGAN achieves superior robustness under seven image manipulations, including hue, brightness, saturation, gamma, contrast, blur, and rotation.The experiments use manipulated ProGAN face images at 1,024 × 1,024 resolution.Source: Deepfake Detection of Manipulated Face… · p. 6

Resizing Robustness

  • 100% performance is maintained from 1,024 × 1,024 through 256 × 256, while performance remains at least 97.8% at 128 × 128 and 64 × 64.FrePGAN outperforms all compared models across the five tested resolutions.Source: Deepfake Detection of Resized Face… · p. 6

Unknown Categories

  • FrePGAN is the most robust model across 20 unknown categories, even as the number of training classes and input types varies.The results indicate that the model detects both the tested category and other categories across varied testing environments.Source: Deepfake Detection of Unknown Categories · p. 6

Unknown Models

  • Across eight generative models, FrePGAN achieves the highest accuracy and average precision for ProGAN, StyleGAN, GauGAN, and Deepfake.It performs similarly when trained with one category or four categories, while the supplied passage continues with results for four additional models.Source: Deepfake Detection of Unknown GAN Models · pp. 6–7

Ablation Study

Ablation and visualization experiments show that frequency-level perturbation generation, adversarial learning, and compression loss jointly support generalization. The perturbations reduce frequency artifacts while largely preserving pixel-level appearance.

Ablation Study

  • The complete framework reaches the best ablation performance, with the frequency-level generator, adversarial learning, and compression loss used together.The reported best performance is (91.8, 94.6).Source: Ablation Study · p. 7
  • Using compression and adversarial losses together improves generalization, while replacing the frequency-level generator with a pixel-level generator sharply reduces performance.The ablation attributes the difference to the quality of the generated perturbation maps.Source: Ablation Study · p. 7

Perturbation Visualization

  • Adding perturbation maps greatly reduces the power-spectrum difference between real and fake images caused by high-frequency artifacts.The visualization supports the framework’s goal of making real and fake images more similar at the frequency level.Source: Visualization of Perturbation Maps · p. 7
  • Perturbation maps preserve the pixel-level image and two-dimensional power spectrum while magnifying high-frequency components in the one-dimensional spectrum.The resulting one-dimensional spectrum becomes similar to that of fake images containing frequency-level artifacts.Source: Visualization of Perturbation Maps · p. 7, Figure 6 · p. 7

Conclusion

FrePGAN targets deepfake-detector generalization by reducing domain-specific frequency artifacts and jointly training the perturbation generator and classifier. Experiments support robust detection across unknown categories, GAN models, manipulations, and resizing.

Conclusion

  • FrePGAN generates perturbations that reduce domain-specific artifacts in generated images and supports generalized detection across varied testing scenarios.The tested scenarios include unknown categories, GAN models, manipulations, and resizing.Source: Conclusion · p. 7
  • Alternate updates of the deepfake classifier and perturbation generator are validated as helpful for improving detector generalization.The framework uses perturbation generation and compression losses during training.Source: Conclusion · p. 7

Related papers

Loading 2202.03347v1…