PaperlayerView exact revision on arXiv

Source-linked AI summary

Fighting deepfakes by detecting GAN DCT anomalies

Oliver Giudice, Luca Guarnera, Sebastiano Battiato

arXiv:2101.09781v4cs.CV

TL;DR

Deepfake detectors are described as lacking generalizability and explainability, while distinguishing fakes from their source images can be difficult. The paper analyzes DCT coefficients to identify GAN-specific frequency fingerprints, and reports that its approach outperforms state-of-the-art methods and generalizes across tested datasets.

  • Problem

    The paper identifies limited generalizability and explainability in existing detectors, and notes that distinguishing generated faces from their source datasets can be difficult.

    Source: §I · p. 1, §V · p. 6

  • Method

    CTF-DCT analyzes DCT AC-coefficient distributions to identify GAN-specific frequency bands and feeds the resulting 63 β features to a gradient-boosting classifier.

    Source: §I · p. 1, §III · p. 5, A. Finalizing the CTF Approach · p. 8

  • Results

    The authors report that CTF-DCT outperforms state-of-the-art methods and that GSF analysis generalizes across the involved datasets, with β values discriminating binary tests.

    Source: §I · p. 1, §V · p. 7

  • Takeaways & Limitations

    The authors report that GSF analysis can help explain unusual Deepfake artifacts, while the full classifier is designed for robustness to plausible image attacks.

    Source: A. Finalizing the CTF Approach · p. 8, §VII · p. 11

Abstract

from arXiv · show

To properly contrast the Deepfake phenomenon the need to design new Deepfake detection algorithms arises; the misuse of this formidable A.I. technology brings serious consequences in the private life of every involved person. State-of-the-art proliferates with solutions using deep neural networks to detect a fake multimedia content but unfortunately these algorithms appear to be neither generalizable nor explainable. However, traces left by Generative Adversarial Network (GAN) engines during the creation of the Deepfakes can be detected by analyzing ad-hoc frequencies. For this reason, in this paper we propose a new pipeline able to detect the so-called GAN Specific Frequencies (GSF) representing a unique fingerprint of the different generative architectures. By employing Discrete Cosine Transform (DCT), anomalous frequencies were detected. The \BETA statistics inferred by the AC coefficients distribution have been the key to recognize GAN-engine generated data. Robustness tests were also carried out in order to demonstrate the effectiveness of the technique using different attacks on images such as JPEG Compression, mirroring, rotation, scaling, addition of random sized rectangles. Experiments demonstrated that the method is innovative, exceeds the state of the art and also give many insights in terms of explainability.

I. INTRODUCTION

The paper proposes CTF-DCT, a frequency-based, white-box approach to Deepfake detection, against methods described as black-box and difficult to generalize. Its introduction also situates the work among GAN generation and detection methods and reports tests of robustness and generalization.

I. INTRODUCTION

  • The paper frames existing deep-learning detectors as black boxes and says robust detection in the wild remains difficult.It also notes their considerable computing-power requirements.Source: §I · p. 1
  • CTF-DCT analyzes DCT coefficients to extract GAN-specific frequency fingerprints and uses gradient boosting to distinguish GAN architectures, which the authors say outperforms state-of-the-art methods.The authors also report generalization to Deepfakes not generated by GAN engines.Source: §I · p. 1
  • The proposed method is described as explainable: estimated Laplacian-distribution parameters identify anomalous frequencies generated by different Deepfake architectures.The paper presents this as a white-box alternative to detectors based on deep architectures.Source: §I · p. 2
  • The introduction reports robustness tests using mirroring, scaling, rotation, and random rectangles, including evaluation on a video dataset not used in training.Source: §I · p. 2

II. RELATED WORKS

  • The related-work discussion names five evaluated GAN architectures: StarGAN, StyleGAN, StyleGAN2, AttGAN, and GDWCT.CelebA supplied images for StarGAN, AttGAN, and GDWCT, while FFHQ supplied images for StyleGAN and StyleGAN2.Source: §II · p. 2, Figure 1 · p. 2

A. Deepfake Generation Techniques of Faces

The section describes face-generation architectures used to produce different kinds of synthetic images. They range from multi-domain attribute translation and style transfer to whole-face synthesis.

A. Deepfake Generation Techniques of Faces

B. Deepfake Detection Techniques

The section reviews detectors that look for generative traces in image frequencies, neurons, or texture. It positions the paper's DCT feature extraction as a different way to investigate Deepfake patterns.

B. Deepfake Detection Techniques

  • Frequency-domain detectors seek anomalous spectral peaks; Zhang et al. emulate GAN up-sampling artifacts, while Durall et al. [26] use frequency analysis with naive classifiers.Source: B. Deepfake Detection Techniques · p. 3
  • FakeSpotter [28] monitors single-neuron behavior and was compared with Zhang et al., achieving over 90% average accuracy across four fake-face types.Source: B. Deepfake Detection Techniques · p. 3
  • DAD-HCNN uses a three-level hierarchy to distinguish unaltered, retouched, or GAN-generated images and identify specific GANs, while Gram-Net analyzes global texture representations.Source: B. Deepfake Detection Techniques · p. 3
  • CNN detectors are described as limited in robustness, generalization, and explainability; Guarnera et al., instead use EM-derived features to distinguish real images, Deepfakes, and FaceApp images.Source: B. Deepfake Detection Techniques · p. 3
  • Unlike the reviewed approaches, this paper investigates discriminative features extracted through the DCT transform to capture Deepfake patterns.Source: B. Deepfake Detection Techniques · p. 3

III. THE CTF APPROACH

The CTF approach analyzes 8×8-block DCT coefficients to identify generative-model patterns through AC-coefficient statistics. It compares those statistics across image sets to derive GAN Specific Frequencies (GSF), which the paper reports can discriminate deepfakes and are not fully resolution-dependent.

III. THE CTF APPROACH

  • The pipeline divides images into non-overlapping 8×8 blocks, applies DCT, and orders the resulting coefficients in JPEG zig-zag order.The DC coefficient represents the block’s average pixel value; the remaining AC coefficients correspond to frequency bands.Source: §III · p. 4
  • AC coefficients are modeled with a zero-centred Laplacian distribution, and the approach uses β statistics inferred from their distributions.The paper contrasts this with modeling the DC coefficient using a Gaussian distribution.Source: §III · pp. 4–5
  • GSF extraction compares dataset AC-coefficient distributions modeled by βDTg using χ² distances, with GSF enabling one-to-one image-set evaluation.The most discriminative DCT frequency is selected greedily, without further frequency ranking or sorting steps.Source: §III · p. 5
  • A single β value can roughly distinguish a deepfake by thresholding a frequency selected under the GSF definition.The reported β trends compare generated images with their source datasets: StyleGAN and StyleGAN2 with FFHQ, and StarGAN, AttGAN, and GDWCT with CelebA.Source: §III · p. 5, Figure 4 · p. 5
  • The paper reports that GSF discriminative power is not fully dependent on the image resolution involved.The approach is motivated by patterns reported in the middle bands of deepfake image frequency spectra.Source: §III · pp. 3, 4, 6

IV. DATASETS DETAILS

Experiments used CelebA and FFHQ as real-image datasets and generated deepfakes with five GAN architectures. The authors collected 3,000 images per architecture and 3,000 images from each real dataset for training and testing.

IV. DATASETS DETAILS

  • StarGAN, GDWCT, and AttGAN manipulated CelebA images, while StyleGAN and StyleGAN2 generated images using FFHQ.The reported output resolutions were 256×256 for StarGAN and AttGAN, 216×216 for GDWCT, and 1024×1024 for both StyleGAN variants.Source: §IV · p. 6
  • The real-image datasets were CelebA, with more than 200,000 celebrity images, and FFHQ, a high-quality face dataset of 1024×1024 PNG images.CelebA images are 178×218 JPEGs; FFHQ images vary in age, ethnicity, and background.Source: §IV · p. 6
  • The experiments collected 3,000 deepfake images per GAN architecture and 3,000 images from each of CelebA and FFHQ for training and testing.The passage leaves the division into training and test sets to the experimental sections.Source: §IV · p. 6

V. DISCUSSION ON GSF

The hardest comparison is between real images and Deepfakes generated from the same source dataset. The discussion uses GSF-based comparisons to examine this challenge, the discriminative value of β, and the approach’s limits.

V. DISCUSSION ON GSF

  • Distinguishing original images from Deepfakes made from those images is a particularly difficult case, including for FFHQ versus StyleGAN.The authors contrast this with easier comparisons such as CelebA versus StyleGAN.Source: §V · pp. 6–7
  • GSF β values were empirically discriminative in binary tests, including the difficult FFHQ–StyleGAN comparison, where the authors report impressive generalization results.The tests used 200 images per set for one-to-one comparisons; the simple classifier used β as its sole feature.Source: §V · p. 7, Figure 6 · p. 7
  • GSF amplification makes abnormal frequencies visible, and the original and amplified Fourier spectra show the same abnormal frequency appearance.The amplification multiplies non-GSF DCT coefficients by k1 and GSF coefficients by k2; the example uses k1 = 0.1 and k2 = 100.Source: §V · p. 7
  • A GSF can identify a set of GAN-generated images, but the authors say it is insufficient on its own for in-the-wild use or fakes not generated by neural approaches.They propose completing it with a more robust feature vector, while noting that this would lose explainability.Source: §V · p. 7

A. Finalizing the CTF Approach

The paper extends GSF analysis into a more robust classifier using the full β feature vector. It also reports that StyleGAN and StyleGAN2 are separable in a binary test.

A. Finalizing the CTF Approach

  • The StyleGAN-versus-StyleGAN2 binary test achieved accuracy close to 99%.The reported GSF values are 63 for each model against FFHQ and 54 for the StyleGAN-versus-StyleGAN2 comparison.Source: A. Finalizing the CTF Approach · pp. 7–8, Figure 7 · p. 8
  • The robust classifier uses all 63 β values as input to Gradient Boosting to identify the most probable GAN and provide visual explainability.The authors selected Gradient Boosting as the best classifier for their data.Source: A. Finalizing the CTF Approach · p. 8

VI. EXPERIMENTAL RESULTS

The experimental section evaluates the finalized CTF approach under image attacks and on FaceForensics++ Deepfake videos. The paper also illustrates abnormal-frequency inspection using a StarGAN image.

VI. EXPERIMENTAL RESULTS

  • The robust classifier was tested on FaceForensics++ Deepfake videos, which were used only during testing to classify real versus fake.The training and validation data comprised 3,000 real and fake images, with 10% used for training and the remainder for testing; cross-validation was carried out.Source: §VI · p. 8
  • Figure 8 presents a StarGAN image, its Fourier spectrum, and an abnormal frequency shown through GSF amplification.The caption says examples of GSF amplification for the generation processes are reported in supplementary material.Source: Figure 8 · p. 8
  • The experiment included multiple image attacks, with examples of attacked images shown in Figure 9.The section also reports testing in a different scenario using FaceForensics++ videos.Source: §VI · p. 8

A. Testing with Noise

The noise tests report that many attacks preserve high detection accuracy, while transformations that remove or alter frequency information can reduce accuracy. The effects depend in part on image resolution and attack settings.

A. Testing with Noise

  • Attacks in the robustness test generally do not destroy GSF, and the authors report high accuracy values for the tested conditions.Table I covers precision, recall, F1-score, and accuracy across raw images and the listed attacks.Source: A. Testing with Noise · p. 8, Table I · p. 10
  • Larger Gaussian blur kernels reduce accuracy, particularly for lower-resolution AttGAN, GDWCT, and StarGAN images.The authors explain that increasing kernel size preserves low frequencies while nearly deleting high frequencies.Source: A. Testing with Noise · p. 8
  • Rotations at 90°, 180°, and 270° preserve frequencies, whereas interpolated rotations reduce accuracy except for the two higher-resolution StyleGAN models.Interpolation adds new image information that can affect frequency information.Source: A. Testing with Noise · p. 9
  • Mirroring does not alter image frequencies and yields extremely high detection accuracy.The described mirror attacks reflect pixels horizontally, vertically, or along both axes.Source: A. Testing with Noise · p. 9
  • Downscaling by 50% can remove high-frequency components, especially in already small images, while upscaling by 50% preserves main frequencies and yields high accuracy.The authors describe the downscaling effect as similar to low-pass filtering.Source: A. Testing with Noise · pp. 9–10
  • JPEG compression at QF 100 does not affect results; accuracy is affected as quality factor decreases, but remains high at QF 50.The authors note that QF 50 preserves mid-level frequencies and is used by common social platforms such as WhatsApp and Facebook.Source: A. Testing with Noise · p. 10

B. Comparison and Generalization Tests

The CTF approach is reported to outperform prior methods on shared GAN architectures and to generalize to FaceForensics++ without training on that dataset. The tests also suggest that GAN-specific frequencies reflect reenactment as well as the generative process.

B. Comparison and Generalization Tests

  • 99.9% accuracy: the CTF approach achieves the best reported results against AutoGAN, FakeSpotter, and Expectation-Maximization on shared GAN architectures.Table II compares real CelebA and FFHQ images with Deepfake images.Source: B. Comparison and Generalization Tests · p. 10, Table II · p. 10
  • FaceForensics++ images had a GSF value of 61, close to StyleGAN's, suggesting GSF depends on reenactment as well as the generative process.The authors relate this to Face2Face's analytical reenactment and StyleGAN's trained reenactment component.Source: B. Comparison and Generalization Tests · pp. 10–11
  • The FaceForensics++ test included 3,000 face images spanning four manipulation categories and resolutions from 162 × 162 to 895 × 895 pixels.Images were cropped from videos at different compression levels.Source: B. Comparison and Generalization Tests · p. 10
  • On FaceForensics++, the CTF approach beats the state of the art despite the dataset being used only for testing, not classifier training.The robust classifier trained in Section VI-A was applied to cropped faces from the dataset.Source: B. Comparison and Generalization Tests · pp. 10–11, Table III · p. 11

VII. CONCLUSIONS

The conclusion presents CTF as a fast, explainable Deepfake image detector that uses DCT coefficient statistics to identify GAN-specific frequency bands. Its robust classifier generalizes to FaceForensics++ without training on it, while further work could investigate reenactment signals and other media.

VII. CONCLUSIONS

  • CTF uses overall DCT coefficient statistics to discriminate among known GANs through GAN Specific Frequency bands.The authors report that the approach is fast, explainable, and does not require intense training computation.Source: §VII · p. 11
  • All frequency bands must be considered for higher accuracy, and the finalized CTF approach uses a G-boost classifier.Source: §VII · p. 11
  • The G-boost classifier is reported robust to attacks and able to generalize to FaceForensics++ videos not used during training.Source: §VII · p. 11
  • Further work could investigate GSF signals from reenactment and apply the analysis to attribute manipulation, expression swaps, audio, and video.Source: §VII · p. 11

Related papers

Loading 2101.09781v4…