PaperlayerView exact revision on arXiv

Source-linked AI summary

Global Texture Enhancement for Fake Face Detection in the Wild

Zhengzhe Liu, Xiaojuan Qi, Philip Torr

arXiv:2002.00133v3cs.CV

TL;DR

Fake-face detection remains underexplored, particularly for images from unknown GANs or altered in the wild. This paper introduces Gram-Net, which uses global texture features, and reports stronger performance than recent approaches and baselines across in-domain, cross-GAN, and cross-dataset settings.

  • Problem

    Fake-face detection is underexplored, especially for images from unknown GAN sources that may undergo distortions such as blur, noise, downsampling, or JPEG compression.

    Source: §1 · p. 1

  • Method

    Gram-Net uses Gram-matrix global texture statistics to capture long-range texture features that complement a CNN backbone.

    Source: §1 · p. 2, §4.2 · p. 5

  • Results

    Gram-Net significantly outperforms recent approaches and baselines in in-domain, cross-GAN, and cross-dataset settings, with more than 10% improvement on fake faces from GANs unseen during training.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The findings indicate that global texture statistics are more robust to image editing and transferable across fake faces from different GANs and datasets.

    Source: §1 · p. 2, §6 · p. 8

    The texture analysis covers only texture contrast, and differences between real and fake faces extend beyond that analysis.

    Source: §3.3 · p. 4

Abstract

from arXiv · show

Generative Adversarial Networks (GANs) can generate realistic fake face images that can easily fool human beings.On the contrary, a common Convolutional Neural Network(CNN) discriminator can achieve more than 99.9% accuracyin discerning fake/real images. In this paper, we conduct an empirical study on fake/real faces, and have two important observations: firstly, the texture of fake faces is substantially different from real ones; secondly, global texture statistics are more robust to image editing and transferable to fake faces from different GANs and datasets. Motivated by the above observations, we propose a new architecture coined as Gram-Net, which leverages global image texture representations for robust fake image detection. Experimental results on several datasets demonstrate that our Gram-Net outperforms existing approaches. Especially, our Gram-Netis more robust to image editings, e.g. down-sampling, JPEG compression, blur, and noise. More importantly, our Gram-Net generalizes significantly better in detecting fake faces from GAN models not seen in the training phase and can perform decently in detecting fake natural images.

1. Introduction

The paper studies fake-face texture and proposes Gram-Net to improve detection across unknown GAN sources and image distortions. Gram-Net achieves state-of-the-art detection and stronger robustness and generalization in the reported experiments.

1. Introduction

  • Real-world detection is challenging because fake faces may come from unknown GAN sources and undergo distortions such as downsampling, blur, noise, or JPEG compression.The paper frames reliable fake-face detection as desirable because generated faces can deceive people and create social and security risks.Source: §1 · p. 1
  • Studies find fake-face texture differs substantially from natural faces; humans rely on visible shape/color artifacts, while CNNs focus on texture and nearly perfectly detect untouched fakes with same-source training and testing.The paper also uses ablations and low-level statistics analysis to diagnose the CNN discriminator.Source: §1 · p. 1
  • Global texture information is more robust to distortions and invariant across GANs, motivating Gram-Net’s Gram layer to compute multilevel global texture representations that complement the CNN backbone.The paper notes CNNs cannot fully capture long-range or global cues because of their limited effective receptive field [21].Source: §1 · pp. 1–2
  • Gram-Net achieves state-of-the-art fake-face detection, with 10%, 15%, 13%, and 9% improvement under resizing, blurring, noise, and JPEG compression, respectively.It surpasses compared approaches by more than 10% on fakes from unseen GANs and improves detection of fake natural images by 10%.Source: §1 · p. 2

2. Related work

Prior work spans GAN-based face generation and fake-face detection, while this paper presents global texture and Gram matrices as a route to robust, generalizable detection.

GANs for human face generation

  • High-resolution GANs PGGAN [12] and StyleGAN [13] can generate face images that may fool humans, while image-to-image GANs include StarGAN [19] [36] [4].Earlier random-vector GANs [29] [15] [2] could generate high-quality low-resolution images but suffered from mode collapse at high resolution.Source: §2 · p. 2

Fake GAN face detection

  • Prior detectors used color cues [17] [26], GAN fingerprints, co-occurrence matrices [27], decoder artifacts [34], neuron coverage [30], or incremental learning [25], with stated limitations.Fingerprint methods may fail on unseen GANs, decoder-artifact detection on different architectures [34], and neuron coverage can be time-consuming and hard to deploy [30]; incremental learning requires many GAN models in training [25]. Data augmentation [32] offered limited further improvement, while landmark alignment [18] [33] targeted face-swapping edits.Source: §2 · p. 2
  • This work proposes a simple fake-face detection framework that analyzes fake faces and is described as more robust with significantly better generalization.The authors state it can handle fake images from GANs with decoder architectures drastically different from those seen during training.Source: §2 · p. 2

Textures in CNNs

  • The work uses the Gram matrix as a global texture descriptor for discriminative models, unlike prior Gram-matrix methods for image generation [5].The authors connect their focus on texture to evidence that CNNs are biased toward textures rather than shapes [7].Source: §2 · p. 2

3. Empirical Studies and Analysis

In-domain experiments found that ResNet-18 distinguished fake from real faces with over 99.9% accuracy, while human participants were easily fooled. The analysis identified texture as a key cue and found stronger contrast in real faces than fake faces.

3.1. Human vs. CNN

  • In the user study, 20 participants each rated 1,000 images, and users typically cited visible shape or color artifacts when judging faces as fake.Evaluation took around 5.14 seconds per image on average.Source: §3.1 · p. 3
  • Class activation maps showed CNNs relied mainly on skin and hair texture, while visible artifacts contributed little to fake-face classification.Source: §3.1 · p. 3, Figure 2 · p. 4

3.3. What are the differences between real & fake faces in terms of texture?

  • GLCM analysis found that real faces retained stronger texture contrast than fake faces at every measured pixel distance.The authors suggest CNN-based generators correlate nearby pixel values, limiting contrast, and note that other texture differences remain beyond this analysis.Source: §3.3 · pp. 3–4

3.1. Human vs. CNN

  • ResNet-18 achieved more than 99.9% accuracy in all in-domain fake/real face experiments, while human participants were easily fooled.Source: §3.1 · p. 3, Table 1 · p. 4

3.2. Is texture an important cue utilized by CNNs for fake face detection?

  • Skin-region inputs performed on par with full images, grayscale had little effect, and L0-filtering small textures reduced performance by around 20%.The experiments used cheek skin regions; L0 filtering smooths small textures while retaining shape and color information.Source: §3.2 · p. 3, Table 1 · p. 4

4. Improved Model: Better Generalization Ability, More Robust

The section examines cross-GAN detection and image-editing robustness, then introduces Gram-Net to capture global texture and long-range information. It argues that global texture is more robust to editing and useful across GANs, while the CNN baseline struggles in these settings.

4.1. Generalization and Robustness Analysis

  • ResNet performance fell 22% in the worst case after 64 × 64 downsampling and JPEG compression, and to around 64%–75% on low-resolution cross-GAN tests.The reported performance drop indicates that the CNN discriminator is not robust to image editing or cross-GAN evaluation.Source: §4.1 · p. 4
  • Texture contrast between edited and original images correlated more strongly at larger pair distances, suggesting global texture is robust to editing and informative across GANs.The authors motivate long-range modeling because CNNs’ effective receptive fields are much smaller than their calculated receptive fields [21].Source: §4.1 · pp. 4–5, Figure 4 · p. 5

4.2. Gram-Net Architecture

  • A Gram matrix summarizes spatial statistics by discarding spatial and content information, providing a stationary texture description and a whole-feature-map descriptor beyond CNN receptive-field limits.Its diagonal records individual filter responses, while off-diagonal elements represent coherence between filters.Source: §4.2 · p. 5
  • Gram-Net adds six Gram Blocks to ResNet at the input and before each downsampling layer to incorporate global texture at multiple semantic levels.Each block aligns feature dimensions, calculates a Gram matrix, refines the representation with conv-bn-relu layers, and applies global pooling.Source: §4.2 · p. 5, §4.1 · p. 5, Figure 5 · p. 6
  • Gram-Net’s reconstructed inputs show larger texture patterns than ResNet’s, indicating that it captures long-range texture for discrimination.The visualization compares reconstructions for the “res-block2” and “avg-pool” features in both models.Source: §4.2 · pp. 5–6, Figure 6 · p. 6

5. Experiments

Experiments find that Gram-Net outperforms detector baselines across high-resolution image edits, low-resolution GANs, and tests involving a GAN discriminator with Gram-Blocks. The section also examines cross-dataset transfer and texture differences between natural and BigGAN images.

Robustness and cross-GAN generalization experiments on high-resolution GANs

  • More than 20%: Gram-Net outperforms Co-detect [27] on average, and improves on the ResNet baseline by around 7% on average in StyleGAN-vs-CelebA-HQ settings.The reported ResNet improvement applies to both in-domain and cross-GAN settings.Source: §5 · pp. 6–7, Table 3 · p. 7
  • Gram-Net’s features are more invariant across GAN models and robust to downsampling, JPEG compression, blur, and noise.The evaluations include downsampling to 64×64, JPEG compression, and Gaussian blur or noise.Source: §5 · pp. 6–7

Generalize to StyleGAN trained with Gram-Block in discriminator

  • Gram-Net still outperforms baseline methods when StyleGAN’s discriminator contains Gram-Blocks.The StyleGAN discriminator was fine-tuned with eight Gram-Blocks before evaluation.Source: Table 5 · p. 7, §5 · p. 7

Generalize to low-resolution GANs

  • Around 10% and 15%: Gram-Net exceeds the ResNet baseline and Co-detect [27], respectively, in accuracy across low-resolution GAN settings.The baselines degrade to around 50% to 75%; evaluation includes fake faces from StarGAN, an image-to-image translation model.Source: §5 · p. 7, Table 4 · p. 8

Cross-dataset experiments

  • Gram-Net outperforms baselines in cross-dataset testing, and training on FFHQ with StyleGAN transfers better to CelebA-HQ with StyleGAN or PGGAN than the reverse.A ResNet distinguishes real CelebA-HQ and FFHQ images with more than 99.9% accuracy, illustrating that the datasets differ substantially.Source: Table 6 · p. 8, §5 · p. 8

Generalize to natural images

  • GLCM analysis finds that real natural images retain stronger texture contrast than BigGAN images at every measured pair distance.The face-trained model was also evaluated on 10K ImageNet images and 10K BigGAN images.Source: Table 7 · p. 8, §5 · p. 8

6. Conclusion

The paper proposes Gram-Net, which uses global texture features for fake-face detection, based on findings that fake and real faces differ in texture and that global texture statistics are robust to editing and invariant across GANs. Gram-Net outperforms recent approaches and baselines across in-domain, cross-GAN, and cross-dataset settings, and generalizes to fake natural images.

6. Conclusion

  • Gram-Net significantly outperforms recent approaches and baseline models in in-domain, cross-GAN, and cross-dataset settings.The authors also report better generalization in detecting fake natural images.Source: §6 · p. 8
  • Fake faces have different textures from real faces, while global texture statistics are more robust to image editing and invariant across GANs.Low-level texture statistical analysis further verifies the findings.Source: §6 · p. 8
  • Gram-Net leverages global texture features to improve robustness and generalization in fake-face detection.The authors describe this as a promising direction for understanding GAN-generated fake images and improving real-world fake-face detection.Source: §6 · p. 8

Related papers

Loading 2002.00133v3…