Source-linked AI summary
Global Texture Enhancement for Fake Face Detection in the Wild
Zhengzhe Liu, Xiaojuan Qi, Philip Torr
TL;DR
Realistic GAN faces can deceive humans, while detectors must handle unknown GAN sources and image distortions. The paper studies texture differences and introduces Gram-Net, which uses global texture representations for detection. Gram-Net improves robustness to editing and generalizes better across unseen GANs, datasets, and fake natural images.
Problem
Detecting GAN-generated faces remains under-explored when images come from unknown GANs and undergo distortions such as resizing, blur, noise, or JPEG compression.
Method
Gram-Net incorporates Gram Blocks into a CNN backbone to compute global texture representations at multiple semantic levels.
Results
Gram-Net significantly outperforms recent approaches and baselines across in-domain, cross-GAN, and cross-dataset settings.
Takeaways & Limitations
Global texture statistics provide a basis for more robust and transferable fake-image detection across edits, GANs, and datasets.
Takeaways & Limitations
The texture analysis focuses only on texture contrast, while differences between real and fake faces extend beyond that analysis.
Abstract
from arXiv · showhide
Generative Adversarial Networks (GANs) can generate realistic fake face images that can easily fool human beings.On the contrary, a common Convolutional Neural Network(CNN) discriminator can achieve more than 99.9% accuracyin discerning fake/real images. In this paper, we conduct an empirical study on fake/real faces, and have two important observations: firstly, the texture of fake faces is substantially different from real ones; secondly, global texture statistics are more robust to image editing and transferable to fake faces from different GANs and datasets. Motivated by the above observations, we propose a new architecture coined as Gram-Net, which leverages global image texture representations for robust fake image detection. Experimental results on several datasets demonstrate that our Gram-Net outperforms existing approaches. Especially, our Gram-Netis more robust to image editings, e.g. down-sampling, JPEG compression, blur, and noise. More importantly, our Gram-Net generalizes significantly better in detecting fake faces from GAN models not seen in the training phase and can perform decently in detecting fake natural images.
1. Introduction
The paper studies why CNNs detect GAN-generated faces and proposes Gram-Net to improve robustness to image distortions and generalization across unseen GANs.
- Motivation: GANs can generate face images that deceive humans, while detecting such fakes remains under-explored under unknown sources and image distortions.Practical distortions include downsampling, blur, noise, and JPEG compression.
- Empirical findings: Texture statistics of fake faces differ substantially from those of natural faces.
- Empirical findings: Humans emphasize visible shape and color artifacts, whereas CNNs focus more on texture regions when detecting fake faces.
- Empirical findings: CNNs use texture as an important cue, with ResNet performing almost perfectly on untouched faces from the same source.
- Approach: Gram-Net adds Gram Blocks to a CNN backbone to compute global texture representations at multiple semantic levels.The Gram layer complements the backbone CNN by capturing global texture information.
- Results: Gram-Net achieves state-of-the-art fake-face detection and improves robustness to resizing, blurring, noise, and JPEG compression.Reported improvements are 10%, 15%, 13%, and 9%, respectively.
2. Related work
Prior work uses color, GAN-specific fingerprints, augmentation, handcrafted co-occurrence features, decoder artifacts, or neuron coverage, but these approaches have stated generalization or deployment limitations.
- GANs for human face generation: High-resolution GANs such as PGGAN and StyleGAN can generate face images that fool human beings.
- Fake GAN face detection: Color-based detectors exploit color information, although CNN performance changes little when color is removed.
- Fake GAN face detection: GAN-fingerprint methods identify the generating source but cannot generalize to GAN models absent from training data.
- Fake GAN face detection: Data augmentation improves generalization, but further gains are limited by the detection algorithm.
- Fake GAN face detection: Handcrafted color co-occurrence features lose raw-data information, while decoder-artifact methods fail on unseen GANs with drastically different decoder architectures.
- Fake GAN face detection: Neuron-coverage detection is time-consuming and difficult to deploy in real systems.
3. Empirical Studies and Analysis
The empirical studies compare human and CNN evidence for fake-face detection and analyze texture as a distinguishing signal. Controlled skin-region and texture-statistics experiments show that CNNs rely strongly on texture, while real and fake faces differ in texture contrast.
- 3.1. Human vs. CNN: Human observers typically use recognizable shape and color artifacts, including asymmetrical eyes, irregular teeth, and irregular letters.The user study involved 20 participants, each rating 1,000 images after viewing the training set.
- 3.1. Human vs. CNN: More than 99.9% accuracy is achieved by ResNet-18 in all in-domain fake/real face experiments, while human beings are easily fooled.The CNN is trained and tested on fake images from the same GAN.
- 3.1. Human vs. CNN: CNN activation maps concentrate on texture regions such as skin and hair, whereas visible artifacts contribute little to classification.The reported artifacts include asymmetrical earrings, irregular letters, and irregular teeth.
- 3.2. Is texture an important cue utilized by CNNs for fake face detection?: Skin regions perform on par with full images for in-domain detection, and grayscale conversion changes the result little, but L0 filtering reduces performance by around 20%.The controlled inputs were original skin, grayscale skin, and L0-filtered skin, with examples shown in Figure 3.
- 3.3. What are the differences between real & fake faces in terms of texture?: Real faces retain stronger texture contrast than fake faces at every measured distance in the GLCM analysis.The analysis computes contrast across multiple pixel distances and orientations; larger contrast indicates sharper, clearer texture.
4. Improved Model: Better Generalization Ability, More Robust
Cross-GAN testing and image editing expose weaknesses in CNN fake-face detectors. Gram-Net addresses these limits by modeling global texture statistics through Gram Blocks, improving robustness and long-range texture capture.
- Generalization and Robustness Analysis: ResNet performance drops by 22% in the worst case after downsampling and JPEG compression, and falls to around 64%–75% in cross-GAN evaluation.These reductions indicate limited robustness to editing and generalization across GAN sources.
- Generalization and Robustness Analysis: Large texture remains strongly correlated between edited and original images, while real–fake texture-contrast differences persist at larger pair distances.The paper therefore treats global texture as a robust cue for edited images and cross-GAN detection.
- Gram-Net Architecture: Gram-Net adds Gram Blocks to ResNet to compute global texture representations at multiple semantic levels and model long-range information.Blocks are inserted on the input image and before each downsampling layer, with convolution, Gram-matrix, refinement, and global-pooling components.
- Gram-Net Architecture: The Gram matrix summarizes channel-wise spatial statistics while discarding spatial and content information, providing a stationary texture description beyond CNN receptive fields.Its diagonal entries represent individual filter responses, whereas off-diagonal entries represent coherence between filters.
- Gram-Net Architecture: Reconstructed inputs from Gram-Net exhibit larger texture patterns than those from ResNet, indicating that Gram-Net captures long-range texture for discrimination.The comparison covers reconstructed features at “res-block 2” and “avg-pool.”
5. Experiments
Experiments evaluate Gram-Net in in-domain, cross-GAN, cross-dataset, low-resolution, and natural-image settings. Gram-Net consistently improves robustness and generalization over baseline detectors, including under image editing and unseen GAN conditions.
- High-resolution GANs: Gram-Net outperforms compared methods across original, edited, in-domain, and cross-GAN high-resolution settings.Evaluations include downsampling, JPEG compression, blur, and noise.
- Low-resolution GANs: Around 10% accuracy improvement over ResNet and around 15% over [27] shows better generalization to low-resolution GANs.Baseline ResNet and [27] degrade to around 50% to 75% in this setting.
- Gram-Block in the GAN discriminator: Gram-Net still outperforms baselines when the StyleGAN discriminator also contains Gram-Blocks.This tests whether the detector remains effective when the GAN discriminator uses the same architectural component.
- Cross-dataset experiments: Gram-Net outperforms baselines in cross-dataset experiments, where real CelebA-HQ and FFHQ images are distinguishable at more than 99.9% accuracy.The experiments train and test across StyleGAN, PGGAN, CelebA-HQ, and FFHQ combinations.
- Natural images: Texture contrast differs between real and GAN-generated natural images, and the face-trained model is directly evaluated on ImageNet versus BigGAN.Real natural images retain stronger texture contrast than GAN images at all measured distances.
6. Conclusion
The paper studies fake and real faces, finds different texture statistics, and proposes Gram-Net to exploit global texture features. Across evaluated settings, Gram-Net improves robustness and generalization, including to fake natural images.
- Conclusion: Fake faces have texture statistics that differ from real faces, while global texture information is more robust to editing and invariant across GANs.These findings motivate Gram-Net for fake-face detection.
- Conclusion: Gram-Net leverages global texture features to improve robustness and generalization in fake-face detection.The architecture is motivated by empirical analysis of human and CNN discrimination behavior.
- Conclusion: Gram-Net significantly outperforms recent approaches and baseline models in in-domain, cross-GAN, and cross-dataset settings.The model also shows better generalization when detecting fake natural images.