Source-linked AI summary
Exposing GAN-synthesized Faces Using Landmark Locations
Xin Yang, Yuezun Li, Honggang Qi, Siwei Lyu
TL;DR
Realistic GAN-synthesized faces raise concerns because faces carry identifiable information, and GAN-generated facial parts may lack coherent placement. The method detects facial landmarks, normalizes their locations, and uses the resulting features in an SVM classifier. The SVM classifier achieved 94.13% AUROC, and landmark-feature SVM achieved 0.83 AUROC for individual-frame classification on FaceForensics.
Problem
Realistic GAN-synthesized faces raise concerns because faces carry identifiable information, and GAN-generated facial parts may lack coherent placement.
Source: §1 · p. 1Method
Results
Takeaways & Limitations
The conclusion reports that the method uses low-dimensional inputs, lightweight models, and is robust to scale variation.
Source: §6 · p. 5The evaluation excludes STGAN because its best-quality images are conditioned on existing images rather than generated directly from random noise.
Source: §2.1 · p. 2
Abstract
from arXiv · showhide
Generative adversary networks (GANs) have recently led to highly realistic image synthesis results. In this work, we describe a new method to expose GAN-synthesized images using the locations of the facial landmark points. Our method is based on the observations that the facial parts configuration generated by GAN models are different from those of the real faces, due to the lack of global constraints. We perform experiments demonstrating this phenomenon, and show that an SVM classifier trained using the locations of facial landmark points is sufficient to achieve good classification performance for GAN-synthesized faces.
1 INTRODUCTION
GANs can produce highly realistic faces, but their facial parts may lack coherent placement. The paper proposes using facial-landmark locations to detect these synthetic faces.
- Fabricated or manipulated faces are especially concerning because faces carry identifiable information, motivating detection methods for GAN-synthesized faces.Source: §1 · p. 1
- Existing detection approaches use low-level features such as color disparities or feed whole images to neural networks for holistic features.Source: §1 · p. 1
- GANs can render realistic facial details without explicit constraints on where facial parts are placed, producing configurations unlike those of real faces.The paper compares this placement problem to Fukuwarai, in which face parts are placed without seeing the blank face.Source: §1 · p. 1
- The proposed detector uses automatically detected facial landmarks, normalizes their locations, and classifies them with an SVM; it reports reasonable PGGAN detection using low-dimensional features and a lightweight model.Source: §1 · p. 1
2 RELATED WORKS
The related work traces GAN face synthesis from early models to high-resolution PGGAN and contrasts existing image-based detectors with landmark-based cues. The paper also limits its synthesis scope to models that generate faces directly from random noise.
2.1 GAN-based Face Synthesis Methods
- PGGAN faces show mismatched eye size and location, a mouth shifted left relative to the nose, and an abnormal eye canthus.Source: Figure 2 · p. 2
- PGGAN progressively grows the generator and discriminator from 4×4 images to 1024 × 1024, producing high-resolution faces with fine details.The paper presents GAN, DCGAN, COGAN, PGGAN, and Style-GAN as examples of improving face-synthesis quality over time.Source: §2.1 · p. 2, Figure 1 · p. 2
- The evaluation excludes STGAN because its best-quality images are conditioned on existing images rather than generated directly from random noise.Source: §2.1 · p. 2
2.2 Detection Methods for GAN-synthesized Images
- Prior detectors use color statistics or color-channel disparities, which can be removed by post-processing, or deep convolutional classifiers that require GPUs and do not reveal their classification mechanisms.Source: §2.2 · pp. 2–3
- Figure 3 compares normalized landmark-coordinate distributions for 200k+ CelebA and 100k PGGAN faces, while Figure 4 shows a 68-landmark-to-136D classification pipeline.Source: Figure 3 · p. 3, Figure 4 · p. 3
3 METHOD
The method detects irregular configurations of facial parts in GAN-generated faces through aligned facial-landmark locations. Joint landmark distributions show differences between real and synthesized faces, and the resulting feature is low-dimensional.
3 METHOD
- GAN-synthesized faces show asymmetric or displaced facial parts, including unequal eyes, a shifted mouth, and an unnatural eye corner.These examples are reported for high-resolution PGGAN faces.Source: §3 · p. 3
- Aligning landmarks to a standard coordinate system makes their joint locations usable as features for distinguishing real from GAN-synthesized faces.The alignment uses an affine transform estimated from central facial landmarks, excluding the contour; joint landmark distributions show more prominent differences than marginal distributions.Source: §3 · p. 3
- Landmark features are low-dimensional and image-size independent, while their abnormalities reflect GAN synthesis mechanisms that may be difficult to fix without more complex constraints.The feature dimension is twice the number of extracted landmarks.Source: §3 · pp. 3–4
4 EXPERIMENTS
Experiments evaluate landmark-based SVM detection using PGGAN-generated and CelebA real faces, alongside image-based neural-network comparisons. The SVM achieves a reported AUROC of 94.13%, while image resizing has relatively little effect on landmark-based performance.
4 EXPERIMENTS
- The evaluation uses 100K PGGAN-generated faces and more than 200K CelebA real faces, with 75% of each dataset used for training and 25% for testing.The SVM uses standardized, flattened 136-dimensional landmark features and an RBF kernel selected by five-fold cross-validation.Source: §4 · p. 4
- 94.13% AUROC: the landmark-based SVM outperforms VGG19 and XceptionNet in the reported comparison.The comparison uses resized images; the cited table covers PGGAN and CelebA at different image widths.Source: Table 1 · p. 5, §4 · p. 5
- The neural-network comparison has a data caveat: its results were taken from a published paper, and the training and testing data may differ.The compared methods use resized images, and the effect of resizing on their classification results was not studied.Source: §4 · p. 5
- Landmark-based classification performance is relatively indifferent to resizing the PGGAN and CelebA images.Figure 6 reports performance across image widths; the method's landmark features are independent of image size.Source: §4 · p. 5, Figure 6 · p. 5
5 RESULTS ON FACE FORENSICS
The landmark-based SVM is also tested on FaceForensics videos manipulated with Face2Face. Averaging frame-level predictions raises AUROC from 0.83 for individual frames to 0.90 for videos.
5 RESULTS ON FACE FORENSICS
- 0.90 AUROC for video-level predictions, compared with 0.83 AUROC for classification of individual frames.The evaluation uses FaceForensics test videos manipulated with Face2Face.Source: §5 · p. 5
6 CONCLUSION
The paper concludes that aligned facial landmarks can distinguish PGGAN-generated faces by capturing errors in facial-part configuration. Experiments indicate effectiveness with low-dimensional features, lightweight models, and scale variation.
6 CONCLUSION
- The proposed method uses aligned facial-landmark locations to distinguish PGGAN-synthesized faces from real faces.The authors attribute landmark errors to GANs' ability to depict facial-part details without constraints on component configuration.Source: §6 · p. 5
- The reported results indicate effectiveness with low-dimensional input, lightweight models, and robustness to scale variation.Figure 7 presents FaceForensics testing results, including ROC/AUROC and examples of predictions for both classes.Source: §6 · p. 5, Figure 7 · p. 6