Source-linked AI summary
Detecting GAN-generated Imagery using Color Cues
Scott McCloskey, Michael Albright
TL;DR
Detecting GAN-generated imagery matters because synthetic images can support online disinformation. This paper analyzes generator color processing to derive two forensic cues, finding that saturation statistics discriminate GAN imagery from camera imagery more effectively than color statistics.
Problem
Image forensics needs methods for detecting GAN-generated imagery because modern data-driven approaches make artificial imagery easier to create and disseminate.
Method
The paper analyzes GAN generator color processing, focusing on intensity normalization and feature-to-RGB conversion, to develop two forensic cues.
Results
AUC was 0.7 for fully GAN-generated images and 0.61 for GAN-generated faces spliced into camera images, while color-statistics forensics were less useful.
Takeaways & Limitations
Analyzing common generator operations, including normalization and feature-to-color conversion, can support forensics for GAN-generated imagery.
Takeaways & Limitations
The approach may not generalize to future GANs because rapidly evolving generator structures are difficult to predict.
Abstract
from arXiv · showhide
Image forensics is an increasingly relevant problem, as it can potentially address online disinformation campaigns and mitigate problematic aspects of social media. Of particular interest, given its recent successes, is the detection of imagery produced by Generative Adversarial Networks (GANs), e.g. `deepfakes'. Leveraging large training sets and extensive computing resources, recent work has shown that GANs can be trained to generate synthetic imagery which is (in some ways) indistinguishable from real imagery. We analyze the structure of the generating network of a popular GAN implementation, and show that the network's treatment of color is markedly different from a real camera in two ways. We further show that these two cues can be used to distinguish GAN-generated imagery from camera imagery, demonstrating effective discrimination between GAN imagery and real camera images used to train the GAN.
1. Introduction
The paper develops forensic methods for detecting GAN-generated imagery, motivated by the growing role of synthetic images in online disinformation. It identifies two generator color-processing differences from cameras: limited saturated pixels and camera-inconsistent channel-combination weights.
- Image forensics is increasingly important because social media facilitates fake-news dissemination and computer vision makes fake imagery easier to generate.
- The paper analyzes GAN generator architecture to detect imagery that differs from camera images despite appearing indistinguishable to the GAN discriminator.
- The generator normalizes internal values to constrain outputs, limiting the frequency of saturated pixels.
- Its multi-channel representation collapses into red, green, and blue channels using weights unlike a camera’s analogous spectral sensitivities.
- The color-based approach complements visual-artifact detection, whose effectiveness may decline as GAN architectures mitigate artifacts such as checkerboard patterns.Checkerboard artifacts have been mitigated by replacing deconvolution with up-sampling followed by convolution.
2. Related Work
GANs train competing generator and discriminator networks to create and classify synthetic imagery, motivating forensic methods that distinguish GAN images from real images. Existing detection approaches include exploiting absent blinking in DeepFake videos and using machine learning rather than semantic cues.
- GANs: GANs use competing generator and discriminator networks: one produces artificial imagery resembling a real dataset, while the other classifies images as real or generated.This framework was introduced in 2014 and has become valuable across computer vision applications.
- Forensic Detection: Forensic researchers develop methods to determine whether images or videos were generated by GAN-based networks rather than originating as real images.The paper refers to these detection targets as GAN images and seeks to distinguish them from real images.
- Forensic Detection: One detection method exploits the lack of blinking in DeepFake-type videos, while other approaches use machine learning and neural networks instead of semantically meaningful cues.The supplied passage introduces these approaches as examples of GAN-media detection methods.
3. GAN Generator Architecture
The generator’s final layer converts K > 3 depth layers into RGB channels, creating color-formation cues that differ from camera imaging. Later-layer color weights and normalization outputs provide two proposed forensic cues for distinguishing GAN imagery from real camera imagery.
- GAN generators commonly use repeated upsampling and convolutions, while later-layer cues are emphasized because subsequent processing is less likely to modulate them.The representative architecture produces a high-resolution image through repeated spatial upsampling, 3x3 convolutions, and a final 1x1 convolution.
- The final generator layer maps a K-by-W-by-H depth representation, with K > 3, to a 3-by-W-by-H RGB image.The conversion from depth layers to red, green, and blue channels provides two potential forensic cues.
- Unlike camera spectral responses, GAN channel weights can be negative and substantially overlapping because synthetic generators do not need to count photons.Camera responses are constrained by limited channel overlap and non-negativity, whereas learned GAN weights for face synthesis share a common peak and correlate at several non-peak values.
- GAN generators apply pixel-wise or layer-wise normalization to produce relatively uniform unit-interval values before transforming them into RGB intensities.In one implementation, pixel-wise normalization fixes the magnitude of each pixel’s depth vector; another normalizes individual depth planes using learned β and γ parameters.
- Camera irradiance values are generally logarithmically distributed rather than neatly constrained, requiring high dynamic range imaging and distinguishing camera-based intensity formation from GAN normalization.The contrast between normalized generator values and camera irradiance distributions motivates normalization as a forensic cue.
4. Detection Methods
The section proposes two GAN-image detection methods derived from color analyses: pixel-wise RGB chromaticity correlations and suppressed exposure extremes. The methods use pretrained or low-dimensional models suitable for relatively small training sets.
- Chromaticity correlations: GAN-generated images are expected to show unusually high r–g chromaticity correlations without spatial dependence because color conversion is applied independently at each pixel.The chromaticity coordinates are r = R/(R + G + B) and g = G/(R + G + B).
- Chromaticity correlations: A pretrained INH classifier is fine-tuned on r-vs.-g histograms from GAN imagery and camera imagery used to train the GAN.The approach adapts Chen et al.’s bivariate-histogram forensic method.
- Exposure-frequency features: 1387 GAN-generated images and real ImageNet camera images provide training data for a linear SVM using exposure-frequency features.The GAN images are randomly subsampled from 30 LSUN categories generated by the GAN in [7].
5. Evaluation
Evaluation on GAN Crop and GAN Full benchmark datasets shows that over-exposure features detect GAN imagery substantially better than random, especially fully generated images, while combining exposure features or using color-image forensics performs worse. The color-image forensic results are near-random, potentially because evaluation camera images contain focus manipulations.
- Datasets and protocol: The evaluation used GAN Crop images with entirely GAN-generated or non-GAN regions and GAN Full images consisting mostly of camera content with GAN-generated face replacements.Features were computed over each entire image, including GAN Full images whose manipulated regions are localized around faces.
- Over-exposure detection: Over-exposure features produced performance significantly better than random, reaching 0.7 AUC on fully GAN-generated images.The feature measures the proportion of saturated pixels, which is diluted by non-GAN regions in GAN Full images.
- Over-exposure detection: Combining under- and over-exposed pixel frequencies reduced AUC from 0.70 to 0.67 and lowered correct detection at a 0.5 false alarm rate.The authors hypothesize that under-exposed features are less predictive, potentially because of the asymmetric shape of the Rectified Linear Unit.
- Color-image forensics: Color-image forensic ROC curves were little better than random, with AUCs of 0.56 and 0.54 on the two datasets.The results provided little evidence that the classifier learned useful color statistics distinguishing GAN-generated from camera-generated imagery.
- Color-image forensics: Focus manipulations in some evaluation camera images may explain the weak color-forensic results, because the pre-trained INH network was intended to detect such manipulations.An example shows blurred left and top image edges in a celebrity face image.
6. Conclusion
The paper analyzes two color-related forensics arising from how GAN generators transform feature representations into RGB intensities. These cues distinguish GAN-generated imagery from camera imagery, while the authors note that future generator architectures may require similar analysis.
- Forensic basis: A forensic based on over-exposed pixel frequency provides good discrimination between GAN-generated and camera imagery.The evaluation used an independently generated challenge dataset.
- Forensic basis: The proposed forensics examine how GAN generator networks transform multi-channel feature maps into three-channel color images.Their analysis compares generator feature weights with color filter array integration over the visible spectrum.
- Forensic basis: The saturation forensic is motivated by generator normalization, which limits the range of generated intensities.The passage attributes this forensic to the generators’ normalization behavior.
- Limitations and outlook: Because GAN innovation is advancing rapidly, the authors find future generator structures difficult to predict and encourage similar analyses of future GANs.They frame this work as contributing to an important problem affecting increasingly large parts of society.