Source-linked AI summary
Detecting GAN generated Fake Images using Co-occurrence Matrices
Lakshmanan Nataraj, Tajuddin Manhar Mohammed, Shivkumar Chandrasekaran, Arjuna Flenner, Jawadul H. Bappy, Amit K. Roy-Chowdhury, B. S. Manjunath
TL;DR
GAN-based image manipulation makes reliable detection important because generated images can be difficult to distinguish from authentic ones. The paper detects them by feeding RGB pixel co-occurrence matrices into a deep CNN, achieving over 99% accuracy on two datasets and strong cross-dataset performance.
Problem
GAN-based manipulation can make doctored images difficult to distinguish from normal images, while the robustness of existing image-forensics tools on GAN-generated images is uncertain.
Method
The method computes co-occurrence matrices directly on RGB image pixels and passes the resulting tensor through a deep convolutional neural network.
Results
Testing accuracy reaches 99.71% on cycleGAN and 99.37% on StarGAN, while cross-dataset testing reaches 99.45% and 93.42%.
Takeaways & Limitations
The approach is effective on two diverse GAN datasets and generalizes when trained on one dataset and tested on the other.
Abstract
from arXiv · showhide
The advent of Generative Adversarial Networks (GANs) has brought about completely novel ways of transforming and manipulating pixels in digital images. GAN based techniques such as Image-to-Image translations, DeepFakes, and other automated methods have become increasingly popular in creating fake images. In this paper, we propose a novel approach to detect GAN generated fake images using a combination of co-occurrence matrices and deep learning. We extract co-occurrence matrices on three color channels in the pixel domain and train a model using a deep convolutional neural network (CNN) framework. Experimental results on two diverse and challenging GAN datasets comprising more than 56,000 images based on unpaired image-to-image translations (cycleGAN [1]) and facial attributes/expressions (StarGAN [2]) show that our approach is promising and achieves more than 99% classification accuracy in both datasets. Further, our approach also generalizes well and achieves good results when trained on one dataset and tested on the other.
Introduction
GANs make image manipulation increasingly difficult to distinguish from authentic content, motivating detection methods based on altered pixel statistics. The paper proposes combining pixel co-occurrence matrices with deep learning to detect GAN-generated images and reports promising cross-dataset generalization.
- GAN-based editing can make doctored images difficult to distinguish from normal images.
- GAN applications include image-to-image translation, style transfer, facial attributes, and facial expressions.
- GAN-generated images alter pixel statistics, so deviations from natural image statistics can support detection.
- The proposed method feeds pixel co-occurrence matrices into deep learning rather than relying on residual computation or image filters.
- The approach achieves promising results and generalizes when the GAN model is unknown during training.
Related Work
GAN-image detection is a relatively new image-forensics area with limited prior work. Existing approaches use residual or high-pass-filtered features, whereas this paper directly computes co-occurrence matrices on color channels and learns detection features with a CNN.
- GAN-image detection is a new image-forensics area with relatively few published studies.
- Prior work includes residual features combined with deep learning for detecting cycleGAN images.
- Other approaches compute co-occurrence matrices from residuals of high-pass-filtered images and concatenate them into feature vectors.
- Unlike these approaches, the proposed method directly computes co-occurrence matrices on three color channels and passes them through a deep CNN.
Methodology
The method computes RGB pixel co-occurrence matrices and uses them as input to a multilayer CNN. This design lets the network learn features directly from the matrices without first computing image residuals.
- The network learns features from co-occurrence matrices instead of using image residuals or multiple filtering steps.
- Co-occurrence matrices are computed directly from the red, green, and blue image channels.
- The RGB co-occurrence matrices form a 3x256x256 tensor before CNN processing.
- The tensor is processed by a multilayer CNN containing convolution, ReLU, and max-pooling layers.
Experiments
The experiments evaluate detection separately on CycleGAN and StarGAN datasets, then test cross-dataset generalizability and robustness to JPEG compression. The method also receives comparison with existing approaches on original CycleGAN images.
- Datasets: 36,302 images comprise the CycleGAN dataset, evenly split between 18,151 GAN and 18,151 non-GAN images.The dataset contains unpaired translations involving objects, scenes, and styles.
- Evaluation protocol: The evaluation uses separate dataset experiments, cross-dataset testing, and 50% training, 25% validation, and 25% testing splits.Models were trained for 50 epochs with batch size 40.
- Separate-dataset results: 99.71% testing accuracy is obtained on CycleGAN, while StarGAN reaches 99.37% testing accuracy.These results follow high training and validation accuracy during separate dataset evaluation.
- Generalizability: 99.45% accuracy results from training on CycleGAN and testing on StarGAN, compared with 93.42% for training on StarGAN and testing on CycleGAN.The authors attribute the lower reverse-direction accuracy to StarGAN’s non-uniform class distribution and CycleGAN’s diverse image sources and categories.
- Comparison with prior methods: 97.84 accuracy is achieved on average for original CycleGAN images, outperforming or matching the compared top methods on most categories.Performance is poorer on cityscapes and facades, where JPEG compression may affect classification accuracy.
- JPEG robustness: JPEG compression substantially affects detection, although training on compressed images raises accuracy to 87.31% even at quality factor 75.The experiments compare training and testing conditions involving original and JPEG-compressed images.
Conclusions
The paper proposes detecting GAN-generated fake images by combining pixel co-occurrence matrices with deep learning. Experiments on two diverse GAN datasets indicate effectiveness and generalizability, while future work targets manipulated-pixel localization.
- Conclusions: The proposed method computes co-occurrence matrices on image color channels and trains a deep convolutional neural network to distinguish GAN-generated images from real ones.The approach operates directly on pixel-derived co-occurrence matrices.
- Conclusions: Experiments on two diverse GAN datasets show that the approach is effective and generalizable.The paper identifies manipulated-pixel localization as future work.
Author Biography
The author biographies describe researchers working across image forensics, machine learning, computer vision, image processing, and related applications. Their backgrounds span academia, industry, and government research.
- Authors: Lakshmanan Nataraj is a Senior Research Staff Member at Mayachitra Inc. leading research projects on malware detection and media forensics.He earned engineering and doctoral degrees in electrical and computer engineering.
- Authors: Tajuddin Manhar Mohammed is a Research Staff Member at Mayachitra Inc. whose recent efforts involve deep learning and computer vision for image forensics and cybersecurity.He holds degrees from IIT Hyderabad and UCSB.
- Authors: B. S. Manjunath is a UCSB Distinguished Professor directing a multimodal big data science and healthcare center.His research interests include image processing, machine learning, computer vision, and media forensics.
- Authors: Shivkumar Chandrasekaran is a UCSB professor with training in physics and computer science.His biography describes prior mathematics teaching and research experience.
- Authors: Arjuna Flenner is a research physicist at NAVAIR whose work includes machine learning.He received a Ph.D. in physics and received a Navy Scientist and Engineer award.
- Authors: Jawadul H. Bappy is a scientist at JD.Com whose research interests include media forensics, deep generative models, and advanced machine learning.He earned his Ph.D. in electrical and computer engineering from UC Riverside.
- Authors: Amit K. Roy-Chowdhury is a UC Riverside professor whose research spans computer vision, image processing, and vision-based statistical learning.His applications include cyber-physical, autonomous, and intelligent systems.