Source-linked AI summary
Efficient GAN-Based Anomaly Detection
Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, Vijay Ramaseshan Chandrasekhar
TL;DR
GAN-based anomaly detection is promising for complex, high-dimensional data, but conventional GAN approaches require costly latent-representation recovery at test time. The paper jointly learns an encoder with the GAN to make anomaly detection efficient, achieving competitive or state-of-the-art performance on MNIST and KDD99 while greatly accelerating inference.
Problem
GANs suit complex, high-dimensional anomaly-detection data, but their implicit distributions make test-time latent-representation recovery costly and GAN-based anomaly detection remains relatively unexplored.
Method
The method uses a BiGAN-based model that jointly learns an encoder, generator, and discriminator, then scores anomalies using reconstruction and discriminator-based losses.
Results
The method achieves competitive or state-of-the-art anomaly-detection performance on MNIST and KDD99, with inference approximately 800x faster on MNIST and 700x to 900x faster on KDD99 than AnoGAN.
Takeaways & Limitations
Jointly learning an encoder eliminates costly latent-representation recovery and supports efficient test-time anomaly detection on high-dimensional, complex datasets.
Takeaways & Limitations
Future work includes more extensive evaluation, other training strategies, and studying how encoder accuracy affects anomaly-detection performance.
Abstract
from arXiv · showhide
Generative adversarial networks (GANs) are able to model the complex highdimensional distributions of real-world data, which suggests they could be effective for anomaly detection. However, few works have explored the use of GANs for the anomaly detection task. We leverage recently developed GAN models for anomaly detection, and achieve state-of-the-art performance on image and network intrusion datasets, while being several hundred-fold faster at test time than the only published GAN-based method.
1 INTRODUCTION
Anomaly detection requires modeling complex, high-dimensional normal-data distributions, making GANs a promising approach. The paper develops an efficient GAN-based method using encoder-learning GANs and evaluates it on image and network-intrusion data.
- Anomaly detection is important across manufacturing, medical imaging, and cybersecurity because methods must model complex, high-dimensional normal-data distributions.
- GANs are promising for anomaly detection because they have successfully modeled complex, high-dimensional distributions, particularly for natural images.
- GAN-based anomaly detection normally requires costly test-time optimization to recover an input’s latent representation, limiting practicality for large datasets and real-time applications.
- The proposed method learns an encoder during GAN training, is evaluated on MNIST and KDD99, and achieves competitive performance with other approaches.
- The method is reported as the first GAN-based anomaly-detection approach to achieve state-of-the-art results on KDD99.
2 RELATED WORK
Prior anomaly-detection research includes classical, reconstruction-based, and deep-learning methods, while GAN use remains relatively limited despite GANs’ suitability for complex, high-dimensional data.
- Established anomaly-detection methods include clustering, nearest neighbors, one-class classification, and reconstruction fidelity.
- One-class SVMs learn a discriminative boundary around normal data, representing a one-class classification approach.
- PCA and its kernel and robust variants use dimensionality-based representations for anomaly detection.
- Deep anomaly-detection methods include autoencoders, variational autoencoders, energy-based models, and deep autoencoding Gaussian mixture models.
- Aside from AnoGAN, GAN-based anomaly detection has been relatively unexplored despite GANs’ ability to model complex, high-dimensional real-world distributions.
3 EFFICIENT ANOMALY DETECTION WITH GANS
The method jointly learns an encoder, generator, and discriminator so test-time anomaly scoring avoids latent-representation recovery. Scores combine reconstruction error with discriminator-based evidence.
- BiGAN-based training jointly learns an encoder E, generator G, and discriminator D, with E mapping inputs x to latent representations z.
- Learning the encoder jointly with the generator avoids computationally expensive latent-representation recovery at test time.
- The discriminator evaluates both samples and latent representations, unlike a regular GAN discriminator that considers only inputs.
- Training solves a minG,E maxD optimization of the GAN objective V(D,E,G), using data, latent, encoder-induced, and generator-induced distributions.
- The anomaly score A(x) is a convex combination of reconstruction loss LG and discriminator-based loss LD.
- LD can use discriminator cross-entropy or feature matching, and larger A(x) values indicate examples more likely to be anomalous.
4 EXPERIMENTS
Experiments evaluate the method on MNIST image anomalies and KDD99 network intrusions using task-specific metrics. The method outperforms or competes strongly with baselines while providing substantially faster inference than AnoGAN.
- MNIST: MNIST experiments create ten tasks by designating each digit class as anomalous and train models only on normal digits.
- MNIST: MNIST performance is measured with area under the precision-recall curve because the datasets are imbalanced.
- MNIST: The proposed model significantly outperforms the VAE baseline and outperforms AnoGAN on MNIST.
- MNIST: Approximately 800x faster inference than AnoGAN is reported for the MNIST experiment.
- KDD99: The KDD99 evaluation uses precision, recall, and F1-score, classifying the 20% highest-scoring samples as anomalies.
- KDD99: The method is highly competitive with state-of-the-art KDD99 approaches, achieves higher recall, and outperforms AnoGAN.
- KDD99: KDD99 inference is 700x to 900x faster than AnoGAN.
5 CONCLUSION
The method achieves state-of-the-art anomaly-detection performance on high-dimensional, complex datasets while remaining efficient at test time. Its jointly learned encoder removes the costly latent-representation recovery procedure, but broader evaluation remains future work.
- The GAN-based method achieves state-of-the-art anomaly-detection performance on high-dimensional, complex datasets while being efficient at test time.
- A jointly learned encoder eliminates the costly procedure required to recover a latent representation for each input.
- Future work includes more extensive evaluation, alternative training strategies, and studying how encoder accuracy affects anomaly-detection performance.
A EXPERIMENT DETAILS
The experiments implemented AnoGAN and the BiGAN-based method in TensorFlow and aligned their architectures and learning hyperparameters as far as possible for comparison.
- AnoGAN and the BiGAN-based method were implemented in TensorFlow using matched architectures and learning hyperparameters where possible.
- Both models used α = 0.9 in the anomaly score A(x), while AnoGAN used stochastic gradient descent for 500 iterations.
- Exponential moving averages of model parameters were used at test time, with a decay of 0.999 for MNIST and an additional dataset-specific setting.
B INFERENCE TIME DETAILS
Inference time was measured as the average over 100 batches, with KDD experiments run on NVIDIA Tesla K40 GPUs using TensorFlow 1.1.0.
- KDD inference experiments used NVIDIA Tesla K40 GPUs and TensorFlow 1.1.0 with Python 3.5.3.
- Inference time was measured by averaging performance over 100 batches.
C MNIST EXPERIMENTS DETAILS
The MNIST experiment details specify pixel preprocessing and identify the discriminator layer used for feature-matching anomaly scoring, alongside the GAN and BiGAN architecture specifications.
- MNIST pixels were scaled to the range [-1,1].
- The outputs of the discriminator’s starred layer were used for the feature-matching scoring variant.
- Table 3 specifies the MNIST GAN architecture and hyperparameters.
- Table 4 specifies the MNIST BiGAN architecture and hyperparameters.
D KDD99 EXPERIMENT DETAILS
The KDD99 experiments preprocess mixed continuous and categorical data into scaled features and specify GAN and BiGAN architectures and hyperparameters.
- Preprocessing: KDD99 samples have 41 dimensions: 34 continuous and 7 categorical features.Categorical features are one-hot encoded, producing 121 features before min-max scaling.
- Preprocessing: One-hot encoding expands the categorical variables to a total of 121 features, followed by min-max scaling.
- Preprocessing: The FM scoring variant uses outputs from the discriminator’s starred layer.
- GAN configuration: Table 5 specifies the KDD99 GAN architecture and hyperparameters.
- BiGAN configuration: Table 6 specifies the KDD99 BiGAN architecture and hyperparameters.