PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarially Learned Anomaly Detection

Houssam Zenati, Manon Romain, Chuan Sheng Foo, Bruno Lecouat, Vijay Ramaseshan Chandrasekhar

arXiv:1812.02288v1cs.LGstat.ML

TL;DR

Effective anomaly detection for complex, high-dimensional data remains difficult. ALAD uses bidirectional GANs, learned encodings, cycle-consistency regularization, and feature-space reconstruction errors, achieving strong performance across image and tabular datasets while improving test-time efficiency.

  • Problem

    Anomaly detection methods still struggle to model complex and high-dimensional data effectively.

  • Method

    ALAD learns an encoder and generator with data- and latent-space adversarial regularization, then scores samples using reconstruction errors in discriminator feature space.

  • Results

    ALAD achieves state-of-the-art performance across image and tabular datasets and is several hundred-fold faster at test time than the only published GAN-based method.

  • Takeaways & Limitations

    ALAD demonstrates that bidirectional GANs with adversarially learned features are a promising approach to anomaly detection on complex, high-dimensional data.

Abstract

from arXiv · show

Anomaly detection is a significant and hence well-studied problem. However, developing effective anomaly detection methods for complex and high-dimensional data remains a challenge. As Generative Adversarial Networks (GANs) are able to model the complex high-dimensional distributions of real-world data, they offer a promising approach to address this challenge. In this work, we propose an anomaly detection method, Adversarially Learned Anomaly Detection (ALAD) based on bi-directional GANs, that derives adversarially learned features for the anomaly detection task. ALAD then uses reconstruction errors based on these adversarially learned features to determine if a data sample is anomalous. ALAD builds on recent advances to ensure data-space and latent-space cycle-consistencies and stabilize GAN training, which results in significantly improved anomaly detection performance. ALAD achieves state-of-the-art performance on a range of image and tabular datasets while being several hundred-fold faster at test time than the only published GAN-based method.

I. INTRODUCTION

ALAD addresses anomaly detection for complex, high-dimensional data by combining GAN modeling with efficient learned inference and reconstruction-based scoring.

  • Anomaly detection remains challenging for complex and high-dimensional data despite its practical significance and extensive study.
  • Standard GANs model data distributions with a generator mapping latent variables to data and a discriminator distinguishing real from generated samples.
  • GAN-based anomaly detection can require costly likelihood estimation or per-example latent optimization through the generator.
  • ALAD learns an encoder during training, enabling faster inference than methods that optimize a latent representation separately for each test example.
  • ALAD combines adversarially learned representations with reconstruction-based anomaly detection and evaluates them on tabular and image data.

IV. ADVERSARIALLY LEARNED ANOMALY DETECTION

Standard GANs support efficient sampling but do not directly provide latent representations for observed samples, making anomaly scoring computationally expensive.

  • Standard GANs efficiently sample from the learned distribution but do not directly compute likelihoods or latent representations for given data samples.

A. GAN architecture

The ALAD architecture combines bidirectional generator–encoder modeling with adversarial cycle-consistency constraints in data and latent spaces.

  • ALAD uses an encoder E that maps data samples to latent variables alongside a generator G mapping latent variables back to data.
  • An additional discriminator Dxx regularizes the encoder and generator to encourage data-space cycle consistency, G(E(x)) ≈ x.
  • BiGAN and AliGAN train G, E, and a joint-space discriminator Dxz to match encoder and generator joint distributions.
  • In practice, imperfect saddle-point convergence can violate cycle consistency and create problems for reconstruction-based anomaly detection.

B. Stabilizing GAN training

ALAD stabilizes training by regularizing the latent-space conditional and applying spectral normalization within a multi-discriminator adversarial objective.

  • Spectral normalization constrains discriminator weight-matrix spectral norms to stabilize GAN training efficiently.
  • ALAD regularizes the latent-space conditional Hπ(z|x) with an additional adversarial discriminator Dzz.
  • The complete training objective jointly optimizes G and E against discriminators Dxz, Dxx, and Dzz.
  • The final model simultaneously learns the generator, encoder, and three discriminators.

C. Detecting anomalies

ALAD detects anomalies by comparing samples with GAN reconstructions in a learned discriminator feature space. Its score uses feature-space reconstruction error rather than raw data-space distance.

  • C. Detecting anomalies: ALAD reconstructs each sample through an encoder-generator pair and treats larger reconstruction discrepancies as stronger anomaly evidence.The inference procedure encodes x, reconstructs it as G(E(x)), and returns an L1 feature discrepancy.
  • C. Detecting anomalies: ALAD models normal data with a GAN and uses cycle-consistency regularization to obtain latent representations that support faithful normal-sample reconstructions.The model includes symmetric conditional-entropy cycle-consistency terms to learn the data manifold and precise latent representations.
  • C. Detecting anomalies: ALAD measures reconstruction error in the CNN-code feature space of the cycle-consistency discriminator Dxx instead of using Euclidean distance in data space.The feature layer is defined as the layer before Dxx’s logits, and the score uses L1 distance between feature activations.
  • C. Detecting anomalies: The anomaly score A(x) compares Dxx features for the original pair (x, x) with those for the reconstructed pair (x, G(E(x))).The inference algorithm computes fδ=fxx(x, x̂) and fα=fxx(x, x), then returns ∥fδ−fα∥1.
  • C. Detecting anomalies: The proposed score is related to feature-matching loss, but uses features from Dxx rather than the discriminator of a standard GAN.ALAD applies the feature-matching idea to anomaly scoring at inference time.

Why not use the output of the Dxx discriminator?

The supplied passages motivate evaluating discriminator-based anomaly detection across public tabular and image benchmarks, while noting that Dxx can lose discriminative power at an ideal GAN solution.

  • Why not use the output of the Dxx discriminator?: At the GAN saddle-point solution, a perfectly learned generator and encoder make Dxx unable to distinguish real sample pairs from reconstructions.This motivates using feature loss rather than Dxx’s output for the anomaly score.
  • Why not use the output of the Dxx discriminator?: The evaluation uses public tabular and image datasets, including KDD99, Arrhythmia, SVHN, and CIFAR-10.KDD99 is a network-intrusion dataset, Arrhythmia contains cardiac-arrhythmia data, and the image datasets contain house-number and ten-class natural images.
  • Why not use the output of the Dxx discriminator?: The tabular evaluation labels the highest-scoring 20% of KDD samples and 15% of Arrhythmia samples as anomalies.These proportions correspond to the datasets’ stated experimental anomaly setups.

B. Baselines

ALAD is evaluated against classic, deep-learning, and GAN-based anomaly detectors on tabular and image datasets. It is competitive or superior across many tasks, while its image-task weaknesses arise in visually confusable normal-class settings.

  • Tabular data: ALAD outperforms DAGMM and significantly outperforms all other methods on the large KDD99 dataset.Isolation Forests is competitive on KDD99 and achieves state-of-the-art results on the small Arrhythmia dataset.
  • Image data: On SVHN, ALAD outperforms all other methods; on CIFAR-10, it wins 5 of 10 tasks and is competitive on the remaining 5.The image results are reported for individual tasks and averaged across tasks.
  • Inference efficiency: ALAD is orders of magnitude faster than AnoGAN at inference on both SVHN and CIFAR-10.Inference time is reported for the first task on each dataset.

E. Ablation studies

Ablation studies assess latent penalization, spectral normalization, and a latent-space discriminator. Their effects vary by dataset, with the strongest image gains appearing on SVHN.

  • Ablation design: The ablations remove spectral normalization and conditional entropy regularization, while the full ALAD model includes spectral normalization and a latent-space discriminator.The experiments systematically remove each additional component from the baseline ALICE model.
  • Tabular datasets: Latent penalization improves Arrhythmia results but does not affect KDD99 performance.These effects are reported in the tabular ablation study.
  • Image datasets: On SVHN, adding spectral normalization and the latent-space discriminator improves performance, whereas both have minimal effect on CIFAR-10.Image results are averaged over all tasks, with task-specific results reported separately.

F. Exploration of different anomaly scores

ALAD compares discriminator-feature reconstruction scores with logits and non-adversarial L1/L2 errors. Features are strongest on tabular data and CIFAR-10, while matching L1/L2 on SVHN; the broader method remains competitive across datasets and efficient at test time.

  • Score definitions: ALAD compares Logits, L1, L2, and discriminator-based Features anomaly scores using reconstructions x′ = G(E(x)).Features use representations computed by the reconstruction discriminator, whereas L1 and L2 use non-adversarial reconstruction errors.
  • Tabular results: Discriminator-based adversarially learned features are more suited to anomaly detection on tabular data.
  • Image results: On CIFAR-10, discriminator features detect anomalies better than the alternative scores, while on SVHN they perform comparably to L1 and L2.The full SVHN and CIFAR-10 results are reported in Table VIII and Figures 9–10.
  • Overall method: ALAD learns an encoder during training, making test-time inference significantly more efficient than the only published GAN-based anomaly detector.
  • Ablation findings: Ablation analyses show that additional discriminators and spectral normalization improve anomaly detection performance.Spectral normalization is incorporated to stabilize GAN training, while additional discriminators improve the encoder.
  • Overall results: ALAD is highly competitive with state-of-the-art methods across tabular and image datasets and often outperforms them.The conclusion also identifies other data modalities, including speech and sensor data, as future directions.

APPENDIX

The appendix records preprocessing, architectures, and inference settings for the image experiments and identifies the discriminator layer used for anomaly scoring.

  • Preprocessing: Image pixels are scaled to the range [-1,1].
  • Image architecture: CIFAR-10 and SVHN use a convolutional architecture with one convolutional layer, max-pooling, and a 128-unit fully connected layer.The convolution uses a 3-sized kernel, stride 2, 64 filters, and same padding.
  • Baseline settings: The experiments use the official DCGAN architecture and hyperparameters, with exponential moving average inference decay set to 0.999.
  • Anomaly scoring: The outputs of an underlined discriminator layer are used to compute the anomaly score.All convolutional layers use same padding unless otherwise specified.
  • Configuration tables: Appendix tables provide ALAD architecture and hyperparameter specifications for CIFAR-10 and SVHN.
  • Tabular preprocessing: Categorical features are one-hot encoded, increasing a 41-dimensional dataset to 121 features.The dataset contains 34 continuous and 7 categorical dimensions before encoding.

C. Arrhythmia Experiment Details

The supplied appendix material identifies the Arrhythmia data dimensionality and lists experiment figures and architecture tables, but does not provide their detailed results or encodings.

  • Data preparation: The Arrhythmia dataset contains 274 dimensions, and the methods are applied to the raw data.
  • Ablation studies: Figures 7 and 8 are labeled as ablation studies on SVHN and CIFAR-10, respectively.
  • Score comparisons: Figures 9 and 10 are labeled as performance comparisons of different anomaly scores on SVHN and CIFAR-10, respectively.
  • Architecture specifications: The appendix lists separate GAN and ALAD architecture and hyperparameter tables for KDD99 and Arrhythmia.
Loading 1812.02288v1…