PaperlayerView exact revision on arXiv

Source-linked AI summary

DOPING: Generative Data Augmentation for Unsupervised Anomaly Detection with GAN

Swee Kiat Lim, Yi Loo, Ngoc-Trung Tran, Ngai-Man Cheung, Gemma Roig, Yuval Elovici

arXiv:1808.07632v2cs.LGstat.ML

TL;DR

Unsupervised anomaly detection lacks labels for identifying minority events, and infrequent normal samples are associated with false positives. DOPING uses an AAE to generate such samples from the edge of a latent distribution, and the paper reports consistent performance improvements across several real-world datasets.

  • Problem

    In unsupervised anomaly detection, labels for identifying minority events are unavailable, and infrequent normal samples are poorly characterized and associated with false positives.

    Source: §I · pp. 1–2

  • Method

    DOPING uses an adversarial autoencoder to impose a unimodal latent distribution with well-defined tails, then samples its edge to generate infrequent normal examples.

    Source: §I · p. 2, §VII · p. 10

  • Results

    Experiments show that DOPING improves anomaly-detection performance across a variety of datasets, and iForest with DOPING performs best across all datasets except Mammography.

    Source: §VII · p. 10, B. Results and Discussion · p. 10

  • Takeaways & Limitations

    The authors present latent-space sampling as a unified approach for generating infrequent normal samples across datasets with complex distributions.

    Source: §I · p. 2, §VII · p. 10

Abstract

from arXiv · show

Recently, the introduction of the generative adversarial network (GAN) and its variants has enabled the generation of realistic synthetic samples, which has been used for enlarging training sets. Previous work primarily focused on data augmentation for semi-supervised and supervised tasks. In this paper, we instead focus on unsupervised anomaly detection and propose a novel generative data augmentation framework optimized for this task. In particular, we propose to oversample infrequent normal samples - normal samples that occur with small probability, e.g., rare normal events. We show that these samples are responsible for false positives in anomaly detection. However, oversampling of infrequent normal samples is challenging for real-world high-dimensional data with multimodal distributions. To address this challenge, we propose to use a GAN variant known as the adversarial autoencoder (AAE) to transform the high-dimensional multimodal data distributions into low-dimensional unimodal latent distributions with well-defined tail probability. Then, we systematically oversample at the `edge' of the latent distributions to increase the density of infrequent normal samples. We show that our oversampling pipeline is a unified one: it is generally applicable to datasets with different complex data distributions. To the best of our knowledge, our method is the first data augmentation technique focused on improving performance in unsupervised anomaly detection. We validate our method by demonstrating consistent improvements across several real-world datasets.

I. INTRODUCTION

Unsupervised anomaly detection lacks the labels needed by conventional oversampling, while infrequent normal samples are linked to false positives. DOPING addresses this by generating such samples without labels, using latent-space sampling designed for complex data distributions.

I. INTRODUCTION

  • Unlike supervised oversampling methods such as SMOTE and SPO, DOPING targets unsupervised anomaly detection without requiring class labels.The methods described as conventional oversampling rely on minority-class labels to select samples.Source: §I · p. 1
  • Infrequent normal samples—normal events occurring with very small probability—are poorly characterized and linked to false positives near the imprecise normal–anomalous boundary.The paper hypothesizes that targeted oversampling of these samples will be most effective for reducing false-positive errors.Source: §I · p. 2
  • DOPING uses an adversarial autoencoder to map complex, potentially multimodal data into a unimodal latent distribution with well-defined tail probability, then samples there to generate infrequent normal examples.The authors present latent-space sampling as a way to handle diverse datasets with complex distributions.Source: §I · p. 2
  • DOPING is dataset augmentation before detector training, rather than a standalone anomaly detector, and is intended to complement anomaly detection algorithms.The paper contrasts targeted synthesis of infrequent normals with indiscriminate interpolation across the training set.Source: Figure 1 · p. 1, §I · p. 2

II. RELATED WORK

Prior GAN-based anomaly detectors typically train on known normal data, while GAN data augmentation has mainly served supervised classification. DOPING instead uses an unlabeled AAE latent space to generate samples for anomaly-detection augmentation.

II. RELATED WORK

  • Prior GAN-based anomaly detectors typically train on a known normal dataset, while GAN augmentation has mainly generated samples for supervised classification.Earlier autoencoder approaches also include methods trained with normal and anomalous labels or methods combining autoencoders with robust PCA.Source: §II · p. 2
  • DOPING differs from prior approaches by requiring neither a known normal set nor labels, and is presented as augmentation applicable to any dataset or anomaly detector.The paper contrasts this with minority-class oversampling methods that require label information.Source: §II · pp. 2–3

III. DOPING

  • DOPING means Doping with Infrequent Normal Generator, an augmentation technique that adds synthetically generated infrequent normal samples to the original training set.The generated samples are produced with adversarial autoencoders.Source: §III · p. 3
  • The AAE matches the aggregated encoder posterior q(z) to a chosen prior p(z) using an adversarial discriminator.The paper notes that this mechanism can yield latent spaces with varied distributions, including mixtures of Gaussians and a swiss roll.Source: §III · p. 3

A. Data augmentation with DOPING: Details

DOPING uses an unlabeled adversarial autoencoder to identify edge-region latent vectors, interpolate them with nearest neighbors, and decode synthetic samples for anomaly-detector training.

A. Data augmentation with DOPING: Details

  • DOPING trains an Unlabeled AAE on the full dataset with a multivariate Gaussian prior, then encodes the training data into latent vectors.The method uses no labels during AAE training.Source: A. Data augmentation with DOPING:… · p. 3
  • It filters latent vectors by norm to select tail-region samples, then interpolates each selected vector with its nearest latent-space neighbor.The edge region is defined by α < ∥z∥ < β; α and β are set using validation data.Source: A. Data augmentation with DOPING:… · p. 3
  • DOPING decodes the interpolated vectors into synthetic samples and trains the anomaly detector on the augmented dataset.The augmented dataset combines the original data with the synthesized samples.Source: A. Data augmentation with DOPING:… · p. 3

B. Use of Adversarial Autoencoders

The AAE maps complex data distributions to a shared latent representation that supports systematic sampling. Synthetic-data experiments examine this mapping and compare sampling across datasets and latent magnitudes.

B. Use of Adversarial Autoencoders

  • The AAE enables explicit encoding and decoding and permits control over the latent prior, allowing analysis to guide sample synthesis across datasets.Unlike methods requiring iterative latent estimation, the AAE couples an encoder and decoder; its adversarial prior-matching requires samples rather than the prior’s exact functional form.Source: B. Use of Adversarial Autoencoders · p. 4
  • DOPING uses a multivariate Gaussian prior to transform complex data distributions into a unimodal latent distribution with well-defined tail probability.Sampling at the latent distribution’s edge is intended to generate infrequent normal samples for augmentation.Source: B. Use of Adversarial Autoencoders · p. 4

IV. ANALYSIS OF DOPING WITH SYNTHETIC DATASETS

  • Across synthetic datasets, the AAE maps different input distributions to similar latent distributions, including the challenging dataset where normal and anomalous distributions switch.The analysis reports a consistent sampling region for improving anomaly detection despite differences among the data distributions.Source: §IV · p. 4

A. Dataset and Experimental Design

  • The three synthetic datasets each contain 1,000 training and 1,000 test samples, with 95% normal and 5% anomalous samples.Dataset A is a 2-D Gaussian setup, Dataset B is its 3-D version, and Dataset C places Gaussian anomalies inside a ring-shaped normal distribution.Source: A. Dataset and Experimental Design · p. 4
  • The magnitude-based analysis decodes samples from latent vectors at increasing l2-norms, adds 100 samples per setting, and evaluates detector ROC AUC.The tested norms run from 5.0 to 100.0 in increments of 5.0.Source: A. Dataset and Experimental Design · pp. 4–5

B. Mapping of Data Spaces to Latent Distributions

AAE training maps distinct synthetic data distributions to similar latent distributions, making latent-space sampling comparable across datasets. Sampling at the normal-distribution boundary improves AUC, while sampling farther out can worsen it.

B. Mapping of Data Spaces to Latent Distributions

  • Training an AAE with the same prior maps the three distinct synthetic datasets to similar latent distributions, even when Dataset C’s normal and anomalous distributions switch.Dataset C has a ring-shaped normal distribution surrounding Gaussian anomalies.Source: B. Mapping of Data Spaces to Latent… · p. 5

C. Optimal Latent Space Sampling for Synthesizing Samples

V. ANALYSIS OF DOPING WITH MNIST EXPERIMENT

The MNIST experiments assess DOPING with an Unlabeled AAE, compare latent priors, and test against other augmentation methods using iForest. The paper reports consistent and substantial improvement across state-of-the-art anomaly detectors.

V. ANALYSIS OF DOPING WITH MNIST EXPERIMENT

  • DOPING gives consistent and substantial improvement over other data augmentation methods across several state-of-the-art anomaly detectors in the MNIST experiments.The section also describes MNIST as a higher-dimensional, more complex dataset and reports comparisons among latent priors.Source: §V · p. 6

A. Dataset and Experimental Setup

  • The experiment uses 5,000 MNIST images, with 4,750 class-4 images designated normal and 250 images from the other nine classes designated anomalous.iForest is the detector, and its contamination parameter is varied from 0.01 to 0.69 in increments of 0.02 to generate the ROC curve.Source: A. Dataset and Experimental Setup · p. 6
  • The augmentation baselines are random pixel changes and elastic deformation, with 500 augmented samples added to the original dataset.Elastic deformation uses σ = 4 and α = 34; detectors are trained on 5,500 samples and tested on 5,000 unaugmented samples.Source: A. Dataset and Experimental Setup · p. 6
  • The prior comparison replaces the Gaussian prior with generalized Gaussians at varying β values or an 8-D isotropic Gaussian prior.The analysis first examines magnitude-based sampling and then evaluates the full DOPING method.Source: A. Dataset and Experimental Setup · p. 7

B. Optimal Sampling with Unlabeled AAE

With an Unlabeled AAE, MNIST samples follow a 2-D Gaussian latent distribution, whose boundary yields infrequent but normal-looking images. iForest performance peaks when sampling at that boundary, while sampling farther out worsens performance.

B. Optimal Sampling with Unlabeled AAE

  • MNIST training-image encodings from the Unlabeled AAE follow a 2-D Gaussian, with normal and anomalous samples mixed despite some anomaly clustering.The AAE was trained without labels.Source: B. Optimal Sampling with Unlabeled AAE · p. 7
  • Decoded samples near the latent center look more common and normal, samples outside degrade, and boundary samples resemble infrequent normal images.Edge-based samples qualitatively resemble samples decoded at magnitude 20, which is also at the boundary.Source: B. Optimal Sampling with Unlabeled AAE · p. 7, Figure 6 · p. 7
  • AUC peaks at latent magnitude 15.0, corresponding to the distribution boundary, and edge-based sampling performs similarly to the best magnitude-based result.At increasing magnitudes, AUC worsens and eventually falls below the unaugmented iForest result.Source: B. Optimal Sampling with Unlabeled AAE · pp. 7–8, Figure 7 · p. 7
  • Adding 500 DOPING samples reduces False Positive Rate from 0.31 to 0.26 while True Positive Rate remains 0.80.The samples are generated using edge-based sampling.Source: B. Optimal Sampling with Unlabeled AAE · p. 8

C. Comparison of Prior Distributions

Across the tested DOPING priors, iForest AUC is consistently better than on the original MNIST dataset, and DOPING performs comparably to or better than other augmentation methods. Sampling at the latent distribution’s edge can be done without manually tuning magnitude.

C. Comparison of Prior Distributions

  • Across the tested priors, DOPING’s best iForest AUC results are consistently better than iForest on the original MNIST dataset.The comparison samples across a range of latent-vector l2-norms.Source: C. Comparison of Prior Distributions · p. 8, Table I · p. 8
  • As β increases, the optimal sampling magnitude decreases along with the generalized Gaussian’s boundary; the 8-D Gaussian’s optimal magnitude exceeds the 2-D Gaussian’s.The text relates the 8-D result to the increase in the mean l2-norm as Gaussian dimension rises.Source: C. Comparison of Prior Distributions · p. 8

D. Comparison Against Other Data Augmentation Methods

  • Edge-based sampling reliably selects latent vectors at the distribution edge without manually tuning magnitude.Edge-based decoded samples qualitatively resemble magnitude-20 samples, which also come from the boundary.Source: D. Comparison Against Other Data… · p. 8
  • Both 2-D and 8-D DOPING achieve comparable or much better iForest AUC than other augmentation methods on MNIST, with 8-D DOPING performing far better.The authors suggest the higher latent dimension may encode the dataset better.Source: D. Comparison Against Other Data… · p. 8, Table II · p. 8
  • DOPING samples form ringlike latent distributions that overlap strongly with false positives, which are denser near the distribution edge.The authors state that these samples increase false-positive density in training so detectors can better recognize them as normal, without a prior anomaly-detection test to identify them.Source: D. Comparison Against Other Data… · p. 9, Figure 8 · p. 9

VI. EVALUATION WITH REAL-WORLD DATASETS

The paper evaluates the formal DOPING algorithm on several real-world datasets and reports best-F1 and G-measure comparisons.

VI. EVALUATION WITH REAL-WORLD DATASETS

  • The formal DOPING algorithm is evaluated on several real-world datasets.Source: §VI · p. 9
  • The real-world results include comparisons of best F1.Source: Table III · p. 9
  • The real-world results also include comparisons of G-measure.Source: Table IV · p. 9

A. Dataset and Experimental Setup

The evaluation uses four ODDS anomaly-detection datasets, clean normal-only training splits, and iForest. Each augmentation adds synthetic samples equal to 10% of the original training-set size.

A. Dataset and Experimental Setup

  • DOPING results are reported on four anomaly-detection datasets from the ODDS repository; the cited dataset descriptions identify Mammography (6 features), Thyroid (6 features), and Lymphography (18 features).Source: A. Dataset and Experimental Setup · p. 9
  • Each run uses a random 50% training split and reserves the remaining 50% for testing, with only normal samples used to train models.iForest is the detector; contamination varies from 0.01 to 0.69 in increments of 0.02 to calculate AUC and search for best F1.Source: A. Dataset and Experimental Setup · p. 9
  • iForest is trained on the original set or one of three augmented sets: DOPING, a SMOTE variant, or an INOS variant.Each augmentation synthesizes samples equal to 10% of the original training-set size.Source: A. Dataset and Experimental Setup · p. 9
  • For these datasets, DOPING uses an Unlabeled AAE with a 2-D regular Gaussian prior of standard deviation 10.0 on both dimensions, then edge-based sampling.Source: A. Dataset and Experimental Setup · p. 10
  • The SMOTE and INOS comparison variants interpolate randomly selected training samples across the whole set because labels are unavailable.The SMOTE variant uses nearest-neighbor interpolation; INOS uses r = 0.8, with 80% of synthetic samples from ESPO and 20% from ADASYN.Source: A. Dataset and Experimental Setup · p. 10

B. Results and Discussion

DOPING performs best among tested augmentation methods on every dataset except Mammography, and the experiments report improved anomaly detection across varied datasets. The paper presents the experiments as demonstrating potential and notes that they use a fixed Gaussian prior.

B. Results and Discussion

  • Augmented datasets mostly outperform baseline Isolation Forest, which the authors associate with increased density and variety of normal training samples.The authors offer this density-and-variety explanation for results obtained with a clean training set.Source: B. Results and Discussion · p. 10
  • DOPING achieves the best performance across all datasets except Mammography.The comparison is reported for Isolation Forest with and without augmentation using F1 score and G-measure.Source: B. Results and Discussion · p. 10

VII. CONCLUSION AND FUTURE WORK

  • DOPING uses an adversarial autoencoder to impose a multivariate Gaussian latent space and decode at its edge, generating infrequent normal samples to address their misclassification.Source: §VII · p. 10
  • The AUC-versus-l2-norm/magnitude trend is consistent across datasets, and experiments report improved anomaly detection across a variety of datasets.DOPING uses the consistent trend to synthesize specific samples for a dataset.Source: §VII · p. 10
  • The experiments demonstrate potential using a plain multivariate Gaussian prior with fixed mean and covariance; varying prior parameters and distribution types remains future work.Source: §VII · p. 10
Loading 1808.07632v2…