PaperlayerView exact revision on arXiv

Source-linked AI summary

Anomaly Detection using One-Class Neural Networks

Raghavendra Chalapathy, Aditya Krishna Menon, Sanjay Chawla

arXiv:1802.06360v2cs.LGcs.NEstat.ML

TL;DR

Anomaly detection needs methods that identify unusual instances in complex datasets, where conventional shallow approaches can be limited. The paper proposes OC-NN, combining deep representation learning with a one-class SVM-like objective, and reports performance on par with state-of-the-art methods while outperforming conventional shallow methods in some scenarios.

  • Problem

    Anomaly detection on complex, high-dimensional datasets motivates methods beyond conventional OC-SVM and hybrid pipelines whose feature learning is not influenced by anomaly detection.

  • Method

    OC-NN integrates a one-class SVM-like loss into a neural network so hidden-layer representations are customized for anomaly detection.

  • Results

    OC-NN performs on par with state-of-the-art methods on complex datasets and outperforms conventional shallow methods in some scenarios.

  • Takeaways & Limitations

    OC-NN jointly learns progressively rich representations and a one-class envelope around normal data for anomaly detection.

Abstract

from arXiv · show

We propose a one-class neural network (OC-NN) model to detect anomalies in complex data sets. OC-NN combines the ability of deep networks to extract a progressively rich representation of data with the one-class objective of creating a tight envelope around normal data. The OC-NN approach breaks new ground for the following crucial reason: data representation in the hidden layer is driven by the OC-NN objective and is thus customized for anomaly detection. This is a departure from other approaches which use a hybrid approach of learning deep features using an autoencoder and then feeding the features into a separate anomaly detection method like one-class SVM (OC-SVM). The hybrid OC-SVM approach is sub-optimal because it is unable to influence representational learning in the hidden layers. A comprehensive set of experiments demonstrate that on complex data sets (like CIFAR and GTSRB), OC-NN performs on par with state-of-the-art methods and outperformed conventional shallow methods in some scenarios.

1 Anomaly detection: motivation and challenges

The paper addresses anomaly detection on complex, high-dimensional data by integrating a one-class objective into a neural network. It proposes OC-NN, whose anomaly-specific representation learning is evaluated through extensive experiments.

  • Hybrid approaches using autoencoder features and a separate anomaly detector cannot influence representational learning in hidden layers.
  • OC-NN derives a one-class neural network model whose one-class SVM-like loss drives neural-network training.
  • OC-NN proposes an alternating minimization algorithm, with a subproblem equivalent to quantile selection.
  • Extensive experiments evaluate OC-NN against other state-of-the-art deep-learning approaches on complex image and sequence datasets.
  • The paper develops the model, describes experiments and metrics, analyzes results, and concludes with future-work directions.

2 Background and related work on anomaly detection

The related work situates OC-NN among unsupervised anomaly-detection methods, including OC-SVM, hybrid deep-feature approaches, and robust autoencoders. These methods differ in how they represent data and separate normal observations from anomalies.

  • Unsupervised anomaly detection seeks rules separating normal and anomalous data without labels.
  • OC-SVM constructs a smooth boundary around most of the data’s probability mass to detect anomalies.
  • Robust deep autoencoders decompose input X into latent representation LD and noise or outlier component S.
  • Robust autoencoder optimization combines backpropagation with ADMM, and the experiments compare OC-NN with robust-autoencoder approaches.
  • OC-SVM separates mapped data points from the origin in a reproducing kernel Hilbert space and maximizes the hyperplane’s distance from that origin.
  • The parameter ν controls the trade-off between hyperplane distance from the origin and the number of data points allowed to cross it.

3 From One Class SVM to One Class Neural Networks

OC-NN adapts the one-class SVM objective into a neural network so learned representations can be refined for anomaly detection. Training alternates neural-network optimization with quantile-based threshold selection.

  • 3.1 One-Class Neural Networks (OC-NN): The model uses a feed-forward network with one hidden layer, linear or sigmoid activation, and one output node.The paper states that deeper architectures can be generalized straightforwardly.
  • 3.1 One-Class Neural Networks (OC-NN): OC-NN replaces the OC-SVM feature dot product with a neural-network transformation g(VX_n:) before computing the output score.This enables anomaly-specific feature refinement but makes the objective non-convex, so optimization is not guaranteed to reach a global optimum.
  • 3.2 Training the model: Training alternates between optimizing w and V with backpropagation and optimizing r using the υ-quantile of the predicted scores.The resulting decision score is S_n = ŷ_n − r; nonnegative scores are classified as normal and negative scores as anomalous.
  • 3.2 Training the model: The algorithm initializes r, repeats parameter and threshold updates until convergence, then returns decision scores for all input points.The classification rule assigns normal status when S_n ≥ 0 and anomaly status otherwise.
  • 3.2 Training the model: For the example set y = {1, 2, 3, 4, 5, 6, 7, 8, 9} with υ = 0.33, the objective minimum occurs at f(3).The example illustrates the quantile-selection step used to optimize r.

4 Experimental Setup

The experiments evaluate OC-NN against shallow and deep anomaly-detection baselines on synthetic and real-world datasets, emphasizing high-dimensional image data. The setup also uses autoencoder-derived representations and dataset-specific one-class constructions.

  • Datasets: The study focuses primarily on non-trivial high-dimensional images while noting that OC-NN can also apply where autoencoders provide feature representations.Speech is given as an example of another possible application context.
  • Methods compared: OC-NN is compared with OC-SVM/SVDD, Isolation Forest, KDE, DCAE, AnoGAN, Deep SVDD, RCAE, and other state-of-the-art methods.The listed baselines include both shallow and deep approaches.
  • Baseline Model Parameters: The OC-NN model is implemented alongside the baselines using Keras and TensorFlow, with publicly available implementations for OC-SVM and Isolation Forest.Baseline hyperparameters and model configurations are selected according to described implementation procedures and prior recommendations.
  • Datasets: The comparison includes synthetic data and four real-world datasets, including MNIST, GTSRB, and CIFAR-10.The synthetic dataset contains 190 normal points and 10 anomalous points; MNIST and CIFAR-10 are organized into one-class tasks.
  • Model architecture: OC-NN first obtains representative features with a deep autoencoder, then feeds copied encoder layers into a one-hidden-layer network whose encoder weights remain trainable.The decoder for the DCAE baseline is constructed symmetrically using upsampling in place of max-pooling.

5 Experimental Results

Experiments evaluate OC-NN on synthetic, image, and adversarial data, finding performance comparable to state-of-the-art methods and stronger than conventional shallow methods in some scenarios.

  • OC-NN performs on par with state-of-the-art methods on complex data sets and outperforms conventional shallow methods in some scenarios.The experiments cover synthetic and real data sets.
  • 5.1 Synthetic Data: In synthetic data with 190 normal and 10 anomalous 512-dimensional points, OC-NN accurately identifies nearly all anomalies and performs on par with classical OC-SVM.Anomalies have decision scores that are negative.
  • 5.2 OC-NN on MNIST and CIFAR-10: The visual comparisons include most normal and in-class anomalous examples detected by RCAE and OC-NN on MNIST and CIFAR-10.These figures provide qualitative comparisons between the two methods.
  • 5.2 OC-NN on MNIST and CIFAR-10: MNIST and CIFAR-10 experiments construct one-class tasks by treating one class as normal and the remaining nine classes as anomalies.Training uses approximately 6,000 normal MNIST instances and 5,000 normal CIFAR-10 instances, with anomalies included at 1% and 10%, respectively.
  • The evaluation also includes adversarial stop-sign examples from GTSRB, with average AUCs reported separately for MNIST, CIFAR-10, and adversarial attacks.Figures identify normal and anomalous stop signs, while Tables 3 and 4 report average AUCs with standard deviations.

6 Conclusion

The paper proposes OC-NN, which trains neural representations with an OC-SVM-like objective for anomaly detection. Its experiments support performance comparable to state-of-the-art approaches on complex data, while its alternating minimization procedure provides a parameter-learning method.

  • OC-NN trains a neural network with an OC-SVM-like loss so hidden-layer features are constructed specifically for anomaly detection.This differs from hybrid approaches that use generic deep features with a separate anomaly detector.
  • Hybrid approaches use generic, task-unaware feature extraction, whereas OC-NN makes representation learning aware of the anomaly detection objective.
  • The proposed alternating minimization approach learns OC-NN parameters, with one subproblem equivalent to quantile selection.
Loading 1802.06360v2…