Source-linked AI summary
Adversarially Learned One-Class Classifier for Novelty Detection
Mohammad Sabokrou, Mohammad Khalooei, Mahmood Fathy, Ehsan Adeli
TL;DR
Novelty detection lacks reliable negative-class data, making end-to-end deep one-class classification difficult. The paper introduces an adversarial end-to-end system whose refinement and discrimination networks learn from the target class, and reports state-of-the-art performance across image and video applications.
Problem
End-to-end deep one-class classification is difficult when novelty-class data are unavailable, while novelty detection must model a target class without a well-defined negative class.
Method
An adversarial, unsupervised architecture uses R to reconstruct and enhance target-class samples and D to distinguish them from reconstructed inputs in an end-to-end setting.
Results
The framework learns without novelty-class training samples and achieves state-of-the-art performance in image outlier detection and video anomaly detection.
Takeaways & Limitations
Using both trained modules during testing supports novelty detection across outlier detection, image novelty detection, and video anomaly detection applications.
Takeaways & Limitations
Finding the optimal network structure and training procedure can be tedious, and performance depends critically on when joint training is stopped.
Abstract
from arXiv · showhide
Novelty detection is the process of identifying the observation(s) that differ in some respect from the training observations (the target class). In reality, the novelty class is often absent during training, poorly sampled or not well defined. Therefore, one-class classifiers can efficiently model such problems. However, due to the unavailability of data from the novelty class, training an end-to-end deep network is a cumbersome task. In this paper, inspired by the success of generative adversarial networks for training deep models in unsupervised and semi-supervised settings, we propose an end-to-end architecture for one-class classification. Our architecture is composed of two deep networks, each of which trained by competing with each other while collaborating to understand the underlying concept in the target class, and then classify the testing samples. One network works as the novelty detector, while the other supports it by enhancing the inlier samples and distorting the outliers. The intuition is that the separability of the enhanced inliers and distorted outliers is much better than deciding on the original samples. The proposed framework applies to different related applications of anomaly and outlier detection in images and videos. The results on MNIST and Caltech-256 image datasets, along with the challenging UCSD Ped2 dataset for video anomaly detection illustrate that our proposed method learns the target class effectively and is superior to the baseline and state-of-the-art methods.
1. Introduction
Novelty detection identifies observations outside a target class, but absent novelty data makes end-to-end deep one-class classification difficult. The paper proposes an adversarial end-to-end architecture in which refinement and discrimination collaborate to separate inliers from novelties.
- Novelty detection identifies unusual observations as inliers or outliers and relates to outlier, denoising, and image- and video-anomaly detection.
- One-class classification models settings where the negative class is absent, poorly sampled, or not well defined, while target-class training instances are available.
- End-to-end deep one-class classification is difficult because training data from the negative class are unavailable.
- The proposed GAN-inspired architecture uses refinement network R and detector D, which compete while collaborating to make target and novelty samples more separable.R refines inputs and injects discriminative material, while D performs detection.
- R reconstructs positive samples and tries to fool D, whereas D distinguishes original positive samples from reconstructed ones using only target-class training data.This training lets D learn the concept represented by the positive-sample space.
- The framework learns without novelty-class training samples and reports state-of-the-art performance across image outlier detection and video anomaly detection applications.
2. Related Works
Related work models novelty through self-representation, statistical descriptions of the target class, and adversarial learning. These approaches differ in whether they use reconstruction, distributional distance, low-dimensional structure, or competing networks.
- Self-Representation: Self-representation methods detect novelties through sparse or low-rank representations, assuming outliers are not well represented by target-class samples.
- Self-Representation: Reconstruction-based self-representation classifies samples using reconstruction error, with higher error indicating greater likelihood of being an outlier.
- Statistical Modeling: Statistical modeling fits a distribution to target-class features and treats samples far from that distribution as outliers or novelties.
- Statistical Modeling: Robust statistical approaches use low-dimensional subspaces, mutual coherence, or convex optimization to detect outliers despite corrupted entries.
- Deep Adversarial Learning: GAN-based methods train a generator and discriminator concurrently, with the generator producing realistic data and the discriminator distinguishing real from generated data.
3. Proposed Approach
The proposed one-class classifier jointly trains a refinement/reconstruction network R and discriminator D in an adversarial, unsupervised, end-to-end framework. R enhances target-class inputs and degrades outliers, while D assigns target-likelihood scores, improving separability for novelty detection.
- Framework overview: The framework comprises adversarially trained networks R and D, with R reconstructing inputs and D distinguishing target-class samples from outliers.Training uses only target-class samples; R attempts to fool D, while D learns the target-class concept.
- R network: R is trained as a decoder-encoder CNN that reconstructs target-class inputs but poorly reconstructs novelty inputs, enhancing inliers and decimating outliers.Its convolutional encoder is followed by deconvolutional decoding layers.
- D network: D uses convolutional layers to distinguish novelty samples without supervision and outputs a scalar likelihood that its input follows the target-class distribution.The output is interpreted as a target-likelihood score for any given input.
- Adversarial training: The adversarial training adds Gaussian noise to inputs so R learns to map noisy samples toward the target-class distribution and become more robust to distortions.The noise is sampled from N(0, σ^2I), and λ controls the relative importance of the loss terms.
- Detection behavior: In most cases, D(R(X)) provides greater target-versus-novelty score separation than D(X), and its reject region is smaller.The reported comparison indicates that R supports D by making the outputs more separable than the original images.
- One-class classification: At inference, novelty detection can use D alone or the proposed refined score D(R(X)) compared with a predefined threshold.R is incorporated as a testing-time refinement step because it reconstructs and enhances target-class samples.
4. Experiment Results
The framework is evaluated on three image and video datasets to demonstrate its applicability to outlier-image and video-anomaly detection.
- Experiments cover outlier-image detection and video-anomaly detection across three image and video datasets.The reported results are compared with state-of-the-art techniques.
4.1. Setup
The reported experiments use fixed network structures and implementation settings across tasks, with λ set to 0.4.
- The experiments use TensorFlow and Python on an NVIDIA TITAN X, with fixed D and R structures across tasks.The detailed network structures are described in Sections 3.2 and 3.1.
- λ is set to 0.4, while batch normalization uses ϵ = 10^-6 and decay = 0.9.
4.2. Outlier Detection
The outlier-detection experiments evaluate the joint R+D network on MNIST and Caltech-256, including varying target-class and outlier settings. Results indicate that R refines inliers, distorts outliers, and improves separability for detection.
- 4.2. Outlier Detection: The method learns the shared concept among inlier samples to identify outliers in computer-vision data.The motivation is that outliers and noise can substantially degrade machine-learning performance when not handled.
- 4.2.1. Outlier Detection: D(R(X)) generally produces greater inlier–outlier separability than D(X), supporting improved novelty discrimination.The transformed samples yield a smaller reject region than the original samples in the reported comparison.
- 4.2.1. Experimental Setup: MNIST experiments use each digit category as the target class and simulate outliers from other categories at 10% to 50%.
- 4.2.1. Outlier Detection Results: On MNIST, D(R(X)) is more efficient than LOF and DRAE, remains robust as outlier proportions increase, and improves further over D(X).D(X) itself also outperforms the cited state-of-the-art methods in the reported experiment.
- 4.2.2. Outlier Detection Results: On Caltech-256, experiments use one, three, or five randomly chosen inlier categories and 50% outliers from the clutter category.Table 1 reports results for both D(X) and D(R(X)).
- 4.2.2. Outlier Detection Results: Both D(X) and D(R(X)) outperform other methods in most cases as the number of inlier categories increases from 1 to 3 and 5.The reported results describe robust learning of the inlier concepts across these settings.
4.3. Video Anomaly Detection
Video anomaly detection is evaluated on UCSD Ped2 using patches from normal training frames. The results show that R reconstructs normal patches, distorts abnormal ones, and improves discrimination over D(X) alone.
- 4.3. Video Anomaly Detection: UCSD Ped2 contains outdoor pedestrian scenes where cars, skateboarders, wheelchairs, and bicycles are treated as anomalies.The evaluation uses the Ped2 subset recorded with a static camera.
- 4.3.1. Experimental Setup: The video frames are divided into 30×30 patches, with training patches containing only normal behavior.Testing patches are evaluated by the joint R+D network.
- 4.3.2. Anomaly Detection Results: R successfully refines and reconstructs normal patches while distorting abnormal patches that it cannot adequately reconstruct.The reported examples show this behavior on testing patches.
- 4.3.2. Anomaly Detection Results: D(R(X)) produces more distinguishable likelihood scores than D(X), improving the one-class model for video anomaly detection.The comparison is especially relevant for difficult normal patches with partially visible human subjects.
- 4.3.2. Anomaly Detection Results: The method is reported as comparable to state-of-the-art methods on frame-level EER while using only spatial patch-based features.Other methods cited in the comparison may use more complex spatial and temporal features.
4.4. Discussion
The R+D framework performs competitively, but its effectiveness depends on selecting and training the joint network appropriately. The method assumes predominantly inlier training data and differs from typical GAN use by retaining both modules.
- R+D detects novelty samples at least as well as state-of-the-art methods or better in many cases.
- Finding the optimal network structure and training procedure is tedious and cumbersome.The authors report that modifying convolutional-layer size and order improved results by 0.02 to 0.04 over Table 1 results.
- Stopping training too early produces immature weights, whereas overtraining confuses R and yields undesirable outputs.The stopping criterion in Section 3.3 is intended to balance joint-network maturity and target-class understanding.
- Training without novelty samples is treated as weak supervision and is generally acceptable when available data are predominantly inliers.The authors assume target-class samples substantially outnumber outliers, while noting that the model can still work with a small number of outlier samples.
- R is less exposed to conventional GAN mode collapse because it directly observes all possible target-class samples and implicitly learns their manifold.
5. Conclusion
The paper proposes an adversarially trained framework for one-class classification and novelty detection in images and videos. Its Reconstructor and Discriminator learn the target-class concept together, enabling detection of novel samples without novelty-class training data.
- The framework uses a Reconstructor and Discriminator trained adversarially for one-class classification and novelty detection.The Reconstructor learns to reproduce target-class images, while the Discriminator is trained to recognize them.
- R reconstructs target-class samples correctly but distorts samples lacking the target-class concept.This distortion helps D discriminate testing samples more effectively.
- The method is applied to outlier and anomaly detection in images and videos.
- Results on several datasets show that the method detects samples outside the target class despite having no novelty-class samples during training.