PaperlayerView exact revision on arXiv

Source-linked AI summary

One-Class Classification: A Survey

Pramuditha Perera, Poojan Oza, Vishal M. Patel

arXiv:2101.03064v1cs.CVcs.LG

TL;DR

OCC asks how to recognize a positive class when training data contain only that class, a setting in which learning features and defining boundaries are challenging. This survey reviews classical and deep-learning methods for visual OCC, finding that methods perform well on simpler datasets but generalize poorly to more complex ones.

  • Problem

    OCC must recognize a positive class using training data from that class alone, which makes feature learning and defining classification boundaries more challenging than in multi-class classification.

    Source: §I · p. 1

  • Method

    The paper surveys classical and recent deep-learning OCC methods for visual recognition, emphasizing deep features and classifiers.

    Source: §I · p. 2

  • Results

    Reported OCC methods perform very well on simpler datasets such as MNIST and COIL100 but fail to generalize well to more complex datasets such as CIFAR10.

    Source: §VI · p. 16

  • Takeaways & Limitations

    The authors identify generalization to complex visual datasets as an open problem and call for more semi-supervised OCC methods that leverage additional data.

    Source: §VI · p. 16

    Recent work shows deep network-based OCC can be vulnerable to adversarial attacks, and more study of their effects on one-class models is needed.

    Source: §VI · p. 16

Abstract

from arXiv · show

One-Class Classification (OCC) is a special case of multi-class classification, where data observed during training is from a single positive class. The goal of OCC is to learn a representation and/or a classifier that enables recognition of positively labeled queries during inference. This topic has received considerable amount of interest in the computer vision, machine learning and biometrics communities in recent years. In this article, we provide a survey of classical statistical and recent deep learning-based OCC methods for visual recognition. We discuss the merits and drawbacks of existing OCC approaches and identify promising avenues for research in this field. In addition, we present a discussion of commonly used datasets and evaluation metrics for OCC.

I. INTRODUCTION

OCC learns to recognize a positive class from positive-only training data, making feature learning and boundary definition harder than in multi-class recognition. The survey organizes OCC methods and reviews their features, classifiers, and research directions.

I. INTRODUCTION

  • OCC trains on a single positive class, then must distinguish it from other objects at inference, making feature learning and boundary definition challenging.The survey also relates OCC to novelty detection, outlier detection, and open-set recognition, distinguishing their training assumptions.Source: §I · p. 1

II. TAXONOMY

  • The proposed taxonomy groups image-based OCC methods by training data, feature representation, and classification algorithm, with methods potentially spanning multiple categories.The categories include positive-only, positive-and-unlabeled, or OOD training data; handcrafted, statistical, or deep features; and statistical, representation-based, or deep classifiers.Source: §II · p. 2

III. FEATURES FOR OCC

  • Useful OCC features should compactly represent images of the same class while distinguishing images from different classes.The absence of non-positive training examples makes learning or selecting such features harder than in multi-class classification.Source: §III · p. 2

A. Statistical Features

  • Handcrafted features, and label-independent PCA, KPCA, and sparse coding can be used for OCC, while label-driven LDA and deep feature learning do not fit its positive-only setting.PCA and KPCA embeddings can describe the training image space but are not necessarily compact.Source: §III · p. 2, A. Statistical Features · p. 4

B. Deep learning-based Features

The section surveys deep feature-learning strategies for OCC, including reconstruction, transformation prediction, metric learning, and learning with labeled OOD data. These approaches seek representations that are compact for positive-class data and descriptive of data outside that class.

Deep Auto-encoders

  • Auto-encoders minimize input-to-output distance; a bottleneck can retain reconstruction-essential information, and adding noise can reduce overfitting and improve generalizability.A denoising auto-encoder reconstructs a clean image from a noisy input.Source: B. Deep learning-based Features · p. 4

Geometric Transformation based Self-supervision

  • Geometric self-supervision trains a network to predict which randomly selected transformation was applied, producing features described as suitable for OCC.The transformation set includes the identity transformation.Source: B. Deep learning-based Features · pp. 4–5
  • Golan et al. [28] score test-image normality using ground-truth transformation-class likelihoods, and also derive a more effective Dirichlet-based score for OCC.The first score combines log-likelihood probabilities across all k transformations.Source: B. Deep learning-based Features · p. 5

Deep Metric Learning with OOD Data

  • Contrastive metric learning uses OOD data, or Gaussian-noise-generated substitutes, to encourage similar positive embeddings and separate different-class embeddings by at least a margin.The loss also permits zero loss when both images are from the same class and neither is OOD.Source: B. Deep learning-based Features · p. 5

Feature Learning With OOD Data (DOC)

  • OOD-data training jointly optimizes positive-feature compactness and OOD descriptiveness after training the network on OOD data.The compactness loss uses within-batch positive-feature distances, while descriptiveness is measured with OOD cross-entropy.Source: B. Deep learning-based Features · p. 5

IV. OCC ALGORITHMS

The section describes representation-based OCC methods that classify queries through reconstruction error or distance from training representations, then focuses on Foley–Sammon Transform methods. In the one-class formulation, the coordinate origin serves as the negative class and a null-space projection supports threshold-based recognition.

A. Representation-based Methods

  • Representation-based OCC can classify by reconstructing a query and measuring image-space error, or by scoring its feature distance from training data.The section places sparse coding and PCA in the reconstruction strategy and kernel-based OCC in the distance strategy.Source: A. Representation-based Methods · p. 5
  • Null Foley–Sammon Transform seeks a mapping with zero within-class distances and positive between-class distances; its one-class formulation treats the coordinate origin as the negative class.A single null-space vector is learned from positive data.Source: A. Representation-based Methods · p. 6
  • Kernelizing the product terms maps data to a higher-dimensional space and relaxes the small-sample-size condition.The stated condition is n ≤ N.Source: A. Representation-based Methods · p. 6
  • NFST inference projects mean-removed training and query data onto the learned vector, then labels the query positive when their projection difference is below a threshold.The query projection is compared with the training projection.Source: A. Representation-based Methods · p. 6

B. Statistical Methods

The section surveys statistical OCC classifiers built around separating positive data from the origin, enclosing it in a hypersphere, or using probability and multiple-boundary formulations. It also describes kernelization, robustness assumptions, and subspace extensions for these methods.

One Class Support Vector Machine (OCSVM)

  • OCSVM finds a hyperplane furthest from the origin with positive training data in its positive half-space, and its kernelized dual supports kernel-based inference.The inference decision is expressed using dual variables, training images, and a kernel function.Source: B. Statistical Methods · pp. 6–7

Support Vector Data Descriptor (SVDD)

  • SVDD instead fits the smallest hypersphere around most training samples, classifies queries inside it as positive, and becomes equivalent to OCSVM for kernels depending only on vector differences.Its hypersphere is characterized by a center and radius.Source: B. Statistical Methods · p. 7

One Class Mini-max Probability Machine (OCMPM)

  • One-class MPM uses training mean and covariance to seek a hyperplane that contains data with probability at least τ in the worst case, without assuming Gaussian data.Its optimization can be expressed as a second-order cone program.Source: B. Statistical Methods · p. 7
  • Robust 1-MPM assumes bounded discrepancies between sample and true covariance and mean, modifying its covariance and probability-bound terms accordingly.The stated bounds are ρ for covariance difference and ν² for Mahalanobis distance between means.Source: B. Statistical Methods · p. 7

Dual slope Mini-max Probability Machine (DS–OCMPM)

  • Dual-slope MPM adds a second hyperplane to maximize projected variance and uses clustering to address sub-distributions while retaining global-statistics constraints.The method uses Ward’s clustering and obtains negative labels only when both hyperplane tests fall below their thresholds.Source: B. Statistical Methods · p. 8

Generalized One-class Discriminative Sub-spaces (GODS)

  • GODS extends OCSVM with two hyperplanes, placing most data on opposite sides; its generalized form uses subspaces and a dataset-specific hyperparameter for the number of hyperplanes.The distance constraint prevents a trivial solution.Source: B. Statistical Methods · p. 8

1) Discriminative Methods:

Discriminative OCC methods use losses inspired by statistical classifiers or regularization to adapt neural-network training. The figures illustrate statistical-classifier decision boundaries and the One-Class CNN training strategy.

1) Discriminative Methods:

  • Discriminative methods use losses inspired by OC-SVM and SVDD, or regularization that makes traditional neural-network training more compatible with OCC.Source: 1) Discriminative Methods · p. 8
  • Figure 5 presents decision boundaries obtained by different statistical one-class classifiers.Source: Figure 5 · p. 9
  • Figure 6 depicts One-Class CNN training inspired by OC-SVM, comparing features against a zero-centered Gaussian distribution as OC-SVM separates one-class data from the origin.Source: Figure 6 · p. 9

One Class CNN (OCCNN)

One Class CNN combines a feature extractor with a two-output classifier. Training contrasts features from positive examples with Gaussian noise sampled in feature space.

One Class CNN (OCCNN)

  • The classifier has positive and negative outputs, while its feature extractor is learned using an OOD dataset or another one-class feature-learning method.The feature extractor is trained on an OOD dataset in [58], though the survey notes that any One Class feature-learning method can be used.Source: One Class CNN (OCCNN) · p. 9
  • Training labels positive-example features as class 0 and Gaussian noise sampled from feature space as class 1, then trains the classifier with binary cross-entropy.The two batches are concatenated before classification; p0 and p1 denote the outputs for classes 0 and 1.Source: One Class CNN (OCCNN) · p. 9

Deep SVDD (DSVDD)

Deep SVDD learns a representation that encloses positive examples in a small hypersphere, while HRN combines log-likelihood training with feature regularization and instance normalization.

Deep SVDD (DSVDD)

  • Deep SVDD seeks a representation that encloses positive embeddings within the smallest possible hypersphere.Its loss encourages a smaller radius, penalizes points outside the sphere, and includes weight decay; ν bounds the outlier fraction and the fraction of samples outside or on the boundary.Source: Deep SVDD (DSVDD) · p. 9
  • For the typical setting where most training data are positive, a second Deep SVDD loss encourages lower intra-class variance, and inference labels a query positive when its embedding is within a selected distance threshold.The distance is measured from the hypersphere center.Source: Deep SVDD (DSVDD) · p. 9
  • Deep SVDD avoids trivial solutions by fixing the sphere center to the initial-pass mean, omitting network biases, and avoiding bounded activations.The authors report that a learnable center, bias terms, and bounded activations can each lead to trivial solutions.Source: Deep SVDD (DSVDD) · p. 9

Holistic Approach to One Class (HRN)

  • HRN trains a deep classification network with log-likelihood loss and holistic feature regularization, adding 2-norm instance normalization to address differing feature scales.The normalization also subtracts each instance's feature mean; the survey says this strategy performed better for OCC than commonly used instance normalization.Source: Deep SVDD (DSVDD) · pp. 9–10

2) Generative Models:

Generative models form one family of one-class learning methods. The surveyed AnoGAN-related approaches use adversarial generation and discrimination to model positive data or identify unusual inputs.

2) Generative Models:

  • Generative approaches to one-class learning include denoising auto-encoders, GANs, autoregressive models, and adversarial auto-encoders.Source: 2) Generative Models · p. 10

AnoGAN

  • AnoGAN trains a GAN on positively labeled data, with a generator producing image-like samples and a discriminator distinguishing real from generated images.Source: AnoGAN · p. 10
  • AnoGAN approximates a query's latent vector by back-propagating a loss that compares the query and generated image in pixel and discriminator-feature spaces; the resulting loss is used as a novelty score.A query is assigned positive when that score is below a selected threshold.Source: AnoGAN · p. 10
  • ALOCC uses a conditional GAN with an auto-encoder generator that denoises Gaussian-corrupted inputs, while a discriminator distinguishes clean images from denoised outputs.At inference, a query is positive when the discriminator score exceeds a predetermined threshold.Source: AnoGAN · p. 10
  • Old is Gold fine-tunes the ALOCC architecture using bad-quality examples and pseudo-anomaly images as unusual inputs for discriminator training.The pseudo-anomaly image interpolates two bad-quality examples produced using an older generator snapshot.Source: AnoGAN · p. 11

One Class GAN (OCGAN)

The surveyed methods use generative or representation-learning designs to distinguish positive-class data from atypical or novel inputs. They include OCGAN, GPND, latent-space autoregression, and inter-class splitting.

OCGAN

  • OCGAN addresses cases where diverse positive data can make autoencoder reconstruction-based novelty detection fail.Source: One Class GAN (OCGAN) · p. 11
  • OCGAN adversarially constrains latent codes to a uniform range and uses a visual discriminator so generated latent codes produce positive-class images; its classifier guides informative latent sampling.Training uses real, reconstructed, and randomly generated fake images.Source: One Class GAN (OCGAN) · p. 11

GPND

  • GPND models the positive-data manifold probabilistically, using an adversarial autoencoder to regularize latent codes and generated images toward their respective distributions.Source: One Class GAN (OCGAN) · pp. 11–12

Latent Space Auto-regression (AND)

  • Latent-space autoregression jointly learns an autoencoder representation and the positive-class latent density; reconstruction error and negative log-likelihood train the networks, and the resulting loss measures query novelty.A query is assigned positive when its novelty loss is below a selected threshold.Source: One Class GAN (OCGAN) · pp. 12–13

Inter Class Splitting (ICS)

  • Inter-class splitting identifies atypical positive-class samples by reconstruction quality, then trains feature, distance, and classification subnetworks to distinguish typical from atypical examples.The method uses SSIM for image-quality assessment and pairwise latent-space losses alongside binary classification.Source: One Class GAN (OCGAN) · p. 13

3) Knowledge Distillation:

Knowledge distillation methods train student networks against a teacher to improve one-class learning. US-OCL adds metric and compactness losses, while P-KDGAN uses progressive teacher–student training.

3) Knowledge Distillation:

  • US-OCL [9] trains an ensemble of student networks to regress a generative teacher’s encoder representation, then scores queries by student–teacher prediction errors.The teacher is trained on one-class data; US-OCL also uses triplet metric loss and a compactness loss that minimizes correlations among mini-batch samples.Source: 3) Knowledge Distillation · pp. 13–14
  • P-KDGAN [105] trains teacher and student GANs with reconstruction, latent-space, and adversarial losses before progressively distilling teacher representations into the student.The networks share an architecture but differ in channel count; first the student learns with fixed teacher parameters, then both are trained and the student is fine-tuned with distillation loss.Source: 3) Knowledge Distillation · p. 14, Figure 13 · p. 14

V. DATASETS AND EVALUATION PROTOCOLS

The section describes standard multi-class datasets and protocols for evaluating one-class classifiers, alongside a dataset pairing normal PASCAL images with abnormal examples.

A. Multi-class Benchmark Datasets

  • Standard benchmarks treat each dataset class in turn as positive, train on its training split, and test against test splits from all classes as negatives or positives.The survey lists COIL100, MNIST, fashion-MNIST, and CIFAR10 as commonly used multi-class benchmarks.Source: A. Multi-class Benchmark Datasets · pp. 14–15
  • COIL100 has 100 object classes with varied poses and typically fewer than 100 images per class; MNIST and fashion-MNIST are 28 × 28 images, while CIFAR10 has 10 classes.CIFAR10 contains four vehicle and six animal classes; its images are centered and cropped.Source: A. Multi-class Benchmark Datasets · pp. 14–15

B. Abnormal 1001 Dataset

  • Abnormal 1001 supplies at least 100 abnormal images for each of six PASCAL object classes, with abnormality judged through Amazon Mechanical Turk responses.Models train on PASCAL images and test on same-class PASCAL positives against negatives sampled from Abnormal 1001; Figure 14 shows sample images.Source: B. Abnormal 1001 Dataset · p. 15, Figure 14 · p. 15

C. MVTec-AD Dataset

The section describes MVTec-AD and common OCC evaluation metrics, then identifies open research challenges involving generalization, training data, robustness, and model design.

C. MVTec-AD Dataset

  • MVTec-AD covers 15 industrial texture and object classes; models train on normal images and test on same-class normal and abnormal images.It has 3,629 training images and a test set of 467 normal and 1,258 abnormal images; Figure 15 shows normal and defective samples.Source: C. MVTec-AD Dataset · p. 15, Figure 15 · p. 15

D. Evaluation Metrics

  • AUC-ROC measures performance across operating points, with 1.0 ideal and 0.5 corresponding to random guessing.ROC curves represent the relationship between false positive rate and true positive rate.Source: D. Evaluation Metrics · p. 15
  • FPR at a selected TPR includes EER as the equal-FPR-and-TPR case; HTER summarizes errors assuming equal probabilities of positive and negative queries.HTER is commonly reported in biometrics when FPR and TPR are equal.Source: D. Evaluation Metrics · pp. 15–16

VI. DISCUSSION AND FUTURE RESEARCH DIRECTIONS

  • Existing methods perform well on simpler datasets such as MNIST and COIL100 but fail to generalize well to more complex datasets such as CIFAR10.The survey identifies generalization to complex visual datasets as an open problem.Source: §VI · p. 16
  • Semi-supervised OCC can use labeled out-of-distribution data or a few annotated positive and negative examples, and the survey calls for more methods leveraging such data.Conventional OCC assumes only positively labeled training data.Source: §VI · p. 16
  • Current OCC methods are described as unable to handle adversarial data, and the survey calls for extensive study of attacks on one-class models.Most cited attack and defense research focuses on multi-class models, although a deep OCC approach has been shown vulnerable.Source: §VI · p. 16
  • OCC methods assume inference queries follow the training-image distribution, but domain-generalizable OCC remains largely unexplored.The survey identifies generalization across image distributions as an open problem.Source: §VI · p. 16
  • Future work also includes OCC-specific architecture search, interpretable deep models, and federated training, where multi-class approaches do not directly apply.The survey says no methods had been proposed for training one-class classifiers with decentralized data.Source: §VI · p. 16
Loading 2101.03064v1…