Source-linked AI summary
One-Class Classification: Taxonomy of Study and Review of Techniques
Shehroz S. Khan, Michael G. Madden
TL;DR
OCC addresses classification when negative data are absent, poorly sampled, or undefined, making boundary learning and performance estimation difficult. The paper unifies the field through a taxonomy based on training-data availability, algorithms, and applications, and reviews techniques, limitations, and research directions. It concludes that OCC has broad applications but retains fundamental open problems in evaluation, scaling, error control, and model design.
Problem
OCC must characterize a target class when negative examples are absent, scarce, or unrepresentative, limiting boundary learning and estimation of negative-class errors.
Method
The paper presents a unified taxonomy based on training-data availability, algorithms, and application domains, then reviews OCC techniques, methodologies, significance, limitations, and applications.
Results
The survey provides a unified, in-depth view of OCC and organizes its research into broad categories for studying algorithms and applications.
Takeaways & Limitations
The taxonomy can guide selection and improvement of OCC techniques according to data availability, algorithm use, and application domain.
Takeaways & Limitations
OCC still has open problems in classifier description and training, scaling, error control, outlier handling, dimensionality reduction, and fair comparison with multi-class errors.
Abstract
from arXiv · showhide
One-class classification (OCC) algorithms aim to build classification models when the negative class is either absent, poorly sampled or not well defined. This unique situation constrains the learning of efficient classifiers by defining class boundary just with the knowledge of positive class. The OCC problem has been considered and applied under many research themes, such as outlier/novelty detection and concept learning. In this paper we present a unified view of the general problem of OCC by presenting a taxonomy of study for OCC problems, which is based on the availability of training data, algorithms used and the application domains applied. We further delve into each of the categories of the proposed taxonomy and present a comprehensive literature review of the OCC algorithms, techniques and methodologies with a focus on their significance, limitations and applications. We conclude our paper by discussing some open research problems in the field of OCC and present our vision for future research.
1 Introduction to One-class Classification
One-class classification addresses decisions about a target class when negative examples are absent, scarce, or unreliable. The paper motivates OCC through applications where obtaining representative negative data is difficult or costly, then outlines a taxonomy and review.
- Motivation: Traditional classifiers assign unknown objects to predefined categories, even when an object belongs to an entirely different domain.A fruit-versus-vegetable classifier may incorrectly label a cat as one of those classes.
- OCC setting: OCC characterizes a positive or target class using training instances while the negative or outlier class is absent, sparse, or statistically unrepresentative.This setting defines the classification problem primarily from knowledge of the target class.
- Applications: Machine-fault monitoring illustrates OCC because normal-operation measurements are available while many possible faults have not occurred and may be costly or dangerous to await.The passage links missing fault examples to potential machine malfunction, high costs, and operator risk.
- Applications: Medical diagnosis and homepage classification likewise face difficult negative-data collection because untested patients and improperly sampled non-homepages may not represent true negatives.Rare diseases also make positive samples difficult to collect until many patients are affected.
- Paper structure: The paper reviews OCC by proposing a taxonomy, surveying algorithms and applications, and discussing open research problems and future directions.Its later sections compare OCC with multi-class classification, review related work, and organize the state of the art.
2 One-class Classification Vs Multi-class Classification
Unlike conventional multi-class classification, OCC must determine a boundary with limited or absent negative data. This restriction complicates classifier evaluation because the negative-class error components cannot generally be estimated.
- Comparison: Conventional multi-class classifiers use data from multiple classes to support decision boundaries, but can fail when a class is severely undersampled or absent.The conventional setting often assumes classes are more or less balanced.
- Comparison: In OCC, only one side of the classification boundary can be determined from positive data or a small amount of negative data.This makes generalization and other conventional classification concerns more prominent.
- Related concepts: OCC research is related to outlier detection and novelty detection, which are treated as established neighboring research themes.The passage directs readers to dedicated surveys of both areas.
- Performance measurement: Because only the positive-class density is known, true-positive and false-negative quantities can be estimated, whereas false positives and true negatives cannot be estimated without outlier data.A limited amount of outlier-class data is therefore required to estimate performance and generalize accuracy.
- Performance measurement: Table 1 presents the confusion matrix used to compute one-class classifier performance.The matrix is attributed to Tax (2001).
3 Related Review Work in OCC
Earlier OCC reviews addressed selected algorithms, data types, temporal relations, or application domains. This paper broadens the scope through a more comprehensive taxonomy and review across OCC research.
- Earlier reviews: Mazhelis’s taxonomy organizes classifiers by internal model, data type, and ability to represent temporal feature relations.The internal-model categories are density-, reconstruction-, and boundary-based approaches.
- Earlier reviews: Mazhelis reviews many techniques but covers only a sub-spectrum of OCC problems and targets applicability to mobile-masquerader detection.Its taxonomy is therefore suited to a specific application domain rather than the entire field.
- Application-focused reviews: Brew et al. review OCC algorithms with Gaussian Mixture models for speaker verification, emphasizing speech processing, feature extraction, and speaker and impostor modeling.Their review is centered on a speech-data application.
- Application-focused reviews: Other reviews cover statistical, neural-network, and support-vector-machine methods for credit scoring, as well as density and boundary approaches for biometric applications.These examples show that OCC reviews have been organized around both algorithms and application areas.
- This survey: This paper extends earlier work with a more comprehensive and detailed survey that is not restricted to one application domain or specific data-dependent algorithms.It identifies research areas, raises open questions, and discusses significant contributions.
4 Proposed Taxonomy
The taxonomy organizes OCC research by training-data availability, algorithms, and application domains, while acknowledging overlap among categories. The review covers diverse methods, including SVMs, neural networks, nearest neighbours, and ensembles.
- The proposed OCC taxonomy unifies research according to training data, algorithms, and application domains.
- The taxonomy categories overlap, but cover almost all major OCC research contexts and application domains.
- Algorithms used: OSVM methods construct boundaries around positive data, while alternative approaches include neural networks, nearest neighbours, and ensembles.
- Algorithms used: SVDD uses a hyper-sphere around positive data, whereas OSVM uses a hyper-plane separating data-containing regions from regions without data.
- Algorithms used: Particle swarm optimization can match or surpass grid-search OSVM performance while using lower CPU time.
- Limitations: OSVM requires larger training sets to induce accurate boundaries with positive and unlabeled data, and increased support vectors can cause overfitting.
- Algorithms used: One-class ensembles improve retrieval precision, support missing-feature prediction, and combine dissimilarity representations or parameter settings.
5 Conclusions and Open Research Questions
The paper unifies one-class classification through a taxonomy organized by data availability, algorithms, and application domains, while identifying substantial open problems and future research directions.
- The taxonomy organizes OCC research according to training-data availability, classification algorithms, and application domains, supporting a unified literature survey.The survey covers techniques and methodologies while emphasizing their limitations, importance, and applications.
- Open problems span classifier description and training, scalability, error control, outlier handling, non-representative negatives, classifier combination, subspace generation, dimensionality reduction, and fair multi-class comparisons.These challenges remain despite the field's increasing maturity.
- Ensemble methods remain underexplored in OCC, especially boosting, random-subspace approaches, and one-class variants of decision-tree and nearest-neighbour classifiers.The paper specifically identifies further exploration of ensemble methods as a research direction.
- Further research is suggested on innovative and domain-specific OSVM kernels, efficient kernel-parameter tuning, missing target-class data, and Bayesian approaches for abundant unlabeled data with some positives.The paper also notes promising kernel use as a distance metric in one-class nearest-neighbour classifiers.
- OCC cost-sensitive learning is largely unexplored because outlier priors and error costs are often unknown, while most existing work focuses on numerical or continuous data.The paper suggests preference elicitation for inferring error costs and calls for more attention to categorical or mixed data.
- Density-estimation methods may reject positive objects when target data occupy low-density regions, motivating alternative OCC formulations.Raising thresholds can instead increase outlier acceptance in such scenarios.