Deep One-Class Classification via Interpolated Gaussian Descriptor

Yuanhong Chen; Yu Tian; Guansong Pang; Gustavo Carneiro

Source-linked AI summary

Deep One-Class Classification via Interpolated Gaussian Descriptor

Yuanhong Chen, Yu Tian, Guansong Pang, Gustavo Carneiro

arXiv:2101.10043v5Jan 25, 2021cs.CV

TL;DR

One-class classifiers can overfit when normality descriptions are learned from small or contaminated training sets. IGD learns a Gaussian anomaly classifier with adversarially interpolated training samples, and it reports stronger detection performance and robustness than current state-of-the-art methods across diverse benchmarks.

Problem
State-of-the-art one-class classifiers can overfit training data, particularly when the training set is small or contains anomalous samples.
Method
IGD learns a one-class Gaussian anomaly classifier with adversarially interpolated training samples to build a smooth normality description.
Results
IGD performs consistently better than current state-of-the-art detectors across six diverse anomaly-detection benchmarks and is more robust with small or contaminated training sets.
Takeaways & Limitations
The results support using representative normal samples and interpolation-based regularisation for robust one-class anomaly detection across semantic, industrial, and medical applications.
Takeaways & Limitations
Some state-of-the-art methods are excluded from comparisons because their larger backbones, slower inference, or more complex training would make the comparison unfair.

Abstract

from arXiv · show

One-class classification (OCC) aims to learn an effective data description to enclose all normal training samples and detect anomalies based on the deviation from the data description. Current state-of-the-art OCC models learn a compact normality description by hyper-sphere minimisation, but they often suffer from overfitting the training data, especially when the training set is small or contaminated with anomalous samples. To address this issue, we introduce the interpolated Gaussian descriptor (IGD) method, a novel OCC model that learns a one-class Gaussian anomaly classifier trained with adversarially interpolated training samples. The Gaussian anomaly classifier differentiates the training samples based on their distance to the Gaussian centre and the standard deviation of these distances, offering the model a discriminability w.r.t. the given samples during training. The adversarial interpolation is enforced to consistently learn a smooth Gaussian descriptor, even when the training data is small or contaminated with anomalous samples. This enables our model to learn the data description based on the representative normal samples rather than fringe or anomalous samples, resulting in significantly improved normality description. In extensive experiments on diverse popular benchmarks, including MNIST, Fashion MNIST, CIFAR10, MVTec AD and two medical datasets, IGD achieves better detection accuracy than current state-of-the-art models. IGD also shows better robustness in problems with small or contaminated training sets. Code is available at https://github.com/tianyu0207/IGD.

Introduction

Existing one-class classifiers can overfit small or contaminated training sets. IGD addresses this with a Gaussian anomaly classifier and adversarial interpolation to learn a smoother, more robust normality description.

Introduction: Current state-of-the-art OCC models minimise a hyper-sphere enclosing training representations, but this description can overfit when data are limited or contaminated.DSVDD is cited as showing this behaviour in Figure 1.
Introduction: IGD targets overfitting in one-class classifiers by learning normality from representative samples rather than fringe or anomalous samples.The method is designed for settings with small or contaminated training sets.
Introduction: IGD combines a Gaussian anomaly classifier with adversarially interpolated representations to enforce a smooth normality description.The model also uses an encoder, decoder, critic, and multi-scale reconstruction constraints.
Introduction: The Gaussian classifier distinguishes samples using their distance to the estimated Gaussian centre and the standard deviation of those distances.Its distribution parameters are estimated during the EM optimisation rather than explicitly optimised.
Introduction: IGD is reported to generalise across six anomaly-detection benchmarks and perform consistently better than current state-of-the-art detectors.The benchmarks cover semantic anomalies, industrial defects, and malignant lesions.
Introduction: The paper introduces a benchmark for evaluating robustness when training sets are small or contaminated with anomalous samples.This setting is presented as relevant to industrial applications and early medical-disease detection.

Experiments

IGD is evaluated across standard vision, industrial, and medical anomaly-detection benchmarks, with additional tests of localisation, component effectiveness, and robustness to limited or contaminated training data. Across these settings, the reported results show stronger performance than competing methods and improved robustness under adverse training conditions.

Experiments on MNIST, Fashion MNIST and CIFAR10: IGD outperforms current state-of-the-art methods for most pre-training settings across MNIST, Fashion MNIST, and CIFAR10.Table 1 compares mean AUC results for Scratch, self-supervised, and ImageNet-pretrained models.
Experiments on MVTec AD: IGD achieves at least 2% higher accuracy and 5% higher AUC than previous state-of-the-art methods on MVTec AD, regardless of pre-training.Its localisation performance also exceeds CAVGA-Ru by 2% with ImageNet pre-training and 4% with self-supervised pre-training.
Experiments on Medical Datasets: IGD obtains the best AUC results on both Hyper-Kvasir and LAG, surpassing CAVGA-Ru by 0.9%–3.8%.The evaluation covers polyp and glaucoma detection, whose appearances and structural anomalies differ from standard image benchmarks and MVTec AD.
Ablation Study: Each proposed component improves standard anomaly-detection AUC by 1.7%–11.6%, while GAC and INTER provide additional gains under small or contaminated training settings.On CIFAR10, GAC improves over the REC baseline by 13% with 20% training data and 10.4% with 10% contamination; INTER adds 3.7% and 3.1%, respectively.
Experiments on Small/Contaminated Training Sets: IGD remains comparatively stable as training data decreases, whereas DSVDD and DSVDD+REC deteriorate dramatically on smaller CIFAR10 and MVTec training sets.The robustness comparison uses 20%, 60%, and 100% subsamples of the original training sets and also evaluates contamination rates of 1%, 5%, and 10%.

Discussion

IGD reports stronger MVTec and high-resolution anomaly-detection results than several compared methods, while comparisons with some SOTA methods are excluded as unfair because of substantially larger backbones or heavier training procedures.

Comparisons with PANDA, CSI, and DROC are omitted from some tables because of larger backbones, slower or more complex training, or gains mainly attributable to self-supervised pre-training.
93.4% AUC on MVTec exceeds CSI's 63.6% and PANDA's 86.5%, while 93% localisation AUC trails PANDA's 96% but exceeds DROC's 90%.
93.7% AUC on Hyper-Kvasir exceeds the 91.6% AUC obtained by CSI trained by the authors.
Small- and contaminated-training-set experiments provide additional benchmarks that the compared CSI, PANDA, and DROC results do not address.

Conclusion

The paper concludes that IGD is an unsupervised anomaly-detection and segmentation model using a Gaussian classifier trained with adversarial interpolation. It reports strong performance across diverse datasets and greater robustness than DSVDD variants under small or contaminated training sets.

IGD achieves the best performance across MNIST, CIFAR10, Fashion MNIST, MVTec AD, and two large-scale medical datasets for anomaly detection and localisation.
IGD is a one-class Gaussian anomaly classifier trained with adversarially interpolated samples to describe normality using representative rather than fringe or anomalous samples.
IGD is more robust than DSVDD and image-reconstruction-constrained DSVDD on small or contaminated training sets.
The optimisation is formulated as an EM algorithm that is theoretically correct and converges to a stationary solution under certain conditions.

Datasets

The evaluated datasets span standard image benchmarks, industrial inspection imagery, and medical or high-resolution anomaly-detection settings. MVTec AD supplies defect-free normal images and more than 70 defect categories.

MVTec AD contains 5,354 high-resolution images covering 15 industrial objects and textures.
CIFAR10 has 60,000 images across 10 classes, while MNIST and Fashion MNIST each have 70,000 images across 10 classes.
MVTec's normal class contains 3,629 defect-free training images and 467 defect-free testing images.
MVTec's anomalous class includes more than 70 defect categories, including dents, structural failures, and contamination.

Global and Local IGD Models

The multi-scale localisation example combines global and local IGD outputs for MVTec AD images. The global model is smoother but can make mistakes, whereas the local model is jagged but avoids those global errors.

Combining global and local IGD outputs produces a smooth and correct anomaly heatmap in the illustrated MVTec AD example.
The global IGD model tends to produce smooth localisation results but makes some mistakes.
The local IGD model produces jagged results but avoids the mistakes made by the global model.

(MS-SSIM) Score

The MS-SSIM score compares image patches across multiple scales using luminance, contrast, and structural components, with separate global and local patch configurations.

(MS-SSIM) Score: MS-SSIM combines luminance, contrast, and structural similarity components across multiple image scales.The global score uses five scales, while the local score uses four.
(MS-SSIM) Score: Global patches are centered at ω and sized 11×11×3, whereas local patches use 3×3×3 neighborhoods.
(MS-SSIM) Score: The score uses predefined constants and patch statistics including means, variances, and covariance between original and reconstructed patches.The pixel range is set to L = 4.7579, with C1, C2, and C3 derived from K1 = 0.01 and K2 = 0.03.

Implementation Details

The implementation uses self-supervised or ImageNet knowledge-distillation pre-training, followed by fixed-backbone anomaly modeling and empirical comparison against competing methods.

Implementation Details: Self-supervised pre-training runs from scratch for 2,000 epochs with SGD, learning rate 0.01, weight decay 10^-1, and batch size 32.The MLP layer is removed afterward and a fixed-parameter linear layer is attached to the backbone.
Implementation Details: ImageNet knowledge-distillation pre-training matches 512-dimensional encoder features to an intermediate ResNet18 layer using Adam for 50,000 iterations.It uses learning rate 0.0001, weight decay 10^-5, and batch size 64 before fixing the encoder.
Implementation Details: IGD’s t-SNE representation shows more compact normal samples and fewer anomalous samples inside the normal cluster than DSVDD and GAC.This visual comparison is reported for the MVTec bottle class.

Convergence Conditions Proof

The convergence argument relies on a nondecreasing, bounded-above sequence of log-likelihood values and a condition controlling successive parameter updates.

Convergence Conditions Proof: The trained parameter sequence is assumed to have a bounded-above conditional log-likelihood and satisfy a parameter-update condition.These assumptions support convergence to some θ⋆ under the stated conditions.
Convergence Conditions Proof: The log-likelihood sequence is nondecreasing and bounded above, so it converges to a finite limit L⋆.
Convergence Conditions Proof: The Cauchy criterion bounds cumulative log-likelihood changes after a sufficiently large iteration index by any ε > 0.The proof combines this bound with nonnegative per-iteration expected log-likelihood improvements.

Class-level Results

IGD achieves strong class-level anomaly-detection results across MVTec AD, MNIST, FMNIST, CIFAR10, and localisation evaluation, outperforming reported baselines on several comparisons.

Class-level Results: IGD surpasses previous MVTec AD methods by at least 5.3% in AUC and achieves the best results in eleven categories.The ImageNet KD model also gives the best accuracy in ten categories, while comparisons exceed LSA and λ-VAEu by more than 7%.
Class-level Results: On MVTec AD, ImageNet KD and SSL pre-training outperform CAVGA-Du and CAVGA-Ru by the reported accuracy margins of 6%, 2%, 7%, and 3%.
Class-level Results: IGD’s CIFAR10 AUC is around 10% higher than the Student-Teacher approach, while MNIST results are competitive with Student-Teacher and better than earlier methods.
Class-level Results: The MVTec localisation table reports IGD results, but comparable class-level results from other approaches were unavailable.

Qualitative Localisation Results

The supplementary results provide qualitative anomaly visualisations for Hyper-Kvasir and MVTec AD, alongside class-level detection and localisation AUC tables. Figures mark anomalous regions in red or by high anomaly probability.

Qualitative Localisation Results: Figures S3 and S4 provide qualitative anomaly visualisations for Hyper-Kvasir polyp segmentation and MVTec AD defect results.Figure S3 uses red to denote anomalies, while Figure S4 uses red for high anomaly probability.
Qualitative Localisation Results: Table S4 reports class-level testing AUC for anomaly detection on CIFAR10 using state-of-the-art and proposed methods.
Qualitative Localisation Results: Table S5 reports class-level pixel-wise localisation AUC on anomalous MVTec AD testing images for the proposed methods.