Source-linked AI summary
CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
Jihoon Tack, Sangwoo Mo, Jongheon Jeong, Jinwoo Shin
TL;DR
OOD detection requires distinguishing samples outside the training distribution, but learning representations tailored to that distinction remains under-explored. CSI addresses this by contrasting distributionally shifted augmentations alongside other instances and using a specialized detection score. It outperforms baseline methods across one-class and multi-class settings, with reported gains on several image benchmarks.
Problem
OOD detection must identify test samples outside the training distribution, while representations specifically suited to distinguishing in- and out-of-distributions remain under-explored.
Method
CSI extends contrastive learning by contrasting distributionally shifted augmentations of each sample and defines a detection score using the resulting representation and shifted-instance information.
Results
CSI outperforms baseline methods across unlabeled one-class, unlabeled multi-class, and labeled multi-class OOD detection settings, including one-class AUROC gains on CIFAR-10, CIFAR-100, and ImageNet-30.
Takeaways & Limitations
CSI provides a unified OOD detection framework that also improves classifier confidence calibration while maintaining classification accuracy.
Takeaways & Limitations
Naively contrasting external OOD samples can degrade performance because the contrastive loss also discriminates among those samples, adding unnecessary learning burden.
Abstract
from arXiv · showhide
Novelty detection, i.e., identifying whether a given sample is drawn from outside the training distribution, is essential for reliable machine learning. To this end, there have been many attempts at learning a representation well-suited for novelty detection and designing a score based on such representation. In this paper, we propose a simple, yet effective method named contrasting shifted instances (CSI), inspired by the recent success on contrastive learning of visual representations. Specifically, in addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Based on this, we propose a new detection score that is specific to the proposed training scheme. Our experiments demonstrate the superiority of our method under various novelty detection scenarios, including unlabeled one-class, unlabeled multi-class and labeled multi-class settings, with various image benchmark datasets. Code and pre-trained models are available at https://github.com/alinlab/CSI.
1 Introduction
The paper frames OOD detection as identifying samples outside the training distribution and targets the under-explored problem of learning representations specifically suited to that distinction. CSI extends contrastive learning by contrasting distribution-shifted augmentations and introduces a corresponding detection score, achieving strong results across multiple OOD settings.
- Motivation: OOD detection identifies whether a test input lies far from the training distribution, using only training data despite the diverse space of possible OOD samples.The task is important for applications including medical diagnosis, fraud detection, and autonomous driving.
- Motivation: Prior approaches model normality or define detection scores, while self-supervised learning provides useful inductive biases for learning discriminative OOD features.The literature includes density-based, reconstruction-based, one-class, and self-supervised methods.
- Method: CSI contrasts distributionally shifted augmentations of each sample in addition to other instances, encouraging representations to distinguish in-distribution from shifted samples.The method uses hard augmentations, such as rotation, that standard contrastive learning typically avoids.
- Method: CSI combines a new shifted-instance training scheme with a detection score that uses the learned representation and information from contrasting shifted instances.The paper also applies CSI to classifier confidence calibration while maintaining classification accuracy.
- Results: CSI outperforms baselines across unlabeled one-class, unlabeled multi-class, and labeled multi-class OOD settings and improves one-class AUROC on CIFAR-10 from 90.1% to 94.3%.Reported one-class gains also include CIFAR-100 from 79.8% to 89.6% and ImageNet-30 from 85.7% to 91.6%.
2 CSI: Contrasting shifted instances
CSI adapts contrastive learning for OOD detection by treating distribution-shifted augmentations as negatives and combining representation learning with specialized detection scores. It also extends the approach to confidence-calibrated classifiers.
- Problem setup: OOD detection learns from training data to distinguish in-distribution samples from samples outside the training distribution, without relying on external OOD data.The broad and variable space of OOD samples motivates avoiding assumptions about specific prior OODs.
- Contrastive learning: SimCLR learns representations by contrasting independently augmented views of each sample as positives against other samples as negatives.The method uses an encoder and, in these experiments, an additional projection layer for the contrastive feature.
- Contrasting shifted instances: CSI treats hard, distribution-shifting augmentations as negatives relative to the original sample, unlike vanilla SimCLR, which treats augmented views as positives.The shifting transformations are intended to represent OOD samples with respect to the original input, while preserving semantic meaning.
- CSI training objective: CSI combines a contrasting-shifted-instances loss with an auxiliary classifier that predicts which shifting transformation was applied.The final objective is LCSI = Lcon-SI + λ · Lcls-SI, with λ = 1 in the experiments.
- Detection scores: The proposed detection scores use nearest-training-sample cosine similarity and representation norm, with additional scores incorporating shifting transformations and their classifier.The scores can also be ensembled over random augmentations, and a coreset can reduce computation and memory cost.
- Confidence-calibrated classifiers: CSI is extended to supervised contrastive learning for training classifiers whose predictions are accurate in-distribution and whose confidence is lower for OOD or misclassified inputs.This extension adapts CSI to labeled datasets and targets confidence calibration.
3 Experiments
Experiments evaluate CSI across unlabeled one-class, unlabeled multi-class, and labeled multi-class OOD detection settings. CSI consistently outperforms prior methods, while ablations show that shifting transformations, complementary losses, and combined scores are important.
- Main results: CSI is evaluated on unlabeled one-class, unlabeled multi-class, and labeled multi-class OOD detection benchmarks.The experiments also analyze shifting transformations and ablate the proposed training objectives and detection scores.
- Main results: CSI significantly outperforms prior methods across all tested unlabeled multi-class benchmarks, including hard near-distribution OOD samples.The authors also report notable performance on Places-365, where prior methods can reach random-guess AUROC.
- Main results: CSI consistently improves AUROC and ECE across labeled benchmarks and also improves test accuracy.Ensembling predictions over shifting transformations provides further improvement but is four times slower when using rotation.
- Ablation study: “Perm” and “Rotate” are the most distinguishable transformations under vanilla SimCLR and are also the most effective shifting transformations for CSI.This links higher measured OOD-ness with better CSI performance for these transformations.
- Ablation study: Using transformations as shifts consistently improves detection, whereas aligning them as standard augmentations degrades or matches detection performance.Combining multiple shifting transformations can further improve performance.
- Ablation study: The best shifting transformation depends on the dataset: Gaussian noise helps on rotation-invariant data, while rotation can degrade performance there.For DTD versus Textile, rotation does not shift the distribution sufficiently, making Gaussian noise more suitable.
- Ablation study: CSI is more specialized for OOD detection than standard classification representation learning, with linear evaluation accuracy of 90.19% versus 90.48% for SimCLR.The reported comparison is on unlabeled CIFAR-10.
- Ablation study: Contrasting and classifying shifted instances each outperform vanilla SimCLR, while combining them yields further improvements in training and detection scores.Cosine similarity and feature norm are likewise complementary, with their combination performing best.
4 Related work
Related work spans unsupervised OOD detection, confidence calibration, and self-supervised representation learning. CSI belongs to the self-supervised line and extends contrastive learning toward OOD detection.
- OOD detection: Unsupervised OOD detection methods include density-based, reconstruction-based, one-class classifier, and self-supervised approaches.CSI is positioned within the self-supervised approach because it uses representations learned from self-supervision.
- Confidence-calibrated classifiers: Confidence-calibrated classifiers use prediction confidence as an uncertainty estimator for in-distribution and out-of-distribution inputs.Prior work improves calibration through inference or training schemes, and specialized detection scores can be built on pretrained classifiers.
- Self-supervised learning: Contrastive learning and instance discrimination have achieved strong visual representation-learning results, but prior work rarely investigates uncertainty or robustness.CSI contributes to the concurrent effort applying contrastive learning to OOD detection.
- Related work: The paper provides additional surveys and discussions of prior work in Appendix B.
5 Conclusion
CSI extends contrastive learning for OOD detection and demonstrates strong performance across varied detection scenarios. The authors present it as a baseline for future work in OOD detection and self-supervised learning.
- Conclusion: CSI extends contrastive learning for OOD detection and demonstrates outstanding performance across various OOD detection scenarios.
- Conclusion: The authors position CSI as an important baseline that may guide future directions in OOD detection and self-supervised learning.
Broader Impact
OOD detection is framed as important for safe and reliable intelligent systems. The paper argues that its findings distinguish representations useful for OOD detection from those optimized for classification.
- Motivation: OOD detection is presented as an essential ingredient for building safe and reliable intelligent systems.
- Rethinking representation for OOD detection: Representations for classification can differ from representations for OOD detection, and hard augmentations can help OOD detection despite harming standard contrastive representation learning.
- Towards reliable intelligent systems: Reliable intelligent systems must address uncertain environments and malicious adversaries, with outlier detection also relevant to medical diagnosis, autonomous driving, and manufacturing inspection.
A Experimental details
The experiments evaluate CSI across unsupervised and supervised OOD-detection settings using image benchmarks, contrastive representations, shifting transformations, and AUROC/ECE metrics. The method builds on contrastive learning while using distributionally shifted augmentations and a specialized detection score.
- Experimental setup: CSI uses a ResNet-18 encoder and a 2-layer projection head with 128-dimensional embeddings.Training minimizes LCSI with temperature τ = 0.5 and follows the SimCLR optimization scheme.
- Data augmentation: SimCLR augmentations include Inception crop, horizontal flip, color jitter, and grayscale, while rotation is used as a distribution-shifting transformation.The augmentation pipeline separates random augmentations T from shifting transformation S.
- Datasets and settings: The experiments cover one-class CIFAR-10, CIFAR-100 super-classes, ImageNet-30, and unlabeled or labeled multi-class CIFAR-10 and ImageNet-30 settings.The one-class datasets contain 10, 20, and 30 image classes respectively, while multi-class experiments use CIFAR-10 and ImageNet-30.
- Evaluation metrics: AUROC measures separation between in-distribution and OOD images, while ECE measures softmax-classifier confidence calibration.AUROC varies a threshold over true- and false-positive rates; ECE groups predictions into confidence bins.
- Problem setting: The unsupervised OOD setting gives the detector only in-distribution samples while requiring detection of unseen OOD samples.The paper distinguishes this setting from semi-supervised and supervised settings that provide some OOD information.
- Method: CSI contrasts distributionally shifted augmentations in addition to other instances and defines a score using the learned representation and this training scheme.Unlike prior self-supervised approaches centered on transformation classification, CSI incorporates contrastive learning for OOD detection.
C Additional one-class OOD detection results
Additional one-class experiments show that CSI performs consistently across CIFAR-10, CIFAR-100 super-classes, and ImageNet-30. The reported results include class-level confusion patterns and comparisons against prior methods.
- CIFAR-10: CSI’s hard CIFAR-10 pairs align with human intuition: car is confused with ship and truck, while cat is confused with dog.The confusion matrix identifies these pairs as difficult OOD distinctions.
- CIFAR-100: CSI outperforms prior methods for all 20 CIFAR-100 super-classes.Table 9 reports AUROC values for every super-class, with the final row giving the mean.
- ImageNet-30: CSI consistently performs well across all 30 one-class ImageNet-30 classes.Table 10 reports AUROC values for the selected ImageNet-30 classes and corresponding results.
D Ablation study on random augmentation
The random-augmentation ablation studies how augmentation sampling and coreset selection affect CSI’s OOD-detection efficiency. Controlled augmentation subsets provide the best performance for a fixed sample count, while small training coresets remain effective.
- Random augmentation: Controlled random augmentations outperform random sampling from the entire augmentation set for a given number of samples.The controlled set is formed from the most common augmentation samples, improving sample efficiency.
- Coreset selection: A coreset containing only a few training samples, such as 1%, achieves sufficiently good OOD-detection results.Coresets are constructed by K-means clustering normalized training features using cosine similarity.
F Ablation study on the balancing terms
The balancing terms in CSI’s loss consistently improve results, especially on CIFAR-100 super-classes and ImageNet-30. Combining multiple shifting transformations provides further gains beyond rotation alone.
- Balancing terms: Balancing terms improve AUROC consistently across the evaluated one-class datasets.CIFAR-10 shows little gain because its balancing coefficients have similar values, whereas CIFAR-100 and ImageNet-30 show larger gains because they vary more.
- Multiple shifting transformations: Combining shifting transformations by composing S1 and S2 can further improve OOD detection.The study combines Noise, Blur, and Perm with Rotate and reports additional gain.
- Transformation choice: The experiments use rotation as the shifting transformation because of its simplicity, while noting that better transformation combinations remain possible.The reported combination study suggests that transformation composition is an avenue for further investigation.
H Discussion on the features of the contrastive score (6)
The discussion identifies nearest-neighbor cosine similarity and feature norm as complementary OOD cues under contrastive representations. Combining both produces the strongest detection performance.
- Combined score: The final score combines both features under the proposed training objective, whose discrimination trend remains consistent across models.The reported comparison uses the final training loss to match prior experiments.
- Cosine similarity: Nearest-neighbor cosine similarity separates OOD samples because they lie farther from training samples than in-distribution test samples.The score uses max_m sim(z(x_m), z(x)).
- Feature norm: Feature norm is discriminative: contrastive training makes in-distribution representations relatively larger than OOD representations.The norm gap increases with training epoch and appears consistently across models and settings.
- Feature norm: Increasing feature norm may help maximize cosine similarity by reducing the relative distance between augmented representations.This is offered as an explanation for the norm phenomenon despite SimCLR computing its loss on normalized features.
- Combined score: Cosine similarity and feature norm are complementary, so the combined sim+norm score achieves additional improvements over either feature alone.The comparison includes sim-only, norm-only, and sim+norm AUROC scores.
I Rethinking OOD detection benchmarks
The paper argues that common OOD benchmarks can be detected through artificial visual artifacts rather than semantic differences. It therefore constructs fixed-resize LSUN and ImageNet datasets intended to be more visually realistic, while also examining smoothness-based detection.
- Benchmark problem: Resized LSUN and ImageNet contain artificial noise from broken image operations, making them detectable through simple image statistics.This creates a benchmark concern because detection may not require semantic representations.
- Smoothness score: The smoothness score measures an image’s total-variation difference from training samples to detect unusually noisy inputs.The method treats higher input-space variation as evidence of lower natural-image smoothness.
- Smoothness score: A simple smoothness score detects current benchmark datasets surprisingly well.Table 16 reports AUROC values for the score under unlabeled CIFAR-10.
- Revised benchmarks: The authors construct LSUN (FIX) and ImageNet (FIX) using fixed resizing to reduce visual artifacts in the benchmark datasets.LSUN (FIX) samples 1,000 images from each of ten classes, while ImageNet (FIX) samples 10,000 images from ImageNet-30 after excluding four CIFAR-10-overlapping classes.
- Revised benchmarks: The revised datasets are presented as more visually realistic than the original resized LSUN and ImageNet benchmarks.Figures 5 and 6 show the former and fixed datasets, respectively.
- Additional datasets: Additional examples cover rotation-invariant manufacturing and aerial imagery, including steel and textile images.These examples broaden the application settings beyond natural-image benchmarks.