Source-linked AI summary
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, Balaji Lakshminarayanan
TL;DR
Near-OOD detection exposes a weakness of the simple Mahalanobis distance approach, despite its usefulness and ease of deployment. The paper introduces relative Mahalanobis distance, which contrasts class-specific and background distances, and reports consistent improvements across challenging benchmarks, including gains of up to 16.5 points with pretrained models. The method is less advantageous after task-specific fine-tuning on most evaluated tasks, with genomics OOD as an exception.
Problem
Mahalanobis distance often fails to detect near-OOD samples that are semantically similar to in-distribution data, although it is effective for far-OOD detection.
Method
Relative Mahalanobis distance subtracts a class-independent background distance from each class-specific Mahalanobis distance, forming a likelihood-ratio-style confidence score without complicated retraining or OOD training data.
Results
RMD consistently improves AUROC over MD across benchmarks, with margins of 3.17 to 16.5 points for different pretrained models.
Takeaways & Limitations
RMD provides a simple near-OOD detection method that works across vision, language, and biology benchmarks and is more robust than MD during training.
Takeaways & Limitations
After task-specific fine-tuning, MD is close or competitive with RMD on most evaluated tasks, except genomics OOD.
Abstract
from arXiv · showhide
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks. We analyze its failure modes for near-OOD detection and propose a simple fix called relative Mahalanobis distance (RMD) which improves performance and is more robust to hyperparameter choice. On a wide selection of challenging vision, language, and biology OOD benchmarks (CIFAR-100 vs CIFAR-10, CLINC OOD intent detection, Genomics OOD), we show that RMD meaningfully improves upon MD performance (by up to 15% AUROC on genomics OOD).
1 Introduction
OOD detection matters for safety-critical deployment, but simple Mahalanobis distance methods that work well for far-OOD inputs often fail on more challenging near-OOD inputs. The paper proposes relative Mahalanobis distance as a simple, retraining-free fix for near-OOD detection.
- OOD detection is critical for deploying machine-learning models in safety-critical applications.
- Existing improvements may require an additional model, OOD training data, or OOD data for hyperparameter selection.
- Mahalanobis distance is popular because it is simple, requires no model retraining, and works out of the box with trained models.
- MD is effective for far-OOD detection but often fails on semantically similar near-OOD samples such as CIFAR-100 versus CIFAR-10.
- Relative Mahalanobis distance is a simple fix that improves near-OOD detection without complicated retraining or OOD training data.
2 Methods
The paper reviews MD, which compares test features with class-conditional Gaussian distributions, and introduces RMD by contrasting class-specific distance with a class-independent background distance. RMD is framed as a likelihood-ratio-style modification that remains simple and hyperparameter-free.
- Mahalanobis distance based OOD detection: MD uses intermediate neural-network feature maps and fits one Gaussian distribution per in-distribution class with class-specific means and shared covariance.
- Mahalanobis distance based OOD detection: For a test feature, MD computes distances to all class distributions, uses the minimum as uncertainty, and negates it to obtain confidence for OOD classification.
- Relative Mahalanobis distance: RMD subtracts the distance to a class-independent Gaussian fitted to all training data from each class-specific Mahalanobis distance.
- Relative Mahalanobis distance: RMD is equivalent to a likelihood ratio between class-specific Gaussian fits and a Gaussian fit using data from all classes.
- Related work: Unlike Partial Mahalanobis distance, RMD is described as hyperparameter-free and is designed to perform better for near-OOD detection.
3 Failure Modes of Mahalanobis distance
Mahalanobis distance can fail near-OOD detection because nondiscriminative dimensions accumulate distance and overwhelm informative dimensions. Relative Mahalanobis distance suppresses these shared contributions, improving separation between IND and OOD data.
- MD decomposes into independent eigen-basis contributions, with each dimension weighted by the inverse covariance eigenvalue.The projected coordinate is l_d = v_d^T(z′ − µ), and its contribution is l_d^2/λ_d.
- In CIFAR-100 versus CIFAR-10, only the top 120 largest-eigenvalue dimensions show distinct mean distances between IND and OOD.The remaining dimensions have similar mean distances for both groups.
- 74.98% AUROC: MD’s later dimensions contribute substantially despite being nondiscriminative, overwhelming the informative dimensions.The final MD score sums distance across dimensions, analogous to the area under the distance curve.
- RMD fits a class-independent one-dimensional Gaussian background model per dimension, reducing contributions from later shared dimensions nearly to zero.The top dimensions retain their distinction between IND and OOD.
- 81.08% AUROC: RMD improves near-OOD separation in the CIFAR-100 versus CIFAR-10 experiment.
- In a simulated high-dimensional Gaussian setting, MD reaches 83.13% AUROC, whereas RMD detects OOD perfectly at 100% AUROC.The simulation assigns different means to IND and OOD in one discriminative dimension and the same mean in all remaining dimensions.
4 Experiments and Results
Across four near-OOD benchmarks, RMD consistently improves AUROC over MD, including with pretrained features, while its robustness is especially apparent during training. Fine-tuning can narrow the gap between MD and RMD on most tasks, limiting RMD’s advantage in that setting.
- Models without pre-training: RMD improves AUROC over MD by 1.2 to 15.8 points across all four near-OOD benchmarks.Compared with MSP, RMD gains 2.5 points on Genomics OOD and shows partial gains on CIFAR-10/100.
- Additional analysis: RMD extends beyond Gaussian densities, achieving 78.34% AUROC with a masked autoregressive flow compared with 76.10% for the corresponding alternative.
- Models with pre-training: RMD also outperforms MD with pretrained models without fine-tuning, with margins ranging from 3.17 to 16.5 points.These pretrained models achieve AUROC scores close to or better than the models in Table 1.
- Models with pre-training: 84.60% AUROC is achieved by BiT with RMD for CIFAR-100 versus CIFAR-10, compared with 81.01% for the Wide ResNet baseline.For CIFAR-10 versus CIFAR-100, pretrained CLIP with RMD achieves 91.19% AUROC.
- Models with pre-training: Fine-tuning substantially improves MD, leaving it close or competitive with RMD on most tasks, except Genomics OOD.The authors conjecture that fine-tuning removes disruptive non-discriminative features or yields representations that implicitly model foreground and background distributions.
- Relative Mahalanobis is more robust: 66.19% AUROC for MD at step 50k falls to 53.10% at step 500k, whereas RMD increases and stabilizes with model accuracy.The same early-peak pattern was observed for MD during fine-tuning of the BERT genomics model, where its peak AUROC was 77.49%.
A Pseudocode for Relative Mahalanobis distance
The pseudocode computes RMD from class-conditional and class-independent Gaussian models fitted to in-distribution features. It scores test inputs by comparing the nearest class-specific MD with the background MD, then evaluates AUROC.
- The method takes labeled in-distribution training features, an in-distribution test set, and a feature extractor as inputs.
- It fits K class-conditional Gaussians N(µk, Σ) using the labeled in-distribution training set.
- It fits a background Gaussian N(µ0, Σ0) on the same training features while ignoring class labels.
- For each test feature, it computes class-specific MD scores and a background MD score.
- The RMD confidence score is −min_k{MD_k(z′) − MD_0(z′)} for each test input.
- The method computes AUROC between in-distribution test inputs and out-of-distribution test inputs using the RMD scores.
B Additional Experimental Details
The experiments use image, genomics, and language models trained or fine-tuned on in-distribution data. The setups include Wide ResNet, pretrained vision models, CNNs, and BERT-style models.
- CIFAR-10/100 experiments use a Wide ResNet 28-10 trained from scratch and pretrained ViT-B 16, BiT R50x16, and CLIP models with classification heads.The pretrained models are fine-tuned on in-distribution data, except CLIP, which is not fine-tuned because it requires paired text-image training data.
- The genomics benchmark contains 10 in-distribution bacteria classes, 60 OOD classes, and sequences of 250 base pairs over A, C, G, and T.A 1D CNN with 2000 filters of length 20 is trained from scratch for 1 million steps using Adam and learning rate 10^-4.
- A BERT-style genomics model is pretrained by masking input tokens and predicting them, then fine-tuned on in-distribution data for 100,000 steps.Masked-token prediction accuracy is 48.35%, and classification accuracy after fine-tuning is 89.84%.
- For CLINC Intent OOD, a standard pretrained BERT model is fine-tuned on in-distribution data for three epochs at learning rate 10^-4.The resulting classification accuracy is 96.53%.
C Performance of Partial Mahalanobis distance
Partial Mahalanobis distance depends strongly on which eigen-bases are selected, whereas RMD is hyperparameter-free and slightly outperforms its best setting on CIFAR-100 versus CIFAR-10.
- Top eigen-bases: 79.72% AUROC is PMD’s peak using the top eigen-bases at d = 76, after which performance decreases as more dimensions are included.This makes PMD sensitive to its eigen-basis hyperparameter.
- Comparison with RMD: 81.08% AUROC is achieved by hyperparameter-free RMD, slightly exceeding PMD’s peak on CIFAR-100 versus CIFAR-10.
- Smallest eigen-values: AUROC decreases as top eigen-bases are excluded from PMD, indicating that these bases are more important for near-OOD detection.The authors associate top eigen-bases with discriminative features and the remaining bases with features shared by IND and OOD.
- PMD variants: PMD evaluates AUROC using either the first eigen-bases with the largest eigenvalues or the last eigen-bases with the smallest eigenvalues.
D Simulation study for the failure mode of Mahalanobis distance
The simulation shows that many non-discriminative dimensions can obscure the signal from discriminative features, causing MD scores to overlap for IND and OOD inputs. RMD removes the shared-feature effect and restores separation.
- Simulation setup: 1024 dimensions are simulated, but only x1 is class-specific; the remaining dimensions are independent, non-discriminative features shared across classes.The class means differ only in the first dimension, with σ = 0.25.
- MD failure: MD scores for IND and OOD inputs largely overlap even though OOD inputs generally have greater distance.
- Failure mechanism: The summed log-likelihood contributions from many non-discriminative dimensions can overwhelm the contribution from the discriminative feature.
- RMD mechanism: RMD fits a class-independent Gaussian alongside class-conditional Gaussians, canceling contributions from features shared by IND and OOD.The resulting score compares maxk{log pk(x)} with log p0(x).
- Simulation outcome: RMD perfectly separates the simulated IND and OOD test inputs, with OOD values positive and IND values concentrated around zero.
- Neural-network features: The same pattern appears with neural-network feature maps: MD distributions overlap, whereas RMD recovers perfect separation.The authors expect intermediate representations in image, text, and genomics models to contain non-discriminative features.