PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks

Shiyu Liang, Yixuan Li, R. Srikant

arXiv:1706.02690v5cs.LGstat.ML

TL;DR

The paper asks how pretrained neural networks can distinguish in-distribution from out-of-distribution images despite high confidence on unfamiliar inputs. It proposes ODIN, which combines temperature scaling and small input perturbations without retraining, and reports consistent gains across architectures and dataset pairs, including a reduction in FPR from 34.7% to 4.3% at 95% TPR.

  • Problem

    The problem is distinguishing in-distribution from out-of-distribution images when neural networks may make high-confidence predictions on unfamiliar inputs.

  • Method

    ODIN combines temperature scaling and small gradient-based input perturbations to separate softmax scores without retraining the pretrained network.

  • Results

    ODIN significantly improves detection across neural architectures and diverse dataset pairs, reducing FPR from 34.7% to 4.3% at 95% TPR in one DenseNet evaluation.

  • Takeaways & Limitations

    ODIN provides a simple detector that is compatible with diverse architectures and dataset pairs while retaining the pretrained network for in-distribution classification.

  • Takeaways & Limitations

    Large input perturbations can degrade performance because higher-order terms in the Taylor expansion become non-negligible.

Abstract

from arXiv · show

We consider the problem of detecting out-of-distribution images in neural networks. We propose ODIN, a simple and effective method that does not require any change to a pre-trained neural network. Our method is based on the observation that using temperature scaling and adding small perturbations to the input can separate the softmax score distributions between in- and out-of-distribution images, allowing for more effective detection. We show in a series of experiments that ODIN is compatible with diverse network architectures and datasets. It consistently outperforms the baseline approach by a large margin, establishing a new state-of-the-art performance on this task. For example, ODIN reduces the false positive rate from the baseline 34.7% to 4.3% on the DenseNet (applied to CIFAR-10) when the true positive rate is 95%.

1 INTRODUCTION

Out-of-distribution detection is difficult because deployment data may differ from training data, while neural networks can remain highly confident on unfamiliar inputs. ODIN addresses this challenge without retraining and improves detection across architectures and dataset pairs.

  • Neural networks can make high-confidence predictions on unrecognizable or irrelevant inputs outside the training distribution.
  • 1,000s of possible out-of-distribution examples make exhaustive retraining computationally expensive and potentially intractable.The passage describes the number of out-of-distribution examples as potentially infinite.
  • Temperature scaling and small controlled input perturbations enlarge the softmax-score gap between in- and out-of-distribution examples.
  • 34.7% to 4.3%: ODIN reduces FPR on DenseNet trained on CIFAR-10 against TinyImageNet when 95% of in-distribution images are correctly classified.
  • ODIN detects out-of-distribution examples without retraining and is implementable across modern neural architectures.
  • ODIN consistently outperforms the baseline by a large margin across state-of-the-art architectures and diverse in- and out-of-distribution dataset pairs.

2 PROBLEM STATEMENT

The paper formulates out-of-distribution detection as distinguishing whether an image drawn from a mixture comes from the training distribution or a distinct out-distribution. It focuses on detection while preserving the pretrained network’s original in-distribution classification behavior.

  • The task is to distinguish images from the in-distribution PX and out-distribution QX using a pretrained neural network.
  • An input X is drawn from a mixture distribution whose latent indicator identifies whether it came from PX or QX.
  • The method focuses on detecting out-of-distribution images rather than changing the network’s predictions for in-distribution images.
  • Once an image is detected as in-distribution, the original image can be passed through the neural network for ordinary class prediction.

3 ODIN: OUT-OF-DISTRIBUTION DETECTOR

ODIN combines temperature scaling with gradient-based input preprocessing to separate in- and out-of-distribution softmax scores. It classifies an image using a threshold on the calibrated score after preprocessing.

  • ODIN consists of two components: temperature scaling and input preprocessing with small perturbations.
  • Temperature Scaling: Temperature scaling uses T to calibrate softmax outputs and can separate scores between in- and out-of-distribution images.
  • Input Preprocessing: The perturbation magnitude ε controls an input modification computed from the input gradient of the cross-entropy loss.
  • Input Preprocessing: The perturbation aims to increase the softmax score of any input without requiring a class label.
  • Out-of-distribution Detector: The detector preprocesses each image, computes its calibrated softmax score, and compares that score with threshold δ.
  • Out-of-distribution Detector: T, ε, and δ are selected so the true positive rate for correctly identified in-distribution images is 95%.

4 EXPERIMENTS

ODIN is evaluated on diverse architectures, datasets, and out-of-distribution settings using multiple detection metrics. It consistently improves over the baseline, while performance becomes harder as in- and out-of-distribution datasets become more similar.

  • Evaluation metrics: Detection effectiveness is measured with FPR at 95% TPR, detection error, AUROC, AUPR-In, and AUPR-Out.FPR and detection error use the operating point where TPR is 95%, whereas AUROC and AUPR are threshold-independent metrics.
  • Comparison with baseline: 30 percentage points: ODIN reduces FPR at TPR=95% from 34% to 4.2% on DenseNet-BC-100 distinguishing CIFAR-10 from TinyImageNet (crop).The comparison is shown through ROC curves for the baseline and ODIN.
  • Main results: ODIN significantly improves detection performance across all reported in- and out-of-distribution dataset pairs.The main experiments use T=1000, with perturbation magnitudes selected on a separate validation dataset.
  • Data distributional distance vs. detection performance: Performance is negatively correlated with MMD distance: detection becomes harder as in- and out-of-distribution images become more similar.Cropped datasets tend to have larger MMD distances than resized datasets because cropping removes global image patterns.

5 DISCUSSIONS

The discussion explains why temperature scaling and input perturbation improve separation between in- and out-of-distribution images, while noting that overly large perturbations can degrade performance.

  • Temperature scaling: Sufficiently large temperature improves detection performance, although the benefit diminishes when T becomes too large.The paper analyzes this behavior using a Taylor expansion of the softmax score.
  • Temperature scaling: U1 measures the largest output’s deviation from the remaining outputs, whereas U2 measures the remaining outputs’ deviation from each other.These quantities provide an interpretation of the output patterns used to distinguish datasets.
  • Temperature scaling: In-distribution images show larger deviation in their largest output and higher conditional expectations of U2 than out-of-distribution images.For similar U1 values, in-distribution images tend to have more separated remaining class outputs.
  • Input preprocessing: With T = 1000, appropriate perturbation magnitudes substantially improve detection performance on most datasets.Figure 4 examines perturbation magnitude for DenseNet and Wide-ResNet-28-10 trained on CIFAR-10.
  • Input preprocessing: Input preprocessing can increase the softmax score of in-distribution images more than that of out-of-distribution images when their original scores are similar.The proposed explanation relies on larger softmax-gradient norms for in-distribution images.
  • Input preprocessing: Perturbations that are too large can degrade performance because second- and higher-order Taylor terms are no longer insignificant.The degradation is shown in Figure 4.

6 RELATED WORKS AND FUTURE DIRECTIONS

The paper places image out-of-distribution detection alongside established low-dimensional methods and highlights two neural-network observations motivating its approach.

  • Related work: Low-dimensional out-of-distribution detection commonly uses density estimation, nearest neighbors, and clustering analysis.These methods have been studied across multiple contexts.
  • Future directions: The approach is motivated by larger output variance across class labels and larger log-softmax gradient norms on in-distribution images.The paper suggests that understanding these phenomena may yield further insights into detection.

7 CONCLUSIONS

The conclusion presents ODIN as a retraining-free detector that improves over a baseline across architectures and dataset pairs, while identifying broader applications as future work.

  • Conclusion: ODIN detects out-of-distribution samples without retraining and improves on the Hendrycks–Gimpel baseline across architectures and dataset pairs.The paper also reports empirical analysis under different parameter settings and provides intuition for the approach.
  • Future directions: Future work will explore ODIN in applications such as speech recognition and natural language processing.

A SUPPLEMENTARY RESULTS IN SECTION 5.1 AND 5.2

Supplementary analyses compare output and gradient behavior across several CIFAR-10-trained architectures, with and without temperature scaling, and present the softmax Taylor expansion.

  • Output analysis: The supplementary figures compare E[U2|U1] across DenseNet, Wide-ResNet-28-10, and Wide ResNet-40-4.All networks are trained on CIFAR-10.
  • Gradient analysis: Gradient norms conditioned on softmax scores are evaluated without temperature scaling across three network architectures.The architectures are DenseNet, Wide-ResNet-28-10, and Wide ResNet-40-4.
  • Gradient analysis: The same conditional gradient-norm analysis is repeated with the optimal temperature, T = 1000.The comparison uses the same three CIFAR-10-trained architectures.
  • Analysis: The supplementary material presents the Taylor expansion of the softmax score function.

C PROPOSITION 1

Proposition 1 characterizes the large-temperature limit of ODIN without input preprocessing. As temperature grows, detection error approaches a constant, while a threshold can maintain 95% true positive rate with a corresponding limiting false positive rate.

  • Limit of detection error: As T becomes sufficiently large with ε=0, detection error Pe(T, 0) approaches a constant c.The constant depends on U1, the in-distribution PX, and the out-of-distribution QX.
  • Threshold selection: For a fixed α > 0, selecting δT = 1/(N − α/T) defines the threshold used to analyze asymptotic detection rates.The supplied analysis connects this threshold choice to the limiting false positive and true positive rates.
  • Asymptotic rates: Choosing α∗ so that the in-distribution probability is 0.95 yields TPR(T) → 0.95 as T → ∞.The condition is PX((N − 1)U1(X) > α∗) = 0.95.
  • Asymptotic rates: Under the same choice of α∗, FPR(T) converges to QX((N − 1)U1(X) > α∗) as T → ∞.The limiting false positive rate is determined by the out-distribution probability under the corresponding condition.
  • Limit of detection error: The limiting detection error is 0.05P(Z = 0) + P(Z = 1)QX((N − 1)U1(X) > α∗) = c.This expression combines the 95% true positive-rate threshold with the out-distribution limiting probability.

D ANALYSIS OF TEMPERATURE

The analysis introduces score differences relative to the largest-output class to simplify notation for studying temperature effects.

  • Notation: For each non-maximum class i, the score difference is defined as ∆i = f̂y − fi.The collection ∆ = {∆i}i≠ŷ contains these differences, and ¯∆ denotes their mean.

E ADDITIONAL RESULTS ON DISTANCE MEASUREMENT

The additional results examine distributional distances, threshold effects, and the relationship between detection difficulty and classification difficulty. The supplied figure captions specify threshold-rate curves and accuracy comparisons for CIFAR-10-trained networks.

  • Distance measurement: Energy distance between distributions P and Q is defined using cross-distribution distances minus within-distribution distances.The formula uses expectations of pairwise norms across P and Q and within each distribution.
  • Distance measurement: The energy-distance analysis is applied to two datasets represented as V = {V1, ..., Vm}.The supplied passage introduces the dataset-level form but does not provide its complete expression.
  • Distance measurement: The distance comparison organizes results by in-distribution datasets, out-of-distribution datasets, MMD distance, and energy distance.These are the table's stated row or column labels.
  • Threshold effects: Figure 10 plots FPR and TPR across thresholds δ with T = 1,000 and ε = 0.0014 for a DenseNet trained on CIFAR-10.The figure compares detection rates under different threshold settings.
  • Classification and detection difficulty: Figure 11 reports test accuracy for images above and below the softmax threshold associated with a specified TPR.All networks in the figure are trained on CIFAR-10.
  • Threshold effects: The threshold analysis evaluates FPR, the fraction of out-of-distribution images misclassified as in-distribution, alongside other detection metrics.It also studies how performance changes when parameters are tuned on one out-of-distribution dataset and evaluated on others.
  • Classification and detection difficulty: The classification-difficulty experiment compares test accuracy above and below softmax thresholds corresponding to specified true positive rates.The experiment fixes temperature and perturbation magnitude before varying the threshold.
Loading 1706.02690v5…