PaperlayerView exact revision on arXiv

Source-linked AI summary

ReAct: Out-of-distribution Detection With Rectified Activations

Yiyou Sun, Chuan Guo, Yixuan Li

arXiv:2111.12797v1cs.LG

TL;DR

OOD detection is challenged by neural networks’ overconfident predictions on unknown inputs, motivating methods that improve ID-OOD separation. ReAct clips abnormal penultimate-layer activations at test time, and achieves strong benchmark performance, including a 25.05% FPR95 reduction on ImageNet versus the previous best method.

  • Problem

    Neural networks can be highly confident on OOD inputs, undermining reliable separation of unknown samples from in-distribution data in safety-critical deployment.

  • Method

    ReAct is a post hoc method that truncates high penultimate-layer activations using a threshold chosen from ID activation percentiles.

  • Results

    ReAct achieves state-of-the-art performance across common OOD benchmarks and reduces ImageNet FPR95 by 25.05% versus the previous best method.

  • Takeaways & Limitations

    ReAct offers a plug-and-play OOD detection approach compatible with different network architectures and scoring functions.

  • Takeaways & Limitations

    The theoretical analysis assumes OOD activation patterns with positive skewness, motivated by observations on real OOD data.

Abstract

from arXiv · show

Out-of-distribution (OOD) detection has received much attention lately due to its practical importance in enhancing the safe deployment of neural networks. One of the primary challenges is that models often produce highly confident predictions on OOD data, which undermines the driving principle in OOD detection that the model should only be confident about in-distribution samples. In this work, we propose ReAct--a simple and effective technique for reducing model overconfidence on OOD data. Our method is motivated by novel analysis on internal activations of neural networks, which displays highly distinctive signature patterns for OOD distributions. Our method can generalize effectively to different network architectures and different OOD detection scores. We empirically demonstrate that ReAct achieves competitive detection performance on a comprehensive suite of benchmark datasets, and give theoretical explication for our method's efficacy. On the ImageNet benchmark, ReAct reduces the false positive rate (FPR95) by 25.05% compared to the previous best method.

1 Introduction

OOD detection matters for safety-critical deployment because neural networks can be overconfident on unknown inputs. ReAct addresses this problem by truncating abnormal activations, improving separation between ID and OOD data and achieving strong benchmark results.

  • Motivation: OOD detection is important for safety-critical systems because unknown inputs can cause failures if they are not recognized.Examples include autonomous driving and health care.
  • Challenge: Modern neural networks can produce overconfident predictions on OOD inputs, making ID-OOD separation non-trivial.Prior work has therefore focused on defining more suitable OOD uncertainty measures.
  • Observation: OOD data can trigger penultimate-layer unit activation patterns that differ substantially from those of ID data.OOD activations show larger variation across units and a bias toward sharp positive values.
  • Method: ReAct attenuates outsized hidden-unit activations by rectifying them at an upper limit, without modifying model training.The resulting uncertainty distributions become more separated while ID activations are largely preserved.
  • Contributions: ReAct generalizes across network architectures and OOD scores, including MSP, ODIN, and energy score.It is presented as a post hoc approach for OOD detection.
  • Results: 25.05% FPR95 reduction was achieved on ImageNet compared with the previous best method.The paper reports extensive evaluation, empirical ablations, and theoretical analysis.

2 Background

OOD detection classifies test inputs according to whether they come from the training distribution or an unknown distribution. Its difficulty depends on how well the in-distribution and OOD distributions can be separated.

  • Problem Setup: The setup distinguishes an in-distribution marginal Din from an OOD test distribution Dout over inputs.A neural network trained on samples from P produces logits used for classification.
  • Decision Function: The OOD decision function assigns inputs to either Dout or Din at test time.The intended output is a binary distinction between unknown and in-distribution samples.
  • Difficulty: OOD detection becomes impossible to separate better than chance when Din and Dout are identical.In practice, Dout often represents unknown samples from an irrelevant distribution whose labels do not overlap with the task labels.

3 Method

ReAct modifies a pretrained network at test time by clipping penultimate-layer activations above a threshold. The rectified output can then support several standard OOD scoring functions.

  • Rectified Activation: ReAct operates on the penultimate-layer feature vector of a pretrained neural network before the output layer.The feature vector has dimension m and is mapped to K class logits by W.
  • Rectified Activation: ReAct(x; c) = min(x, c) truncates each activation above threshold c to limit the effect of noise.The operation is applied element-wise to the feature vector.
  • Threshold Selection: The threshold c is selected from a percentile of activations estimated on ID data to preserve ID activations while rectifying OOD activations.For example, p = 90 means 90% of ID activations are below c.
  • Threshold Selection: With c = ∞, rectification is removed and the model output equals the original output.Higher c values apply less activation truncation.
  • OOD Scoring: During testing, ReAct supports softmax confidence, ODIN, and energy scores, which classify samples by thresholding their scores.The threshold is typically chosen to correctly classify a high fraction of ID data, such as 95%.

4 Experiments

Experiments evaluate ReAct across large-scale ImageNet and CIFAR OOD benchmarks, network architectures, layers, thresholds, and scoring functions. ReAct establishes strong performance while remaining compatible with varied models and OOD scores.

  • ImageNet benchmark: ReAct improves OOD detection on a challenging ImageNet benchmark with 1,000 classes and diverse OOD domains.The OOD datasets include Places365, Textures, iNaturalist, and SUN, spanning fine-grained, scene, and textural images.
  • ImageNet benchmark: 25.05% FPR95 reduction is achieved by ReAct versus the best baseline on the ImageNet benchmark.The evaluation averages results across four OOD test datasets and compares post hoc methods.
  • Threshold ablation: ReAct improves OOD detection across thresholds when the percentile is sufficiently large, whereas overly small percentiles degrade OOD performance.A sufficiently large percentile can improve detection while maintaining comparable ID classification accuracy.
  • Network architectures: 9.18% FPR95 reduction is obtained on MobileNet-v2 compared to the best baseline.The rectification threshold is selected using the 90-th percentile of activations.
  • Layer selection: Penultimate-layer rectification is most effective because deeper layers develop more distinctive semantic activation patterns between ID and OOD data.Early layers capture lower-level features whose activations can be similar between ID and OOD data.
  • Scoring-function compatibility: ReAct consistently outperforms versions without rectification across softmax confidence, ODIN, and energy scoring functions on CIFAR evaluations.Energy scoring with ReAct gives the best performance among the considered scoring functions.

5 Theoretical Analysis

Theoretical analysis models ID and OOD activations to explain why ReAct reduces OOD activations and output logits more strongly than ID activations. The effect increases with positive skewness and chaoticness, widening the separation used by logit-based OOD scores.

  • ReAct models ID and OOD activations as rectified Gaussian distributions and derives their post-rectification distributions.
  • OOD activations experience greater mean reduction than ID activations because they are more positively skewed.
  • Increasing positive skewness or chaoticness produces larger activation reductions after ReAct.
  • The larger OOD activation reduction transfers to model outputs under the sufficient condition W1 > 0.
  • ReAct enlarges the gap between OOD and ID logit-based scores, improving threshold-based separation.

6 Discussion and Further Analysis

Further analysis attributes abnormal OOD activations partly to mismatched BatchNorm statistics and examines ReAct across normalization schemes. ReAct approaches an OOD-statistics oracle without requiring OOD batches and remains effective with WeightNorm and GroupNorm.

  • Mismatched BatchNorm statistics estimated on ID data can trigger abnormally high OOD unit activations.
  • Using true OOD BatchNorm statistics produces well-behaved activation patterns and serves as an impractical oracle upper bound.
  • ReAct favorably matches oracle performance without requiring a batch of OOD data, supporting single-input testing.
  • ReAct is evaluated with WeightNorm and GroupNorm, whose activation patterns are also analyzed.
  • 23.54% and 14.7% reductions in average FPR95 are reported for models trained with WeightNorm and GroupNorm, respectively.

7 Related Work

Related work covers discriminative and generative OOD uncertainty estimation, neural-network activation analysis, and broader distributional-shift distinctions. ReAct is positioned as a post hoc activation-rectification approach addressing high OOD activations without auxiliary data.

  • Discriminative OOD methods include ODIN, OpenMax, Mahalanobis distance, and energy-based scores.
  • Prior activation analyses include work on activation spaces and arbitrary high activations for inputs far from training data.
  • ReAct rectifies unit activations to alleviate OOD overconfidence, works with pre-trained models, and does not rely on auxiliary data.
  • Generative approaches detect OOD inputs using estimated density, although deep generative models can assign high likelihood to OOD data.
  • OOD detection commonly concerns label-space shifts, whereas other distribution-shift work considers label-distribution shifts.

8 Conclusion

ReAct is a test-time activation-rectification strategy for OOD detection that truncates high activations to attenuate their outsized influence on model outputs. The paper combines theoretical and empirical analysis and reports improved OOD detection across benchmarks and large-scale image classification models.

  • ReAct truncates high activations during test time to improve OOD detection.The method is designed as a simple activation rectification strategy.
  • Rectifying activations attenuates hidden units’ outsized contribution to OOD outputs, resulting in stronger separability from ID data.
  • ReAct combines empirical and theoretical insights to characterize how activation rectification improves OOD uncertainty estimation.
  • Extensive experiments show significant OOD-detection improvements on common benchmarks and large-scale image classification models.
  • The paper anticipates extending OOD-detection research beyond image classification.

9 Societal Impact

The project targets more dependable and trustworthy machine-learning systems by improving OOD uncertainty estimation. It emphasizes applications involving classification models, releases code, and presents ReAct as a plug-and-play tool for practitioners.

  • The project aims to improve the dependability and trustworthiness of modern machine-learning models.
  • OOD uncertainty estimation is presented as relevant to digital content understanding, driver assistance, autonomous vehicles, and unseen disease identification.
  • Many of the named applications require classification models in operation.
  • The authors hope their work and released code provide researchers with a new methodological perspective and practitioners with a plug-and-play tool for OOD safety.

B Descriptions of Baseline Methods

The paper reviews post-hoc OOD scores computed from models trained only on in-distribution data, including output-based, perturbed-input, energy, and feature-level approaches. Scores conventionally increase with ID-ness, except that energy is negated to follow this convention.

  • Common OOD scores are derived post hoc from neural networks trained only with in-distribution data.
  • Softmax score: The maximum softmax probability uses the largest class prediction probability as the OOD score.
  • ODIN score: ODIN combines temperature scaling and input preprocessing with a score computed from the perturbed input.
  • Energy score: The energy score maps logits to a scalar that is relatively lower for ID data, so the negative energy is used for OOD detection.
  • Mahalanobis distance: The Mahalanobis score models feature-level distributions as class-conditional Gaussians with class means and tied covariance.

C Selected Categories in OOD Datasets

The OOD evaluation uses categories from iNaturalist, SUN, Places365, and Textures, with the listed concepts curated to be disjoint from ImageNet-1k labels. Experiments are conducted on NVIDIA GeForce RTX 2080Ti GPUs.

  • Dataset sources: OOD test concepts are selected from iNaturalist, SUN, and Places365 and curated to be disjoint from ImageNet-1k labels.
  • Dataset sources: The Textures evaluation uses the entire dataset.
  • iNaturalist: iNaturalist categories include numerous plant and botanical species such as Coprosma lucida, Cucurbita foetidissima, and Mitella diphylla.
  • SUN and Places365: SUN and Places365 categories include scenes and environments such as forests, deserts, fields, waterways, gardens, and waterfalls.
  • Hardware: All experiments are conducted on NVIDIA GeForce RTX 2080Ti GPUs.

E Ablation Study on Different Layers

ReAct is most effective when applied to layers whose activation patterns distinguish ID from OOD data. Early layers show weaker distinctions, while Gaussian-noise inputs also produce overly high activations in ResNet-50.

  • Layer selection: Early layers show less distinctive activation signatures between ID and OOD data, and ReAct performs worse than the baseline on layer 1.The comparison uses intermediate residual-block outputs in ResNet-50.
  • Layer selection: The ablation evaluates ReAct across residual-block outputs using ResNet-18 trained on CIFAR-100 and ResNet-50 trained on ImageNet.Results are reported over multiple OOD test datasets.
  • Activation patterns: Figure 6 compares per-unit penultimate-layer activation distributions for ImageNet ID data and iNaturalist OOD data across four ResNet-50 residual-block outputs.Each layer corresponds to the output of one residual block.
  • Normalization analysis: Using mismatched ID BatchNorm statistics on OOD data can trigger abnormally high unit activations, whereas true OOD statistics yield near-constant means and standard deviations.True OOD statistics require access to a batch of OOD images, while ReAct can operate on a single OOD instance.
  • Activation patterns: Gaussian-noise OOD inputs can produce overly high unit activations in ResNet-50, consistent with the activation patterns observed elsewhere.Gaussian noise is used as validation data in Figure 7.
Loading 2111.12797v1…