PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Importance of Gradients for Detecting Distributional Shifts in the Wild

Rui Huang, Andrew Geng, Yixuan Li

arXiv:2110.00218v2cs.LG

TL;DR

OOD detection is important for reliable deployment, but existing approaches largely overlook gradient-space information. GradNorm uses gradient norms derived from KL divergence to a uniform target and reduces average FPR95 by up to 16.33% versus the previous best method.

  • Problem

    Existing OOD detection approaches primarily derive scores from output or feature space while largely overlooking gradient-space information needed to distinguish ID from OOD inputs.

  • Method

    GradNorm uses the vector norm of gradients backpropagated from the KL divergence between softmax predictions and a uniform distribution as the OOD score.

  • Results

    Up to 16.33% lower average FPR95 was achieved than the previous best method, with superior performance across ImageNet and common OOD benchmarks.

  • Takeaways & Limitations

    Gradient space provides useful OOD signals, while GradNorm offers a simple, label-agnostic and OOD-agnostic approach without additional training or outlier data.

  • Takeaways & Limitations

    GradNorm relies on the operating hypothesis that KL-divergence gradient norms are higher for ID data than for OOD data.

Abstract

from arXiv · show

Detecting out-of-distribution (OOD) data has become a critical component in ensuring the safe deployment of machine learning models in the real world. Existing OOD detection approaches primarily rely on the output or feature space for deriving OOD scores, while largely overlooking information from the gradient space. In this paper, we present GradNorm, a simple and effective approach for detecting OOD inputs by utilizing information extracted from the gradient space. GradNorm directly employs the vector norm of gradients, backpropagated from the KL divergence between the softmax output and a uniform probability distribution. Our key idea is that the magnitude of gradients is higher for in-distribution (ID) data than that for OOD data, making it informative for OOD detection. GradNorm demonstrates superior performance, reducing the average FPR95 by up to 16.33% compared to the previous best method.

1 Introduction

OOD detection addresses whether deployed models are making reliable predictions on familiar data or unreliable predictions on unseen distributions. This paper explores gradient space as an additional source of OOD signals and introduces GradNorm, reporting stronger detection performance than prior output- and feature-based approaches.

  • OOD detection determines whether an input is in-distribution or out-of-distribution, supporting models that recognize uncertainty beyond their training distribution.
  • Modern neural networks can produce overconfident predictions on OOD inputs, making ID–OOD separation non-trivial.
  • Prior OOD methods primarily derive uncertainty measurements from model outputs or feature representations, while gradient space remains largely unexplored.
  • GradNorm uses the vector norm of gradients backpropagated from KL divergence between the softmax output and a uniform distribution as an OOD score.
  • 16.33% reduction in FPR95 is reported for GradNorm relative to the previous best method, alongside theoretical analysis linking feature and output information.

2 Preliminaries

The paper formulates supervised learning with inputs, labels, training data, a neural network, and cross-entropy risk minimization. OOD detection is then posed as binary classification using a score that distinguishes samples from the training distribution from other samples.

  • The input space is X = R^d, the label space is Y = {1, 2, ..., C}, and training data are drawn from an unknown joint distribution P.
  • A neural network f(x; θ) maps inputs to R^C and minimizes empirical risk using cross-entropy loss.
  • The temperature T is used in the model's softmax-related formulation, while f_y(x) denotes the output corresponding to the ground-truth label.
  • OOD detection is formulated as binary classification that determines whether a sample x ∈ X comes from the in-distribution P or not.
  • The scoring function S(x) captures OOD uncertainty, with γ commonly selected so that a high fraction, such as 95%, of ID data is correctly classified.

3 Gradient-based OOD Detection

GradNorm detects OOD inputs by using gradient norms computed from a KL divergence between softmax predictions and a uniform distribution. The method is designed to distinguish ID from OOD data while remaining label-agnostic and applicable to pretrained models.

  • The gradients are obtained by backpropagating KL divergence between the softmax output and a uniform distribution.The uniform target is u = [1/C, 1/C, ..., 1/C], while the predictive distribution is the softmax output.
  • KL divergence measures how far the predictive distribution is from uniformity, with ID predictions expected to have larger values because they concentrate on the ground-truth class.
  • The gradient of the KL divergence is equivalent to averaging categorical cross-entropy derivatives across all labels.The entropy term H(u) is constant and has zero gradient.
  • GradNorm uses the vector norm of gradients as the OOD score.The gradients are computed with respect to selected network parameters.
  • The operating hypothesis is that gradient norms are higher for ID data than OOD data, and Figure 1 illustrates this separation.Figure 1 depicts inputs in the xy-plane and gradient norms in the z dimension, with ID data shown in light green and OOD data in deep blue.
  • GradNorm is label-agnostic and can be used during inference without ground-truth labels.It can be implemented using cross-entropy between predicted softmax probabilities and a uniform target.

4 Experiments

Experiments evaluate GradNorm across datasets, architectures, gradient subsets, target distributions, norms, temperatures, and scoring functions. The results show strong performance and identify practical choices that improve OOD detection.

  • Comparison with existing methods: GradNorm outperforms the best output-based baseline, Energy, by 16.33% in FPR95 and the feature-based Mahalanobis method by 26.99%.It requires only a simple gradient calculation without hyperparameter tuning or additional training, whereas Mahalanobis requires collecting intermediate-layer features over the training set.
  • Gradient subset: Last-layer gradients achieve the best results among tested gradient subsets, while deeper layers generally outperform shallower layers.Using only the last fully connected layer is computationally efficient and incurs negligible overhead.
  • Target distribution: Uniform targets produce larger ID gradient magnitudes and better ID–OOD separability than one-hot targets, whose scores overlap substantially.The uniform-target score uses gradient norms derived from KL divergence between the softmax output and a uniform distribution.
  • Alternative architecture: On DenseNet-121, GradNorm consistently outperforms the best baseline, Energy, by 10.29% in FPR95.The evaluation uses a model trained on ImageNet-1k and compares post hoc methods applicable to pretrained models.
  • Gradient norm: L1-norm performs best across all four OOD datasets, whereas L∞-norm performs worst among the evaluated norms.The analysis attributes this pattern to L1 treating all gradient dimensions equally, unlike higher-order norms that emphasize larger elements.
  • Temperature scaling: Temperature T = 1 is optimal, while increasing or decreasing temperature degrades performance; GradNorm can therefore be hyperparameter-free with T = 1.The paper explains that large temperatures smooth the softmax distribution and reduce ID–OOD distinguishability.
  • Scoring function: GradNorm significantly outperforms directly using KL divergence as an OOD score, supporting the value of gradient-space information.The comparison uses an ablation between gradient-norm scoring and KL divergence extracted directly from the output space.

5 Analysis of Gradient-based Method

The analysis decomposes GradNorm’s last-layer gradient norm into feature- and output-space factors and finds that their combination improves ID–OOD separability. It also examines how gradient distributions and score components relate to detection performance.

  • Gradient formulation: GradNorm’s analysis focuses on the L1-norm of gradients from the network’s last fully connected layer.The last-layer formulation uses logits produced from input features through weights and biases.
  • Gradient formulation: The L1 gradient norm decomposes into two summations whose distributions are compared for ID and OOD data.Figure 7 visualizes the two decomposed summations for both data types.
  • Joint information: The factors U and V measure feature-space and output-space distributions, respectively, so GradNorm captures their joint information.Their multiplication produces stronger ID–OOD separability than either factor alone.
  • Joint information: U and V used individually as scoring functions are less competitive than GradNorm.The comparison is reported in Table 5 for a ResNetv2-101 trained on ImageNet-1k.

6 Discussion

The discussion distinguishes GradNorm from earlier gradient-related OOD methods by using gradient information directly rather than for perturbation or auxiliary classification. It highlights computational and methodological differences while identifying output–gradient combinations as future work.

  • Comparison with ODIN: GradNorm uses gradient-space information directly, unlike ODIN, which uses input gradients to perturb inputs before deriving output-based OOD scores.ODIN’s perturbations aim to increase the softmax score of the predicted label and widen ID–OOD score gaps.
  • Future direction: Combining gradient-space and output-space information is left as a question for future exploration.GradNorm itself uses information solely from the gradient space.
  • Comparison with Lee and AlRegib: Lee and AlRegib train an auxiliary binary classifier on gradient information using OOD data, whereas GradNorm avoids that OOD-dependent setup.The paper characterizes the auxiliary classifier’s use of OOD datasets as potentially overfitting test data and unsuitable for OOD-agnostic settings.
  • Design choices: GradNorm’s ablations find that L1-norm gradients outperform L2-norm gradients by up to 22.31% in FPR95.The study also reports that last-layer gradients yield the best performance among gradient-set selections, with negligible computational cost.

7 Related Work

Related work covers discriminative and generative approaches to OOD uncertainty estimation, alongside broader distributional-shift settings. These approaches differ in whether they use classifier scores, feature distances, density estimates, or input-space shifts.

  • Discriminative models: Discriminative OOD methods include abstention, ODIN, and Mahalanobis distance-based confidence scores.This line of work addresses uncertainty estimation with classification models.
  • Generative models: Generative OOD methods estimate density and classify inputs in low-likelihood regions, but deep generative models can assign high likelihood to OOD data.The high-likelihood finding complicates density-based OOD detection.
  • Distributional shifts: OOD detection commonly concerns label-space shifts with disjoint ID and OOD labels, while other work studies covariate shifts in input space.The paper distinguishes these distributional-shift problem types.

8 Conclusion

The paper concludes that GradNorm uses gradient-space information for OOD uncertainty estimation and improves OOD detection performance. Its ablations further analyze design choices and motivate future work on gradient-space signals.

  • Conclusion: GradNorm improves OOD detection performance by up to 16.33% in FPR95.The paper presents this as an experimental result of its gradient-based method.
  • Conclusion: Extensive ablations provide further understanding of the gradient-based approach and support continued study of gradient space for OOD uncertainty estimation.The authors hope this work inspires future research using gradient-space information.

9 Societal Impact

The project aims to improve the reliability and safety of modern machine learning models, with potential relevance across consumer, transportation, and health-care applications.

  • OOD uncertainty estimation is positioned as important for applications including digital content understanding, driver assistance, autonomous vehicles, and unseen disease identification.The paper also releases code and describes GradNorm as an easy-to-use tool for practitioners.

A Evaluation on CIFAR Benchmarks

The evaluation covers standard CIFAR-10 and CIFAR-100 OOD benchmarks alongside implementation and baseline details. GradNorm remains competitive on CIFAR and is reported as hyperparameter-free and suitable for OOD-agnostic use.

  • CIFAR benchmark: 8.77% lower average FPR95 is reported for GradNorm on CIFAR-10 than the best baseline.The benchmark uses CIFAR-10 as an in-distribution dataset and evaluates four common OOD datasets.
  • CIFAR benchmark: 14.47% lower FPR95 is reported for GradNorm on CIFAR-100 than the best baseline, energy score.CIFAR-100 is evaluated with the same standard benchmark setup described for CIFAR-10.
  • CIFAR benchmark: GradNorm remains competitive across the CIFAR benchmark and overall demonstrates superior performance compared with competitive methods in the literature.The paper notes that some baselines require validation datasets, whereas GradNorm is hyperparameter-free and suitable for OOD-agnostic settings.
  • CIFAR benchmark: The CIFAR evaluation uses CIFAR-10 and CIFAR-100 as ID datasets with 50,000 training images and 10,000 test images.The OOD datasets include SVHN, LSUN, Places365, and Textures.
  • Additional evaluations: L1-norm achieves the best overall performance among the additional Lp-norms evaluated with a ResNetv2-101 architecture pretrained on ImageNet-1k.The comparison extends beyond the six norms shown in Figure 4.
  • Additional evaluations: T = 1 achieves the best average performance among the additional scaling temperatures evaluated with a ResNetv2-101 architecture pretrained on ImageNet-1k.The temperature comparison is reported in Table 8.
Loading 2110.00218v2…