PaperlayerView exact revision on arXiv

Source-linked AI summary

Extremely Simple Activation Shaping for Out-of-Distribution Detection

Andrija Djurisic, Nebojsa Bozanic, Arjun Ashok, Rosanne Liu

arXiv:2209.09858v2cs.LGcs.CV

TL;DR

OOD detection asks whether models can recognize inputs outside their training distribution, while existing methods may require extra data, training, or network changes. ASH addresses this with post-hoc, on-the-fly activation shaping that prunes and lightly adjusts late-layer features, achieving state-of-the-art OOD detection and the best reported trade-off with ID accuracy.

  • Problem

    OOD detection must distinguish out-of-distribution from in-distribution samples, but this remains difficult and existing methods can require extra data, training, or network modifications.

  • Method

    ASH prunes a large portion of an input’s late-layer activation and scales or otherwise lightly adjusts the remaining values during inference without modifying trained network weights.

  • Results

    ASH reaches state-of-the-art OOD detection across the evaluated benchmarks and provides the best trade-off between OOD detection and ID classification accuracy.

  • Takeaways & Limitations

    ASH is a simple, post-hoc, plug-and-play framework that can improve OOD detection while preserving ID performance and combining with existing methods.

Abstract

from arXiv · show

The separation between training and deployment of machine learning models implies that not all scenarios encountered in deployment can be anticipated during training, and therefore relying solely on advancements in training has its limits. Out-of-distribution (OOD) detection is an important area that stress-tests a model's ability to handle unseen situations: Do models know when they don't know? Existing OOD detection methods either incur extra training steps, additional data or make nontrivial modifications to the trained network. In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly activation shaping method, ASH, where a large portion (e.g. 90%) of a sample's activation at a late layer is removed, and the rest (e.g. 10%) simplified or lightly adjusted. The shaping is applied at inference time, and does not require any statistics calculated from training data. Experiments show that such a simple treatment enhances in-distribution and out-of-distribution distinction so as to allow state-of-the-art OOD detection on ImageNet, and does not noticeably deteriorate the in-distribution accuracy. Video, animation and code can be found at: https://andrijazz.github.io/ash

1 INTRODUCTION

OOD detection asks whether models recognize inputs outside their training distribution, but existing solutions often require extra data, training, or network modifications. ASH instead reshapes late-layer activations post hoc and achieves strong OOD detection while preserving ID accuracy.

  • OOD detection tests whether neural networks know when they do not know, yet distinguishing OOD from ID samples is harder than expected.
  • Existing approaches include activation clipping and weight sparsification, but can require network or representation modifications.
  • ASH removes most late-layer activations using top-K pruning, then scales the survivors or assigns them a constant value.A typical configuration removes 90% and adjusts the remaining 10%.
  • ASH is post-hoc, one-shot, plug-and-play, and operates in activation space without modifying trained network weights or requiring training-data thresholds.
  • Across 3 ID datasets and 10 OOD datasets, ASH improves OOD detection and establishes state-of-the-art performance with an optimal ID-OOD trade-off.
  • The paper reports extensive ablations covering ASH placement, pruning strength, shaping treatments, and combinations with other methods.

2 THE OUT-OF-DISTRIBUTION DETECTION SETUP

The OOD setup freezes pretrained classifiers, scores inputs for ID-versus-OOD discrimination, and evaluates detection alongside ID accuracy. Experiments cover CIFAR and ImageNet benchmarks with multiple datasets, architectures, scores, and metrics.

  • The standard OOD pipeline trains and freezes a model, feeds it OOD data, derives a detection score, and evaluates the detector with metrics.
  • Datasets and models: Experiments use CIFAR-10 and CIFAR-100 with 6 OOD datasets, plus ImageNet-1k with 4 OOD datasets and 3 pretrained architectures.The architectures are DenseNet-101 for CIFAR and ResNet50 or MobileNetV2 for ImageNet.
  • Datasets and models: All networks are pretrained on ID data and remain unchanged during the OOD detection phase.
  • Detection scores: The default detector uses the negative Energy score from logits, while Softmax and other scoring methods provide comparison or ablation settings.
  • Evaluation metrics: Detection is evaluated with AUROC, AUPR, and FPR95, alongside ID classification accuracy such as ImageNet Top-1 accuracy.
  • ID-OOD tradeoff: On ImageNet, Figure 2 compares average AUROC across 4 OOD datasets against ID Top-1 accuracy to show the ID-OOD tradeoff.

3 ACTIVATION SHAPING FOR OOD DETECTION

ASH simplifies an intermediate activation on each input during inference, pruning values below a percentile threshold and optionally transforming the survivors. It is mainly applied at late layers, where high pruning can improve OOD detection without severe ID degradation.

  • ASH argues that over-parameterized neural representations can be simplified with little classification deterioration while improving OOD detection.
  • Activation shaping: ASH first removes activations below the pth percentile, then applies pruning-only, binarization, or scaling treatment to the remaining values.
  • Activation shaping: ASH operates on each input’s feature representation at an intermediate layer before routing the modified representation through the rest of the network.
  • Shaping treatments: The scaling variant rescales surviving activations using the ratio of the original and post-pruning sums, while the binary variant assigns a positive constant.
  • Placement of ASH: The main ImageNet placement is after average pooling, while CIFAR experiments apply ASH after the penultimate layer.
  • The p parameter: Pruning percentages from 60 to 90 show relatively steady performance, with best configurations using p = 65 or 90 on ImageNet and p = 85, 90, or 95 on CIFAR.

4 RESULTS

ASH improves OOD detection while preserving much of the original in-distribution accuracy across ImageNet and CIFAR benchmarks. Its performance depends on the shaping variant and pruning strength, with extreme randomized shaping remaining competitive.

  • ASH offers the best ID-OOD tradeoff: ASH establishes a new Pareto frontier by improving OOD detection while preserving in-distribution accuracy on ImageNet.ASH-B and ASH-S outperform or compare favorably with existing methods across OOD datasets, while retaining stronger ID accuracy than representation-modifying baselines.
  • OOD detection on ImageNet and CIFAR: ASH-B and ASH-S establish state-of-the-art OOD detection across almost all ImageNet datasets and metrics on ResNet.On MobileNet, they perform comparably with DICE+ReAct while using a simpler algorithm.
  • OOD detection on ImageNet and CIFAR: All ASH variants profoundly outperform existing baselines on CIFAR benchmarks averaged across 6 OOD datasets.The evaluation uses pretrained DenseNet-101 models and follows the experimental setting of Sun and Li.
  • On preserving in-distribution accuracy: 76.13% to 76.01%: at 65% pruning, ASH-S and ASH-P cause only a slight drop in ImageNet Top-1 validation accuracy.At 90% pruning, both variants maintain 74.98% ID accuracy, while accuracy reaches 64.976% at 99% pruning.
  • On preserving in-distribution accuracy: ASH-RAND remains comparable to ASH-B and ASH-S and consistently beats Energy, Softmax, and ODIN on ImageNet.It assigns un-pruned activations random values between 0 and 10, with OOD performance averaged across 4 datasets.
  • On preserving in-distribution accuracy: ASH-B performs best between 80% and 90% pruning but completely destroys the classifier at 0% pruning.At low pruning rates, binarization modifies most feature-map values; at 0% pruning, every activation is set to its average.

5 ABLATION STUDIES

The ablations examine threshold selection, layer placement, and compatibility with existing scoring and detection methods. Local thresholds and later-layer placement perform best, while ASH improves several established methods.

  • Threshold selection: Local thresholds consistently outperform global thresholds and achieve the best overall performance despite slight inference overhead.Local thresholds are computed per image on the fly and require no training or test-data information.
  • Compatibility: ASH can be combined with Softmax, Energy score, ODIN, and ReAct to provide immediate improvement over these methods.The compatibility experiments cover CIFAR-10, CIFAR-100, and ImageNet.
  • Compatibility: The compatibility results average CIFAR-10 and CIFAR-100 performance across six OOD tasks and ImageNet performance across four OOD tasks.All methods and experiments in the comparison were reimplemented by the authors.

6 RELATED WORK

ASH relates to post-hoc model enhancement and sparse representations. Unlike training-time sparse activation approaches, it reshapes activations during inference and leaves adversarial defense outside this paper’s scope.

  • Post-hoc model enhancement: Post-hoc model enhancement modifies trained representations at test time, paralleling practices such as Monte Carlo dropout and randomized smoothing.The cited examples use stochastic or perturbed predictions to address uncertainty or robustness.
  • Sparse representations: ASH parallels activation pruning by removing low-magnitude activations and scaling retained ones, but applies this operation during inference rather than training.Activation pruning and sparse representations provide the conceptual comparison for ASH.
  • Sparse representations: Adversarial defense is outside this paper’s scope, although the relationship between ASH and activation pruning suggests it as future work.The paper distinguishes ASH from training-time top-K activation functions that remove ReLU.

7 CONCLUSION

The paper presents ASH as a simple inference-time activation-shaping method for classification and OOD detection. Across broad experiments, it reports strong OOD detection and a favorable trade-off with ID accuracy.

  • Conclusion: ASH is a post hoc, on-the-fly, plug-and-play method that prunes much of an activation and lightly adjusts what remains at inference.The shaped representation is passed through the rest of the network to produce classification and OOD-detection scores.
  • Conclusion: Across 3 ID datasets, 10 OOD datasets, and 4 metrics, ASH reaches state-of-the-art OOD detection while preserving the best reported trade-off with ID accuracy.The method is also compatible with and beneficial to existing methods.

A FULL RESULTS ON ASH PLACEMENTS

The full placement study evaluates ASH throughout ResNet-50 and finds that placement strongly affects both accuracy preservation and OOD detection.

  • Placement results: The penultimate layer gives the best combination of ID accuracy preservation and OOD detection among the evaluated ResNet-50 placements.The main experiments place ASH at the penultimate layer for ResNet-50 and MobileNet.
  • Placement results: Moving ASH toward earlier network layers degrades both ID accuracy and OOD detection rates.The study evaluates placements in the first through fourth ResNet-50 blocks, before the activation function.
  • Variant selection: Layer-specific variants are required because ASH-S drastically degrades performance in earlier blocks; on Layer 3, scaling drops ID accuracy from 75% to 5%.ASH-S@90 is used for the penultimate layer and fourth layer, while ASH-P@90 is used for Layers 1–3.
  • Evaluation setup: OOD results are averaged across iNaturalist, Places, Textures, and Sun, with arrows indicating whether larger or smaller values are better.All values in the placement table are percentages.

B DETAILED CIFAR RESULTS

Tables 7 and 8 provide supplementary CIFAR OOD results, expanding the main-text Table 2 with dataset-level detail.

  • Tables 7 and 8 supplement Table 2 with the full CIFAR OOD results.
  • The supplementary results are reported separately for models trained on CIFAR-10 and CIFAR-100.
  • Each table displays results for all six OOD datasets used in these experiments.

C ID-OOD TRADEOFF FOR ADDITIONAL ARCHITECTURES

Additional architecture experiments examine ASH's ID-OOD tradeoff on MobileNetV2, alongside detailed six-dataset ImageNet results. ASH preserves ID accuracy strongly while achieving OOD performance comparable to leading methods on MobileNetV2.

  • MobileNetV2: On MobileNetV2, ASH-S and ASH-B offer superior ID accuracy preservation while providing OOD performance comparable to DICE+ReAct.
  • Detailed benchmark results: Tables 7 and 8 report detailed results on six common OOD benchmark datasets.
  • MobileNetV2: Figure 6 plots average OOD AUROC across iNaturalist, SUN, Places365, and Textures against ImageNet validation Top-1 accuracy.
  • MobileNetV2: The MobileNetV2 tradeoff compares OOD detection rate with in-distribution classification accuracy across methods and variants.

D MORE EXPERIMENTS ON GLOBAL VS LOCAL THRESHOLD

The experiments compare local and global pruning thresholds, interpret ASH as post-hoc feature cleaning or a data-dependent activation function, and examine how pruning changes energy-score separation.

  • Global vs local thresholds: Local thresholds consistently outperform global thresholds for both ASH-S and ASH-B on ImageNet with ResNet-50.
  • Interpretation: ASH can be viewed as post-hoc feature cleaning or regularization applied to overrepresented learned representations.
  • Interpretation: ASH-P combined with the subsequent ReLU is equivalent to a modified ReLU with an input-dependent threshold.
  • Energy-score distributions: Increasing ASH pruning strength morphs the ID and OOD energy-score distributions toward greater separation, measured by Intersection Over Union.

F NOTES ON IMPLEMENTATION

The implementation notes document settings across datasets, architectures, scoring functions, and detector combinations. They also report broader compatibility results and identify unanswered questions about ASH's explanation and transfer beyond vision.

  • Scaling functions: ASH-S achieves its best reported scaling-function performance with an exponential function.
  • Additional architectures: On additional ImageNet architectures, ASH-S@90 and ASH-B@65 are best for DenseNet-121, while ASH-B@65 consistently outperforms benchmarks for VGG-16.
  • Compatibility experiments: ASH improves KNN-based OOD detection across pruning settings, with ASH-S more reliable than ASH-B in the reported experiment.
  • Comparison with DICE: DICE alone is reported to perform less well than when combined with ReAct, which doubles hyperparameter-tuning overhead.
  • Comparison with DICE: ASH is described as on-the-fly, requiring neither DICE-style precomputation nor access to training data, while preserving ID accuracy relative to DICE.
  • Open questions: The paper calls for future work explaining ASH's effectiveness and testing whether it applies to other domains and tasks.
Loading 2209.09858v2…