PaperlayerView exact revision on arXiv

Source-linked AI summary

Anti-coherence Regularized Absolute Scaling for Out-of-Distribution Detection

Donghwan Kim, Youngwoong Jin, Hyunsoo Yoon

arXiv:2610.32255v2cs.LGcs.AI

TL;DR

OOD detection seeks to flag inputs that depart from the training distribution, but SCALE’s percentile ratio can fail when negative activations cancel in signed sums. The paper uses absolute-valued activations to correct the ratio and regularizes its scale with Anti-coherence. Under OpenOOD v1.5, the method ties or outperforms training-data-dependent activation-shaping methods on CIFAR-10 and ImageNet without requiring training data or gradients.

  • Problem

    SCALE’s signed-sum percentile ratio can lose its intended meaning on backbones with negative penultimate activations, limiting its use across architectures.

    Source: §1 · p. 2, §3.2 · p. 4

  • Method

    The method computes SCALE’s ratio over absolute-valued activations and regularizes the scale using the fraction of sign-mismatched max-logit contributions.

    Source: §1 · p. 2, §4 · p. 6

  • Results

    Under OpenOOD v1.5 on CIFAR-10 and ImageNet, the method ties or outperforms activation-shaping methods that rely on training data or additional gradient computation.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The method requires neither training data nor gradient computation while achieving the reported benchmark comparison.

    Source: §1 · p. 2, §6 · p. 10

    OOD detection offers no strict guarantee of coverage and should not be treated as the sole safeguard in high-stakes deployment settings.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Out-of-distribution (OOD) detection is recognized as a critical problem in AI safety. SCALE is an effective post-hoc OOD detection method in the activation-shaping family that requires no training data and rescales penultimate activations using a percentile-based magnitude ratio. However, this ratio is ill-defined when negative activations are present, since positive and negative contributions cancel within the underlying sum, causing SCALE to degenerate on backbones such as ViT. We address this with an Absolute Fix that computes the ratio over absolute-valued activations rather than discarding negative activations through ReLU. Building on this corrected scale, we propose Anti-coherence regularization, which penalizes the scaling factor by the fraction of a sample's max-logit contributions to the max logit that are sign-mismatched and thereby suppresses unreliable evidence for the predicted class. Despite requiring no access to training data or training labels, our method ties or outperforms activation-shaping methods that rely on training data in terms of AUROC and FPR@95 on CIFAR-10 and ImageNet under the OpenOOD v1.5 protocol.

1 INTRODUCTION

OOD detection helps flag unfamiliar inputs before confident predictions lead to erroneous decisions. The paper corrects SCALE’s handling of negative activations and adds Anti-coherence regularization, reporting performance that ties or exceeds training-data-dependent activation-shaping methods on CIFAR-10 and ImageNet.

Motivation and approach

  • OOD detection is presented as a safeguard against confident predictions on inputs that deviate from the training distribution, including in safety-critical domains.The introduction names robotics, cybersecurity, and medical imaging as examples.Source: §1 · p. 1
  • SCALE’s signed-sum scaling degenerates on backbones with negative penultimate activations, while ReLU rectification improves it by discarding all negative-activation information.The proposed Absolute Fix instead includes negative activations through their absolute values.Source: §1 · p. 2
  • Anti-coherence regularizes the scaling factor using the fraction of a sample’s max-logit contribution that drives the maximum logit downward.It is derived by decomposing contributions to the maximum logit.Source: §1 · p. 2

Contributions and results

  • Under OpenOOD v1.5 on CIFAR-10 and ImageNet, the method ties or outperforms activation-shaping methods that rely on training data or additional gradient computation.The method requires neither training data nor additional gradient computation.Source: §1 · p. 2

2 RELATED WORKS

Related work spans activation-shaping, representation-based, and logit-based OOD methods. Their approaches vary in how they manipulate activations, use training statistics, or derive scores from logits.

Activation Shaping Methods

  • Activation-shaping methods truncate, prune, or scale activations; some approaches estimate scaling or typical sets from training data.ReAct truncates above a percentile, VRA also truncates a lower bound, DICE truncates connected weights, and ASH combines truncation with scaling.Source: §2 · pp. 2–3

Representation-based Methods

  • Representation-based methods score feature-space distances or related structure, and the described approaches require access to training data.Examples include class-wise Mahalanobis distance, k-nearest-neighbor distance, decision-boundary distance, and Neural Collapse-based approaches.Source: §2 · p. 3

Logit-based Methods

  • Logit-based methods derive OOD scores from logits, including maximum softmax probability, energy, generalized entropy, and scaling logits using ReLU-rectified activations.Source: §2 · p. 3

3 ABSOLUTE FIX: HANDLING NEGATIVE ACTIVATIONS IN Q/Qp

SCALE estimates activation concentration from a percentile-based ratio and uses the rescaled activations to compute an OOD score. Because signed sums can cancel when activations are negative, the Absolute Fix computes the ratio from absolute values.

3.1 PRELIMINARIES

  • SCALE computes a ratio from the sum of all penultimate activations and the sum of those above a percentile, then rescales activations before scoring.The resulting energy score uses a threshold set to yield 95% true positive rate on ID samples; FPR@95 measures the OOD samples incorrectly classified as ID.Source: §3.1 · pp. 3–4

3.2 ABSOLUTE-VALUE PERCENTILE SCALING

  • On ViT and Swin, negative activations let positive and negative terms cancel, so SCALE’s ratio can lose its intended magnitude meaning and fail to separate ID from OOD.On ResNet, whose penultimate activations are non-negative, the ratio is visibly higher for ID than OOD.Source: §3.2 · p. 4, Figure 1 · p. 4
  • The Absolute Fix computes the percentile-based ratio using absolute-valued activations, reflecting the symmetric heavy-tailed activation distribution observed for ID and OOD on backbones with negative activations.For strictly non-negative activations, the corrected ratio reduces to the original ratio.Source: §3.2 · p. 5, Figure 2 · p. 5
  • The corrected ratio separates ID from OOD in the intended direction on ViT-B/16, Swin-B, and ConvNeXt-B, while results are less clear on DeiT3-B and BEiTv2-B.EfficientNet-B1 also benefits despite lacking the symmetric heavy tail described for LayerNorm-based backbones.Source: §3.2 · p. 5

4 REGULARIZING SCALE WITH ANTI-COHERENCE

Scaling makes the predicted class’s max logit account for an increasing share of the energy score, motivating analysis of that logit’s positive and negative contributions. Anti-coherence uses the sign-mismatched share to regularize scaling; it tends to be higher for OOD than ID samples and requires neither training data nor gradients.

4 REGULARIZING SCALE WITH ANTI-COHERENCE

  • The analysis first considers how activation scaling affects the energy score, then focuses on the max logit after scaling makes it dominate that score.The mean fraction of the energy score attributable to the max logit is compared for ID and OOD ImageNet samples under OpenOOD v1.5.Source: §4 · pp. 5–6
  • As r increases, the max logit’s share of the energy score is non-decreasing and approaches 1; after r_abs scaling, it dominates for ID and OOD samples across backbones.This motivates analyzing the max logit rather than the full energy score.Source: §4 · p. 6
  • Anti-coherence A(z) is the fraction of max-logit contributions that exert a negative influence, using sign-mismatched contributions in the decomposition.Under the stated zero-bias assumption, scaling leaves the predicted class and this decomposition unchanged, so A(z) is independent of r.Source: §4 · p. 6
  • Across all backbones, A(z) tends to be smaller for ID than OOD samples, making a larger value a signal used to regularize the scaling factor.Figure 3 shows this distributional pattern across six backbones; SwinV2-B is omitted because its pattern resembles Swin-B.Source: §4 · p. 6, Figure 3 · p. 7
  • The regularization suppresses the scaling factor in proportion to sign-mismatched contributions, using only one test sample’s activation and classifier weight, without training data or gradient computation.λ controls the regularization strength.Source: §4 · p. 6

5 EXPERIMENTS

Under OpenOOD v1.5, the method performs strongly across the CIFAR-10 and ImageNet benchmarks, while ablations assess the Absolute Fix and Anti-coherence separately. The results also show a weaker CIFAR-100 setting and different gains across backbone types.

5.1 EXPERIMENTAL SETUP

  • The evaluation uses CIFAR-10 with ResNet-18 and ImageNet across ten convolutional and transformer backbones, comparing OOD detection metrics against post-hoc baselines and activation-shaping methods.CIFAR-10 includes far- and near-OOD datasets; ImageNet results average five OOD datasets per backbone.Source: §5.1 · p. 7

5.2 MAIN RESULTS

  • On ImageNet, the method achieves the lowest average FPR@95 across ten backbones and the highest average AUROC on six, while tying or nearly tying training-data-dependent methods in AUROC.The comparison averages results over five OOD datasets.Source: Table 2 · p. 8, §5.2 · p. 8
  • On CIFAR-10, the method achieves the highest average AUROC among baselines and the best overall average performance despite trailing some resource-intensive methods in FPR@95 on MNIST and Places365.The largest stated exception is AdaSCALE-L on Places365, by 1.50 points.Source: Table 3 · p. 8, §5.2 · pp. 8–9
  • On CIFAR-100, the method underperforms several baselines, while most methods show lower absolute performance than on CIFAR-10 or ImageNet with ResNet backbones.The authors suspect limited images per class contribute to this pattern.Source: §5.2 · p. 9

5.3 ABLATION STUDIES

  • Across ten ImageNet backbones with negative penultimate activations, the Absolute Fix improves over SCALE on every model and consistently improves over the ReLU-based correction.It leaves AUROC unchanged on DeiT3-B, which the authors attribute to fluctuations in activation density at large absolute values.Source: Table 4 · p. 9, §5.3 · p. 9
  • Anti-coherence improves every metric except FPR@95 on ViT-B/16 relative to SCALE+Abs, with larger improvements on backbones having negative penultimate activations.Backbones with only non-negative activations show comparatively modest gains.Source: Table 5 · p. 9, §5.3 · p. 9

6 CONCLUSION

The method combines an Absolute Fix for negative penultimate activations with Anti-coherence regularization, and is evaluated without training data or gradient computation. Under OpenOOD v1.5, it performs strongly against training-free methods and ties or outperforms resource-dependent methods on most backbones, while a gap remains against representation-based methods on negative-activation architectures.

6 CONCLUSION

  • The method achieves the best average AUROC and FPR@95 among training-free methods on every ImageNet backbone and the best average AUROC on CIFAR-10.It ties or outperforms methods relying on training data or gradients on most backbones in both settings, but trails representation-based methods on negative-activation architectures.Source: §6 · p. 10
  • Anti-coherence regularizes the scaling factor using the fraction of each sample’s max-logit contribution that drives the logit downward.The Absolute Fix computes the scaling factor over absolute-valued activations, preserving negative-activation magnitude.Source: §6 · p. 10
  • The authors report using generative AI tools for implementation assistance, related-work retrieval, proof checking, and editing, with author review of the assisted work.They state that the paper’s core ideas originated with the authors.Source: AI USE STATEMENT · p. 10
  • The evaluation uses established public benchmark datasets under the OpenOOD v1.5 protocol and includes no new data collection.The authors caution that OOD detection offers no strict coverage guarantee and should not be the sole safeguard in high-stakes deployment.Source: ETHICS STATEMENT · p. 10
  • The paper provides its proof assumptions, hyperparameter search spaces, checkpoint details, implementation details, and supplementary source code for reproducibility.It uses publicly available ImageNet-1K pretrained checkpoints and the OpenOOD v1.5 benchmark.Source: REPRODUCIBILITY STATEMENT · p. 10

A PROOF OF LEMMA 4.1

Lemma 4.1 establishes that increasing the scaling factor makes the max logit account for a progressively larger fraction of the energy score. Under the stated assumptions, that fraction is non-decreasing and approaches 1.

A PROOF OF LEMMA 4.1

  • The max-logit fraction ρ(r) of the energy score is non-decreasing in r and tends to 1 as r tends to infinity.The lemma assumes a unique positive maximum logit and zero classifier bias.Source: A PROOF OF LEMMA 4.1 · p. 15
  • As r increases, the contribution ξ(r) from non-maximal logits is non-increasing and remains bounded.With a unique maximizing class, the non-maximal terms vanish in the limit.Source: A PROOF OF LEMMA 4.1 · p. 15
  • The scaled maximum logit increases without bound, so the ratio of non-maximal contributions to it falls to zero.This establishes why the max-logit share of the energy score approaches 1.Source: A PROOF OF LEMMA 4.1 · p. 15

B ADDITIONAL ABLATIONS ON THE ABSOLUTE FIX

The Absolute Fix restores useful ID–OOD ordering of the scaling ratio across most tested backbones with negative penultimate activations, where the original ratio often fails. On ReLU-based backbones, Anti-coherence separates ID and OOD values, but its performance gain is modest because the scaling factor already provides sufficient separability.

B ADDITIONAL ABLATIONS ON THE ABSOLUTE FIX

  • After the Absolute Fix, ID samples have higher r than OOD samples on all six shown backbones except DeiT3-B; original r often fails to separate them or reverses the ordering.The figure compares original r in the top row with r_abs after the fix in the bottom row across six backbones.Source: Figure 4 · p. 16, B ADDITIONAL ABLATIONS ON THE ABSOLUTE… · p. 16
  • SCALE+Abs achieves the best average AUROC and FPR@95 across seven backbones, except that its average AUROC ties original SCALE on DeiT3-B.SCALE can underperform Energy by over 21 points in average AUROC on SwinV2-B; Table 6 reports AUROC/FPR@95 (%) on ImageNet under OpenOOD v1.5.Source: Table 6 · p. 17, B ADDITIONAL ABLATIONS ON THE ABSOLUTE… · p. 17

C VISUALIZATION OF THE ANTI-COHERENCE ON RELU-BASED BACKBONES

  • On ReLU-based backbones, OOD samples tend to have higher Anti-coherence A(z) than ID samples, but the performance gain over SCALE is modest.The paper attributes the limited gain to sufficient separability from the scaling factor under the ImageNet ID setting, leaving little additional signal for Anti-coherence.Source: C VISUALIZATION OF THE ANTI-COHERENCE… · p. 18, Figure 5 · p. 18

D COMPARISON WITH REPRESENTATION-BASED METHODS

Against representation-based methods requiring training statistics, the method performs best among those compared on non-negative-activation architectures, but still trails the strongest methods on architectures with negative penultimate activations. Closing that gap without training data remains an open direction for activation-shaping methods.

D COMPARISON WITH REPRESENTATION-BASED METHODS

  • Closing the performance gap on negative-activation architectures without relying on training data remains an important direction for activation-shaping methods.The comparison indicates that activation shaping remains strongest on architectures such as ResNet.Source: D COMPARISON WITH REPRESENTATION-BASED… · p. 18
  • On ResNet-50, ResNet-101, and MobileNetV2, the training-data-free method achieves the best performance among the methods compared, but it trails CADRef and Mahalanobis on negative-activation architectures.Table 7 reports AUROC/FPR@95, averaged over five ImageNet OOD datasets, against representation-based methods requiring training statistics.Source: Table 7 · p. 18, D COMPARISON WITH REPRESENTATION-BASED… · p. 18

E EXPERIMENTS OF HYPERPARAMETER SENSITIVITY

Anti-coherence strength affects AUROC differently depending on the backbone’s activation signs: negative-activation backbones are comparatively stable across the tested range, while positive-activation backbones can decline at higher strengths.

E EXPERIMENTS OF HYPERPARAMETER SENSITIVITY

  • On Swin-B, ViT-B/16, ConvNeXt-B, and BEiTv2-B, AUROC rises with λ and then changes gradually, while EfficientNet-B1 and MobileNetV2 peak at moderate λ and can decline afterward.The decline is most notable on Texture, suggesting excessive regularization can be detrimental on backbones dominated by positive activations.Source: E EXPERIMENTS OF HYPERPARAMETER… · p. 19, Figure 6 · p. 19

F.1 METHOD IMPLEMENTATIONS

This section specifies how the compared OOD methods shape activations or logits and score them, then describes implementation settings and validation-based hyperparameter selection.

F.1 METHOD IMPLEMENTATIONS

  • MSP scores the largest post-softmax class probability, MaxLogit uses the largest raw logit, and Energy aggregates logits with log-sum-exp at temperature T = 1.The cited methods are MSP, MaxLogit, and Energy (Liu et al., 2020).Source: §F.1 · p. 20
  • LTS applies the squared ReLU-based SCALE factor to logits, whereas VRA uses a two-sided activation function with lower and upper quantile bounds.LTS and VRA use energy scoring; VRA’s bounds come from pooled ID training activations.Source: §F.1 · pp. 20–21
  • ReAct clips activations above a training-feature percentile, while ASH-S prunes below a percentile and SCALE applies exponential rescaling without pruning.These are ReAct, ASH-S, and SCALE; each uses energy scoring.Source: §F.1 · pp. 20–21
  • OptFS learns a closed-form activation scaling vector from ID training statistics, while LAPS clips channels using asymmetric bounds based on per-channel and average ID statistics.OptFS groups activations into bins and applies each bin’s learned factor; LAPS tunes its base level and deviation weights.Source: §F.1 · pp. 21–22
  • AdaSCALE estimates a sample-specific percentile from gradient perturbations and an ID-validation reference distribution, then applies its factor to features or logits.AdaSCALE-A rescales features before recomputing logits; AdaSCALE-L scales the original logits by the factor squared.Source: §F.1 · p. 22
  • Anti-coh computes r_abs over absolute-valued penultimate activations and regularizes it using the sign-mismatched fraction of the predicted max logit’s contribution.The predicted class comes from the unscaled logit including bias; denominators are clamped at 10^-8, and the shaped activation is scored with energy.Source: §F.1 · p. 22
  • ImageNet backbones use ImageNet-1K-pretrained checkpoints, while CIFAR-10 and CIFAR-100 results use a single OpenOOD v1.5 checkpoint without standard deviations.Source: §F.1 · p. 22

F.2 HYPERPARAMETER SEARCH AND SELECTION

  • Hyperparameters are selected by validation AUROC on OpenOOD v1.5 validation splits containing ID and OOD samples, without using training data for selection.The section points to the search space and selected values for ImageNet and CIFAR-10 in Tables 8–10.Source: §F.2 · p. 23, Table 8 · p. 23, Table 9 · p. 23

G FULL PER-BACKBONE RESULTS

The appendix reports per-backbone OOD results and notes weaker CIFAR-100 performance, while identifying remaining gaps against representation-based methods and on fine-grained near-OOD data.

G FULL PER-BACKBONE RESULTS

  • CIFAR-100 results are noticeably lower than CIFAR-10 for most methods, and MNIST detection is substantially lower across methods despite CIFAR-100 having no digit-related classes.The authors suggest limited training images per class may leave penultimate activations insufficiently discriminative and take the MNIST pattern as evidence suggestive of inadequate feature information.Source: G FULL PER-BACKBONE RESULTS · p. 24, Table 11 · p. 24
  • Tables 12–17 report AUROC/FPR@95 results under OpenOOD v1.5 for ResNet-50, ResNet-101, Swin-B, SwinV2-B, ViT-B/16, and ConvNeXt-B.Source: Table 12 · p. 24, Table 13 · p. 25, Table 14 · p. 25
  • Tables 18–21 report AUROC/FPR@95 results under OpenOOD v1.5 for DeiT3-B, EfficientNet-B1, MobileNetV2, and BEiTv2-B.Source: Table 18 · p. 27, Table 19 · p. 28, Table 20 · p. 28

H LIMITATION

  • Despite requiring no training data, the method trails the strongest representation-based methods on backbones with negative activations and underperforms several baselines on CIFAR-100.The authors also report lower performance on SSB-Hard and identify fine-grained near-OOD detection as future work.Source: H LIMITATION · p. 29

Related papers

Loading 2610.32255v2…