PaperlayerView exact revision on arXiv

Source-linked AI summary

MOOD: Multi-level Out-of-distribution Detection

Ziqian Lin, Sreya Dutta Roy, Yixuan Li

arXiv:2104.14726v1cs.CV

TL;DR

Existing OOD detection methods often require a full network pass, imposing fixed computation. MOOD uses intermediate classifiers and complexity-based exits; across 10 OOD datasets, it reports better detection than final-output approaches while reducing computation by up to 71.05%.

  • Problem

    Existing OOD detectors commonly score penultimate- or output-layer statistics, requiring a full feedforward pass and fixed computation for each test input.

    Source: §1 · p. 1

  • Method

    MOOD attaches OOD detectors to intermediate classifiers, selects exits using input complexity, and applies an adjusted energy score at each exit.

    Source: §1 · pp. 1–2

  • Results

    Across 10 OOD datasets, MOOD reports better OOD detection performance than approaches relying on final outputs and reduces computations by up to 71.05%.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    Easy OOD examples can be detected at early layers while harder examples continue deeper, with computation adaptable to input complexity and changing budgets.

    Source: §1 · p. 1

Abstract

from arXiv · show

Out-of-distribution (OOD) detection is essential to prevent anomalous inputs from causing a model to fail during deployment. While improved OOD detection methods have emerged, they often rely on the final layer outputs and require a full feedforward pass for any given input. In this paper, we propose a novel framework, multi-level out-of-distribution detection MOOD, which exploits intermediate classifier outputs for dynamic and efficient OOD inference. We explore and establish a direct relationship between the OOD data complexity and optimal exit level, and show that easy OOD examples can be effectively detected early without propagating to deeper layers. At each exit, the OOD examples can be distinguished through our proposed adjusted energy score, which is both empirically and theoretically suitable for networks with multiple classifiers. We extensively evaluate MOOD across 10 OOD datasets spanning a wide range of complexities. Experiments demonstrate that MOOD achieves up to 71.05% computational reduction in inference, while maintaining competitive OOD detection performance.

1. Introduction

The introduction frames fixed full-pass OOD scoring as costly, then proposes MOOD’s multi-exit, complexity-based framework and adjusted per-exit energy scoring.

1. Introduction

  • Existing OOD scores use penultimate or output-layer statistics and fixed full passes; overparameterized networks amplify inference demand, while prior resource-efficient methods target ID classification.The paper links higher computational cost to latency in safety-critical applications such as self-driving cars.Source: §1 · pp. 1–2
  • MOOD places OOD detectors at intermediate network depths, enabling dynamic inference that can use early layers for easy examples and deeper layers for complex ones.The framework also aims to let computation adjust to changing budgets or different devices.Source: §1 · p. 1
  • Its complexity-based exit strategy uses model-agnostic scores to select exits, with early exits favored for easier OOD data and deeper exits for more complex data.The paper notes that this is harder than choosing exits for ID classification because OOD data are unseen during training.Source: §1 · pp. 1–2
  • MOOD uses an adjusted energy score at each intermediate classifier to mitigate energy-score fluctuations and improve comparability across exits.The paper reports empirical and mathematical support for the score’s suitability with multiple classifiers at different depths.Source: §1 · p. 2
  • Across 10 OOD datasets, MOOD reduces computation by up to 71.05% while maintaining competitive OOD detection performance.The reported average computation deduction is 22.02% across the datasets.Source: §1 · p. 2

2. Method

MOOD combines intermediate OOD detectors, complexity-based exit selection, and adjusted energy scoring to make OOD inference adaptive across network depths.

2. Method

  • MOOD combines intermediate OOD detectors, complexity-based exit selection, and exit-wise OOD scoring into an adaptive inference framework.For inputs classified as in-distribution, it also classifies using the current exit; inference is described as parameter-free.Source: §2 · p. 2, §2.1 · p. 3, §2.2 · p. 3

2.1. OOD Detector at Early Exits

  • The network attaches k−1 intermediate classifiers to a CNN, with an OOD detector at each exit and thresholds chosen using in-distribution data.A threshold can be set so that a high fraction, such as 95%, of in-distribution data is correctly classified by the detector; the framework adds no parameters beyond the classification network.Source: §2.1 · p. 3

2.2. Complexity-based Exit for OOD Detection

  • MOOD estimates image complexity with compressed-file size and assigns each input to an exit according to its normalized complexity range.The exit function maps normalized complexity into k levels, capped at the final exit.Source: §2.2 · p. 3

2.3. OOD Scoring Function

  • MOOD adjusts each exit’s energy score by its in-distribution mean to make scores more comparable across classifiers.This supports using one threshold across exits, chosen so that 95% of in-distribution data scores above it.Source: §2.3 · pp. 4–5

3. Experiments

The experiments evaluate MOOD’s OOD detection performance and computational cost, including ablations of different algorithmic components.

3. Experiments

  • The experiments assess MOOD’s OOD detection performance and computational cost, then use ablations to examine different aspects of the algorithm.The following sections cover experimental setup, main performance results, and ablation analysis.Source: §3 · p. 5

3.1. Setup

The evaluation uses CIFAR-10 and CIFAR-100 as in-distribution datasets and ten OOD datasets spanning a range of image complexities. It measures inference computation, OOD detection, and ID classification performance.

3.1. Setup

  • MOOD is evaluated with CIFAR-10 and CIFAR-100 as in-distribution datasets, using the standard 50,000-image training and 10,000-image test splits.All images are 32 × 32.Source: §3.1 · p. 5
  • The OOD evaluation covers 10 datasets ordered by increasing image complexity, from MNIST and K-MNIST through iSUN and LSUN (resize).Images are resized to 32 × 32, and each dataset’s full test split is used.Source: §3.1 · p. 5
  • Evaluation metrics are inference-time FLOPs, FPR95 on OOD data at a 95% ID true-positive rate, and AUROC.Source: §3.1 · p. 5
  • The classification model uses the default MSDNet with five blocks and four layers per block, trained for 300 epochs with Gradient Equilibrium.Training follows the settings in [16].Source: §3.1 · p. 5
  • Model-capacity experiments vary the number of layers per block across 2, 3, 4, 6, 8, and 12.Source: §3.1 · p. 6

3.2. Results

Across ten OOD datasets, MOOD’s dynamic exits reduce computation while achieving better OOD detection than final-exit baselines. Results and ablations examine exit choice, scoring functions, capacity, ID classification, and alternative exit strategies.

3.2.1 How does MOOD compare with existing OOD detection methods?

  • 22.02% average FLOPs reduction across 10 OOD datasets, with up to 71.05% reduction on the low-complexity dataset, accompanies better OOD detection than final-exit baselines.The comparison uses the same MSDNet network; the reported results are averaged over ten datasets.Source: §3.2.1 · p. 6
  • 0.79 × 10^8 FLOPs for MOOD accompanies competitive OOD detection performance, compared with 4.09 × 10^9 FLOPs for Glow-based models and 2.78 × 10^10 FLOPs for the PixelCNN-based model.The comparison is with generative-based OOD detection approaches.Source: §3.2.1 · p. 6, Table 2 · p. 6

3.2.2 What is the effect of complexity-based dynamic exiting strategy?

  • Early exits are more frequent for less complex OOD datasets such as MNIST, while deeper exits occur more often for complex datasets such as iSUN.Dynamic exiting achieves OOD detection performance comparable to the best constant exit across all ten datasets, while reducing FLOPs and maintaining high ID classification accuracy.Source: §3.2.2 · p. 6, Table 3 · p. 6

3.2.3 Is MOOD compatible with other scoring functions?

  • Adjusted energy scoring gives optimal average AUROC of 91.29% on CIFAR-10, and MOOD consistently improves over final-output MSDNet Exit@last regardless of scoring function.MOOD also supports MSP and ODIN; ODIN performs slightly better on CIFAR-100 but requires hyperparameter tuning, whereas adjusted energy is parameter-free.Source: §3.2.3 · pp. 6–7, Table 4 · p. 7

3.2.4 How does model capacity affect MOOD performance?

  • As MSDNet depth increases, MOOD’s improvement over the last-layer baseline becomes more pronounced, supporting its applicability across model capacities.The experiments vary steps per block with five blocks fixed, and separately vary the number of blocks.Source: §3.2.4 · p. 7, Figure 4 · p. 7

3.2.5 How does MOOD affect in-distribution classification accuracy?

  • MOOD uses the complexity-based exit for ID test inputs too, applying the corresponding classifier’s predictions while maintaining ID classification accuracy.The paper presents this alongside OOD safety and computational efficiency as combined framework properties.Source: §3.2.5 · p. 7

3.2.6 Comparison with Greedy-based and Random Exit Strategy

  • The greedy exit strategy has 9.26% higher FPR95 than complexity-based exiting on MSDNet with five blocks; randomized exits substantially reduce both OOD detection and ID classification performance.The greedy strategy uses exit-specific thresholds chosen so that 95% of ID data is correctly classified, while the model is not trained with OOD data.Source: §3.2.6 · p. 7, Table 5 · p. 7

4. Related Work

Prior adaptive inference methods largely target in-distribution tasks, while MOOD focuses on adaptive OOD detection using intermediate classifiers. Earlier intermediate-layer OOD approaches do not make adaptive exit decisions.

4. Related Work

  • Prior adaptive inference approaches use multiple paths, branches, gating, or learned policies to reduce computation, but were designed mainly for ID classification or segmentation.MOOD instead adds intermediate OOD detectors at different network depths.Source: §4 · p. 8
  • Existing OOD scoring methods commonly use final or penultimate layer outputs, while some intermediate-feature or output methods lack adaptive exit decisions and retain the computational bottleneck.The cited intermediate methods include Mahalanobis distances [28] and early-layer detectors requiring auxiliary training data [1].Source: §4 · p. 8
  • Prior work used input complexity for novelty detection or generative-model OOD scoring, whereas MOOD connects complexity to the appropriate exit in dynamic inference.Compression algorithms such as PNG, JPEG2000, and FLIF can proxy for intractable Kolmogorov complexity.Source: §4 · p. 8

5. Conclusion

MOOD dynamically uses early classifier outputs to detect easier OOD examples sooner while sending harder examples deeper. Across 10 datasets, it improves OOD detection over final-output approaches while reducing computation by up to 71.05%.

5. Conclusion

  • MOOD dynamically exploits early classifier outputs, detecting easy OOD examples earlier while propagating hard examples to deeper layers.The paper relates OOD data complexity to the optimal exit level.Source: §5 · p. 8
  • Across 10 OOD datasets spanning varied complexities, MOOD achieves better OOD detection performance than approaches relying on final outputs.Source: §5 · p. 8
  • MOOD reduces computation by up to 71.05% while maintaining competitive OOD detection performance.Source: §5 · p. 8

A. Additional Information of Datasets

The dataset appendix describes test-set use and preprocessing, and reports dataset complexities based on PNG-compressed byte counts. It also identifies the hardware and timing used for compression and inference measurements.

A. Additional Information of Datasets

  • The experiments use each dataset’s entire test split, with OOD inputs normalized using in-distribution means and standard deviations.Source: A. Additional Information of Datasets · p. 11
  • Preprocessing pads MNIST, fashion-MNIST, and K-MNIST, resizes and center-crops several image datasets to 32×32, and creates LSUN crop and resize variants.Compression latency is measured on an Intel i7-7820X CPU; average inference time is 20 ms per sample on an Nvidia 1080 Ti GPU.Source: A. Additional Information of Datasets · p. 11, Table 6 · p. 11
  • The 12 datasets are listed in increasing complexity, measured in bytes after PNG compression.Source: Table 6 · p. 11

B. Experiment on JPEG2000

The appendix evaluates JPEG2000 as an alternative lossless compressor for MOOD and presents complexity distributions across 12 datasets. JPEG2000 provides competitive OOD detection results relative to PNG, but uses more inference time.

B. Experiment on JPEG2000

C. Detailed Results for 10 OOD Datasets

The appendix reports detailed evaluations across 10 OOD datasets, including both fixed-exit and MOOD dynamic-exit results. The reported metrics are AUROC and FPR@95.

C. Detailed Results for 10 OOD Datasets

Related papers

Loading 2104.14726v1…