PaperlayerView exact revision on arXiv

Source-linked AI summary

Inhibited Softmax for Uncertainty Estimation in Neural Networks

Marcin Możejko, Mateusz Susik, Rafał Karczewski

arXiv:1810.01861v2cs.LGstat.ML

TL;DR

Overconfident neural networks motivate uncertainty estimation and out-of-distribution detection. The paper extends softmax with Inhibited Softmax, and reports that it outperforms baselines, performs comparably to other methods, and does not deteriorate classifier predictive performance.

  • Problem

    Neural networks can be overconfident, motivating uncertainty estimates that can help address potentially harmful misclassifications and detect out-of-distribution inputs.

    Source: §1 · p. 1

  • Method

    Inhibited Softmax extends a softmax output layer with an additional output used to represent uncertainty, without requiring additional parameters or multiple stochastic forward passes.

    Source: §1 · p. 1, §7 · p. 9

  • Results

    The method outperforms baseline methods, performs comparably to other methods, and does not deteriorate classifier predictive performance.

    Source: §7 · p. 9

  • Takeaways & Limitations

    In the IMDB/Movie Reviews experiment, the classifier can remain useful when an observation comes from another probability distribution and the uncertainty measure detects it.

    Source: §7 · p. 9

    The authors identify applying the method to larger architectures and more varied tasks as further work, and say a full mathematical explanation of the architectural adjustments—especially aleatoric-uncertainty modelling—is still lacking.

    Source: §6 · p. 7

Abstract

from arXiv · show

We present a new method for uncertainty estimation and out-of-distribution detection in neural networks with softmax output. We extend softmax layer with an additional constant input. The corresponding additional output is able to represent the uncertainty of the network. The proposed method requires neither additional parameters nor multiple forward passes nor input preprocessing nor out-of-distribution datasets. We show that our method performs comparably to more computationally expensive methods and outperforms baselines on our experiments from image recognition and sentiment analysis domains.

1 Introduction

The introduction motivates uncertainty estimation as a response to overconfident predictions and their potential harms, and presents Inhibited Softmax as a way to estimate uncertainty and detect out-of-distribution inputs. It also previews the paper’s rationale, architectural improvements, and benchmarks.

1 Introduction

  • Overconfident predictions can cause harmful misclassifications, while uncertainty estimates are also useful when active-learning data collection is costly or time-consuming.The introduction cites misclassified traffic signs and an image tagger that classified two African Americans as gorillas as examples.Source: §1 · p. 1
  • Because uncertainty methods are rarely adopted, the introduction argues for a method without additional computational overhead that could suit settings focused on quick training or inference.The authors suggest that simplicity and ease of implementation could encourage practitioners to develop danger-aware systems.Source: §1 · p. 1
  • Inhibited Softmax is proposed for uncertainty estimation and out-of-distribution detection, with experiments reported to outperform baselines and match more computationally expensive methods on out-of-distribution detection.The introduction previews a mathematical explanation, additions to improve uncertainty approximation, and benchmarks against baseline and contemporary methods.Source: §1 · p. 1

2 Related Work

Related work spans confidence scores, Bayesian and other uncertainty methods, and approaches to distributional uncertainty. The paper positions its method against these approaches using criteria that avoid added parameters, repeated passes, extra out-of-distribution data, and input preprocessing.

2 Related Work

  • Maximum class probability has been used to represent certainty [Hendrycks and Gimpel, 2017], but deep neural networks can be overconfident [Guo et al., 2017].The authors note that this makes maximum probability potentially inadequate as a certainty measure.Source: §2 · p. 1
  • Bayesian neural networks model uncertainty using distributions over weights and predictive entropy or mutual information from stochastic predictions [Blundell et al., 2015, Hern´andez-Lobato and Adams, 2015, Louizos and Welling, 2017, Malinin and Gales, 2018, Wang et al., 2016, Hafner et al., 2018, Zhang et al., 2018, Khan et al., 2018; Buntine and Weigend, 1991, Neal, 1993; Gal and Ghahramani, 2015, 2016; Leibig et al., 2017; Smith and Gal, 2018].Other approaches include ensembles, student networks, GANs, Monte Carlo Batch Normalization, and nearest-neighbour analysis [Lakshminarayanan et al., 2017; Korattikara Balan et al., 2015; Wang et al., 2018; Teye et al., 2018; Mandelbaum and Weinshall, 2017].Source: §2 · pp. 1–2
  • Distributional uncertainty concerns test observations whose distribution differs from training, and a common benchmark distinguishes the original test set from out-of-distribution data [Malinin and Gales, 2018; Hendrycks and Gimpel, 2017].ODIN uses gradient-based input preprocessing [Liang et al., 2018], while DeVries and Taylor add one densely connected layer and use one forward pass [DeVries and Taylor, 2018].Source: §2 · p. 2
  • The method meets the stated criteria: it requires no additional learnable parameters, no extra out-of-distribution or adversarial observations, no input preprocessing, and only a single forward pass.Another work meeting these criteria is based on a Dirichlet interpretation of the softmax output [Sensoy et al., 2018] and subjective logic, replacing softmax with a ReLU layer to model belief masses within the posterior distribution.Source: §2 · p. 2

3 Inhibited Softmax

Inhibited Softmax adds an output interpreted as a certainty factor, with the training objective linking that factor to the training distribution. The authors modify the network architecture and regularize activations to address trivial solutions and improve uncertainty estimation.

3 Inhibited Softmax

  • Inhibited Softmax maps an input vector x and scalar a to n outputs, with the standard softmax S(x) used to define the certainty factor.The authors introduce a as an artificial softmax output from the additional channel and describe it as minimized during optimization.Source: §3 · pp. 2–3
  • The Inhibited Softmax objective combines classification loss with maximizing the certainty factor for training examples, motivating its use as an uncertainty estimator.The certainty factor is identified as P^c_a(x); the loss also includes standard-softmax cross-entropy.Source: §3 · p. 3

3.1 Adjustments and Regularisation

  • The authors identify a trivial solution: increasing all output biases can raise the certainty factor without changing standard-softmax classification loss, motivating removal of biases from the inhibited-softmax layer.They also state that the training process should increase certainty because of training examples, not trivial solutions or accidental network structure.Source: §3.1 · p. 3
  • Kernel activations in the penultimate layer are intended to be large only near a learnable region W, leaving out-of-distribution inputs outside that region with a constant base-rate certainty.Evidence regularization adds a penalty based on the l1 norm of penultimate-layer activations, encouraging training examples into W; the authors report these adjustments significantly improve certainty estimation.Source: §3.1 · pp. 3–4

4 Experiments

The experiments assess uncertainty on an XOR toy task and benchmark methods on out-of-distribution detection, predictive performance, and wrong-prediction detection. In selected computer-vision and sentiment tasks, Inhibited Softmax (IS) improves on baselines, while predictive accuracy remains similar to baseline performance.

4 Experiments

  • On the XOR toy task, IS estimates aleatoric uncertainty between and overlapping classes; greater noise widens uncertain regions, while little overlap yields high confidence.The network has four hidden layers of 100 units and uses 500 training samples.Source: §4 · p. 4, Figure 1 · p. 4

4.1 Benchmarks

  • The benchmarks compare methods on shared base networks using ROC AUC and average precision for OOD detection, and accuracy and negative log loss for predictive performance.Wrong-prediction detection uses ROC AUC; average precision is omitted there because differing misclassification rates would distort it.Source: §4.1 · pp. 4–5
  • 0.973 ROC AUC vs 0.958 on NOTMNIST and 0.978 vs 0.956 on Omniglot: IS beats BASE, and its ROC AUC is similar to MCD on two other vision task pairs.IS records 0.973 vs 0.974 on MNIST/NOTMNIST and 0.931 vs 0.927 on CIFAR10/SVHN; all Bayesian methods outperform baselines on computer-vision tasks.Source: Table 3 · p. 6, §4.1 · p. 6
  • 0.877 ROC AUC for IS vs 0.836 for MCD on Movie Reviews; IS also leads on Reuters-21578 (0.829) and Customer Reviews (0.736).The authors report that methods other than IS are near or below the baseline on sentiment analysis.Source: §4.1 · p. 6, Table 3 · p. 6
  • 0.837 ROC AUC for BASE on full Movie Reviews vs 0.94 in Hendrycks and Gimpel (2017): the authors report detection is tougher with full reviews.The earlier setup used single lines from the dataset file, whereas the reviews span a few lines.Source: §4.1 · p. 6
  • IS does not significantly deteriorate predictive performance; its accuracy is similar to baselines, including 0.2% lower on IMDB and 0.5% lower on CIFAR-10.Only the baseline and IS require neither additional parameters nor multiple forward passes; ensembling gives the best predictive performance.Source: §4.1 · p. 6, Table 4 · p. 7
  • 0.689 ROC AUC: IS alone detects sentiment misclassifications better than random, while IS and MCD trail the baseline on CIFAR-10 (0.869 and 0.85 vs 0.875).On MNIST, every method slightly improves over baseline; DE scores 0.987 and BBP 0.979.Source: §4.1 · p. 6, Table 5 · p. 7

5 Visualisation

The latent-space comparison finds substantial overlap between Inhibited Softmax and Bayesian uncertainty estimates, while both methods miss some out-of-distribution regions. The authors suggest investigating samples that may be difficult for recently proposed uncertainty measures to detect.

5 Visualisation

  • Both methods miss out-of-distribution samples in the bottom-left latent-space region and show no high uncertainty where the blue and purple classes intersect.The comparison uses samples generated across a two-dimensional MNIST VAE latent space.Source: §5 · p. 7
  • In the visualization, Inhibited Softmax and Monte Carlo Dropout show similar uncertainty regions across the VAE latent space.The figure overlays uncertainty on generated samples with the encoded test set and class colors.Source: §5 · p. 7
  • The authors hypothesize that some samples may be difficult for any recently proposed uncertainty measure to detect, motivating investigation of how to construct them.They suggest this may be a way to improve uncertainty-sampling performance.Source: §5 · p. 7

6 Further Work & Limitations

The authors identify extensions to regression, hyperparameter reduction, larger architectures and broader tasks as future directions. They also note that the mathematical effects of architecture adjustments, especially on alleatoric uncertainty, remain incompletely explained.

6 Further Work & Limitations

  • An analogous Inhibited Softmax method for regression remains a future direction.Source: §6 · p. 7
  • Reducing the number of required Inhibited Softmax hyperparameters is another proposed direction.Source: §6 · p. 7
  • On deeper networks, Inhibited Softmax has not yet shown the advantage it has over other methods on a shallow sentiment-analysis network.The authors suggest hidden-layer extensions as a possible direction.Source: §6 · p. 7
  • Applying the method to larger architectures and a wider range of tasks, including sophisticated NLP and reinforcement learning, remains future work.Source: §6 · p. 7
  • A full mathematical explanation is still lacking for how architecture adjustments influence model behavior, particularly Inhibited Softmax’s alleatoric-uncertainty capability.The paper points to Figure 1 and Table 5 in connection with this capability.Source: §6 · p. 7
  • Figure 3 maps latent-space coordinates and class-colored test points over grayscale uncertainty, with lighter shades indicating greater uncertainty.It compares Inhibited Softmax (top) with Monte Carlo Dropout (bottom) and notes similar regions.Source: Figure 3 · p. 8

7 Conclusion & Discussion

The paper reframes Inhibited Softmax as an uncertainty-estimation method that can be applied to multilayer networks without extra parameters, multiple stochastic forward passes, or OOD examples. Its reported results outperform baselines, are comparable to other methods, and do not deteriorate classifier predictive performance.

7 Conclusion & Discussion

  • Inhibited Softmax requires no additional parameters, multiple stochastic forward passes, or out-of-distribution examples and is readily applicable to multilayer neural-network architectures.Source: §7 · p. 9
  • The method outperforms the baseline, performs comparably to other methods, and does not deteriorate the classifier’s predictive performance.Source: §7 · p. 9
  • IMDB/Movie Reviews results suggest a network can remain a useful classifier when uncertainty detects an observation from another probability distribution.Conversely, an accurate classifier may still show uncertainty in its predictions.Source: §7 · p. 9

Appendix 1 - Ablation Study

The ablation study finds that evidence regularization matters, but excessive regularization harms CIFAR-10 results. It also compares alternative kernel functions and reports that non-kernel activations such as ReLU do not correctly perform OOD detection.

Appendix 1 - Ablation Study

  • Removing evidence regularization worsens all checked tasks except MNIST wrong-prediction detection, while too much regularization drops all CIFAR-10 experiment results.The reported results average three experimental runs.Source: Appendix 1 - Ablation Study · p. 11
  • The kernel comparison includes a rescaled Gaussian PDF and a custom function equal to min(x + 1, −x + 1) for |x| < 1 and 0 otherwise.Source: Appendix 1 - Ablation Study · p. 11
  • Non-kernel activation functions such as ReLU are not able to correctly perform the out-of-distribution detection tasks.Source: Appendix 1 - Ablation Study · p. 11

Appendix 2 - Experiments details & Preprocessing

The appendix records preprocessing and implementation choices for the sentiment and comparison experiments. It also discusses a limitation in the baseline publication’s Movie Reviews setup.

Appendix 2 - Experiments details & Preprocessing

  • For sentiment analysis, preprocessing removed stopwords and words absent from the pretrained embedding, which represents vocabulary found in OOD sets but not the in-distribution data.The embedding was used to model vocabulary present in out-of-distribution sets but absent from the in-distribution dataset.Source: Appendix 2 - Experiments details &… · p. 11
  • The authors argue that average pooling over zero-padded Movie Reviews lines can make the baseline uncertainty method need to work only near zero to score highly.They report corroborating the baseline publication’s IMDB/Movie Reviews results after splitting reviews into lines and using randomly initialized embeddings; the model was trained on full IMDB reviews.Source: Appendix 2 - Experiments details &… · p. 11
  • Deep Ensembles used five base networks, without adversarial training, which the authors describe as a way to improve any method in the paper.The paper says ensemble size and inference counts for comparison methods follow original algorithm descriptions when possible.Source: Appendix 2 - Experiments details &… · pp. 11–12
  • Monte Carlo Dropout used 50 forward passes for variational prediction, with dropout probabilities and layers varying by MNIST, sentiment, and CIFAR-10.For larger networks, applying dropout to many layers required more epochs to achieve top performance.Source: Appendix 2 - Experiments details &… · pp. 11–12
  • Bayes By Backprop used 10 forward passes, with variance initialization chosen for the authors’ preferred combination of accuracy and OOD detection.The authors observed a trade-off between accuracy and OOD detection that depended on variance initialization.Source: Appendix 2 - Experiments details &… · p. 12

Appendix 3 - Visualization

The visualization appendix explains how uncertainty maps are normalized for display and pairs an uncertainty visualization with images decoded from a latent-space grid.

Appendix 3 - Visualization

  • Predictive entropy and Inhibited Softmax probabilities are ranked into 400 equal bins so their uncertainties can be visually compared.The bins are plotted after ranking the uncertainties for each method.Source: Appendix 3 - Visualization · p. 12
  • In the normalized maps, white marks the most-uncertain bin and black the least-uncertain bin.Source: Appendix 3 - Visualization · p. 12
  • Figure 6 places the uncertainty-measure visualization beside images generated by decoding a grid in latent space.The decoded grid images are presented to aid understanding of the latent space.Source: Figure 6 · p. 12, Appendix 3 - Visualization · p. 12

Appendix 4 - Mathematical derivations

The mathematical appendix states assumptions for lemmas involving the Inhibited Softmax activation and softmax cross-entropy. The following no-evidence-regularisation comparison reports that this variant still beats the baseline on OOD detection, but does not match the other benchmark methods overall.

Appendix 4 - Mathematical derivations

Appendix 5 - No evidence regularisation scenario

  • Without evidence regularisation, Inhibited Softmax still outperforms the baseline on OOD detection, although its overall benchmark performance does not match the other methods.The comparison covers OOD detection and predictive performance; Table 6 identifies ROC AUC and accuracy as their respective metrics.Source: Table 6 · p. 13, Appendix 5 - No evidence… · p. 13

Related papers

Loading 1810.01861v2…