Source-linked AI summary
Reducing Network Agnostophobia
Akshay Raj Dhamija, Manuel Günther, Terrance E. Boult
TL;DR
Unknown inputs can trigger confident false positives because network behavior outside the training set is poorly defined. The paper introduces Entropic Open-Set and Objectosphere losses that use background samples to shape entropy and feature magnitudes. Experiments on MNIST and CIFAR-10 report better handling of unknown inputs from several unseen datasets.
Problem
Deep networks must reject unknown inputs, but their behavior far from the training set is not well defined and standard rejection approaches can be insufficient for unseen classes.
Method
The paper trains Entropic Open-Set and Objectosphere losses that maximize unknown-sample entropy and separate known and unknown representations through feature magnitudes.
Results
The novel losses are reported to improve robustness to out-of-distribution samples and advance open-set image classification across the evaluated settings.
Takeaways & Limitations
Improving the network representation, rather than only modeling rejection at the output, provides a theoretically grounded approach to unknown-class recognition.
Takeaways & Limitations
Background-class methods rely on training samples being representative of unknown test inputs, an assumption that may fail in the real world’s near-infinite negative space.
Abstract
from arXiv · showhide
Agnostophobia, the fear of the unknown, can be experienced by deep learning engineers while applying their networks to real-world applications. Unfortunately, network behavior is not well defined for inputs far from a networks training set. In an uncontrolled environment, networks face many instances that are not of interest to them and have to be rejected in order to avoid a false positive. This problem has previously been tackled by researchers by either a) thresholding softmax, which by construction cannot return "none of the known classes", or b) using an additional background or garbage class. In this paper, we show that both of these approaches help, but are generally insufficient when previously unseen classes are encountered. We also introduce a new evaluation metric that focuses on comparing the performance of multiple approaches in scenarios where such unseen classes or unknowns are encountered. Our major contributions are simple yet effective Entropic Open-Set and Objectosphere losses that train networks using negative samples from some classes. These novel losses are designed to maximize entropy for unknown inputs while increasing separation in deep feature space by modifying magnitudes of known and unknown samples. Experiments on networks trained to classify classes from MNIST and CIFAR-10 show that our novel loss functions are significantly better at dealing with unknown inputs from datasets such as Devanagari, NotMNIST, CIFAR-100, and SVHN.
1 Introduction and Problem Formulation
Deep networks must reject unwanted and unknown inputs, but standard softmax and background-class strategies remain vulnerable to unseen classes. The paper introduces losses that improve feature robustness and proposes an evaluation metric for unknown-sample settings.
- Computer-vision systems must handle unwanted or unknown inputs in addition to recognizing target classes.
- Unknown classes comprise an infinite space, while training can expose the network to only a small subset of background or known-unknown classes.
- The paper introduces Entropic Open-Set and Objectosphere losses to develop deep features that are more robust to unknown inputs.
- Entropic Open-Set maximizes entropy for background samples, while Objectosphere additionally reduces unknown-sample feature magnitude and improves softmax-based rejection.
- The authors also propose an evaluation metric for comparing approaches when unknown samples are present and report advances in open-set image classification.
2 Background and Related Work
Prior approaches address rejection through softmax thresholds, uncertainty estimation, open-set modeling, or background classes, but their assumptions can fail for unseen unknowns. The paper positions its method as a feature-representation improvement beyond these partial solutions.
- Open-set, outlier-rejection, out-of-distribution, selective-prediction, and uncertainty-estimation methods provide partial solutions for rejecting unknown samples.
- Softmax-thresholding methods reject samples when their maximum known-class score is low, but they do not directly model an unknown class.
- Uncertainty ensembles reduce maximum known-class probabilities for unknown inputs, whereas OpenMax estimates unknown probability from deep-feature class models after training.
- Background-class methods assume training background samples sufficiently represent unknown test inputs, an assumption that may fail in the near-infinite real-world negative space.
- The paper reports that its novel approach advances the state of the art in open-set image classification.
3 Visualizing Deep Feature Responses to Unknown Samples
Visualizations compare standard softmax, background-class training, and Objectosphere features when MNIST networks encounter Devanagari unknowns. Objectosphere produces clearer feature separation and more effective score-based rejection.
- LeNet++ maps MNIST samples and Devanagari unknowns into a two-dimensional deep feature space for visualizing network responses.
- With standard softmax, Devanagari features overlap MNIST features, and most unknowns receive a high score for a known class.
- Training with NIST letters as a background class places many Devanagari samples in the background region, but others still overlap known classes and receive high predictions.
- Objectosphere maps background samples toward the origin while pushing MNIST digit representations farther away, producing clearer separation from Devanagari unknowns.
4 Approach
The approach trains networks to separate known and unknown inputs using entropy and deep-feature magnitude, while introducing OSCR to evaluate open-set performance. Objectosphere strengthens this separation by imposing a magnitude margin, addressing limitations of background-class training and existing evaluation metrics.
- Background-class training constrains all unknown samples to one feature-space region, regardless of their similarity to known classes.
- Objectosphere Loss: Objectosphere reduces unknown feature magnitudes while requiring known samples to exceed a minimum magnitude, creating a margin in magnitude and entropy.This permits unknown samples to share features with known classes when their feature magnitudes remain small.
- Entropic Open-Set Loss: Entropic Open-Set loss equalizes logits for background samples, producing uniform softmax scores and maximum entropy.The loss preserves standard softmax training for known samples while modifying training on background samples.
- Objectosphere Loss: The Entropic Open-Set loss can leave modest overlap between known and unknown feature magnitudes because it does not constrain known samples to be large.Objectosphere addresses this by pushing known samples toward large feature magnitudes.
- Objectosphere Loss: Objectosphere provides improved feature-magnitude separation over Entropic Open-Set loss, while its margin parameter ξ can affect feature scaling and learning rate.The paper recommends determining ξ using cross-class validation.
- Evaluating Open-Set Systems: OSCR evaluates correct classification of known samples against false positives on unknown samples using CCR and FPR across score thresholds.Unlike several alternatives, its DIR axis is computed solely from known samples, reducing dataset-bias concerns when unknown-sample counts differ.
5 Experiments
The experiments compare the proposed Entropic Open-Set and Objectosphere losses with background-class, softmax-thresholding, and OpenMax approaches across two network architectures and multiple unknown-data settings.
- LeNet++: LeNet++ experiments use a two-dimensional feature representation and compare the proposed losses with background class, softmax thresholding, and OpenMax.The paper reports that the proposed methods significantly outperform OpenMax in Fig. 1(d).
- Experimental design: The experiments evaluate whether the proposed losses improve rejection of unknown samples across distinct architectures and datasets.The study compares two network architectures and multiple sources of unknown inputs.
- ResNet-18: ResNet-18 classifies CIFAR-10 using a 1024-dimensional feature layer and tests CIFAR-100-derived unknowns together with Street View House Numbers.CIFAR-100 superclasses are split between background training samples and unknown test samples.
6 Discussion and Conclusion
The discussion links improved open-set performance to higher entropy and lower feature magnitudes for unknowns, while noting dependence on architecture, training unknowns, and Objectosphere hyperparameters.
- Findings: Unseen classes generally produce low feature magnitude and high softmax entropy, which the proposed losses amplify to improve robustness to out-of-distribution samples.Table 1 supports the predicted entropy and feature-magnitude behavior, while Table 2 reports robustness using CCR at different FPR values.
- Evaluation: The OSCR evaluation reports Correct Classification Rates at varying False Positive Rates for comparing open-set systems.Its axes use CCR and FPR rather than a single operating point.
- Limitations: Objectosphere training is more complicated because it requires balancing the loss with λ and selecting ξ, the minimum feature magnitude for known samples.The additional magnitude restriction increases training complexity and computational cost.
- Architecture dependence: Performance depends on architecture: some LeNet++ initializations favor Entropic Open-Set, while Scaled-Objectosphere beats the background class at very low FPR for ResNet-18.The paper attributes the LeNet++ variation partly to its narrow two-dimensional feature space.
- Training unknowns: The unknown samples used during training matter: NIST Letters improve robustness to CIFAR images, whereas CIFAR training unknowns do not robustly reject NIST Letters.The paper connects this asymmetry to the visual similarity of NIST Letters to MNIST digits and the role of hard negatives.
- Conclusion: The paper presents theoretically grounded losses and an evaluation metric as steps toward improved representations for unknown classes.The stated contribution targets reducing network agnostophobia through improved network representations.
Supplementary Material: Reducing Network Agnostophobia
For unknown inputs, the Entropic Open-Set loss is minimized when softmax outputs are equal, while the figures illustrate responses to known and unknown samples.
- Lemma 1: The proof constrains deviations above and below the common softmax value, with lower outputs unable to become negative.For decreased responses, δc′ cannot exceed S because softmax responses cannot be below zero.
- Lemma 1: Equal softmax outputs minimize the Entropic Open-Set loss for unknown inputs.The proof argues that any deviation from equality increases the loss, so the minimum occurs when all softmax outputs are equal.
- Figures: Figure 4 depicts LeNet++ responses to MNIST classes and Latin-letter background samples used during training.The figure concerns a network trained to classify MNIST digits and reject Latin letters when exposed to trained classes.
- Figures: Figure 5 contrasts responses to CIFAR images with responses associated with MNIST digits and Latin-letter background classes.CIFAR samples are described as very different from both the classes of interest and the background classes.
- Figures: Figure 6 shows LeNet++ responses to characters from the NotMNIST dataset after training on MNIST digits and Latin-letter rejection.NotMNIST characters serve as an additional unknown-sample condition in the figure set.