Source-linked AI summary
Addressing Failure Prediction by Learning Model Confidence
Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, Patrick Pérez
TL;DR
Reliable failure prediction matters for deploying deep neural networks in safety-sensitive settings, but MCP rankings can overlap for correct and incorrect predictions and softmax confidence has known drawbacks. The paper proposes TCP as a confidence criterion, learns it with ConfidNet because true labels are unavailable at test time, and reports improvements over strong baselines across classification and semantic segmentation.
Problem
Failure prediction needs confidence rankings that reliably distinguish correct from incorrect neural-network predictions, while MCP suffers from overlapping rankings and softmax-related drawbacks.
Method
The paper uses True Class Probability as the target confidence criterion and trains a confidence network, ConfidNet, to predict TCP from training samples.
Results
The approach significantly improves over strong baselines across various classification and semantic segmentation datasets, including MCDropout and other failure-prediction methods.
Takeaways & Limitations
TCP is supported as an adequate failure-prediction criterion, and ConfidNet can learn it across varied tasks, datasets, and network architectures.
Takeaways & Limitations
TCP has no theoretical guarantee against overlap between correct and incorrect predictions when TCP lies in [1/K, 1/2], although the observed overlap is extremely small in practice.
Abstract
from arXiv · showhide
Assessing reliably the confidence of a deep neural network and predicting its failures is of primary importance for the practical deployment of these models. In this paper, we propose a new target criterion for model confidence, corresponding to the True Class Probability (TCP). We show how using the TCP is more suited than relying on the classic Maximum Class Probability (MCP). We provide in addition theoretical guarantees for TCP in the context of failure prediction. Since the true class is by essence unknown at test time, we propose to learn TCP criterion on the training set, introducing a specific learning scheme adapted to this context. Extensive experiments are conducted for validating the relevance of the proposed approach. We study various network architectures, small and large scale datasets for image classification and semantic segmentation. We show that our approach consistently outperforms several strong methods, from MCP to Bayesian uncertainty, as well as recent approaches specifically designed for failure prediction.
1 Introduction
The paper addresses reliable failure prediction for deep neural networks by developing confidence measures whose rankings distinguish correct from incorrect predictions. It identifies limitations of MCP and introduces TCP-based learning as a proposed remedy.
- Motivation: Failure prediction is important for deploying neural networks safely, especially in applications where errors have serious repercussions.A reliable confidence measure could support retaining predictions, handing cases to humans or backup systems, or triggering alarms.
- Proposed direction: Ranking by TCP better separates correct and incorrect CIFAR-10 predictions than ranking by MCP, according to the paper’s Figure 1.The figure reports considerable overlap for MCP and improved separation for TCP using relative-density distributions.
- Limitations of MCP: Softmax-based confidence also has conceptual drawbacks, including non-calibration, adversarial sensitivity, and inadequate detection of in- versus out-of-distribution examples.
- Limitations of MCP: MCP is a common confidence baseline, but its ranking can be unreliable because correct and incorrect samples may have overlapping confidence values.MCP uses the largest softmax output, which can produce high confidence even for erroneous predictions.
- Approach: The proposed approach defines a TCP-based confidence criterion, provides theoretical guarantees, learns the criterion from training data, and evaluates it across tasks, datasets, and architectures.The experiments include image classification and semantic segmentation, with analyses of loss functions, criteria, and learning schemes.
2 Failure prediction by learning model confidence
The paper frames failure prediction as learning confidence scores that separate correct from incorrect predictions, replacing MCP with TCP-based criteria and a learned confidence network. TCP has theoretical guarantees, while ConfidNet learns the unavailable test-time true-class probability from training data.
- Failure prediction objective: Failure prediction evaluates whether confidence scores rank correct predictions above erroneous ones, enabling systems to retain, defer, or flag predictions.The objective is distinguishing successful and erroneous predictions rather than improving the classifier’s accuracy.
- Limitations of MCP: MCP can assign high confidence to both correct and erroneous predictions, making their confidence distributions overlap and weakening failure detection.This follows from selecting the largest softmax output even when the model misclassifies an example.
- TCP and TCPr criteria: TCP uses the probability assigned to the true class as the confidence criterion, so low probability reflects a model error more directly than MCP.The paper also defines a normalized TCPr variant as the ratio between TCP and MCP.
- Theoretical guarantees: TCP guarantees correct classification when TCP(x, y*) > 1/2 and incorrect classification when TCP(x, y*) < 1/K.Between 1/K and 1/2, theory does not guarantee non-overlap, although the observed overlap is extremely small for deep networks.
- Theoretical guarantees: TCPr gives stronger guarantees by assigning correct predictions the value 1 and errors values in [0, 1[, but learning it may be more difficult because all correct examples share one target.This creates a trade-off between separation guarantees and regression difficulty.
3 Experiments
Experiments evaluate ConfidNet across classification and semantic segmentation, comparing it with confidence and uncertainty baselines and testing learning variants. ConfidNet consistently performs strongly, including improved error detection, while results depend on dataset scale and training setup.
- Experimental setup: Failure prediction is evaluated with AUPR-Error, AUPR-Success, FPR at 95% TPR, and AUROC, with emphasis on AUPR-Error.AUPR-Error treats errors as the positive class.
- Comparative results: ConfidNet outperforms baseline methods in every setting, with the largest gaps on small models and datasets.The results support TCP as a confidence criterion and ConfidNet as a method for learning it.
- Comparative results: ConfidNet improves over MCDropout because entropy can assign similar confidence to correctly and incorrectly classified samples with similarly spread softmax distributions.The approach distinguishes correct from incorrect predictions despite comparable distribution entropy.
- Comparative results: ConfidNet provides better coverage potential than baselines at each selected risk on CIFAR-10 and SVHN, especially at high coverage rates.The stronger high-coverage ranges are [0.8; 0.95] for CIFAR-10 and [0.86; 0.96] for SVHN.
- Learning variants: Training TCP confidence on a 10% validation set generally reduces performance, especially on small datasets with very high train and validation accuracy.On CIFAR-100, validation-set training slightly improves confidence estimation by +0.17%, while larger validation sets could harm prediction performance.
- Learning variants: MSE training outperforms binary cross-entropy on CIFAR-10 and CamVid, while focal and ranking losses produce similar results.The comparison concerns AUPR-Error.
- Learning variants: The normalized TCP criterion performs worse than TCP on small CIFAR-10 but better on larger CamVid, while TCP may regularize training with finer-grained prediction-quality information.The comparison is attributed to differences in the availability of incorrect and correct classification examples.
4 Conclusion
The paper introduces TCP and a confidence neural network for failure prediction, combining theoretical guarantees with empirical evidence. Experiments across classification and semantic segmentation show significant improvement over strong baselines.
- TCP provides theoretical guarantees and empirical evidence for addressing failure prediction.
- The proposed method learns TCP using a confidence neural network built upon a classification model.
- Results show significant improvement over strong baselines across classification and semantic segmentation datasets.