Source-linked AI summary
Cautious Deep Learning
Yotam Hechtlinger, Barnabás Póczos, Larry Wasserman
TL;DR
Ordinary classifiers can assign definite labels to outliers, even where the data provide little information. This paper constructs conformal label sets using p(x|y), with coverage guarantees and a null-set option. It reports cautious predictions across image and facial-data settings, including rejection of all 100 random-noise images tested.
Problem
Using p(y|x) can force a most-likely label for outliers despite little information about inputs in low-probability regions.
Source: §2 · p. 4Method
The method thresholds class-conditional p(x|y) estimates to form conformal prediction sets that may be empty and have coverage at least 1 − α.
Source: §1 · p. 2, §5.1 · p. 8Results
All 100 random-noise images were flagged as the null set at α = 0.01, and the method is reported to provide cautious predictions across the evaluated settings.
Source: §6.2 · p. 13, §8 · p. 17Takeaways & Limitations
The approach can provide null predictions for outliers and larger label sets for ambiguous objects, while allowing classes to be added or removed with overall or class-specific coverage control.
Source: §8 · p. 17Constructing conformal sets is computationally expensive, and using unaugmented-data scores replaces the finite-sample guarantee with an asymptotic one under conditions.
Source: §4 · p. 6
Abstract
from arXiv · showhide
Most classifiers operate by selecting the maximum of an estimate of the conditional distribution $p(y|x)$ where $x$ stands for the features of the instance to be classified and $y$ denotes its label. This often results in a {\em hubristic bias}: overconfidence in the assignment of a definite label. Usually, the observations are concentrated on a small volume but the classifier provides definite predictions for the entire space. We propose constructing conformal prediction sets which contain a set of labels rather than a single label. These conformal prediction sets contain the true label with probability $1-\alpha$. Our construction is based on $p(x|y)$ rather than $p(y|x)$ which results in a classifier that is very cautious: it outputs the null set --- meaning "I don't know" --- when the object does not resemble the training examples. An important property of our approach is that adversarial attacks are likely to be predicted as the null set or would also include the true label. We demonstrate the performance on the ImageNet ILSVRC dataset and the CelebA and IMDB-Wiki facial datasets using high dimensional features obtained from state of the art convolutional neural networks.
1 Introduction
Standard classifiers make definite predictions even for difficult inputs, while this paper proposes conformal label sets with a coverage guarantee and a null-set option for unfamiliar inputs. The method estimates p(x|y) and is evaluated on large image and facial datasets using CNN features.
1 Introduction
- Conventional prediction functions provide a definite label even for cases that are hard to classify.The usual goal is to minimize classification error on new observations.Source: §1 · p. 2
- Conformal prediction returns a set of labels with probability at least 1 − α of containing the true label, and can return the null set for unfamiliar inputs.The construction estimates p(x|y) and thresholds class-specific estimates to form the set.Source: §1 · p. 2
- Class-specific p(x|y) estimates can be fitted independently, allowing classes to be added or removed without retraining the whole classifier.The approach is demonstrated on ImageNet ILSVRC, CelebA, and IMDB-Wiki using high-dimensional features from state-of-the-art convolutional neural networks.Source: §1 · p. 2
- The related work distinguishes this method from reject-option classification, which permits abstention but does not provide confidence guarantees.Sadinle et al. (2017) instead seek small prediction sets based on p(y|x) and avoid null sets.Source: §1 · p. 3
- Lee et al. (2018) proposed a p(x|y)-based framework for predicting out-of-distribution inputs and adversarial attacks.Source: §1 · p. 3
2 p(y|x) Versus p(x|y)
Using p(y|x) can force a label for outliers because it selects among tail-event probabilities, while p(x|y)-based sets can leave such inputs unlabeled. The distinction matters when no label must be assigned to every input.
2 p(y|x) Versus p(x|y)
- The usual maximum-posterior prediction balances p(y) and p(x|y) through p(y|x) = p(x|y)p(y)/p(x).Source: §2 · p. 3
- For set-valued classification, accounting for class popularity can tie predictions to prevalence, while normalization by p(x) obscures whether x itself is probable.The paper notes that predictions may need not be restricted by class popularity when there is no limit on the number of classes.Source: §2 · p. 3
- When p(x) is extremely low, p(y|x) still selects the most likely label among tail events, potentially assigning high-probability predictions where the classifier has virtually no information.In such regions, the paper argues that the reasonable set-valued prediction can be the null set.Source: §2 · p. 4
- Conformal methods based on p(y|x) can assign the null set when all class probabilities are equal, even though that case calls for including all classes as possible.The paper says p(x|y)-based conformal predictors can overcome these issues.Source: §2 · p. 4
3 Motivating Example - Iris Dataset
The Iris example contrasts standard classifiers’ unstable, confident boundaries in low-density regions with conformal sets that express ambiguity or abstain. The example visualizes the method using sepal and petal lengths.
3 Motivating Example - Iris Dataset
- For visualization, the Iris example uses sepal and petal lengths from a dataset with four measured features across three species.Source: §3 · p. 4
- Figure 1 compares KNN, an RBF-kernel SVM, and the proposed conformal method based on an estimate of p(x|y).Source: §3 · p. 4
- In low-density regions, KNN and SVM produce substantially different boundaries, with near-certain predictions lacking sound justification.The SVM assigns almost all the space to one class, while KNN produces an infinite strip bounded by nearly affine half-spaces.Source: §3 · p. 4
- The conformal plot uses overlapping regions for multiple classes and white regions for the null set, with α = 0.05.The method outputs the null set when a new input falls in a region with little training data.Source: Figure 1 · p. 5, §3 · p. 5
4 Conformal Prediction
Conformal prediction forms sets by testing candidate labels with permutation-invariant scores and provides distribution-free coverage. The section also surveys examples and notes computational and finite-sample tradeoffs in constructing the sets.
4 Conformal Prediction
- Conformal prediction forms a candidate-label set from p-values computed using permutation-invariant scores on an augmented dataset, with coverage at least 1 − α for all distributions.The conformity score can be chosen flexibly.Source: §4 · pp. 5–6
- Recomputing all conformal scores for each input and candidate label is computationally expensive, especially for deep-learning applications.Data splitting is one proposed way to reduce this burden.Source: §4 · p. 6
- Using scores from the original data without augmentation gives an asymptotic coverage guarantee rather than the finite-sample guarantee, provided conditions hold.The stated bound is P(Y ∈ C(X)) ≥ 1 − α − oP(1).Source: §4 · p. 6
4.1 Examples
- Regression conformal prediction uses ordered residual errors to define a prediction set; consistency of the regression estimate yields asymptotic coverage 1 − α + oP(1).Source: §4.1 · p. 6
- The examples include a density-level-set method for unsupervised prediction and multiclass methods optimizing ambiguity or limiting multiple predictions.Sadinle et al. (2017) minimize expected set size among classifiers achieving the target coverage; Vovk et al. (2003) give an asymptotic universal predictor.Source: §4.1 · pp. 6–7
5 The Method
The method thresholds class-conditional density estimates to form prediction sets, with finite-sample coverage under data splitting. Density-estimation choices, class adaptivity, and class interaction shape the approach.
5.1 The Classifier
- With data splitting, classwise density scores define C(x)={y: bp(x|y)≥bty}, achieving P(Y∈C(X))≥1−α for every distribution and sample size, though splitting can enlarge sets.The threshold is calibrated from a held-out part of the training data.Source: §5.1 · pp. 7–8
5.2 Density Estimation
- Density estimation can use kernel methods or faster k-nearest-neighbor distances; accuracy depends on preserving density-score ordering, not L2 closeness.The paper calls this weaker requirement “ordering consistency” and leaves its implications for density approximation to future work.Source: §5.2 · pp. 8–9
5.3 Class Adaptivity
- Independent classwise training and prediction allow classes to be added or removed ad hoc, and class-specific coverage levels can be set with separate quantiles.The method achieves this class adaptivity by estimating each class independently.Source: §5.3 · p. 9
5.4 Class Interaction
- Independent class estimates discard inter-class relations; correlated estimators can restore them by penalizing regions dense under other classes.On Iris, the illustrated estimator provides 0.95 coverage while producing boundaries resembling standard methods and cautious out-of-sample predictions.Source: §5.4 · p. 9, Figure 2 · p. 10
6 ImageNet Challenge Example
The ImageNet example applies class-specific kernel density estimators to Xception features and examines prediction-set behavior across ordinary, adversarial, and outlier inputs. Its results show a trade-off between accuracy and set size, with targeted attacks and inputs outside the ImageNet categories often receiving null-set predictions.
6 ImageNet Challenge Example
- 0.721 top-1 and 0.863 top-5 accuracy are achieved by selecting the highest and top-five p(x|y) predictions on ImageNet validation.The Xception model obtains 0.79 top-1 and 0.945 top-5 accuracy on the same validation set.Source: §6 · p. 11
- At α = 0.5, the method averages 2.7 predictions per observation and predicts the null set for 0.252 of observations.Smaller α yields higher accuracy but more predicted classes; larger α yields fewer classes and more null-set predictions.Source: §6 · p. 11, Figure 3 · p. 12
6.1 Adversarial Robustness
- Targeted attacks make larger image modifications and mostly produce null-set predictions for both the true and adversarial labels.Untargeted attacks instead yield predictions containing both the true and adversarial classes; Table 1 reports accuracy on 200 adversarial images.Source: §6.1 · p. 13, Table 1 · p. 13
6.2 Outliers
- All 100 random-noise images and all 11 Jackson Pollock paintings receive null-set predictions under the tested settings.The settings were α = 0.01 for random noise and α = 0.5 for paintings.Source: §6.2 · pp. 13–14
- The Muhammad Ali–Sonny Liston image receives a null set at α as low as 0.55 because “Boxing” is not an ImageNet class.Inception-v4 instead selects “Volleyball” with probability 0.38; its top-five sport predictions have probability 0.781.Source: §6.2 · p. 14
7 Gender Recognition Example
The standard gender classifier performs well on CelebA but poorly on IMDB-Wiki, while the conformal method provides tunable set-valued predictions on both datasets. Its CelebA results follow the expected accuracy and null-prediction pattern; on IMDB-Wiki, prediction behavior trades off accuracy, false predictions, and null outputs.
7 Gender Recognition Example
- At α = 0.7, the ImageNet collage shows the first 20 validation images, with only 0.3 accuracy expected by design.Both true and false predictions are described as reasonable.Source: Figure 5 · p. 15
- The standard classifier reaches 0.963 accuracy on CelebA but only 0.577 accuracy on IMDB-Wiki, where broader posture variety contributes to the dataset shift.The authors attribute the discrepancy to preprocessing and note that IMDB-Wiki has a larger variety of human postures.Source: §7 · p. 16
- On CelebA, conformal accuracy is roughly 1 −α and the proportion of null predictions roughly α, with almost no false positives and all errors being null predictions.The method uses features from the network trained on CelebA.Source: §7 · p. 16
- On IMDB-Wiki, high 1 −α yields accuracy much higher than 0.577, while cautious low 1 −α yields fewer false predictions but more null predictions.The number of false positives and accuracy are tunable; high 1 −α can also produce many predictions containing both genders.Source: §7 · p. 16
- Figure 6 compares mean female-face pixel values for CelebA and IMDB-Wiki and plots conformal performance on both datasets.The caption notes that CelebA images have fixed posture.Source: Figure 6 · p. 17
8 Discussion
The discussion presents p(x|y)-based conformal prediction as a cautious approach that distinguishes ambiguity from outlyingness while retaining coverage guarantees and class adaptivity. These properties come with limitations in decision-boundary precision and deployment speed.
8 Discussion
- The method distinguishes ambiguity, represented by multiple plausible labels, from outlyingness, which can produce the null set.It associates an observation with a class only when the observation has high probability under that class.Source: §8 · p. 17
- The conformal framework provides coverage guarantees and class adaptivity, allowing classes to be added or removed and coverage controlled overall or by class.The method learns each class's p(x|y) distribution independently, discarding information about relations among classes.Source: §8 · p. 17
- Independent classwise estimation limits decision-boundary precision, though complex estimation functions can produce boundaries almost equivalent to standard methods.The limitation arises because each class distribution is learned independently and boundaries are indifferent to data outside that class.Source: §8 · pp. 17–18
- Deployment is relatively slow compared with current methods because it requires evaluating many kernel density estimators.The discussion identifies more efficient ordering-consistent approximations deployable on GPUs as a future research direction.Source: §8 · p. 18
A Appendix: Details on Proposition 1
The appendix establishes approximate coverage for the estimated prediction set under density-estimation conditions and a fixed number of classes. If the class count grows, the remainder must still vanish, with a possible adjustment for small classes.
A Appendix: Details on Proposition 1
- The proof assumes density-estimation conditions, compact input space, and that the minimum class count grows without bound.The bandwidth conditions include nh_y^d/(log n)^16 → ∞ and nh_y^(d+4)(log n)^2 → 0.Source: A Appendix: Details on Proposition 1 · p. 21
- Under the stated conditions, the estimated set achieves coverage 1 −α + oP(1).The argument invokes consistency of the conditional-density estimate in the ℓ∞ norm.Source: A Appendix: Details on Proposition 1 · p. 21
- The stated analysis assumes a fixed number of classes; as class count grows, class counts must grow fast enough for the remainder to vanish.Carefully omitting classes with small counts can preserve coverage if the remainder is small across larger classes.Source: A Appendix: Details on Proposition 1 · pp. 21–22