Deep Gamblers: Learning to Abstain with Portfolio Theory

1 Introduction

• Prediction confidence matters for treating difficult cases separately and for machine-learning applications that require uncertainty estimates.The paper cites scientific discovery and reinforcement learning as examples where uncertainty assessment matters.Source: §1 · p. 1
• The paper identifies the lack of an effective, efficient uncertainty method for deep models and sets out four desiderata: end-to-end training, no heavy sampling, no retraining across uncertainty levels, and no architecture changes.The passage names these as framework requirements and notes that existing methods often miss some of them.Source: §1 · pp. 1–2
• The proposed approach follows selective classification and uses a portfolio-inspired loss that requires almost no modification to existing architectures.The paper contrasts this with Bayesian and ensemble approaches that can involve heavy sampling or architecture changes, and selective methods that require objective changes and retraining.Source: §1 · p. 2

2 The Selective Prediction Problem

• A selective prediction model pairs a predictor f with a selection function g that determines whether its prediction is accepted.The supplied definition introduces f as the classifier and g as the selection function, though the passage cuts off before fully specifying their pair.Source: §2 · p. 2
• The model abstains when g(x) falls below threshold h, and g(x) serves as the input’s uncertainty score.The covered set consists of inputs whose selection score meets or exceeds the threshold.Source: §2 · p. 2
• Coverage is the fraction of inputs in the covered set, and lowering coverage can trade off against lower risk.Different methods may use different uncertainty scores g(x).Source: §2 · p. 2

3 Related Work

• Entropy Selection scores uncertainty with predictive entropy, Softmax-Response uses maximum predicted probability, Bayes Dropout estimates uncertainty with sampling, and SelectiveNet trains an uncertainty predictor.SelectiveNet’s objective uses an interior-point method and depends on the target coverage; Bayes Dropout can require extensive sampling.Source: §3 · pp. 2–3
• The paper does not consider ensembling, but notes that ensembles could be used with its method and may increase performance.This is stated as a footnote to the paper’s single-classifier setting.Source: §3 · p. 2
• The paper’s feature summary states that its method is end-to-end trainable and needs no sampling, retraining, or architecture modification.These are the features the authors highlight for their method in the comparison table.Source: Table 1 · p. 3
• The paper uses a limited horse-race form of portfolio theory for classification and leaves a possible regression reformulation to future work.It situates the connection among portfolio theory, gambling, and information theory, citing.Source: §3 · p. 3

4 Learning to Abstain with Portfolio Theory

• The method interprets abstention as a gambler reserving wealth instead of betting, and turns an m-class task into an m+1-class prediction with an extra rejection score.The authors outline a progression from portfolio theory through a horse race to a classification loss.Source: §4 · p. 3

4.1 A Short Introduction to General Portfolio Theory

• A portfolio distributes wealth across assets, and its wealth relative tracks end-of-day wealth against beginning wealth.The paper introduces this portfolio vocabulary as the basis for its later gambling analogy.Source: §4.1 · pp. 3–4

4.2 Horse Race

• In a horse race, the gambler distributes wealth across outcomes; with reservation, an added category keeps unbet wealth at payoff 1.The horse-race wealth after a race depends on the winning outcome and the bets placed.Source: §4.2 · p. 4

4.3 Classification as a Horse Race

• For classification, the network adds a reservation output and maximizes a mini-batch objective using constant payoff o; that extra output supplies the rejection score.The score’s threshold can be calibrated on a validation set, and the loss does not depend on target coverage; higher o encourages confidence, while lower o encourages less confidence.Source: §4.3 · pp. 4–5

5 Information Theoretic Analysis

• Without reservation, proportional betting b*=p achieves the optimal doubling rate.The analysis identifies this as the correspondence between the classification objective and gambling without reservation.Source: §5 · p. 5
• With side information X, prediction selects a gambling strategy by maximizing mutual information with outcome Y.The increase in doubling rate from knowing X is bounded by the mutual information between X and Y.Source: §5 · p. 5
• For uniform return o, if o > m the optimal strategy never reserves, while if o < 1 it reserves all bets.The analysis treats these as trivial regimes; a more realistic setting has 1 < o < m.Source: §5 · p. 5
• The authors describe 1 < o < m as more realistic because the game organizer takes a cut of bets.The effect of varying o is discussed in the appendix.Source: §5 · pp. 5–6
• Figure 3 reports that the proposed method rejects the outlier cluster earlier than entropy-based selection.The threshold h is chosen as the largest value at which the outlier cluster is rejected.Source: Figure 3 · p. 6
• For a fixed prediction probability, the optimal rejection score can be found using Kuhn-Tucker conditions without training the network.The authors argue the loss’s core contribution is learning a qualitatively different hidden representation, rather than the rejection score itself.Source: §5 · p. 6

6.1 Synthetic Gaussian Dataset

• On the synthetic Gaussian task, the method marks the boundary between two overlapping classes as uncertain and rejects outliers as well as boundary points, unlike entropy selection at ordinary exclusion levels.Lowering threshold h widens the uncertain region, providing a way to calibrate coverage.Source: §6.1 · p. 6

6.2 Locating the outlier testing images of MNIST

• In MNIST, the method’s two least-confident examples look unlike numbers, while entropy selection’s two least-certain images are clearly identifiable as 2 and 7.Source: §6.2 · p. 6

6.4 Comparison with Existing Methods

• At coverage (0.85, 1.00), SVHN performance is reported as SOTA; the method also shows very strong results on CIFAR-10 and Cat vs. Dog.Source: Table 3 · p. 7, §6.4 · p. 7, Table 4 · p. 8

6.3 Rotating an MNIST image

• During a 9-to-180-degree rotation, the model changes its label from 9 to 6, labels an intermediate region as 5 with high uncertainty, and has uncertainty peaks at decision-boundary crossings.Source: §6.3 · p. 7

6.4 Comparison with Existing Methods

• Below 0.8 coverage, comparisons become difficult because few images remain; on Cats vs. Dogs, validation selection produced a model that was among the worst on the test set.Source: §6.4 · p. 8

• The method is competitive with current SOTA [15], simpler and more flexible, and outperforms the runner-up SOTA model [14].Source: §7 · p. 8
• The t-SNE comparison shows more separable second-to-last-layer outputs for Deep Gambler than for the baseline; the authors hypothesize this representation helps explain performance.They report six baseline clusters that are not easily separable and describe the Deep Gambler representation as having a large margin.Source: Figure 5 · p. 8, §7 · p. 8
• Possible future directions include scientific applications, studying rejected examples across training epochs, adversarial robustness, label-noise learning, and connections between portfolio theory and deep learning.Source: §7 · p. 8

8.1 Implementation

• For an m-class task, the guide replaces cross-entropy with the modified loss and adds an output dimension for the rejection score, using a softmax over m + 1 outputs.Source: §8.1 · p. 12
• The code applies softmax, separates class predictions from the reservation output, and minimizes the negative mean log doubling rate based on correct-class gain plus reservation divided by reward.Source: §8.1 · p. 12

8.2 Optimizing the deep gambler’s Objective

• On CIFAR10, values o < 6.3 led to trivial abstention; cross-entropy warm-up was used for 100 epochs there, 50 epochs on SVHN when o < 6.0, and 50 epochs on Cats vs Dogs for all o values.Source: §8.2 · p. 12

9 Theorem Proofs

• The optimal doubling rate is achieved by proportional gambling b*=p and equals the expected log odds minus the entropy H(p).H(p) is the entropy of distribution p.Source: §9 · p. 12
• With side information Y, the optimal strategy is conditional proportional betting b(x|y)=p(x|y), and the doubling-rate gain is I(X;Y).The proof equates the gain to H(X)−H(X|Y).Source: §9 · p. 13
• When all outcome odds equal o, the theorem states that o≥m eliminates reservation, while o≤1 makes optimal betting reserve all wealth.The proof explains the low-odds case by showing that moving any bet to the reserve yields a better return.Source: §9 · p. 13

10.1 Datasets

• The experiments use SVHN, CIFAR-10, and Cats vs. Dogs, with official dataset sources for the first two and a 20,000/5,000 train-test split for Cats vs. Dogs.SVHN and CIFAR-10 each have 10 classes; the Cats vs. Dogs images are resized to 64 × 64 × 3.Source: §10.1 · p. 13

10 Experiment Detail

• The implementation uses PyTorch 1.0, and the experiment setting selects the best model for each coverage using validation data after a grid search over o.The search ranges are 2–9.8 for CIFAR-10, 2–8 for SVHN, and 1.2–2 for Cats vs. Dogs.Source: §10 · p. 13, §10.2 · p. 14

11.1 Tuning the hyperparameter

• Lower o improves rejection but increases variance, while large o impairs low-coverage performance; tuning therefore trades off total error against low-coverage error.The authors report best performance often at small or large o, and say appropriate o can outperform cross-entropy baselines by shifting learning toward certain data.Source: §11.1 · p. 14, Figure 6 · p. 14

11.2 Top-30 Least Certain Images

• Using the method, a trained 4-layer CNN identifies the top 30 least-certain MNIST training images at epochs 1, 10, and 30; the model converges at about epoch 10.Source: §11.2 · p. 14

11.3 t-SNE Plots

• On MNIST, the deep gambler’s t-SNE representation is reported as more separable and larger-margin than the normal model’s, while rejected points are nearly identical under SR and the deep gambler.The authors suggest the better representation may explain superior performance; the plots use the second-to-last-layer output.Source: §11.3 · p. 16, Figure 10 · p. 16, Figure 11 · p. 16