PaperlayerView exact revision on arXiv

Source-linked AI summary

Verified Uncertainty Calibration

Ananya Kumar, Percy Liang, Tengyu Ma

arXiv:1909.10155v2cs.LGstat.ML

TL;DR

Calibration methods must produce probabilities reflecting true event frequencies, but continuous scaling methods can underestimate calibration error and histogram binning is sample inefficient. The paper introduces scaling-binning calibration and a debiased estimator to combine measurable calibration with improved sample efficiency.

  • Problem

    Continuous scaling methods can underestimate calibration error, while histogram binning has sample requirements that grow linearly with the number of distinct model outputs B.

  • Method

    The scaling-binning calibrator fits a parametric function, bins its outputs, and averages the fitted values within equal-sized bins; calibration error is estimated with a debiased estimator.

  • Results

    O(1/ε^2 + B) samples achieve calibration error ε, improving on histogram binning's O(B/ε^2) requirement while retaining measurable calibration error.

  • Takeaways & Limitations

    The paper provides a calibration method with improved sample complexity and measurable calibration error, addressing limitations of both scaling methods and histogram binning.

  • Takeaways & Limitations

    Maintaining calibration under dataset shifts without labeled examples from the target dataset remains an open question.

Abstract

from arXiv · show

Applications such as weather forecasting and personalized medicine demand models that output calibrated probability estimates---those representative of the true likelihood of a prediction. Most models are not calibrated out of the box but are recalibrated by post-processing model outputs. We find in this work that popular recalibration methods like Platt scaling and temperature scaling are (i) less calibrated than reported, and (ii) current techniques cannot estimate how miscalibrated they are. An alternative method, histogram binning, has measurable calibration error but is sample inefficient---it requires $O(B/ε^2)$ samples, compared to $O(1/ε^2)$ for scaling methods, where $B$ is the number of distinct probabilities the model can output. To get the best of both worlds, we introduce the scaling-binning calibrator, which first fits a parametric function to reduce variance and then bins the function values to actually ensure calibration. This requires only $O(1/ε^2 + B)$ samples. Next, we show that we can estimate a model's calibration error more accurately using an estimator from the meteorological community---or equivalently measure its calibration error with fewer samples ($O(\sqrt{B})$ instead of $O(B)$). We validate our approach with multiclass calibration experiments on CIFAR-10 and ImageNet, where we obtain a 35% lower calibration error than histogram binning and, unlike scaling methods, guarantees on true calibration. In these experiments, we also estimate the calibration error and ECE more accurately than the commonly used plugin estimators. We implement all these methods in a Python library: https://pypi.org/project/uncertainty-calibration

1 Introduction

Calibration is important because modern models often produce uncalibrated probabilities, while existing recalibration methods trade measurable guarantees against sample efficiency. The paper introduces scaling-binning calibration and improved error estimation to address both problems.

  • Motivation: Modern models often require recalibration because their probability outputs are not calibrated out of the box.Calibration means predicted probabilities should reflect event frequencies, supporting applications such as personalized medicine and weather forecasting.
  • Limitations of existing methods: Continuous-output methods can appear better calibrated than they are, while their true calibration error is difficult to measure.Using more bins can reveal higher calibration error, but finite-bin evaluation cannot reliably measure the true error of continuous methods.
  • Limitations of existing methods: Histogram binning provides measurable calibration error but requires O(B/ε^2) samples, which can be costly when B is large.This issue is especially important for multiclass models, where B can scale with the number of classes and labeled target-domain data may be scarce.
  • Proposed method: The scaling-binning calibrator first fits a function and then averages its values within equal-mass bins, reducing variance while retaining calibration verification.Its binned averages use lower-variance function values rather than directly averaging labels.
  • Sample complexity: O(1/ε^2 + B) samples suffice for scaling-binning calibration, compared with O(B/ε^2) for histogram binning.The bound assumes the function family G is well chosen.
  • Experiments: On CIFAR-10 and ImageNet, scaling-binning achieved lower calibration error than histogram binning, while a debiased estimator measured calibration error more accurately than the plugin estimator.The reported calibration-error improvements were 35% on CIFAR-10 and 5x on ImageNet for B = 100.

2 Setup and background

The paper defines calibration and related objectives for binary and multiclass prediction, then formalizes recalibration as learning a transformation of an uncalibrated model output. It contrasts parametric scaling with histogram binning.

  • Binary classification: A calibrated binary model’s predicted probability should match the true conditional frequency of the label given that prediction.Perfect calibration corresponds to CE(f) = 0.
  • Binary classification: Calibration alone is insufficient because a constant prediction can be calibrated without being useful.The paper therefore also considers mean-squared error, or Brier score, subject to a calibration budget.
  • Multiclass classification: Multiclass calibration includes top-label calibration and marginal calibration, which evaluates calibration for each class.Marginal calibration uses class-specific importance weights and addresses less likely predictions as well as top predictions.
  • Multiclass classification: Top-label calibration can be reduced to binary calibration, while marginal calibration becomes K one-vs-all binary calibration problems.The experiments consider both top-label and marginal calibration.
  • Recalibration methods: Recalibration learns g so that applying it to an uncalibrated model f produces a well-calibrated composition g ◦ f.The recalibration data consist of independently sampled pairs of model outputs and labels.
  • Recalibration methods: Scaling fits a differentiable function from a model family, whereas histogram binning partitions [0, 1] and outputs the average label in each bin.Scaling is fast because it fits few parameters; binning may use equal-width or uniform-mass intervals.

3 Is Platt scaling calibrated?

The paper shows that finite-bin evaluation can substantially underestimate the calibration error of continuous-output models. Experiments on CIFAR-10 and ImageNet support this conclusion as finer binning reveals larger measured errors.

  • Why calibration is hard to measure: Estimating calibration error for continuous outputs requires estimating E[Y | f(X)], which is impossible without unverifiable smoothness assumptions.The paper compares this difficulty with measuring mutual information between continuous signals.
  • Binned evaluation: Binned calibration error evaluates the calibration error of a model whose outputs are averaged within each bin.This evaluation binning differs from histogram binning, where binning is part of the recalibration method.
  • Underestimation result: For any binning scheme and continuous bijective model, binned calibration error can be zero while true calibration error is at least 0.49.Within each bin, over- and underestimation can cancel after averaging.
  • Underestimation result: Binning always produces a calibration error no greater than the original model’s calibration error because errors within bins can cancel.The result applies to arbitrary ℓp calibration errors, including ECE.
  • Experiments: Experiments on CIFAR-10 and ImageNet suggest that prior reports understated the actual calibration error of their models.Increasing the number of bins provided tighter lower bounds, but the actual calibration error remained uncomputable.
  • Experiments: On ImageNet, using 15 bins suggested an error around 0.02 even though the true calibration error was at least twice as high.CIFAR-10 showed similar behavior, and the finding persisted for ECE and alternative binning strategies.

4 The scaling-binning calibrator

The scaling-binning calibrator combines parametric fitting with uniform-mass binning to retain scaling methods’ sample efficiency while making calibration measurable. It requires O(B + 1/ε^2) samples and performs favorably against histogram binning in CIFAR-10 and ImageNet experiments.

  • Method: The scaling-binning calibrator fits a function, constructs equal-mass bins, and outputs the mean fitted value within each bin.Uniform-mass binning is essential for efficiently estimating calibration error.
  • Calibration guarantees: O(B + 1/ε^2) samples suffice for calibration, versus O(B/ε^2) for histogram binning.The bound assumes regularity conditions on the function family G.
  • Calibration guarantees: The method can be at least almost as calibrated as the best function in G, and binning may further reduce calibration error.Binning can worsen sharpness and increase mean-squared error, although many bins limit that increase.
  • Method: The fitted values have lower within-bin variance than labels, reducing estimation error relative to histogram binning.This variance reduction explains how the method estimates B bin outputs with roughly O(1/ε^2) samples.
  • Experiments: 35% lower calibration error is achieved with 100 bins per class on CIFAR-10, while the method also performs better than histogram binning under calibration constraints.The experiments use 1,000 recalibration points and evaluate marginal calibration error.
  • Experiments: When the number of samples is fixed, scaling-binning’s squared calibration error stays nearly constant as B varies, unlike histogram binning’s near-linear increase.For both methods, squared calibration error decreases approximately as 1/n.

5 Verifying calibration

The paper studies how to estimate calibration error for binned models, contrasting the biased plugin estimator with a debiased estimator from meteorology. The debiased approach reduces bias and requires fewer samples, with experiments showing estimates closer to ground truth.

  • Estimators: The plugin estimator’s sample complexity scales linearly with B, whereas the debiased estimator achieves improved scaling for estimating calibration error.The debiased estimator subtracts an approximate bias correction.
  • Extensions: The paper also proposes debiasing the ℓ1 calibration error, or ECE, and reports more accurate estimates on CIFAR-10 and ImageNet.This extension is described in Appendix G.
  • Assumptions: A 2-well-balanced binning scheme requires each output probability to have population mass at least 1/(2B).This condition underlies the stated estimator guarantees.
  • Estimator analysis: The plugin estimator accumulates total bias B/n because each term contributes bias 1/n.For the debiased estimator, lower bias and variance cancellation across bins support improved sample complexity.
  • Experiments: The CIFAR-10 experiment uses separate recalibration and evaluation sets, varying n and using B = 100 or B = 10 bins per class.Estimates are compared with squared calibration error measured on the full evaluation set.
  • Experiments: The debiased estimator is closer to the ground truth than the plugin estimator, especially when B is large or n is small.Figure 4 reports mean-squared error with 90% confidence intervals, where lower values are better.

6 Related work

Calibration research spans machine learning and several applied fields, while recalibration is related to conditional density estimation. The paper distinguishes its debiasing result from prior work that identified bias without improving sample complexity.

  • Calibration research: Calibration has been studied in meteorology, fairness, healthcare, reinforcement learning, natural language processing, speech recognition, econometrics, and psychology.Related evaluation tools include the Hosmer-Lemeshow test and reliability diagrams.
  • Connections: Recalibration resembles conditional density estimation because both aim to estimate E[Y | f(X)].Density-estimation analyses often assume a Lipschitz true density, whereas calibration seeks measurable error without untestable smoothness assumptions.
  • Estimator comparison: The paper’s debiasing result differs from typical statistical debiasing because it improves sample complexity as well as estimator bias.Prior work established that the naive calibration estimator is biased.

7 Conclusion

The paper concludes that continuous calibration methods underestimate calibration error, while scaling-binning combines improved sample complexity with measurable calibration. It also identifies open problems in distribution shift, calibration metrics, multiclass calibration, and binning.

  • Contributions: Continuous methods’ calibration error is underestimated, motivating methods whose calibration can be measured from data.The conclusion frames this as the first contribution.
  • Contributions: The scaling-binning calibrator improves on histogram binning’s sample complexity while retaining measurable calibration error.The authors describe this as combining the advantages of scaling and binning methods.
  • Contributions: The alternative calibration-error estimator has better sample complexity than the plugin estimator.The conclusion lists this as the paper’s third contribution.
  • Future work: Future work includes maintaining calibration under dataset shifts without labeled target examples.The conclusion also asks whether recalibration can support stronger multiclass notions such as joint calibration efficiently.
  • Reproducibility: The methods, code, data, and experiments are made available through a Python library and CodaLab resources.Updated code is also linked through GitHub.

B Proofs for Section 3

The proofs formalize calibration error for ℓp metrics, show that finer binning yields tighter lower bounds, and demonstrate that finite binning can severely underestimate true error.

  • For p ≥1, ℓp calibration error generalizes calibration error beyond the paper’s main CE metric, including ℓ1 calibration error (ECE).
  • A continuous bijective model can have zero binned calibration error while its true ℓp calibration error is at least 0.49.This construction exploits opposing underestimation and overestimation within each bin, which cancel after averaging.
  • The construction can make the model highly miscalibrated at every probability value despite appearing calibrated after binning.The conditional event probability is either 0 or 1 throughout the relevant interval.
  • Finer binning produces a better lower bound: ℓp-CE(fB) ≤ ℓp-CE(fB′) ≤ ℓp-CE(f).The finer binned function can be viewed as a binned version of the coarser one.
  • On ImageNet, binned ℓ1 calibration error increases with the number of bins, suggesting binning is unreliable for measuring ℓ1-CE.
  • On CIFAR-10, the same trend is suggested, but large confidence intervals make the result inconclusive.

C Ablations for Section 3

Additional experiments test whether binning underestimates ℓ1 calibration error across datasets and bin-selection schemes, while the theoretical analysis relies on regularity assumptions for the scaling family.

  • Ablation experiments: On ImageNet, increasing the number of bins uncovers higher ℓ1 calibration error under both equal-frequency and equal-width binning.
  • Ablation experiments: On CIFAR-10, the binning results are inconclusive because large error bars obscure the smaller calibration error being measured.The dataset is smaller and the model has 93.1% accuracy, making the target calibration error smaller.
  • Experimental protocol: The CIFAR-10 evaluation splits data into 1,000 recalibration points, 1,000 bin-selection points, and 8,000 measurement points.The experiment uses a VGG16 model.
  • Assumptions: The sample-complexity analysis assumes a finite-parameter function family with an open parameter set, injectivity, consistency, and regularity.Regularity includes twice differentiable loss, a nonsingular symmetric Hessian, Lipschitz dependence on parameters, and continuous second derivatives.

D.1 Calibration bound (Proof of Theorem 4.1)

The proof of Theorem 4.1 decomposes the calibrator into scaling, population-balanced bin construction, and empirical binning, then combines concentration and convergence bounds under regularity assumptions.

  • Assumptions: Theorem 4.1 assumes finite parameters, injectivity, Lipschitz continuity, consistency, and twice differentiability of the scaling family.
  • Calibration bound: The scaling-binning calibrator satisfies (CE(ĝB))^2 ≤ 2 ming∈G(CE(g))^2 + ε^2 with probability at least 1 − δ.
  • Step 3: Empirical binning: Step 3 empirically bins the fitted function and bounds convergence to its population-binned version in both ℓ2 and ℓ1 norms.The proof uses concentration within bins, sub-Gaussian bounds, and a union bound.
  • Step 2: Bin construction: Step 2 constructs equal-frequency bins and proves they are approximately population-balanced, with each bin’s probability between 1/(2B) and 2/B.
  • Step 1: Scaling: Step 1 fits a low-dimensional scaling function, whose empirical risk minimizer converges to low calibration error when the family contains a good function.
  • Trade-off: Discretizing fitted outputs reduces calibration error and enables calibration-error measurement, but increases mean-squared error by a small amount.
  • Alternative binning schemes: Alternative equal-width bins may measure calibration error inefficiently unless they happen to be 2-well-balanced, motivating hybrid schemes as future work.

E Experimental details and ablations for section 4

The experiments compare scaling-binning with histogram binning across synthetic and multiclass settings, varying recalibration samples and bins. Results broadly support the predicted sample-efficiency behavior, while synthetic cases show that scaling-binning can outperform its underlying scaling method.

  • Experimental protocol: The experimental protocol repeats recalibration 100 times and accounts for randomness in both recalibration data and evaluation data.The CIFAR-10 protocol treats the empirical validation distribution as the true distribution for a precise hypothesis test.
  • Multiclass ablations: The CIFAR-10 top-label ablation favors scaling-binning at high B but histogram binning at low B, motivating evaluation of multiple recalibration methods.The authors attribute the low-B reversal partly to the CIFAR-10 model’s over-93% top-label accuracy.
  • Synthetic experiments: 2.00 ± 0.06 times and 1.98 ± 0.09 times decreases in squared calibration error follow doubling n from 1000 to 2000 for histogram binning and scaling-binning, respectively.With B = 10, both methods show approximately linear growth of 1/ϵ2 with n, matching the theoretical bounds.
  • Synthetic experiments: 3.71 ± 0.15 times versus 2% ± 7% changes in squared calibration error result when increasing bins from 5 to 20 for histogram binning and scaling-binning, respectively.At n = 2000, histogram binning’s error increases with B, whereas scaling-binning remains nearly constant within estimation error.
  • Synthetic experiments: 5.2 ± 1.1 times lower squared calibration error is achieved by scaling-binning than the underlying scaling method in one synthetic setting with B = 10 and n = 3000.The paper notes that binning can help or hurt relative to the underlying scaling method because empirical binning introduces estimation error.

F.1 Analysis of plugin estimator (proof of Theorem 5.3)

The plugin-estimator proof decomposes estimation error into three terms and bounds them using concentration arguments. Its dominant bias term yields linear dependence on the number of bins, establishing the estimator’s sample inefficiency.

  • Proof strategy: The plugin estimator is decomposed into three terms, P1, P2, and P3, which are bounded separately before being combined.The proof mainly uses algebra and standard concentration results, with one term requiring conditioning.
  • Technical conditions: The proof conditions on accurate bin-probability estimates and applies concentration bounds, union bounds, and independence across evaluation samples.The assumptions include a 2-well-balanced binned model and sufficiently large per-bin probabilities.
  • Dominant error term: P3 dominates the plugin estimator’s bound and produces the estimator’s bin-dependent error.The analysis identifies this term as the component improved by the debiased estimator.
  • Sample complexity: O(B/ϵ2) samples are required in the worst case to estimate calibration error with the plugin estimator.The lower-bound argument uses plugin bias at least B/n and variance on the order of O(B/n).

F.2 Analysis of debiased estimator (proof of Theorem 5.4)

The debiased-estimator analysis replaces the plugin estimator’s dominant bias contribution with a centered term and derives improved bin dependence. The same section also introduces a heuristic debiasing approach for ℓ1 calibration error, or ECE.

  • Debiased proof: The debiased estimator uses a three-term decomposition in which C1 and C2 match the plugin terms, while C3 receives the key new bound.The analysis shows E[C3 | Z] = 0 before controlling conditioning-related bias and concentration.
  • Sample complexity: O(√B) bin dependence replaces the plugin estimator’s O(B) dependence in the regime where only constant-factor calibration-error accuracy is required.For finer asymptotic estimation, both estimators retain approximately 1/√n error behavior.
  • ECE debiasing: The ℓ1 calibration-error estimator subtracts a simulated Gaussian approximation of the plugin estimator’s absolute-value bias.The approximation uses bin-level empirical means and variances to generate normal draws.
  • ECE debiasing: The plugin ECE estimate overestimates binned calibration error because finite-sample absolute deviations remain positive even when population deviations are zero.This explains why bias correction is needed for the ℓ1 estimator.
  • Limitation: The ℓ1 debiasing method is heuristic, and the paper leaves provably improved estimation rates for ℓ1 calibration error to future work.The normal simulation is computationally inexpensive but not theoretically analyzed here.
  • Empirical results: The debiased ECE estimates are closer to ground truth than plugin estimates on CIFAR-10 and ImageNet experiments.The ImageNet figure emphasizes stronger improvement when B is large or n is small.

G.2 Additional experiments for estimating calibration error

Additional experiments evaluate debiased calibration-error estimates and their use in model selection. They compare plugin and debiased estimators across bin counts and datasets using error distributions and confidence bounds.

  • Squared calibration error: For B = 10, squared-error estimation results are mixed, whereas for B = 100 the debiased estimates are much closer to ground truth.Figure 14 summarizes absolute estimation-error histograms over 1,000 resamples.
  • ECE: The debiased estimator produces ECE estimates much closer to ground truth than the plugin estimator on ImageNet and CIFAR-10.Figures 15 and 16 provide the corresponding absolute-error histograms.
  • Model selection: The debiased estimator enables selecting models with lower mean-squared error under a calibration-error constraint.The experiment uses bootstrap resamples to construct a 90% upper confidence bound.
  • Model selection: For calibration error below 0.015, the debiased estimator supports 100 bins, while the plugin estimator supports only 15 bins and incurs 13% higher mean-squared error.Figure 17 varies B and compares mean-squared error against 90% upper calibration-error bounds.
Loading 1909.10155v2…