Source-linked AI summary
Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles
Rajeev Verma, Daniel Barrejón, Eric Nalisnick
TL;DR
Multi-expert learning to defer must decide when to defer and which expert to query, while monitoring expert quality. The paper derives two consistent surrogate losses, studies calibration, and proposes conformal expert-subset selection. The consistent losses outperform prior mixture-of-experts systems in accuracy and calibration, though calibration and accuracy tradeoffs vary with the expert pool.
Problem
Multi-expert L2D must choose both whether to defer and which expert to use, making accurate monitoring of expert quality important for safe deployment.
Source: §1 · p. 1Method
Results
The consistent losses yield superior accuracy and calibration to mixture-of-experts systems; with more experts, OvA stays near 2% ECE while softmax is about 4.5%, but softmax is most accurate at 20 experts.
Source: §1 · p. 1, §7.2 · p. 8Takeaways & Limitations
Expert-correctness estimates matter for safe deferral, and conformal inference provides a way to select a subset of experts when the system defers.
Source: §1 · p. 1The naive conformal statistic is not robust to noise when multiple experts are correct, producing inflated, sometimes vacuous, prediction sets.
Source: §5 · p. 5
Abstract
from arXiv · showhide
We study the statistical properties of learning to defer (L2D) to multiple experts. In particular, we address the open problems of deriving a consistent surrogate loss, confidence calibration, and principled ensembling of experts. Firstly, we derive two consistent surrogates -- one based on a softmax parameterization, the other on a one-vs-all (OvA) parameterization -- that are analogous to the single expert losses proposed by Mozannar and Sontag (2020) and Verma and Nalisnick (2022), respectively. We then study the frameworks' ability to estimate P( m_j = y | x ), the probability that the jth expert will correctly predict the label for x. Theory shows the softmax-based loss causes mis-calibration to propagate between the estimates while the OvA-based loss does not (though in practice, we find there are trade offs). Lastly, we propose a conformal inference technique that chooses a subset of experts to query when the system defers. We perform empirical validation on tasks for galaxy, skin lesion, and hate speech classification.
1 INTRODUCTION
Complex tasks may need multiple experts, while hybrid intelligent systems must combine human and computational decisions. Multi-expert learning to defer must decide both when to defer and which expert to use, while accurately monitoring expert quality.
1 INTRODUCTION
- The need for multiple experts arises from tasks requiring distinct specialties or multiple opinions, such as construction projects and difficult medical diagnoses.Hybrid intelligent systems combine computational and human decision making.Source: §1 · p. 1
- Multi-expert learning to defer requires the rejector to decide both when to defer and which expert should handle an instance.Accurately monitoring expert quality is important for safe and effective deployment.Source: §1 · p. 1
- The paper develops the statistical foundations of multi-expert L2D through consistent surrogate losses, calibration analysis, and conformal selection of expert subsets.The methods are evaluated on image classification, galaxy categorization, skin lesion diagnosis, and hate speech detection.Source: §1 · p. 1
2 LEARNING TO DEFER
Learning to defer combines a classifier with a rejector that chooses whether the classifier or an expert should decide. This section reviews single-expert formulations and their softmax and one-vs-all surrogates.
2 LEARNING TO DEFER
- Single-expert L2D assumes labeled examples and expert demonstrations, with each expert prediction usually drawn from the label space.The training sample pairs features and labels with the expert's prediction.Source: §2 · pp. 1–2
- The rejector selects between the classifier's loss and the expert's loss, and under 0–1 loss the Bayes rule depends on the label and expert correctness probabilities.The probability P(m = y|x) is the chance that the expert is correct for input x.Source: §2 · p. 2
- The softmax surrogate unifies class labels and a rejection option in an augmented label space, with the test-time classifier and rejector chosen from their respective scores.Its minimizers agree with the Bayes-optimal solutions for the single-expert 0–1 L2D loss.Source: §2 · p. 2
- The OvA surrogate uses a binary surrogate loss for each class and rejection option, while retaining the same classifier and rejector decisions as the softmax formulation.The cited motivation is better calibration than softmax; the softmax parameterization can overestimate expert correctness in practice.Source: §2 · p. 2
3 L2D TO MULTIPLE EXPERTS
The paper extends L2D to multiple experts, whose rejector must choose among the classifier and individual experts. It derives two consistent surrogate losses and shows that the prior mixture-of-experts loss is inconsistent.
3 L2D TO MULTIPLE EXPERTS
- The multi-expert rejector maps each input to the classifier or one of J experts, and the Bayes-optimal choice depends on the class and each expert's correctness probability.The training data include each expert's prediction for each example.Source: §3 · p. 3
3.1 Softmax Surrogate Loss
- The softmax-based multi-expert surrogate is convex and calibrated for the 0–1 L2D loss, with minimizers corresponding to the Bayes-optimal classifier and rejector.Its construction uses class and expert-specific rejection functions in an augmented label space.Source: §3.1 · p. 3
3.2 One-vs-All Surrogate Loss
- The OvA surrogate is calibrated when its binary loss is strictly proper and has a well-defined continuous inverse link; it is consistent over all measurable functions.Under that hypothesis-class condition, its minimizers agree with the Bayes-optimal classifier and rejector.Source: §3.2 · pp. 3–4
3.3 Inconsistency of Mixture of Experts
- The prior mixture-of-experts model's negative log-likelihood is inconsistent for multi-expert L2D.At test time, that model assigns responsibility according to the largest mixture weight.Source: §3.3 · p. 4
4 CONFIDENCE CALIBRATION
The calibration analysis focuses on whether a rejector estimates each expert's probability of being correct. The softmax parameterization couples those estimates, whereas OvA models each expert's correctness directly.
4 CONFIDENCE CALIBRATION
- Calibrated expert-correctness estimates indicate how often an expert should be right at a stated confidence level, supporting deferral decisions and interpretability about what the model thinks a human knows.The section focuses on the rejector rather than classifier calibration.Source: §4 · p. 4
4.1 Softmax Parameterization
- Softmax estimates can be unbounded and depend on other experts' estimates, so one miscalibrated expert can propagate error to the others.As the sum of the deferral scores approaches one, the estimate can grow to infinity.Source: §4.1 · p. 4
4.2 One-vs-All Parameterization
- OvA directly models each expert's correctness probability, with estimates constrained to (0, 1) and no dependence across experts' deferral functions.The paper expects these properties to improve calibration in practice.Source: §4.2 · p. 5
5 ENSEMBLING EXPERTS WITH CONFORMAL INFERENCE
The section adapts conformal inference to form input-dependent sets of experts that cover the best expert with marginal probability 1−α. These sets can be used as dynamic ensembles, while a regularized statistic addresses noise that inflates naive set sizes.
Conformal Sets of Experts
- Conformal sets of experts target marginal 1−α coverage of the best expert for a new observation.The resulting set has a dynamic size that changes with x and can be used to aggregate the predictions of all included experts.Source: §5 · p. 5
Naive Statistic
- The method ranks experts by estimated correctness and adds them in descending score order until all correct experts are included.The scores are estimated from the rejector under either the softmax or OvA formulation.Source: §5 · p. 5
Regularized Statistic
- When multiple experts are correct, noise can obscure the best expert and make the naive statistic produce inflated, sometimes vacuous sets.The regularized statistic uses conformal risk control to directly control the false negative rate.Source: §5 · p. 5
- The regularized sets use parameters β and κ, with λα chosen to provide 1−α coverage guarantees.κ thresholds low confidences; β is chosen to optimize set size.Source: §5 · pp. 5–6
6 RELATED WORK
This work extends consistent multiclass learning-to-defer losses and calibration analysis to multiple experts, a setting existing approaches do not fully address. It also applies conformal inference to sets of experts, unlike prior human-AI collaboration work described here.
6 RELATED WORK
- The paper extends the consistent softmax and OvA surrogate losses for multiclass learning to defer to the multi-expert setting.Earlier multi-expert extensions lacked consistency guarantees, while Hemmer et al. (2022) was the only existing model for multi-expert L2D described here.Source: §6 · p. 6
- The paper studies native, out-of-the-box calibration of the losses rather than focusing on post-hoc recalibration.The authors note that post-hoc methods, such as adding a temperature parameter to OvA per-expert terms, could potentially be applied.Source: §6 · p. 6
Conformal Inference for Human-AI Collaboration
- Prior conformal human-AI collaboration work passes classifier prediction sets to humans, but does not use conformal inference to obtain sets of experts.The cited workflows include applying conformal inference only to non-deferred samples to reduce set sizes.Source: Conformal Inference for Human-AI… · p. 6
7 EXPERIMENTS
Experiments compare the proposed surrogates and conformal ensembles on real-world classification tasks and CIFAR-10 simulations. OvA generally shows stronger system accuracy or calibration, while conformal ensembling helps most when few oracle experts are available.
7.1 Overall System Accuracy
- OvA accuracy never falls below the one-classifier baseline on HAM10000 and Galaxy-Zoo, while the softmax and MoE accuracies do.The three methods perform comparably on HateSpeech.Source: §7.1 · p. 7, Figure 1 · p. 7
7.2 Confidence Calibration
- OvA stays near 2% ECE as experts increase, while softmax has roughly double the ECE at about 4.5%.OvA has better accuracy with 12 or fewer experts, but softmax has the best accuracy at 20 experts despite worse calibration.Source: §7.2 · p. 8, Figure 2 · p. 8
- With one expert held at random chance, OvA models it well across other experts’ abilities, whereas softmax ECE rises almost in-step with their correctness.OvA also has better calibration across the simulated settings and on the three real-world data sets; the MoE has extremely poor calibration.Source: §7.2 · p. 8
7.3 Conformal Ensembles
- With one or two oracles, conformal ensembles achieve around 90% CIFAR-10 accuracy, versus below 80% for the fixed five-expert ensemble; both tie with three or more oracles.For expert identification, regularized statistics perform well with and without noise, while naive statistics inflate set sizes and perform terribly with noise.Source: §7.3 · p. 9, Figure 3 · p. 9
8 CONCLUSIONS
The paper concludes that it extends learning to defer to multiple experts with two consistent objectives, calibration analysis, and conformal selection of expert ensembles. Its appendix defines consistency as surrogate-risk convergence implying convergence to Bayes-optimal prediction risk.
8 CONCLUSIONS
- The paper extends L2D to multiple experts and proposes two optimization objectives, both proved consistent.The objectives can be embedded into empirical risk minimization frameworks.Source: §8 · p. 9
- The conclusion notes that the softmax-based objective can produce mis-calibrated models in practice and describes a procedure for selecting minimal expert sets to ensemble.Future work includes extending active-learning results to multiple experts and adapting post-hoc correction for settings where expert queries incur additional cost.Source: §8 · p. 9
A Proofs and Derivations
- Consistency means that minimizing the surrogate yields predictions whose expected error converges to the Bayes risk.The proof section states that it derives the multi-expert Bayes-optimal rule and proves consistency of the proposed losses.Source: A Proofs and Derivations · p. 12
A.1 Bayes Rule for Learning to Defer with Multiple Experts
With multiple experts, the Bayes rule chooses the classifier or the expert with the highest correctness confidence. The section establishes surrogate consistency and calibration properties, and shows that the MoE objective can admit non-optimal decisions.
A.1 Bayes Rule for Learning to Defer with Multiple Experts
- The Bayes rule compares classifier and expert expected losses, deferring when the classifier’s expected loss is at least the minimum expert expected loss.Under 0-1 loss, this is equivalent to choosing whichever has the highest confidence.Source: §A.1 · pp. 13–14
A.2 Proof of Theorem 3.1: Consistency of ψJ
- The softmax surrogate’s minimizers agree with the Bayes solution, as shown by the proof’s comparison of the optimal classifier and rejector conditions.The proof assumes the experts’ predictions are independent given the features and label.Source: §A.2 · pp. 14–15
A.4 Inconsistency of the Mixture of Experts Formulation (Hemmer et al., 2022)
- The MoE objective is inconsistent: it can attain its infimum while choosing the classifier over a perfectly accurate expert, yielding excess error.The construction demonstrates faulty solutions that may fail to reach the Bayes-optimal predictor.Source: §A.4 · pp. 16–17
B Choice of Hyperparameters for Regularized Conformal Ensembles
This section describes hyperparameter selection for regularized conformal ensembles and the associated prediction sets. It also states that the regularized sets are no larger than the unregularized sets.
B Choice of Hyperparameters for Regularized Conformal Ensembles
- The procedure selects λ using a grid of 1,500 equally spaced values in [0, 1] and selects κ using the described algorithm.The procedure also has hyperparameters κ and β for the regularized conformal ensemble.Source: B Choice of Hyperparameters for… · p. 17
- The regularized conformal set Cλ is guaranteed to be no larger than C2.Cλ depends on λ and β, while C2 selects experts whose scores meet the threshold κ∗.Source: B Choice of Hyperparameters for… · p. 18
- Thirty percent of deferred samples are reserved to tune β and κ, with the remaining deferred samples used for the regular conformal procedure.β is tuned by grid search.Source: B Choice of Hyperparameters for… · p. 18
C Experimental Setup for Simulated Experts
The experiments simulate expert configurations for three classification datasets. Galaxy-Zoo and HateSpeech experts draw on human annotations and perturbations, while HAM10000 uses constructed expert types because individual annotator predictions are unavailable.
C Experimental Setup for Simulated Experts
- The study simulates 10 experts of increasing ability for the overall system accuracy experiments.The configuration descriptions cover Galaxy-Zoo, HateSpeech, and HAM10000.Source: C Experimental Setup for Simulated… · p. 18
C.1 Hate Speech and Galaxy-Zoo
- Galaxy-Zoo and HateSpeech configurations use human annotations to define human, flipping-human, and probabilistic experts.The experts’ configurations are listed in Table 1.Source: §C.1 · p. 18, Table 1 · p. 18
C.2 HAM10000
- HAM10000 configurations include random experts, category-specialized dermatologist experts, and predictions from an 8-layer MLPMixer with access to additional metadata.The simulated dermatologist configurations progress from a random expert toward one simulating an experienced dermatologist.Source: §C.2 · pp. 18–19, Table 2 · p. 19
D.1 Overlapping expertise among experts
As expertise overlap increases from specialized to fully overlapped CIFAR-10 experts, regularized conformal sets are more dynamic than naive sets. The naive method’s sets remain close to all experts, while regularization yields smaller sets with an accuracy drop at lower overlap.
D.1 Overlapping expertise among experts
- The CIFAR-10 experiment uses 10 experts, each oracle for one class, and varies their overlap on other classes from 10% to 95%.This spans specialized to fully overlapped experts.Source: §D.1 · p. 19
- Naive conformal sets stay close to the total number of experts across specialized and overlapped cases, while regularized sets are more dynamic.The regularized statistic filters confidences below a threshold, allowing smaller sets.Source: §D.1 · p. 19, Figure 4 · p. 20
- Naive conformal accuracy is slightly better than a fixed-size ensemble, while regularized accuracy drops at lower overlap as its smaller sets include fewer chance-correct experts.The adaptive sets are nevertheless less conservative.Source: §D.1 · pp. 19–20, Figure 4 · p. 20