ReCIRC: Rectified Conformal Risk Control

1 Introduction

• ReCIRC inverts each input’s estimated local risk curve to create a shared risk-budget scale, then applies CRC without changing its calibration step.The finite-sample marginal guarantee does not depend on the accuracy of the estimated curves, though poor estimates can reduce local adaptivity.Source: §1 · p. 3
• In polyp segmentation, ReCIRC keeps conditional false-negative rates closer to the target across difficulty levels than global CRC.The comparison uses conditional FNR, defined as the average fraction of omitted ground-truth polyp pixels within each difficulty bin.Source: §1 · p. 3

1.1 Relation to Other Work

• Unlike approaches that coarsen the conditioning target or alter calibration, ReCIRC retains ordinary calibration and learns a conditional mean-loss scale.The paper also distinguishes ReCIRC from adaptive risk control: it is offline and its pointwise and group-level conclusions depend on risk-estimation error.Source: §1.1 · pp. 5–6

1.2 Novelty

• Risk-curve inversion extends risk-based parameterization to bounded monotone losses across general ordered decision families, rather than individual tasks.The same fitted curve can be recalibrated for multiple risk targets.Source: §1.1 · p. 6, §1.2 · p. 7
• Risk-estimation error yields pointwise, most-input, and group-conditional bounds; under attainability and regularity conditions, deployed conditional risk converges to the target.Finite-sample marginal validity holds regardless of risk-estimation accuracy.Source: §1.2 · p. 7
• The risk-scale diagnostic compares realized losses with advertised budgets overall and within covariate groups, using no data beyond calibration.It is a by-product of calibration.Source: §1.2 · p. 7

2.1 Setting and the CRC procedure

• CRC’s setting uses exchangeable calibration and test pairs, a bounded loss, and an ordered family in which larger thresholds are more protective.The calibration pairs are observed; the test response is unobserved.Source: §2.1 · p. 8
• The loss is non-increasing as protection grows; examples include false-negative proportions for segmentation masks and label sets.The family is treated as fixed, with model-fitting data independent of calibration and test points.Source: §2.1 · p. 8

2.2 The limitation: a global threshold is not a risk scale

• CRC’s guarantee averages risk over the covariate distribution, so one global threshold can leave hard inputs above target while putting easy inputs below it.The same raw threshold can intersect easy and hard inputs’ conditional risk curves at different risk levels.Source: §2.2 · p. 9

3 Our approach: ReCIRC

• ReCIRC changes the shared raw threshold into input-specific thresholds chosen to target a common conditional risk, addressing the easy-versus-hard imbalance under global CRC.Figure 2 contrasts shared λ with input-specific thresholds R−1(a | x).Source: §3 · p. 9, Figure 2 · p. 10

3.1 Oracle risk rectification

• Inverting each input’s local risk curve makes a common budget a a conditional-risk upper bound; when the curve is continuous and crosses a, the achieved risk equals a.Otherwise the oracle scale may be conservative.Source: §3.1 · p. 11

3.2 The ReCIRC procedure

• The procedure uses D to estimate local risk curves and C to calibrate the resulting monotone one-parameter family with CRC, preserving marginal validity regardless of curve quality.The largest budget that passes CRC is selected for deployment.Source: §3.2 · p. 12, Figure 3 · p. 13

3.3 Estimating the local risk curve

• A single regression gθ(x, λ) estimates conditional loss from augmented observations; its fitted curve should respect monotonicity, and reusing base-model training data can cause optimistic bias.The bias can be reduced by separating samples, cross-fitting, or using out-of-sample losses.Source: §3.3 · pp. 13–14

3.4 Special case: rectifying conformal prediction scores

• For scalar-score conformal prediction, rectification is local score recalibration through the conditional score CDF, yielding an input-specific threshold for a local miscoverage budget.This recovers a probability-integral-transform construction used for approximate conditional coverage.Source: §3.4 · pp. 14–15

Appendix E.4 extends this quantile interpretation to general bounded monotone losses:

• For general bounded monotone losses, risk rectification is a local quantile transformation of a latent variable that need not be an observed conformity score.This extends the quantile interpretation beyond score-based conformal prediction.Source: Appendix E.4 · p. 15

3.5 A risk-calibration diagnostic for the risk scale

• The risk-scale diagnostic plots realized group risk against budget; deviations from the diagonal reveal mis-scaling, while below-diagonal curves may also reflect discrete predictions or minimum-size limits.The calibration-set overall curve requires no extra data; group curves assess averages within chosen bins, where opposite deviations can cancel.Source: §3.5 · pp. 15–16

4.1 Finite-sample marginal risk control

• Finite-sample marginal risk control holds regardless of estimated-curve quality under exchangeability, a bounded monotone family with a fully protective rule, a finite budget grid, and the stated sample-size condition.The unconditional risk is also at most α.Source: §4.1 · pp. 16–17

4.2 Conditional risk control

• Oracle rectification bounds conditional risk by a; equality requires a continuous local risk curve and a budget no greater than the least-protective rule’s risk.With estimated curves, the conditional bound degrades by the amount of risk underestimation and is calibrated relative to b_a, which can differ from α.Source: §4.2 · pp. 17–18
• With accurate uniformly estimated curves, shrinking threshold and budget grids, increasing training and calibration samples, and attainable target risk, deployed conditional risk approaches α.Under covariate shift preserving Y | X, deployed risk is controlled by the worst local underestimation of the fitted curves.Source: §4.2 · pp. 18–20

5.1 Experimental setup

• The experiments used three synthetic heterogeneity designs and five real-data benchmarks, with ReCIRC’s local risk estimates fit using TabICLv2 and independent risk-training and calibration sets.Evaluation used task-appropriate bounded losses and grouped observations by prediction difficulty; reported summaries include worst-group risk, mean positive group excess, prediction size, and marginal risk.Source: §5 · p. 20, §5.1 · pp. 20–22, §5.2 · pp. 23–24

5.3 Results

• Across all eight settings, ReCIRC attained the lowest average worst-group risk and mean positive group excess, while prediction-size effects varied across applications.Some advantages over AA-CRC were within sampling variability.Source: §5.3 · p. 25, Table 1 · p. 28, Table 2 · p. 28
• In the three synthetic settings, ReCIRC had the lowest average worst-group risk and mean positive group excess across one-dimensional difficulty and XOR interaction heterogeneity.The XOR setting had average worst-group risks of 0.1130 for ReCIRC, 0.1166 for AA-CRC, and 0.1338 for global CRC.Source: §5.3 · p. 25
• Across five real-data benchmarks, ReCIRC had the lowest average worst-group risk and mean positive group excess in every setting, with particularly pronounced gains in RCV1 and Medical Insurance.In Letter Recognition, worst-group risk remained above α for all methods because every top-m set contains at least one class.Source: §5.3 · p. 25
• ReCIRC’s mean marginal risk ranged from 0.0908 to 0.0985 across the eight experiments, close to α = 0.10; its groupwise gains over global CRC were not explained by lower marginal risk.The CRC guarantee concerns expected loss over calibration and a new observation, so test risk need not stay below α in every repetition.Source: §5.3 · p. 26
• In polyp segmentation, held-out risk curves approximately followed the diagonal for budgets a ∈[0.05, 0.20], overall and within five uncertainty groups.At larger budgets, mean curves generally fell below the diagonal, indicating conservative risk relative to the budget.Source: §5.3 · pp. 26–27, Figure 4 · p. 27
• Compared with global CRC, ReCIRC reduced average Letter Recognition set size from 3.100 to 2.210 and interval width by approximately 29% in Medical Insurance and 12% in Superconductor.In some applications, improved groupwise risk came with larger predictions; overall, size effects were application-dependent.Source: §5.3 · pp. 26–28

B Response-model risk-curve estimation

• A model of Y | x can estimate risk curves using the same samples across losses and decision families, preserving pointwise monotonicity without post-processing.When the loss is monotone, reusing the same sampled responses for every threshold makes the Monte Carlo curve automatically monotone; the estimated curve is then inverted on the grid.Source: B Response-model risk-curve estimation · p. 41
• Response-model estimation can be costly when sampling or scoring a large output space, while direct loss regression targets a scalar response.Direct loss regression avoids those costs but requires a loss-specific fit and explicit monotonicity handling.Source: B Response-model risk-curve estimation · pp. 41–42

• All experiments are repeated 20 times.Source: Table 5 · p. 43
• Table 5 summarizes how the AA-CRC feature map parameterizes the input-dependent score threshold.Source: Table 5 · p. 43
• The table reports ReCIRC split sizes for risk training, calibration, and testing, with further setup details deferred to later subsections.Source: Table 5 · p. 43

• The heteroscedastic experiment fits its GAM mean model on an independent sample of 500, separate from the 1,000 risk-training observations.Source: §C.1 · p. 44
• Synthetic multilabel scores come directly from the data-generating mechanism, while polyp segmentation uses precomputed PraNet probability maps without retraining PraNet.Source: §C.1 · p. 44
• RCV1 fits a one-vs-rest logistic classifier on 12,000 documents; Letter Recognition fits logistic regression on a 6,000-observation base split.Source: §C.1 · p. 44
• Medical Insurance, Superconductor, and XOR fit QRFs on D, using out-of-bag predictions for risk-training features and losses, and ordinary predictions for calibration and testing.Source: §C.1 · p. 44

C.2 ReCIRC implementation details

• The implementation uses task-specific threshold and budget grids, with nested prediction families ranging from intervals and score-threshold sets to top-m classes.The asymmetric regression intervals use estimated conditional medians and QRF-derived lower and upper scales.Source: §C.2 · pp. 44–45
• In regression settings, the finite threshold grid ends at λ = 4, with a fully protective endpoint available as a feasibility fallback but unused in reported repetitions.All reported calibrated budgets were positive, so predictions came from the finite numerical grid.Source: §C.2 · pp. 45–46

C.3 Risk-regression representations

• Risk-regression representations combine task-specific covariates and prediction-difficulty summaries, with QRF-derived features for the three QRF settings.Source: §C.3 · p. 46
• TabICLv2 is used throughout, with estimator counts and context or sample limits varying by setting.Source: §C.3 · p. 46

C.4 AA-CRC implementation

• AA-CRC variants use the authors’ objective with SLSQP, while regularization and coefficient constraints vary by setting.Source: §C.4 · p. 47
• AA-CRC fits θ on C and deploys it without further calibration except in polyp segmentation and Letter Recognition, with marginal guarantees only approximate in most settings.For Letter Recognition, a scalar offset is calibrated on C by CRC; in other settings, guarantee slack grows with the dimension of Φ.Source: §C.4 · p. 47

• Evaluation groups are not supplied to any method and are used only to calculate worst-group risk and mean positive group excess.Source: §C.5 · p. 47
• Heteroscedastic and latent-difficulty experiments use five equal-mass bins; polyp segmentation uses five uncertainty bins, while RCV1 and Letter Recognition group by predictive entropy.Source: §C.5 · p. 48
• Medical Insurance uses overlapping smoker, BMI, and age groups, while Superconductor uses intersections of two conductivity-variable splits and XOR uses four sign quadrants.Superconductor’s grouped analysis is exploratory because its splitting variables were selected using their association with the response in the full dataset.Source: §C.5 · p. 48

D ReCIRC Calibration Diagnostics

• Diagnostic groups use fixed within-repetition bins of known simulation quantities or covariate/model-derived scores, except Superconductor groups selected using response correlations.The Superconductor diagnostic is exploratory because its group selection used responses.Source: D ReCIRC Calibration Diagnostics · pp. 48–49
• Departures above the risk-budget diagonal are small (at most about 0.03) and occur mainly in the noisiest heteroscedastic bins and, at small budgets, harder latent-difficulty bins.Discrete choices of m can produce conservative risk even with exact conditional-risk estimates; low-entropy groups plateau when m reaches 1.Source: D ReCIRC Calibration Diagnostics · p. 56

E.1 Most-input and group consequences

• The held-out group-risk calibration curve provides a practical check of the population quantity controlled by Proposition E.1.The proposition's stated consequences hold under its conditions; the supplied passage does not give their explicit bounds.Source: §E.1 · p. 56

E.2 Rate transfer from risk-curve estimation

• Proposition E.2 addresses transfer of risk-curve estimation error to rectified risk under stated regularity conditions.The supplied passage identifies the proposition and its setup but does not provide the bound itself.Source: §E.2 · p. 57

E.3 A general deployed-risk rate

• Theorem E.3 formulates a general deployed-risk rate for growing training and calibration sequences, with a refining budget grid and stated assumptions.The theorem also distinguishes a high-level marginal risk-scale condition from a calibration conclusion.Source: §E.3 · pp. 57–58

E.4 Rectification as a local quantile transformation

• The oracle rectified rule replaces the raw threshold with a local quantile of a latent variable whose upper tail is the conditional risk curve.For miscoverage, the latent variable has the conditional law of the score; for general bounded monotone loss, it need not be an observed score.Source: §E.4 · pp. 58–59

F.1 Proof of Theorem 4.4

• Applying CRC to the rectified family preserves its finite-sample marginal risk guarantee through the ordinary exchangeability argument.The proof uses a finite ordered index set and a feasible maximum satisfying the CRC constraint.Source: §F.1 · p. 60

F.2 Proof of Propositions 4.5 and 4.6

• Under continuity and an attainable budget, the inverse risk curve reaches that budget exactly; otherwise the selected threshold's risk is bounded above by the budget.The proof's equality claim requires a ≤ R(λmin | x).Source: §F.2 · pp. 60–61

F.3 Proof of Proposition E.1

• The group-risk consequence follows by averaging the most-input bound over a measurable group of positive probability.The proof also permits substitution of the calibrated budget under the additional i.i.d. condition.Source: §F.3 · p. 61

F.4 Proof of Proposition E.2

• The rectified risk error is bounded by risk-curve estimation error plus a Lipschitz constant times threshold-grid spacing.The proof gives the bound uniformly over attainable budgets and derives the stated rate under additional conditions.Source: §F.4 · pp. 61–62

F.5 Proof of Theorem E.3

• Nested, nondecreasing losses allow a uniform calibration bound by reducing grid-indexed events to one-dimensional threshold events.Conditional on the risk-training data, the empirical losses are i.i.d. and bounded.Source: §F.5 · pp. 62–63
• The deployed conditional-risk error is O_P(ρm + ηm + hM + n^-1/2 + hn) under the theorem's assumptions.The proof also establishes a uniform calibration-sample error of O_P(n^-1/2) over every finite budget grid, without a condition on grid size.Source: §F.5 · pp. 63–64

F.6 Proof of Corollary 4.7

• Corollary 4.7's argument extends the deployed-risk rate uniformly over inputs under its common-Lipschitz and uniform-bound conditions.The proof places the calibrated budget in the relevant interval with probability tending to one.Source: §F.6 · pp. 64–65

F.7 Proof of Corollary 4.8

• The pointwise conditional-risk bound also holds under Q when the conditional response law is unchanged and QX is absolutely continuous with respect to PX.Integrating the bound over QX establishes the stated distribution-shift result.Source: §F.7 · p. 65

F.8 Proof of Proposition E.4

• A canonical latent variable has a survival function equal to the original local risk curve, yielding the same generalized inverse and rectification map.The construction sets Fx(λ) = 1 − R(λ | x).Source: §F.8 · pp. 65–66

Worked examples

• Rectification changes how much protection each input receives, while the family determines what greater protection means and requires an ordered scalar path.The examples also rely on a fully protective endpoint, attainable budgets, and monotone realized losses unless otherwise stated.Source: style losses · p. 66

Missed positives: masks, label sets, and retrieval sets

• For missed-positive sets, rectification makes the score cutoff input-specific, giving easy images tighter masks and diffuse-relevance queries deeper retrieval.Ambiguous inputs receive larger label sets, while hard segmentation inputs receive larger masks.Source: style losses · pp. 66–67

Hierarchical graph distance

• For hierarchical labels, rectification controls granularity: easier inputs can receive leaf-level neighborhoods and harder inputs larger subtrees or coarser nodes.The loss vanishes once a selected node lies on the true label's ancestral path.Source: style losses · p. 67

Token-level F1 for question answering

• Question-specific candidate counts are possible with token-level F1, but the fully protective endpoint may retain positive loss unless a candidate exactly matches a reference.Rectifying excess loss instead changes the controlled quantity by the endpoint residual risk.Source: style losses · p. 68

Ordinal losses

• Ordinal rectification selects a point on the pre-specified interval path, so it cannot alter that path's geometry.For divergence loss, saturation can leave larger budgets conservative once a singleton already has low loss.Source: style losses · p. 68

False positives and shrinking masks

• False-positive fraction can be controlled with a shrinking-mask family, but false discovery proportion is generally non-monotone along that family.CRC therefore requires a pointwise-monotone loss, a monotone upper envelope, or a method for non-monotone losses.Source: style losses · pp. 68–69