PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling-Score Conformal Prediction for Multi-Target Regression

Sylvain Rousseau, Soundouss Messoudi

arXiv:2609.17091v1cs.AI

TL;DR

Joint multi-target prediction needs regions that cover all response components together, but current methods face trade-offs in scale adaptation, validity, calibration data use, or model requirements. The paper proposes a scaling-score approach that produces nested regions from one calibration set. Across 29 real-world datasets, it reports valid joint coverage and competitive volume, with the SC2 advantage over Max growing with output dimension.

  • Problem

    Constructing small, jointly valid multi-target regions with model-agnostic methods remains challenging because existing approaches have trade-offs in scale adaptation, calibration-set use, validity, or model requirements.

    Source: §2.2 · p. 3

  • Method

    The scaling-score method uses component-wise absolute residuals and one calibration set to produce four nested prediction regions, including a valid staircase and outer rectangle.

    Source: §1 · p. 2, §6 · p. 18

  • Results

    SC2 with γ = 1 −α achieved mean rank 2.03 and ranked first on 12 of 29 datasets; its volume advantage over Max grew with output dimension.

    Source: §5.2 · p. 16, §6 · p. 18

  • Takeaways & Limitations

    Across the evaluated datasets, SC2 offers competitive volume while SCO is the strongest fully rectangular option among the evaluated baselines.

    Source: §5.2 · p. 16, §6 · p. 18

    The exact region Rα has broken decision-line boundaries and requires evaluating all calibration scores for each query.

    Source: §4.5 · p. 11

Abstract

from arXiv · show

Multi-target regression requires a model to simultaneously predict several related outputs. Conformal prediction provides distribution-free, finite-sample marginal coverage guarantees, but extending these to joint multi-dimensional regions in a model-agnostic, sample-efficient manner remains challenging: max-aggregation ignores scale differences, copula-based methods are only asymptotically valid, rectangular methods typically split the calibration set, and quantile or density-based methods require training a specialised model beyond a plain point predictor. We propose the scaling-score conformal method, which is model-agnostic (requires only component-wise absolute residuals), uses a single calibration set, and yields four nested output types: an outer rectangle (SCO) with valid joint coverage, the exact set R $\alpha$ , a staircase (SC 2 ) over approximation of R $\alpha$ , and an inner rectangle (SCI). A single hyperparameter $\gamma$ $\in$ (0, 1) controls the base-rectangle quantile level independently of $\alpha$. We prove downward-closedness and a rectangular sandwich bound and derive a closed-form outer rectangle. Experiments on 29 realworld datasets confirm valid joint coverage; SC 2 with $\gamma$ = 1-$\alpha$ consistently achieves competitive volume relative to baselines, with the advantage growing with output dimension d.

1 Introduction

Joint multi-target decisions need regions that cover all outputs together, but existing approaches face trade-offs in scale adaptation, validity, calibration data use, or model requirements. The paper introduces a single-calibration scaling-score method with nested prediction regions and theoretical and empirical results.

1 Introduction

  • Applications with correlated outputs need joint prediction regions because decisions can depend on the outputs’ joint distribution.Marginal error bars for each target individually are insufficient for these decisions.Source: §1 · p. 1
  • Existing approaches trade off scale adaptation, validity, calibration-set use, or model simplicity.Max aggregation can be over-conservative for heterogeneous target scales; copula guarantees are at best asymptotic; CHR splits calibration data; and quantile or density methods require additional trained models.Source: §1 · pp. 1–2
  • The scaling score uses per-target calibration quantiles to scale a base vector, producing four nested regions from one calibration set.These are the valid but conservative SCO rectangle, exact Rα, valid staircase SC2, and compact SCI without a formal coverage guarantee.Source: §1 · p. 2
  • Experiments on 29 real-world datasets confirm valid joint coverage and competitive efficiency across target dimensions.The experiments cover a range of target dimensions.Source: §1 · p. 2
  • The paper characterizes Rα cell by cell, derives rectangular sandwich bounds with closed-form expressions, and proves downward-closedness.Source: §1 · p. 2

2 Background and Related Work

Conformal prediction provides finite-sample coverage under exchangeability, while multi-target methods must control joint coverage without making regions inefficient. Existing approaches vary in their assumptions, calibration-data requirements, and region geometry.

2.1 Conformal prediction for regression

  • Split conformal prediction uses a held-out calibration set and a score quantile to form regions with finite-sample coverage under exchangeability.Standard regression scores include absolute residuals and conformalized quantile regression scores.Source: §2.1 · pp. 2–3

2.2 Conformal prediction for multi-target regression

  • Joint coverage requires the full response vector to lie in the region, while Max aggregation can over-cover heterogeneous targets with an overly large hypercube.Coordinate-wise Bonferroni correction is described as even more conservative.Source: §2.2 · p. 3
  • Copula methods model residual dependence, but empirical copulas are only asymptotically valid and parametric copulas assume a specified error distribution (Messoudi et al., 2021).Source: §2.2 · p. 3
  • CHR achieves finite-sample joint coverage and asymptotic marginal balance, but its two-stage calibration split reduces the sample available for the conformal quantile (Sampson and Chan, 2024).The passage notes possible region inflation when calibration samples are moderate or output dimension is large.Source: §2.2 · p. 3
  • TSCP is valid, model-agnostic, and uses one calibration set, while density-based methods can model non-rectangular regions but require a conditional density estimator.TSCP standardizes using calibration-residual means and standard deviations; density methods add modeling complexity.Source: §2.2 · p. 3

3 Problem Setup and Notation

The setup considers a pre-trained point predictor and an exchangeable calibration sample, then seeks a prediction region with specified joint coverage.

3 Problem Setup and Notation

  • The response is d-dimensional, and a pre-trained point predictor maps each feature input to a response vector.Source: §3 · p. 4
  • Exchangeability of the calibration observations is the sole assumption underlying the stated validity guarantees.Source: §3 · p. 4
  • For a candidate test response, the component-wise absolute residual vector is used in the goal of constructing a region with coverage at least 1 − α.The setup also defines component-wise ordering for vectors.Source: §3 · p. 4

4 The Scaling-Score Method

The scaling-score method constructs joint prediction regions by dilating a per-target base rectangle, with test-dependent construction analyzed cellwise and conformal coverage established under exchangeability. It relates four regions through a rectangular sandwich, with coverage and containment depending on the calibration regime and γ controlling scale adaptation and staircase structure.

4.1 Scaling score

  • The scaling score is the minimum dilation needed for the base rectangle to contain a residual; thresholding it gives an axis-aligned scaled rectangle.Source: §4.1 · p. 4

4.2 Lower and upper base rectangles

  • Under exchangeability, the conformal region achieves joint marginal coverage of at least 1 − α despite the test-dependent base rectangle.The base sides depend on the test residual through its rank in the augmented calibration set, preserving exchangeability of the scores.Source: §4.2 · p. 5

4.3 Cell decomposition and scores

  • The coordinate thresholds partition residual space into 3^d cells, comprising 2^d corner cells and 3^d − 2^d mixed cells.The base vector is fixed within corner cells and depends on test-error middle coordinates within mixed cells.Source: §4.3 · pp. 6–7

4.4.1 Corner cells

  • On corner cells, the base vector and conformal quantile are fixed, giving the exact prediction region an explicit form.In the all-low corner cell, the region contains the origin and equals the full cell.Source: §4.4.1 · pp. 7–8

4.4.2 Mixed cells

  • On mixed cells, the exact region’s quantile depends on the test error’s middle coordinates, is non-increasing in each, and matches corner-cell values at cell boundaries.The resulting boundary is continuous but generally implicit; the region can be sandwiched between inner and outer axis-aligned rectangles.Source: §4.4.2 · pp. 8–9

4.4.3 Structural properties

  • The exact region Rα is downward-closed: if it contains a residual, it also contains every component-wise smaller residual.Source: §4.4.3 · p. 9

4.5 Nested prediction regions

  • One calibration run yields four nested outputs: valid exact Rα, valid staircase over-approximation, valid conservative SCO rectangle, and compact SCI rectangle without formal coverage.The staircase is an explicit union of at most 2^d rectangles; the exact region’s mixed-cell boundary can require evaluating calibration scores for each query.Source: §4.5 · pp. 9–11
  • All 2^d corner rectangles are computable at calibration time, and retaining only Pareto-optimal rectangles removes those dominated component-wise.The number retained is often well below 2^d.Source: §4.5 · p. 12
  • When λ̂ub ≥ 1, the four prediction regions form a nested chain; the staircase and outer rectangle contain Rα and are valid, while the compact inner rectangle may under-cover.When λ̂ub < 1, the staircase and outer rectangle can be incomparable.Source: §4.5 · p. 12

4.6 Choice of γ

  • Small γ sets thresholds at low marginal quantiles, so the outer quantile must expand the region uniformly and dilutes per-target scale adaptation.The base rectangle then under-represents large errors on each target.Source: §4.6 · p. 13
  • Large γ makes thresholds sensitive to extreme calibration errors; above γrect, the region shrinks inside the base rectangle and loses the non-rectangular staircase structure.In this regime, λ̂ < 1.Source: §4.6 · p. 13
  • Because joint domination is harder than marginal domination unless targets are perfectly co-monotone, γrect ≥ 1 − α, with equality only under perfect positive dependence.At γ = γrect, the inner rectangle approximates the empirical copula rectangle, while the outer rectangle retains a finite-sample conformal guarantee.Source: §4.6 · p. 13
  • The method recommends γ < γrect, with γ = 1 − α as a natural default that experiments show consistently matches or outperforms other γ values across datasets and dimensions.This choice is tied to the coverage level and always satisfies γ ≤ γrect in practice.Source: §4.6 · p. 13

5 Experiments

Experiments across 29 real-world datasets evaluate joint coverage, volume, staircase compactness, and calibration runtime; SC2 with γ = 1 −α leads volume rankings, while all but one reported configuration achieve valid coverage.

5.1 Protocol

  • The protocol uses a 100-tree random forest, a 60/15/25 train-calibration-test split, 20 seeds, and 29 datasets spanning 2–16 targets.Joint coverage targets at least 1 −α, while volume is measured as log10 volume; lower is better at fixed coverage.Source: §5.1 · p. 14
  • Comparisons include Max, CHR, TSCP, SC2 at γ = 1 −α, 0.8, or γrect, and SCO at γ = 1 −α.The γrect setting is selected on calibration data; γ = 0.8 is a fixed coverage-level-agnostic choice.Source: §5.1 · p. 14

5.2 Results

  • All methods achieve joint coverage near the 0.90 target except SC2 with γ = 0.8 on Births 2, where zero residuals violate an assumption.Choosing γ ≥ 1 −α avoids this failure in the reported cases.Source: §5.2 · p. 14, Table 1 · p. 15
  • SC2 with γ = 1 −α achieves the best mean volume rank (2.03), ranks first on 12 of 29 datasets, and gains more over Max as output dimension grows.Among rectangular methods, SCO has a better mean rank than Max, CHR, and TSCP.Source: Table 2 · p. 16, §5.2 · p. 16
  • With γ = 1 −α, the mean fraction of staircase rectangles surviving pruning falls from 0.57 at d = 2 to below 0.001 at d ≥12.For γrect, the staircase collapses to one corner on 25 datasets, but sparse calibration sets on some d = 16 datasets leave thousands of rectangles active.Source: Table 3 · p. 17, §5.2 · p. 17
  • At d = 8 and ncalib = 300, calibration costs remain milliseconds or less; fixed-γ staircase variants take under 3 ms, versus about 7 ms for γrect.The γrect scan adds O(n) candidate base-rectangle levels, each requiring an O(nd) pass.Source: Table 4 · p. 18, §5.2 · p. 18

6 Conclusion

The conclusion presents scaling-score conformal prediction as a single-calibration method yielding nested multi-target regions and theoretical characterizations. Experiments on 29 datasets report valid joint coverage and competitive volume, with the advantage over max aggregation growing with dimension.

6 Conclusion

  • One calibration run produces four nested regions: SCI, exact Rα, staircase SC2, and closed-form outer rectangle SCO.SCI lacks a formal coverage guarantee; the other three are valid.Source: §6 · p. 18
  • The analysis characterizes Rα cell by cell, proves downward-closedness, and derives closed-form rectangular sandwich bounds.Source: §6 · p. 18
  • Across 29 datasets and dimensions 2–16, experiments confirm valid joint coverage; SC2 is competitive in volume and its advantage over max aggregation grows with dimension.SCO has a better mean rank than all hyperrectangular baselines.Source: §6 · p. 18

A Proofs

The proofs establish finite-sample validity under exchangeability, downward-closedness of the exact region, and rectangular bounds relating the exact region to its staircase approximation.

A Proofs

  • Exchangeability makes the test score’s rank uniform, yielding at least 1 −α coverage for Rα.Source: A Proofs · p. 21
  • For coordinate-wise smaller residual vectors, the test score does not increase while the calibration quantile does not decrease, establishing downward-closedness.Source: A Proofs · p. 21
  • Quantile bounds sandwich Rα within axis-aligned rectangles, and the exact region is contained in the staircase; the staircase-to-outer bound requires λ̂ub ≥ 1.Source: A Proofs · pp. 21–22

B Dataset Details

Table 5 identifies the real-world datasets used in the experiments and reports their source and train/calibration/test split sizes, averaged over 20 random seeds.

  • The reported split is 60% training, 15% calibration, and 25% testing, averaged over 20 random seeds.Source: Table 5 · p. 22
  • The table covers real-world datasets used in the experiments.Source: Table 5 · p. 22
  • The table lists each dataset’s target count, total sample size, train, calibration, and test counts, and source.Source: Table 5 · p. 22, B Dataset Details · p. 23

C Full Experimental Results

The full experimental figures organize dataset panels by target count and show joint coverage and log10 volume across methods. Panel titles identify each dataset and its split sizes.

Random forest, α = 0.10

The random-forest figures for α = 0.10 display joint-coverage and log-volume axes for datasets with multiple target counts.

Random forest, α = 0.05

The random-forest figures for α = 0.05 display joint-coverage and log-volume axes across datasets with different target counts.

Linear model, α = 0.10

This section presents linear-model panels for α = 0.10, plotting joint coverage and log₁₀ volume across datasets and dimensions. The supplied text identifies these plotted quantities but does not report readable comparative outcomes.

Linear model, α = 0.10

  • The passage lists dataset counts as ANSUR-II (d=2), 1191/297/498; Bio (d=2), 27438/6859/11433; and Births-1 (d=2), 6000/1500/2500.Source: Linear model, α = 0.10 · p. 27
  • The cited labels report log10 volume and joint coverage, and listed datasets span d=2 to d=16, including ANSUR-II (d=2), OES10 (d=16), and OES97 (d=16).Source: Linear model, α = 0.10 · pp. 27–28

Linear model, α = 0.05

  • The supplied panel snippets label “log10 vol.” and “Joint cov.”, while dataset listings include ANSUR-II (d=2) and Air (d=6), documenting examples at those two dimensions.Source: Linear model, α = 0.05 · pp. 28–29

Linear model, α = 0.10

Loading 2609.17091v1…