PaperlayerView exact revision on arXiv

Source-linked AI summary

SPACE: Sample-cloud Predictive Adaptive Conformal Ellipsoids for Multivariate Time-Series Forecasting

Baishi Li, Kelvin J. L. Koa, Ke-Wei Huang

arXiv:2608.17333v1stat.MLcs.AIcs.LG

TL;DR

Sample-based multivariate forecasts lack reliable joint coverage, while historical-residual conformal methods can adapt slowly to changing uncertainty geometry. SPACE instead estimates ellipsoidal geometry from the current forecast sample cloud and calibrates its radius dynamically, achieving coverage closer to nominal targets across diverse benchmarks.

  • Problem

    Existing methods often shape multivariate conformal regions with historical residual covariance, limiting adaptation when instantaneous uncertainty geometry changes abruptly.

  • Method

    SPACE constructs ellipsoidal regions from time-local covariance in the current predictive sample cloud while using conformal scores to calibrate the radius.

  • Results

    0.2 percentage points: SPACE’s median realized joint coverage gap from nominal across 7 datasets, 8 forecasters, and 5 conformal baselines, while reducing mean gap by over 80% versus MultiDimSPCI.

  • Takeaways & Limitations

    SPACE is the most consistent compared wrapper for bringing realized coverage close to nominal targets across diverse datasets and forecasters.

  • Takeaways & Limitations

    SPACE depends on the predictive sample cloud’s quality, so biased or insufficiently diverse samples can leave calibrated regions valid only with reduced efficiency.

Abstract

from arXiv · show

Modern probabilistic time-series forecasters often express uncertainty through forecast samples. While typically converted into nominal prediction regions using empirical quantiles, these model-implied sets lack formal coverage guarantees and frequently deviate from nominal targets under distribution shift. Existing multivariate conformal methods can calibrate these regions online, but they typically estimate geometry from historical residuals using fixed or accumulating look-back windows. This reliance on the past limits their ability to exploit the instantaneous dependence structure of current predictions and leaves them vulnerable to stale-regime contamination. To address this, we propose SPACE, a conformal wrapper for sample-generating multivariate forecasters. SPACE constructs ellipsoidal joint prediction regions by estimating time-local covariance geometry directly from the current forecast sample cloud, calibrating the region's radius via a dynamic backward window-selection scheme. Across diverse multivariate datasets, probabilistic forecasters, and conformal baselines, SPACE consistently brings realized joint and rolling coverage closer to the nominal target, achieving superior coverage-efficiency tradeoffs relative to competing wrappers.

1 Introduction

SPACE addresses the challenge of calibrating multivariate sample-based prediction regions under non-stationarity by separating forecast-sample geometry from conformal radius calibration. It combines time-local ellipsoidal regions, regime-local adaptive windows, theoretical guarantees, and broad empirical validation.

  • Motivation: Modern multivariate forecasters generate rich sample clouds, but converting them into calibrated joint prediction regions under non-stationarity remains challenging.Raw sample-based regions can be miscalibrated, and realized coverage may drift from the nominal 90% target.
  • Method: SPACE uses the current predictive sample cloud to determine local ellipsoidal geometry while conformal scores determine the region radius.This decoupling reduces adaptation delay caused by relying on historical residual geometry.
  • Method: SPACE adaptively selects the longest recent calibration window whose conformity scores remain statistically compatible with the current regime.The backward same-regime procedure reduces outdated-dynamics influence while balancing local coverage and statistical efficiency.
  • Theory: SPACE establishes finite-sample joint coverage under an idealized same-regime assumption and derives a sequential local coverage-gap bound.The bound separates errors from sample-based geometry estimation, adaptive window selection, and finite calibration sample size.
  • Empirical validation: 7 datasets, 8 probabilistic forecasters, and 5 competing conformal baselines were used to evaluate SPACE, which achieved a median realized joint coverage gap of 0.2 percentage points.The evaluation spans energy, finance, weather, and traffic domains and includes diffusion, flow, and autoregressive architectures.

2 Related Work

Related work spans sample-generating probabilistic forecasters and conformal methods for sequential, localized, multivariate, and ellipsoidal prediction. SPACE is positioned as a wrapper that calibrates multivariate regions directly from current forecast samples while adapting calibration to the current regime.

  • Sample-generating probabilistic forecasters: Sample-generating forecasters encode joint dependencies through predictive sample clouds, but their native regions remain model-implied rather than coverage-calibrated.SPACE treats autoregressive, diffusion, copula, and flow-based forecasters as black boxes and calibrates their raw predictive samples without changing the architecture.
  • Sequential, localized, and online conformal prediction: Sequential conformal methods such as EnbPI, ACI, and Conformal PID adapt calibration thresholds or miscoverage levels online beyond exchangeability.Related extensions address online tracking, rolling risk control, covariate shift, and localized conformal prediction.
  • Multivariate and ellipsoidal conformal prediction: Prior multivariate conformal methods use structured dependence through copulas or ellipsoidal sets, while MultiDimSPCI uses historical residual covariance and fixed-length sequential calibration windows.MultiDimSPCI is identified as the closest benchmark for multivariate time-series prediction.
  • Positioning of SPACE: SPACE operates without auxiliary model fitting, derives geometry from the current predictive sample cloud, and adapts its calibration window to the current regime.These are the structural differences summarized in Table 1.

3 Method: SPACE

SPACE constructs joint ellipsoidal prediction regions causally by estimating time-local covariance geometry from each forecast sample cloud and calibrating the radius through dynamic same-regime window selection. Its design separates sample-based shape estimation from historical scale calibration, enabling formal finite-sample coverage guarantees.

  • Causal construction: SPACE operates causally, using the current predictive sample cloud and realized past data to construct each prediction set sequentially.Its two components estimate local geometry from the current cloud and dynamically select a historical calibration window for radius calibration.
  • Local geometry: The method estimates a median center and standardized-residual covariance, then maps it back to the original scale as time-local multivariate geometry.The resulting sample-derived covariance consistently concentrates around the forecaster’s true conditional geometry.
  • Nonconformity scoring: SPACE measures target deviation using a stabilized pseudo-inverse Mahalanobis-type radial score relative to the forecast cloud’s local covariance geometry.The pseudo-inverse inverts only eigenvalues above an adaptive tolerance, supporting positive-semidefinite covariance estimates.
  • Dynamic calibration-window selection: SPACE searches backward over candidate calibration lengths and extends the window only when full-block and newly added-increment checks pass across three complementary diagnostics.The diagnostics capture radial size, coordinate-wise extremeness, and variation along a dominant direction; the selected length is the last accepted length before rejection.
  • Radius calibration and guarantees: Given the selected window, SPACE calibrates the ellipsoid radius from same-regime scores using an empirical 1 − α_t quantile with α_t updated by adaptive conformal inference.The resulting ellipses rotate, elongate, or contract over time because their shape follows the contemporaneous sample cloud.

4 Theoretical Analysis

The theoretical analysis establishes concentration of current sample-cloud geometry, recovery of a same-regime calibration window, and a sequential decomposition of conditional coverage error. Under assumptions on sample regularity, regime exchangeability, window diagnostics, and adaptive-level validity, SPACE’s error is controlled by identifiable finite-sample and adaptation terms.

  • 4 Theoretical Analysis: The analysis centers on covariance concentration, same-regime window recovery, and sequential coverage-error decomposition, with formal assumptions and proofs deferred to Appendix A.The bounds are formulated conditionally on the immediate past, while unconditional variants follow by integrating over that past.
  • Step 1: Geometry Concentration: Under (A1), Proposition 1 establishes that the empirical sample-cloud covariance reliably estimates the forecaster-implied conditional geometry.The result uses operator-norm concentration for sample covariance matrices of sub-Gaussian vectors.
  • Step 1: Geometry Concentration: The covariance estimation rate has standard unstructured sub-Gaussian dimension dependence, while controllable generative sample size M can drive geometry error toward zero.Once M ≳d, the leading d/M term decays rapidly; faster rates require structural assumptions such as sparsity or low rank.
  • Step 2: Same-regime window recovery: Under (A2), Proposition 2 bounds recovery of the largest current-regime candidate window using same-regime test false-rejection and false-acceptance probabilities.The sequential search is controlled by a union bound over clean tests and rejection of invalid longer windows.
  • Sequential coverage analysis: Theorem 1 decomposes per-time coverage error into window-selection, adaptive-level, finite-calibration, covariance-perturbation, and concentration-failure penalties.The window-selection term satisfies Proposition 2, while the covariance term depends on the stability constant and covariance concentration rate.
  • Sequential coverage analysis: Corollary 1 extends the theorem to expected empirical average coverage by averaging per-time bounds and applying the tower property.Random right-hand-side terms are interpreted through their outer expectations.

5 Experiments

SPACE is evaluated across heterogeneous probabilistic forecasters, real-world multivariate datasets, and conformal baselines using coverage-gap and log-volume criteria. It achieves the strongest aggregate calibration and particularly stable local coverage, while maintaining competitive prediction-region efficiency.

  • Evaluation setup: SPACE is evaluated with eight heterogeneous probabilistic forecasters across seven real-world datasets and compared against five conformal baselines.The baselines are MultiDimSPCI, Local Ellipsoid, CopulaCPTS, Empirical Copula, and Gaussian Copula.
  • Evaluation criteria: The experiments report coverage gap, rolling coverage gap, and mean log-volume to assess calibration and prediction-region efficiency.Coverage gap measures absolute deviation from the nominal target and penalizes both under- and over-coverage.
  • Aggregate comparison: SPACE achieves the best Mean Gap and Median Gap among all compared wrappers, with the smallest standard deviation.Its mean coverage gap is 0.3 percentage points from the nominal target.
  • Aggregate comparison: SPACE provides much better calibration than competing wrappers while remaining competitive in median log-volume.MultiDimSPCI has the lowest median volume overall, but its smaller volume is accompanied by worse calibration and a lower validity ratio.
  • Rolling calibration: Using rolling windows of length w = 30, SPACE shows a clear advantage in mean rolling gap, especially on higher-dimensional and more nonstationary datasets.The largest separation from the best competing method occurs on datasets such as electricity and traffic, which have richer dependence structure and stronger temporal instability.

6 Conclusion

SPACE is a post-hoc conformal wrapper that combines predictive sample-cloud geometry with sequential calibration of prediction-region radii. This decoupling addresses adaptation delays associated with historical residual-based methods while preserving conformal calibration guarantees.

  • Conclusion: SPACE bridges modern probabilistic forecasters’ uncertainty representations with conformal prediction’s rigorous calibration guarantees.It is introduced as a post-hoc conformal wrapper.
  • Conclusion: SPACE extracts multivariate geometry directly from the predictive sample cloud rather than relying on historical residuals.The method decouples geometry estimation from sequential radius calibration.
  • Conclusion: Decoupling geometry estimation from radius calibration helps SPACE overcome adaptation delays inherent in historical residual-based methods.The region’s radius is calibrated sequentially after estimating its multivariate geometry.

7 Limitations · A Proofs for Section 4 · A.1 Assumptions.

SPACE’s limitations arise from reliance on the forecast sample cloud for covariance geometry and imperfect regime adaptation. Its guarantees are developed under regularity, detectability, exchangeability, and adaptive-level assumptions specified for the current regime.

  • 7 Limitations: SPACE can correct prediction-region scale, but its local covariance geometry remains dependent on the generated sample cloud.Biased or insufficiently diverse samples may therefore preserve validity only with reduced efficiency.
  • 7 Limitations: Gradual changes may provide weak diagnostic signals, while frequent changes leave little same-regime history for representative window selection.Either situation can cause SPACE to select a window that does not fully represent the current regime.
  • A.1 Assumptions.: A1 assumes conditionally i.i.d., sufficiently light-tailed forecast samples, K-sub-Gaussian residuals, and local stability of the conformal set under covariance-estimation error.The empirical covariance is centered on the standardized sample cloud and is unchanged when the ideal center is replaced by the method’s median-based center.
  • A.1 Assumptions.: A2 requires same-regime candidate blocks to avoid excessive rejection and contaminated windows crossing regime boundaries to be detected with high probability.The finite-sample tests use deterministic or previsible KS-type thresholds justified by DKW/Massart bounds, with false-rejection control δuni and missed-detection bound βdet.
  • A.1 Assumptions.: The oracle same-regime length L⋆t is the largest candidate whose candidate and probe blocks both lie within the current regime.For L > L⋆t, the candidate necessarily crosses a regime boundary and must be detected by at least one diagnostic.
  • A Proofs for Section 4: A3 assumes piecewise augmented exchangeability conditional on pre-regime information, yielding a uniform rank for the test score among same-regime calibration scores.This is the property used in the conformal coverage proof after conditioning on the regime state.
  • A.1 Assumptions.: A4 combines bounded adaptive-level tracking error with preservation of the conformal rank argument when αt is predictable relative to score ranking.The resulting assumptions separate adaptive level selection from ordinary finite-sample conformal rank validity.

A.2 Proof of Proposition 1 … C.1 Performance across forecasters

The paper establishes concentration and calibration guarantees for SPACE, describes its sequential wrapper, and reports robust calibration improvements across heterogeneous probabilistic forecasters. The theoretical analysis uses standard high-dimensional arguments and shows that the covariance rate is minimax-optimal in the black-box setting.

  • A.2 Proof of Proposition 1: SPACE’s empirical sample-cloud covariance concentration proof follows a standard high-dimensional probability pipeline rather than introducing novel probabilistic machinery.The proof conditions on past information, uses sub-Gaussian assumptions, net arguments, and Bernstein’s inequality.
  • A.2 Proof of Proposition 1: The covariance bound is statistically tight, achieving the optimal O(d/M) rate when M ≥ d without exploitable structural assumptions.Known sparsity or low-rank structure could permit rates based on intrinsic rather than ambient dimension, but SPACE treats the predictive cloud as unstructured.
  • A.3 Proof of Proposition 2: The backward window selector accepts clean candidate lengths while rejecting detectably contaminated extensions, then returns the longest accepted length.The argument combines clean-test and invalid-length union bounds under Assumption (A2).
  • A.4 Proof of Theorem 1: Theorem 1 transfers oracle same-regime conformal coverage to SPACE by combining exchangeability, covariance perturbation, window-selection correctness, and ACI tracking.Replacing ideal scores with empirical scores changes oracle coverage by at most LΣηM,δt/(n⋆t +1).
  • A.5 Proof of Corollary 1: Corollary 1 obtains average coverage by applying Theorem 1 at each prediction time and averaging the resulting per-time bounds.The statement distinguishes deterministic or almost-sure terms from terms requiring outer expectations.
  • B Algorithm: Algorithm 1 sequentially computes forecast centers and residual samples, searches backward over candidate windows, forms the selected calibration set, and updates the internal ACI level.The procedure accepts each window extension only when both same-regime diagnostics pass, stopping at the first rejection.
  • C.1 Performance across forecasters: Across heterogeneous probabilistic forecasters, Table 4 averages dataset results to assess wrapper robustness across differing raw uncertainty behavior.The subsection frames this evaluation as a robustness check on SPACE’s intended application.
  • C.1 Performance across forecasters: SPACE improves calibration for both clearly under-covered forecasters and already competitive raw forecasters, pulling realized coverage toward the nominal target.Examples include GP-Copula, TACTiS-2, TimeGrad, and NsDiff.

C.2 Full result tables

This subsection provides complete numerical comparisons across evaluated conformal wrappers, datasets, forecasters, and three target coverage levels. It reports empirical coverage gaps and efficiency metrics, including log-volume, at the dataset–forecaster level.

  • 90% target: At the 90% target, Table 4 summarizes GP-Copula, TimeGrad, CSDI, TSFlow, DeepAR, NsDiff, TMDM, and TACTiS-2 under SPACE.Coverage columns show average empirical coverage, Gap columns show average absolute coverage gaps, and gap reduction is defined relative to RAWGAP.
  • 90% target: At the 90% target, Table 5 compares coverage gap and log-volume across every dataset–backbone pair and conformal wrapper.Lower values are better for both metrics.
  • 50% and 95% targets: Tables 6 and 7 provide the same detailed wrapper comparisons at the 50% and 95% targets, respectively.Both tables report coverage gap and log-volume for each dataset–backbone pair, with lower values preferred.

D Non-stationarity and Performance Gains Analysis

The appendix defines a local covariance drift index to quantify time-varying multivariate dependence and examines whether stronger drift corresponds to larger SPACE gains. Results show a positive association between drift and coverage-gap improvement, while SPACE’s advantage is smaller in more stationary, lower-dimensional settings.

  • Overall interpretation: Overall, SPACE is especially effective when multivariate dependence geometry is strong and temporally unstable, while its advantage over simpler baselines is smaller for lower-dimensional, more stationary covariance structure.The appendix characterizes SPACE as remaining competitive in these less non-stationary settings.
  • Local covariance drift index: The local covariance drift index measures instability in a target’s joint second-order geometry across temporal regimes, normalized relative to global covariance geometry.Dataset-level drift is summarized using the mean of the top decile of local drift scores.
  • Results and ranking: Datasets rank by decreasing local covariance drift as traffic (4.939), weather (3.326), electricity (3.108), etth1 (1.738), exchange (1.487), ettm2 (1.075), and ettm1 (0.867).Traffic has the strongest reported drift, while ettm1 has the smallest.
  • Association with coverage-gap improvement: At the pair level, drift correlates positively with coverage-gap improvement: Pearson r = 0.357 (p = 0.0080) and Spearman ρ = 0.281 (p = 0.0396).The reported association links stronger local covariance drift with greater SPACE improvement.
  • Association with coverage-gap improvement: At the dataset level, the association remains positive but is less statistically stable, with Pearson r = 0.596 (p = 0.158) for n = 7.The appendix explicitly notes that the dataset-level sample size is necessarily small.
  • SPACE versus all baselines: Figure 4 compares SPACE with the mean of all baselines across datasets ordered by decreasing local covariance drift, reporting mean coverage gap and mean rolling gap.Points are dataset means across dataset–forecaster pairs, with error bars showing mean ±2 standard errors.

E Dynamic Window-Length Diagnostics · F Implementation details of SPACE

SPACE’s dynamic window length contracts around target-side regime changes, can recover during stable periods, and may grow as regimes become older and more stable. Diagnostics use change points detected from the original standardized multivariate target stream rather than from SPACE’s selected-window path.

  • E Dynamic Window-Length Diagnostics: SPACE shortens its selected memory length Lt when the underlying target process changes regime, reducing reliance on stale calibration history.This contraction is the intended mechanism for downweighting or discarding unreliable past calibration information.
  • E Dynamic Window-Length Diagnostics: Regime changes are detected from the original multivariate target stream yt to avoid circularity with the selected Lt path or SPACE diagnostics.For each dataset–forecaster pair at the 90% target level, the valid test times are aligned, targets standardized, reduced by PCA, and analyzed with an energy-divisive multivariate change-point method.
  • E Dynamic Window-Length Diagnostics: Figure 5 visualizes contraction, recovery, and long-term behavior of the dynamic rule around target-side change points.The diagnostic is designed to assess whether Lt decreases near changes, increases afterward, and grows during older stable regimes.
  • E Dynamic Window-Length Diagnostics: After an immediate postchange period, Lt can increase again, indicating recovery of a longer calibration window during renewed stability.The diagnostic specifically evaluates whether the selected window rebuilds after contraction.
  • E Dynamic Window-Length Diagnostics: As a detected regime becomes older and more stable, Lt is expected to exhibit longer-term growth.This behavior reflects SPACE’s design of using longer memory when the local regime appears stable.
  • E Dynamic Window-Length Diagnostics: Each Figure 5 panel aligns Lt, Target PC1, and detected target-side change points across valid test times.Target PC1 is the first principal component of the standardized multivariate target stream, while the bottom subplot shows energy-divisive change points.

F.1 Metric definitions … G Dataset Details

The paper defines global and rolling coverage diagnostics alongside mean log-volume, then evaluates wrappers under fixed computational and calibration settings. SPACE uses sample-based covariance stabilization and dynamic same-regime testing across seven public datasets.

  • F.1 Metric definitions: Empirical joint coverage and global coverage gap measure realized coverage and absolute deviation from the target miscoverage level α.The supplied passages introduce both definitions but do not include their displayed formulas.
  • F.1 Metric definitions: Rolling diagnostics include rolling joint coverage, rolling coverage gap, mean rolling gap, P90 rolling gap, and fraction of bad windows.The fraction of bad windows counts rolling windows whose gap exceeds tolerance δ.
  • F.1 Metric definitions: The main experiments use w = 30 and δ = 0.10, report mean log-volume, and treat lower coverage-gap, rolling-diagnostic, and LOGVOL values as better.Prediction-region size should be interpreted jointly with calibration because small regions are meaningful only near the target coverage.
  • F.2 Compute resources: Wrapper experiments ran on one workstation with an Intel i9-13980HX CPU, RTX 4080 Laptop GPU, and 32GB memory, excluding original forecasting-model training.The benchmark covers all dataset–forecaster pairs and target levels in the full Section C.2 result table.
  • F.3 Experimental configuration: Unless stated otherwise, forecasters use context window 96, pred_len = 1, and M = 100 saved forecast samples per time point.Increasing forecast samples can reduce sample-geometry error; the main benchmark uses one fixed configuration.
  • F.3 Experimental configuration: SPACE searches candidate calibration lengths L ∈ {20, 30, . . . , 300} with h = 10, uses probe-block length p = 20, and updates adaptive conformal inference with η = 0.01.Thus Lmin = 20 and Lmax = 300.
  • F.3 Experimental configuration: The benchmark uses identity shrinkage λshrinkage = 0.30 to stabilize sample-derived covariance estimates when effective sample size is limited or sample clouds are ill-conditioned.The passage characterizes shrinkage as an empirical implementation device rather than a separate conceptual component or requirement of SPACE.
  • G Dataset Details: The evaluation uses seven publicly available datasets, including Electricity, ETTh1, ETTm1, ETTm2, and Exchange.Electricity covers 321 customers, while the ETT datasets contain transformer variables recorded hourly or every 15 minutes; Table 8 summarizes the saved-stream splits.
Loading 2608.17333v1…