PaperlayerView exact revision on arXiv

Source-linked AI summary

Conformal Prediction for Time Series with Deep Sequence Models

Junghwan Lee, Jonghyeok Lee, Yao Xie

arXiv:2610.02357v1stat.MLcs.LG

TL;DR

Time-series dependence generally violates the exchangeability assumption behind conformal prediction, and systematic evidence on how to use deep sequence models in this setting is limited. The paper studies three such approaches and establishes asymptotic conditional coverage guarantees under suitable assumptions. Across three real-world datasets, DSCP-Local achieves the best overall performance while maintaining stable coverage.

  • Problem

    Time-series dependence generally violates conformal prediction’s exchangeability assumption, and systematic understanding of how deep sequence models can be used in this setting remains limited.

    Source: §1 · p. 2

  • Method

    The paper studies deep sequence models for conditional quantile regression, conditional quantile function estimation, and localized conformal prediction.

    Source: §1 · p. 2

  • Results

    DSCP-Local achieves the best overall performance while maintaining stable coverage across three datasets; DSCP-QR often outperforms the strongest baselines but occasionally undercovers.

    Source: §4.2 · p. 12

  • Takeaways & Limitations

    Among the evaluated methods, DSCP-Local combines the strongest overall performance with stable coverage across the three datasets.

    Source: §4.2 · p. 12

    Exact distribution-free conditional coverage with finite-length intervals is generally impossible, motivating an asymptotic conditional-coverage target.

    Source: §1.2 · p. 4

Abstract

from arXiv · show

Recent advances in deep learning for time series prediction have amplified the need for reliable uncertainty quantification. Conformal prediction has gained attention as a distribution-free framework for constructing prediction intervals with coverage guarantees. However, its coverage guarantees rely on data exchangeability, an assumption generally violated in time series. Active research has focused on developing conformal prediction methods for time series that overcome this limitation. While deep sequence models, such as recurrent neural networks and Transformers, have often been used in conformal prediction for time series, limited work has systematically studied how deep sequence models can be utilized in conformal prediction for time series. In this work, we systematically investigate the use of deep sequence models in conformal prediction for time series through three approaches: conditional quantile regression, conditional quantile function estimation, and localized conformal prediction. We provide a theoretical analysis establishing asymptotic conditional coverage guarantees for all three approaches under suitable assumptions. Through comprehensive experiments on real-world datasets, we demonstrate the effectiveness of leveraging deep sequence models into conformal prediction for time series.

1 Introduction

Conformal prediction offers distribution-free coverage guarantees, but time-series dependence generally violates the exchangeability assumption behind them. This section motivates a systematic study of deep sequence models for time-series conformal prediction and frames its conditional-coverage objective.

1 Introduction

  • Temporal dependence generally violates conformal prediction’s exchangeability assumption, while the systematic use and practical effectiveness of deep sequence models in time-series conformal prediction remain insufficiently understood.Conformal prediction is used for distribution-free uncertainty quantification and prediction intervals with coverage guarantees.Source: §1 · pp. 1–2
  • The paper proposes conditional quantile regression, conditional quantile function estimation, and localized conformal prediction using deep sequence models.Source: §1 · p. 2
  • All three proposed methods are shown to have asymptotic conditional coverage guarantees under suitable assumptions.Source: §1 · p. 2

1.1 Related Works

  • Under distribution shift, prior conformal approaches include adaptive online calibration and reweighting, but choosing suitable weights can be difficult when the shift is unknown [25] [26] [32] [33].Source: §1.1 · p. 3
  • Time-series conformal methods have used ensemble calibration, relevance-based reweighting, deep representations, and conditional residual-quantile estimation [12] [13] [23].Other work estimates future residual quantiles using quantile random forests, Transformers, graph neural networks, or distributionally similar bins [36] [19] [22] [14].Source: §1.1 · p. 3

1.2 Problem Setup

  • The setup seeks prediction intervals with coverage conditional on observed context, a stronger target than average marginal coverage, while exact distribution-free conditional coverage with finite-length intervals is generally impossible [9].The setup considers forecasting a future outcome from contextual information at a specified horizon.Source: §1.2 · p. 4

2 Methods

All three methods encode historical context into a latent representation, then use it differently to estimate residual quantiles or calibrate residual distributions.

2.1 Conditional Quantile Regression using Deep Sequence Models

  • DSCP-QR uses a shared sequence encoder and quantile-specific heads; nonnegative increments constrain successive estimates to prevent quantile crossing.The lowest-level head predicts its quantile directly, and each later head adds a nonnegative increment.Source: §2.1 · p. 5

2.2 Conditional Quantile Function Estimation using Deep Sequence Models

  • DSCP-QFE inputs the requested quantile level alongside encoded context, allowing one head to estimate arbitrary levels and retarget interval coverage without retraining.A partially monotonic head makes the estimated quantile function nondecreasing in the level.Source: §2.2 · p. 6

2.3 Localized Conformal Prediction using Deep Sequence Models

  • DSCP-Local uses learned representations to weight calibration residuals by context similarity, estimate a conditional CDF, and obtain quantiles by inversion.Training the encoder with quantile losses is intended to retain information about conditional residual distributions, so nearby representations identify relevant calibration contexts.Source: §2.3 · pp. 6–7

3 Theoretical Analysis

The analysis establishes asymptotic conditional coverage through quantile consistency for DSCP-QR and DSCP-QFE, and conditional CDF consistency for DSCP-Local, under stated assumptions.

3.1 Asymptotic Conditional Coverage for DSCP-QR

  • DSCP-QR's asymptotic conditional coverage follows from quantile consistency under stationary α-mixing data and approximation and optimization conditions.Density and compactness conditions ensure that any fixed uniform quantile error entails a positive loss gap.Source: §3 · p. 7, §3.1 · pp. 7–8

3.2 Asymptotic Conditional Coverage for DSCP-QFE

  • For DSCP-QFE, stationary α-mixing data, full-support quantile-level sampling on a compact interval, and approximation and optimization conditions support asymptotic conditional coverage.Joint continuity, full support, and compactness connect loss convergence to quantile accuracy; the endpoint comparison yields the stated coverage result.Source: §3 · p. 7, §3.2 · pp. 8–9

Appendix A.2).

DSCP-Local's coverage analysis uses a fixed encoder and representation-local calibration; the evaluation also describes its datasets, predictors, and comparison setup.

3.3 Asymptotic Conditional Coverage for DSCP-Local

  • With a fixed encoder and mixing, local-mass, CDF-regularity, and bandwidth conditions, DSCP-Local establishes representation-conditional coverage; context-conditional coverage additionally requires residual sufficiency.Residual sufficiency means the conditional residual distribution given context depends on context only through the representation.Source: §3.3 · pp. 9–10

Datasets.

  • The evaluation uses Air Quality, Solar, and Sapflow datasets, including 24 Sapflow sequences with 15,000–20,000 observations each.The Sapflow sequences vary in sampling frequency.Source: Datasets · p. 10

Base predictors.

  • Base predictors are linear regression, a jointly trained LSTM, and zero-shot Chronos forecasts; the task is one-step-ahead prediction.Each sequence is split approximately 33:45:22 for training, calibration, and testing; Table 1 reports coverage gap, interval width, and Winkler score at 0.9 target coverage.Source: Base predictors · pp. 10–11, Table 1 · p. 11

Evaluation metrics.

The section defines three interval-quality metrics, reports that DSCP-Local performs best overall with stable coverage, and examines encoder and architectural choices. The conclusion identifies empirical conditional-coverage assessment and efficiency-preserving quantile-crossing controls as future needs.

Evaluation metrics.

  • Coverage gap measures deviation from target coverage, interval width measures interval size, and Winkler score combines width with coverage violations.Each metric is averaged across test sequences within each dataset.Source: Evaluation metrics · p. 12

4.2 Results

  • DSCP-Local achieves the best overall performance while maintaining stable coverage across all three datasets at a target coverage of 0.9.DSCP-QR also performs well, often outperforming the strongest baselines, but occasionally undercovers.Source: §4.2 · p. 12

The effect of architectural constraints for preventing quantile crossing on prediction

  • Removing DSCP-QFE’s partially monotonic heads substantially improves its performance, while removing DSCP-QR’s incremental heads has marginal effect but causes quantile crossing.The comparison examines architectural constraints intended to prevent quantile crossing.Source: The effect of architectural… · p. 12

5 Conclusion

  • The conclusion calls for empirical protocols to assess conditional coverage and architectures that prevent quantile crossing without sacrificing interval efficiency.It also identifies developing methods that better leverage deep sequence models as a future direction.Source: §5 · pp. 12–13

A Detailed Theoretical Analysis and Proofs

The appendix supplies the full assumptions and intermediate results underlying Section 3. Its setup fixes the point predictor and specifies which encoders are learned or held fixed.

A Detailed Theoretical Analysis and Proofs

A.1 DSCP-QR

DSCP-QR estimates a finite set of conditional residual quantiles using shared-encoder quantile heads. Under regularity, model-class, and optimization assumptions, its analysis proceeds from risk convergence to quantile consistency and asymptotic conditional coverage.

A.1 DSCP-QR

  • DSCP-QR’s proof proceeds from uniform convergence of training risk to uniform quantile consistency, then to conditional coverage.The point predictor is fixed, while the encoder is learned jointly with the quantile heads.Source: §A.1 · p. 18
  • The shared encoder and its heads define a jointly feasible class of vectors of conditional quantile functions at a fixed finite set of quantile levels.The estimated functions are trained using quantile loss.Source: §A.1 · p. 18
  • The assumptions require stationarity and strong mixing, bounded residuals, regularity around the true conditional quantiles, and compact continuous model classes.The model class need only approximate the true quantile vector in population risk; it need not contain it at any finite sample size.Source: §A.1 · pp. 18–19
  • Approximate empirical risk minimization allows an optimization-error term, while uniform convergence of empirical risk supports convergence of population excess risk.The argument also uses a model-class approximation condition.Source: §A.1 · pp. 19, 20, 22
  • Under the stated assumptions, the estimated conditional quantile functions are uniformly consistent and yield asymptotic conditional coverage at the selected interval endpoints.The coverage result applies to deterministic forecast-origin sequences as the sample size grows.Source: §A.1 · pp. 23, 25

A.2 DSCP-QFE

DSCP-QFE estimates a conditional quantile function over context and quantile level, while DSCP-Local conditions residual calibration on an encoded representation. The QFE analysis establishes conditional coverage under its stated assumptions; the localized approach adds a residual-sufficiency condition for coverage given context.

A.2 DSCP-QFE

  • DSCP-QFE estimates one function of context and quantile level; inference can be conducted at any quantile level in (0, 1), while a compact interval I is introduced for theoretical analysis.Source: §A.2 · p. 26
  • The QFE estimator is trained through quantile-level sampling or an integrated empirical objective, with optimization error accommodated by the assumptions.The quantile-level sampling law has full support on the analysis interval and is independent of the time-series process.Source: §A.2 · pp. 27–28
  • Under the stated assumptions, the QFE analysis establishes uniform consistency of the estimated conditional quantile function and conditional coverage.The coverage theorem applies at fixed interval endpoints whose difference is 1−α.Source: §A.2 · pp. 29–30

A.3 DSCP-Local

  • DSCP-Local first targets the residual distribution conditional on the fixed encoder representation H_t = Encθ′(Z_t).Coverage conditional on Z_t is obtained under an additional residual-sufficiency assumption.Source: §A.3 · p. 31

A.3.1 Coverage conditional on the representation Ht

The localized estimator achieves asymptotic coverage conditional on its representation under stationarity, dependence, kernel, and local-regularity conditions. Coverage conditional on the full context additionally requires residual sufficiency.

A.3.1 Coverage conditional on the representation Ht

  • The guarantee assumes a stationary, β-mixing representation-residual process, a bounded local kernel with adequate local mass, and regular conditional residual CDFs.Local Hölder continuity in the representation metric is given as a sufficient condition for controlling pooling bias.Source: §A.3.1 · p. 31
  • The analysis uses radial-kernel conditions, and several standard kernels, including uniform, triangular, and Epanechnikov, satisfy the stated kernel requirements.The local CDF regularity conditions also require continuity in the residual value for inversion at neighboring probability levels.Source: §A.3.1 · p. 31
  • Under the stated dependence and bandwidth conditions, the localized conditional residual CDF is consistent, enabling the representation-conditional coverage result.The theorem uses a calibration prefix whose size approaches the full sample while separating it from the remaining observations to account for dependence.Source: §A.3.1 · pp. 32, 34, 35
  • Under Assumptions A.21–A.24, the H-conditional coverage error satisfies |P(A_T | H_t^T) − (1−α)| ≤ 2η + P(D_T > η | H_t^T).Source: §A.3.1 · pp. 36–37

A.3.2 Extension to coverage conditional on the context Zt

  • Coverage conditional on context Zt follows under an additional residual-sufficiency assumption: the residual distribution given Zt is preserved by representation Ht.This permits the encoder to discard features that do not affect the conditional residual distribution.Source: §A.3.2 · pp. 37–38

B.1 Datasets

The experiments use three real-world time-series datasets and three base predictors. The proposed approaches use deep sequence encoders for residual quantile estimation or representation-based localization, with design choices intended to prevent quantile crossing or learn useful representations.

B.1 Datasets

  • The study evaluates Air Quality, Solar, and Sapflow time series, summarized by dataset features, sequence lengths, and numbers of series.Air Quality uses PM10 as its outcome; Solar uses diffuse horizontal irradiance, and Sapflow comprises a subset of 24 series.Source: §B.1 · p. 39, Table 2 · p. 40

B.2 Base predictors

  • Linear regression, a global LSTM, and zero-shot Chronos-2 serve as base predictors, with sequential rolling-window forecasts on held-out observations.Chronos-2 is evaluated without fine-tuning and uses a prediction length of 10; the other two predictors use prediction step 1.Source: §B.2 · pp. 40–41

DSCP-QR

  • DSCP-QR estimates residual quantiles with separate heads for target levels, using nonnegative increments to prevent quantile crossing.The encoder representation feeds the heads, and training uses quantile loss.Source: DSCP-QR · pp. 41–42

DSCP-QFE

  • DSCP-QFE inputs the quantile level to a single prediction head, and its partially monotonic construction makes the estimated quantile function nondecreasing.This allows estimation at continuously varying quantile levels; the alternative quantile-embedding construction does not guarantee avoidance of crossing.Source: DSCP-QFE · pp. 42, 44

DSCP-Local

  • DSCP-Local trains a sequence encoder with quantile loss, then uses representation-based softmax weights over calibration residuals to estimate interval quantiles.The experiments found modest improvement from multiple quantile levels over median-only training, with no noticeable further improvement from three to nine levels.Source: DSCP-Local · p. 45

B.4 Implementation Details of Baseline Methods

The baseline implementations use calibration data for interval calibration and tune hyperparameters with method-specific splits or inherited configurations. Several methods adopt the same grid-search procedure used for the proposed methods.

B.4 Implementation Details of Baseline Methods

  • NexCP splits calibration data 85%/15% for hyperparameter selection and uses the proposed methods’ grid-search procedure.The selected configurations are reported by dataset and base predictor.Source: §B.4 · p. 46
  • HopCPT follows its original hyperparameter-selection procedure, splitting the calibration set equally into training and validation subsets.The study uses this procedure because its datasets overlap with those in the original HopCPT experiments.Source: §B.4 · p. 46
  • SPCI uses an 85%/15% calibration-set split for hyperparameter selection and the proposed methods’ grid-search procedure.Its intervals are calibrated using the calibration set.Source: §B.4 · p. 46
  • KOWCPI uses the Epanechnikov kernel and an 85%/15% calibration split for hyperparameter selection.It uses the same grid-search procedure as the proposed methods.Source: §B.4 · p. 46
  • ResCP reuses published Air and Solar configurations and tunes only Sapflow, while DistMatch tunes its KS threshold over {0.01, 0.05, 0.1}.DistMatch fixes residual patch length at 100 and follows the authors’ experimental setup for other listed parameters.Source: §B.4 · pp. 46–47

B.5 Evaluation Metrics

The evaluation uses coverage gap, average prediction interval width, and Winkler score to assess prediction intervals.

B.5 Evaluation Metrics

  • Coverage gap (∆Cov) measures empirical minus target coverage; positive values indicate overcoverage and negative values undercoverage.The gap is averaged across test sequences for each dataset.Source: §B.5 · p. 47
  • Average prediction interval width (PI-W) measures interval width, with smaller widths indicating greater efficiency when target coverage is obtained.Width is averaged across test sequences for each dataset.Source: §B.5 · p. 47
  • Winkler score (WS) combines interval width and coverage, adding a distance-proportional penalty when the true outcome falls outside the interval.The penalty factor is 2/α, and lower scores indicate smaller intervals with fewer or less severe coverage violations.Source: §B.5 · p. 48

Related papers

Loading 2610.02357v1…