PaperlayerView exact revision on arXiv

Source-linked AI summary

SGA: Uncertainty Quantification for Multi-Step Forecasting in Time Series Foundation Models

Xin-Yu Hu, Shuang Liang, Cheng Feng, Shao-Qun Zhang

arXiv:2609.28582v1cs.LGcs.AIstat.ML

TL;DR

Multi-step TSFM forecasts can branch over time, making their credibility difficult to assess, while existing UQ extensions struggle to characterize that structure. SGA models branches in an aligned DAG and integrates topology with TSFM stochasticity; across 11 TSFMs and 27 datasets, it outperforms contenders in ranking predictive errors with uncertainty estimates.

  • Problem

    Existing UQ extensions struggle to characterize forecast branches that emerge at each step and spread through multi-step forecasts, although this topology is central to assessing forecast uncertainty.

    Source: §1 · p. 3

  • Method

    SGA samples and slices forecasts, constructs and aligns a DAG of potential branches, and estimates graph complexity by combining topology with TSFM-inherent stochasticity.

    Source: §3.2 · pp. 5, 7

  • Results

    Across 11 TSFMs and 27 datasets, SGA significantly outperforms contenders in ranking predictive errors with uncertainty estimates.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Larger-scale TSFMs typically have lower multi-step forecast uncertainty estimates, indicating an empirical scaling law for this setting.

    Source: §5 · p. 14

Abstract

from arXiv · show

The recent emergence of Time Series Foundation Models (TSFMs) has significantly advanced multi-step forecasting performance, enabling accurate predictions over extended future horizons. However, existing TSFMs often suffer from significantly inherent uncertainty, which typically manifests as derived forecast branches emerging at each time step and spreading to subsequent steps; different forecast branches often exhibit varying forecasting performance, thereby undermining the credibility of TSFM forecasts. In this paper, we propose the Slicing-Graphing-Alignment (SGA) method to quantify the uncertainty of multi-step TSFM forecasts. The proposed SGA first characterizes the topology of all potential forecast branches using a directed acyclic graph, such that the graph complexity bounds the uncertainty of multi-step forecasts, and then precisely measures the graph complexity by integrating both topological information and TSFM-inherent stochasticity. Experimental results conducted on 11 TSFMs and 27 datasets demonstrate that (i) SGA achieves the best performance when ranking predictive errors with uncertainty estimates; (ii) SGA works with a more extensive and more precise sampling coverage than those of existing UQ methods, deriving a quantification mechanism fundamentally different from those of established ones; and (iii) larger model scales of TSFMs correlate with lower uncertainty estimates of multi-step forecasts, suggesting another empirical scaling law for uncertainty quantification of multi-step TSFM forecasts.

1. Introduction

TSFMs have improved multi-step forecasting, but their uncertain forecasts need credibility measures for risk-benefit trade-offs. Existing approaches struggle to represent branching across forecast horizons; SGA models that structure and quantifies its complexity.

1. Introduction

  • TSFMs have demonstrated superior multi-step predictive accuracy compared with many previous approaches.The paper frames this progress alongside the remaining need for credible forecasts.Source: §1 · p. 2
  • Multi-step TSFM forecasts need uncertainty estimates to assess credibility in risk-benefit trade-offs, especially in high-stakes settings.The introduction illustrates the trade-off with a trading example: ignoring a sustained-growth forecast may forgo profits, while trusting it may risk substantial losses.Source: §1 · p. 2
  • Existing one-step and LLM-inspired UQ extensions struggle to characterize forecast branches that emerge at each step and spread into a complex forecasting space.The authors identify precise characterization of these potential branches as fundamental to multi-step UQ and still challenging.Source: §1 · pp. 2–3
  • SGA samples and slices forecasts, builds and aligns a DAG of branches, then quantifies forecasting-space complexity using topology and TSFM stochasticity.Across 11 TSFMs and 27 datasets, the paper reports stronger error ranking, broader and more precise sampling coverage, and lower uncertainty estimates for larger models.Source: §1 · p. 3

2. Related Work

Prior UQ work focuses on one-step forecasting or adapts sequence-based methods from LLMs. The paper distinguishes TSFMs by their output formats and notes that multi-step UQ remains underexplored.

2. Related Work

  • TSFMs are grouped by output format into vocabulary-based, quantile-based, and trajectory-based models.The paper gives Chronos, TimesFM-2.5, Chronos-2, Timer-S1, Sundial, and Aurora as examples, respectively.Source: §2 · pp. 3–4
  • Vocabulary-based and trajectory-based models average multiple forecast samples for their point forecasts, while quantile-based models typically use the median forecast.The output formats differ: token distributions, quantile-by-time matrices, and mappings from Gaussian samples to trajectories.Source: §2 · p. 4
  • LLM UQ methods model generations as sequences and use information-theoretic or diversity measures, motivating forecast-slice adaptations for multi-step forecasts.Reflexive methods are unsuitable for TSFMs because they cannot be prompted in the same manner.Source: §2 · p. 4
  • Existing UQ studies primarily address one-step forecasts using quantile-interval widths, leaving multi-step forecasting UQ underexplored.Intervals may be produced directly by a model or estimated from multiple samples.Source: §2 · p. 4

3. Our Methods

The method defines horizon uncertainty as a single estimate for a forecast trajectory, then represents possible branches through slicing, graphing, and alignment. It estimates graph complexity by combining slice uncertainty with DAG topology.

3.1. Horizon Uncertainty Quantification

  • HUQ assigns one uncertainty estimate to an entire h-step forecast trajectory, with one-step UQ as the special case h = 1.The task is more challenging because step-wise error accumulation can amplify uncertainty during multi-step forecasting.Source: §3.1 · pp. 4–5

3.2. Slicing-Graphing-Alignment

  • SGA represents potential forecast branches as slices, connects them in a DAG, and aligns similar nodes using a DTW-distance threshold to reveal forecasting-space topology.Slices spanning the same time steps represent potential branches; DAG nodes are slices and edges encode temporal dependency.Source: §3.2 · p. 5
  • For quantile-based TSFMs, SGA constructs a horizon-level joint inverse CDF by linearly interpolating adjacent quantile forecasts, then samples forecast trajectories from it.The sampling procedure is repeated K times; vocabulary-based and trajectory-based models retain their forecast trajectory samples directly.Source: §3.2 · pp. 5–6
  • Each sampled trajectory is segmented into consecutive-time slices, and same-index slices represent potential branches over the corresponding temporal span.The DAG uses the historical series as its root and adds directed edges to encode temporal dependencies between slices.Source: §3.2 · p. 6
  • Alignment merges nodes with the same slice index when their DTW distance falls below a threshold scaled using seasonal-scaled error.The threshold uses a coefficient and dataset seasonality parameter, matching its scale roughly to MASE and simplifying tuning.Source: §3.2 · p. 6
  • Graph complexity integrates node-level stochasticity and topology through uncertainty-aware alpha-centrality, then sums node centralities over the aligned DAG.A larger graph-complexity value reflects both higher TSFM-inherent stochasticity and a more complex forecasting-space topology.Source: §3.2 · p. 7

4. Experiments

The experiments assess SGA’s uncertainty ranking, sampling coverage, and sensitivity to design choices across TSFMs and datasets. They compare SGA with existing uncertainty methods and examine ablations and hyperparameters.

Experimental Setup

  • The evaluation spans 11 TSFMs and 27 Chronos Benchmark II datasets, comparing SGA with NC, adapted sequence-modeling UQ methods, and a random baseline.The methods are evaluated using selective prediction; the contenders include information- and diversity-based approaches adapted with DTW similarity.Source: §4 · p. 9

Evaluation Metrics

  • NEAURC assesses uncertainty-based ranking through selective prediction, with lower scores indicating better ranking; point forecasts are evaluated with MASE.The experiments use 1,000 bootstrap resamples to report mean and standard deviation of NEAURC.Source: §4 · pp. 9–10

Verifications on SGA

  • 34.49%: SGA outperforms the strongest contenders in overall NEAURC across 27 datasets and meets the 3σ rule.The comparison covers SGA and its contenders; the table reports overall NEAURC.Source: Table 1 · p. 10, §4 · p. 10

Comparisons of Sampling Coverage

  • All first-column overlap ratios exceed 90%, while second-column ratios are typically low, especially for vocabulary- and trajectory-based TSFMs.The first-column ratios indicate how much of NC’s sampling coverage SGA encompasses; the second indicate the portion of SGA’s coverage overlapping NC.Source: Table 2 · p. 11, §4 · p. 12

Ablation Analyses

  • Ablations indicate that node alignment is critical to SGA’s performance, while topology contributes more than TSFM-inherent stochasticity to graph complexity.These comparisons average results across 11 TSFMs of three types and 27 datasets.Source: §4 · p. 13, Figure 4 · p. 13

Sensitivity Analyses

  • For favorable efficiency-performance trade-offs, the authors recommend K=20 for C-2, TimesFM-2.5, Sundial, and Aurora, and K=30 for C-T5 and Timer-S1.They recommend ls=4 and λ=0.25 for all 11 TSFMs.Source: §4 · p. 13, Figure 5 · p. 13

5. Conclusions

The conclusion frames multi-step TSFM uncertainty through the topology of potential forecast branches and reports empirical evidence across models and datasets. The appendix provides supplementary materials organized according to the paper’s sections.

5. Conclusions

  • SGA uses a directed acyclic graph to characterize potential forecast-branch topology, with graph complexity bounding uncertainty in multi-step forecasts.It measures that complexity by integrating topological information and TSFM-inherent stochasticity.Source: §5 · p. 14
  • Experiments across 11 TSFMs and 27 datasets revealed that larger TSFM parameter scales typically correspond to lower multi-step forecast uncertainty estimates.Source: §5 · p. 14

Appendix

  • The appendix supplies supplementary materials organized according to the corresponding sections of the paper.Source: Appendix · p. 15

A. Additional Implementation Details of SGA

The implementation uses recent history as model input and configures point forecasts and uncertainty samples according to each model’s output type. SGA also uses fixed slicing and alignment settings, with reported stability around those defaults.

A. Additional Implementation Details of SGA

  • The model input is the most recent history, truncated to t = min(lh, 512), where lh is the available historical-series length before the forecast horizon.Source: A. Additional Implementation Details… · p. 15
  • For MASE point forecasts, vocabulary- and trajectory-based models use 20 forecast samples, while quantile-based models use the median forecast.The median forecast is the 50% quantile.Source: A. Additional Implementation Details… · p. 15
  • Uncertainty estimates use K = 20 forecast trajectories; sampling-coverage comparisons use K = 50 for SGA and NC on vocabulary- and trajectory-based models.Source: A. Additional Implementation Details… · p. 15
  • Vocabulary-based models use autoregressive sampling with T = 1.0 and top-k = 50, while quantile-based models sample q uniformly from [0.1, 0.9] using an 8-step joint inverse CDF.Trajectory-based models retain their default sampling configurations.Source: A. Additional Implementation Details… · p. 15
  • SGA uses slicing length ls = 4 and DTW threshold coefficient λ = 0.25, and the method is reported as relatively insensitive to small variations around those defaults.When ls does not divide the forecast horizon, the final slice contains the remaining forecast values.Source: A. Additional Implementation Details… · p. 16

B. Additional Experimental Details on SGA

The evaluation details define NEAURC for selective prediction and ARMASE for aggregating point-forecast performance across datasets. Their definitions specify how forecast errors, uncertainty rankings, and family-specific baselines enter the metrics.

B.1. Evaluations

  • NEAURC evaluates whether higher uncertainty ranks forecasts with higher predictive error in selective prediction, with lower values indicating better performance.It normalizes the excess area under the risk-coverage curve against random and oracle estimators.Source: §B.1 · p. 16
  • For selective risk, forecasts are ordered by increasing uncertainty and evaluated across coverage levels using their MASE errors.The resulting average selective risk contributes to the risk-coverage evaluation.Source: §B.1 · p. 16
  • ARMASE aggregates relative MASE across datasets using a geometric mean, with Chronos-T5-Tiny and Chronos-2-Small as the respective family baselines.Source: §B.1 · p. 17

B.2. Details on Datasets

The experiments use a heterogeneous 27-dataset benchmark and adapt several types of existing uncertainty methods to forecast trajectories. For LLM-based contenders, the adaptations include DTW-based criteria for trajectory equivalence and similarity.

B.2. Details on Datasets

  • The evaluation covers 27 Chronos Benchmark II datasets across application domains and sampling frequencies, holding out each series’ final h observations as the forecast horizon.The preceding observations form the history; domains include energy, finance, healthcare, nature, retail, and mobility and transport.Source: §B.2 · p. 17

B.3. Details on UQ Contenders

  • For one-step methods, NC uses average widths of central 80% intervals, obtained from model quantiles or interpolated sampled values depending on model type.Source: §B.3 · p. 18
  • Eight representative sequence-modeling UQ methods for LLMs serve as contenders, with all multi-sampling methods using K = 20 forecast trajectories.The implementations are based on LM-Polygraph.Source: §B.3 · p. 18
  • LLM UQ methods are adapted by treating each forecast trajectory as input and replacing token-level distributions with time-step-level distributions.Information measures operate on trajectory sequences; diversity methods use DTW-based equivalence or similarity.Source: §B.3 · p. 18
  • The contender adaptations replace text-based semantic equivalence and similarity with DTW criteria between forecast trajectories.The DTW equivalence criterion uses a threshold τ.Source: §B.3 · pp. 18–19

B.4. UQ Performance Evaluations

Across 27 datasets and 11 models, SGA achieves the best overall UQ rank; the scale-related pattern is not consistently observable on every dataset.

B.4. UQ Performance Evaluations

  • SGA achieves the best overall rank among 10 UQ methods, with ranks averaged across 27 datasets and 11 models.Source: §B.4 · p. 19, Figure 6 · p. 20
  • 260 of 297 dataset-model pairs favor SGA over its contenders on NEAURC, supporting its effectiveness across varied datasets.Source: §B.4 · p. 20

B.5. Relation between Uncertainty and Scale

  • Across some individual datasets, larger TSFMs do not appear in the lower-left of uncertainty-versus-MASE plots, so the scaling pattern is not universal.Source: Figure 7 · p. 26, §B.5 · p. 26

B.6. Additional Ablation Analyses

Ablations across 11 TSFMs and 27 datasets indicate that node alignment is critical and that topology contributes more than inherent stochasticity in selective prediction.

B.6. Additional Ablation Analyses

B.7. Additional Sensitivity Analyses

Sensitivity analyses examine sample count, slicing length, and threshold coefficient; the authors recommend settings that balance performance and efficiency or reflect overall performance.

B.7. Additional Sensitivity Analyses

  • K = 20 is recommended for the C-2 family, TimesFM-2.5, Sundial, and Aurora, while K = 30 is recommended for the C-T5 family and Timer-S1.The recommendations target a favorable efficiency-performance trade-off because larger K increases computational costs.Source: §B.7 · p. 30, Figure 11 · p. 30
  • The recommended slicing length is ls = 4 based on overall performance.Source: §B.7 · p. 31, Figure 12 · p. 31
  • The recommended threshold coefficient is λ = 0.25 based on overall performance.Source: §B.7 · p. 32, Figure 13 · p. 32

Related papers

Loading 2609.28582v1…