PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Analysis for Time Series Foundation Models

Mathis Jander, Wouter van Heeswijk, Martijn Mes

arXiv:2608.24303v1cs.LG

TL;DR

As time series foundation models serve many applications, identifying their shared biases and failure modes before deployment is important, yet validation has focused on benchmarking uncontrolled observations. This paper uses causal interventions on synthetic time series to test Chronos-2 and TimesFM-2.5, finding preserved drift and harmonic patterns alongside persistence bias and failures on regime switches and energy-release patterns.

  • Problem

    As foundation models serve many applications, their shared biases pose systemic risk, while validation has largely benchmarked uncontrolled observations rather than isolated causal mechanisms before deployment.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    The paper formalizes an in-vitro causal analysis framework and measures dose-response behavior in Chronos-2 and TimesFM-2.5 across six generators and five interventions each.

    Source: §2 · p. 3, §6 · p. 11

  • Results

    Chronos-2 and TimesFM-2.5 preserve drift and harmonic patterns in tested configurations, but both overestimate persistence and fail on regime switches; TimesFM-2.5 also loses the energy-release pattern at high thresholds.

    Source: §4 · pp. 7, 9

  • Takeaways & Limitations

    Model selection should account for tested pattern-specific differences, while whether more diverse training data can remedy the identified biases and failures remains to be tested.

    Source: §5 · p. 10, §6 · p. 11

    Transfer of the experimental findings to real-world datasets cannot be assessed from benchmark comparisons based on competitor rankings rather than dataset-level pattern comparisons.

    Source: §5 · p. 10

Abstract

from arXiv · show

Transitioning from bespoke time series models towards time series foundation models changes the relationship of model and application from one-to-one to one-to-many. This shift introduces concentration risk as many, potentially high-risk, forecasting applications are exposed to the same biases and failure modes of a single time series foundation model. At the same time, this centralization allows for economies of scale in model development and validation. In this study we investigate how biases and failure modes of time series foundation models can be identified before deployment. We propose a causal analysis framework to investigate the ability of a time series foundation model to preserve time series patterns. To achieve this, we intervene on parameterized synthetic time series generators and measure the corresponding change in model output under ceteris paribus conditions. We apply our causal analysis framework to Chronos-2 and TimesFM-2.5 and test them across six distinct time series patterns. We find safe configurations for trend and harmonic oscillation patterns. The results also indicate a bias in both models towards overestimating persistence, sudden failures for both models against the regime switch pattern and failure for TimesFM-2.5 against the energy-release pattern. Our review of the original works for both models indicates that the findings might be explained by the data used for pretraining. We conclude our study with suggestions for further model development, recommendations for application-specific model selection, and a discussion of limitations and further research directions.

1 Introduction

Time series foundation models can scale forecasting across applications, but their reduced transparency and shared failure modes create systemic risk, motivating scrutiny before deployment. This study introduces a causal analysis framework and reports persistence bias and generator-specific failures in Chronos-2 and TimesFM-2.5.

1 Introduction

  • Forecasts inform decisions in critical domains, while growing datasets and parameter counts make time-series models less interpretable.The introduction contrasts bespoke application-specific models with pretrained foundation models made available for use across downstream applications.Source: §1 · p. 1
  • Foundation models may bring economies of scale, but reduced insight into their development and shared failure modes can expose many downstream applications to systemic risk.The authors argue this centralization makes deeper scrutiny necessary before deployment to a particular forecasting application.Source: §1 · p. 1
  • The study defines and applies a causal analysis framework, showing dose-response relationships for Chronos-2 (Ansari et al., 2025) and TimesFM-2.5 (Das et al., 2024) across six time-series patterns.These relationships suggest a bias toward overestimating persistence and identify failure modes.Source: §1 · p. 1
  • The analysis finds persistence overestimation in both models and sudden failures for the regime-switch and energy-release generators.The findings inform model selection for downstream applications and development of future time-series foundation models.Source: §1 · p. 1

2 Literature Review

The review finds that existing validation and adjacent methods do not provide causal analysis of time series foundation model biases and failure modes. It motivates adapting staged, ceteris paribus testing practices from other engineering disciplines.

Literature Review

  • Drug and automotive development first isolate causal effects under ceteris paribus conditions, then validate behavior under more realistic conditions (Food and Drug Administration, 2020; European Medicines Agency, 2009; International Organization for Standardization, 2018; Peter, 2022; Zollino et al., 2025).The staged approach trades some realism for causal control before testing integrated systems and monitoring outcomes after release.Source: §2 · pp. 2–3
  • Chronos-2 and TimesFM-2.5 benchmark rankings cannot reveal whether performance differences stem from training data, architecture, or other factors, or whether results persist for future observations (Ansari et al., 2025; Das et al., 2024).The review contrasts TimesFM-2.5’s first-place Monash result with its third-place Darts result, while llmtime ranks second to last on Monash and first on Darts.Source: §2 · p. 2
  • Causal analysis of time series foundation models remains unexplored: prior time-series causal studies infer relationships between variables, not model behavior under interventions on data-generating processes.The authors identify this as the methodological gap their framework aims to address.Source: §2 · pp. 2–3
  • Explainable-AI methods such as SHAP, LIME, and Partial Dependence Plots examine realized input features, whereas this study targets interventions on time-series patterns such as trend, seasonality, and autocorrelation (Lundberg and Lee, 2017; Ribeiro et al., 2016).Partial Dependence Plots can estimate average treatment effects by changing feature values, but the review argues that realized-sample interventions do not directly represent changes to the underlying time-series generator.Source: §2 · p. 2
  • Robustness analysis tests input corruption or distribution shifts, but does not identify biases on uncorrupted samples through interventions on the generating process; defining shifts is also unclear for broadly pretrained models (Ansari et al., 2025; Das et al., 2024).The review distinguishes corruption by noise or adversarial manipulation from shifts in data distribution (Hendrycks and Dietterich, 2019; Goodfellow et al., 2015; Koh et al., 2021).Source: §2 · p. 2

3 Methodology

The study uses controlled interventions on synthetic time-series generators to test whether foundation models preserve parameter-linked patterns. It applies this framework to two models across six generators, using matched noise realizations and parameter-specific statistics.

3.1 Causal Analysis Framework

  • The framework represents a generator as a parameterized function producing noisy trajectories, and a foundation model as mapping an input trajectory to an output trajectory.A parameter statistic compresses a trajectory to a scalar related to a generator parameter; the causal structure is θ → y → M(y).Source: §3.1 · p. 3
  • Intervening on one generator parameter while holding other parameters and noise constant isolates its causal effect on statistics of the input and model output.The framework maps these responses across an interval of the intervened parameter to establish a dose-response relationship.Source: §3.1 · p. 3

3.2 Experimental Configuration

  • Chronos-2 (Ansari et al., 2025) and TimesFM-2.5 (Das et al., 2024) were tested across six generators; all setups used T = 200, H = 200, and n = 50 independent noise realizations.Source: §3.2 · p. 3
  • The drift, AR(1), and harmonic-oscillator experiments intervene on drift, autoregressive coefficient, and wavelength, respectively, and estimate each corresponding parameter from trajectories.Their sweeps are μ ∈ {−0.05, −0.025, −0.005, 0.005, 0.025, 0.05}, β ∈ {−0.5, 0.0, 0.3, 0.6, 0.85, 0.98}, and λ ∈ {5, 10, 25, 50, 75, 100}; statistics are empirical mean first differences, estimated β, and FFT-based wavelength.Source: §3.2 · pp. 3–4
  • The regime-switch, energy-release, and fractional-Brownian-motion experiments intervene on dwell time, reset threshold, and Hurst exponent, estimating each parameter from the generated trajectory.The sweeps are τ ∈ {5, 10, 25, 50, 75, 100}, κ ∈ {5, 10, 25, 50, 75, 100}, and H ∈ {0.15, 0.3, 0.45, 0.55, 0.7, 0.85}; their statistics estimate dwell time, average peak values, and Hurst exponent, respectively.Source: §3.2 · p. 4

4 Results

Across the tested configurations, Chronos-2 and TimesFM-2.5 preserve random-walk drift and harmonic wavelength but show biases toward persistence and failures on regime-switch and energy-release patterns. Chronos-2 better preserves the energy-release threshold pattern, while TimesFM-2.5 deviates strongly at higher thresholds.

4 Results

  • Both models preserved harmonic wavelength across interventions, while random-walk outputs smoothed noise and TimesFM-2.5 underestimated drift magnitude relative to Chronos-2.Harmonic amplitude showed small variations along interventions; for random walks, Chronos-2’s parameter-statistic distribution aligned more closely with the trajectory distribution.Source: §4 · p. 4, Figure 2 · p. 5, Figure 3 · p. 5
  • Both models produced near-zero, flatlined outputs for most AR(1) interventions and consistently overestimated lag-one autocorrelation, indicating bias toward persistence.The outputs were close to zero except at do(β =0.98), and model-output autocorrelation estimates were high even for low β values.Source: §4 · p. 4, Figure 6 · p. 6, Figure 7 · p. 6
  • Both models failed to preserve regime switches at higher τ: Chronos-2 failed for τ ≥50 and TimesFM-2.5 for τ ≥25.At higher τ, TimesFM-2.5 showed greater variance and deviation; it appeared to overestimate τ while Chronos-2 underestimated it.Source: §4 · p. 7, Figure 11 · p. 7, Figure 12 · p. 7
  • Chronos-2 preserved the energy-release threshold pattern across interventions better than TimesFM-2.5, which began smoothing at κ ≥25 and lost the pattern at κ ≥50.Both models’ parameter statistics aligned at κ estimates around 5 and 10, began dispersing around 25, and underestimated κ around 50; at an estimate around 100, TimesFM-2.5 underestimated more than Chronos-2.Source: §4 · p. 7, Figure 14 · p. 8, Figure 15 · p. 8
  • Both models smoothed fractional Brownian motion and overestimated H, consistent with persistence bias; Chronos-2 underestimated H more at low values but overestimated it less overall.TimesFM-2.5 showed underestimation only at higher H values.Source: §4 · p. 7, Figure 18 · p. 9, Figure 19 · p. 9
  • Both models preserved drift and harmonic wavelength in tested settings, but overestimated persistence and failed regime switches; TimesFM-2.5 also strongly deviated from high-threshold energy-release patterns.Chronos-2 deviated only weakly from the energy-release threshold pattern.Source: §4 · p. 9

5 Discussion

The discussion finds prior benchmarks broadly consistent with, but insufficient to validate, the experimental results; it links possible biases to training-data coverage and recommends pattern-aware development and model selection.

5 Discussion

  • Prior benchmarks do not contradict the experiments, but provide limited evidence that their findings generalize to real-world datasets.Chronos-2 case studies show smoothing of jagged sales patterns and forecasts aligned with cyclical energy prices, but each reports only one sample without quantitative evaluation. TimesFM-2.5 has mixed dataset rankings across Darts and Monash; the authors recommend validating against real-world datasets selected for assumed patterns.Source: §5 · pp. 9–10
  • The authors suggest training-data emphasis on trend and cyclical patterns may explain preserved drift and wavelength, while weaker coverage may relate to other failures and persistence bias.Chronos-2 training includes energy and transportation datasets and synthetic data combining trend, seasonality, and irregularity; TimesFM-2.5 training is described as mostly periodic, with synthetic piecewise-linear trends, ARMA processes, and seasonal patterns.Source: §5 · p. 10
  • For future development, the authors recommend more diverse training data targeting identified failure modes and biases, potentially using parameterized synthetic generators to span configurations.They note that experimentation is needed to determine whether training diversity remedies the issues, since architecture or other factors may be responsible; Chronos-2 synthetic-only training performed close to training on real and synthetic data (Ansari et al. (2025)).Source: §5 · p. 10
  • For application-specific selection, the authors argue that pattern-level behavior may be more useful than rankings across heterogeneous datasets; Table 2 summarizes recommendations by pattern.For example, foot-traffic forecasting with expected day-night and weekday-weekend cycles may call for comparing how well models preserve wavelength.Source: §5 · p. 10, Table 2 · p. 11

6 Conclusion

The study presents a causal analysis framework for time series foundation models, reports model-specific findings, and offers application-specific selection recommendations. Its conclusions are bounded by the experimental configurations and motivate broader, domain-specific, and real-world validation.

6 Conclusion

  • Across six generators and five interventions, both models showed potential bias toward overestimating persistence, alongside idiosyncratic failure modes.The study identifies its framework as the first formalization of causal analysis for time series foundation models, to the authors’ knowledge, and makes application-specific model-selection recommendations.Source: §6 · p. 11
  • Findings are limited to the models and configurations tested, including fixed generators and interventions, limited parameter sweeps, one noise scale, and a 200-step evaluation window.The authors note that it remains unclear whether Experiment 4’s failure reflects nominal dwell time or the number of observable regime switches.Source: §6 · p. 11
  • Future work could test more models and generators, develop domain-specific experiments, analyze links between model structure and behavior, mitigate identified failures, and validate findings on real-world data.The authors suggest that structured in vitro causal analysis may improve model capabilities and inform application and regulation.Source: §6 · p. 11

Related papers

Loading 2608.24303v1…