PaperlayerView exact revision on arXiv

Source-linked AI summary

On the Encoding of Calendar Features in Covariate-Informed Time Series Foundation Models

Alexander Kreusel, Maximilian Beichter, Alexandra Nikoltchovska, Benjamin Schäfer, Ralf Mikut, Frederik Zahn, Veit Hagenmeyer

arXiv:2610.13676v1cs.LG

TL;DR

The effect of calendar-feature encoding in covariate-informed TSFMs remains underexplored, despite encoding being an inference-time design choice. The study compares encodings across models and datasets and finds that their effects vary across models and data regimes.

  • Problem

    Calendar-feature encoding in covariate-informed TSFMs has not been systematically compared, leaving its effect on forecast performance unquantified.

    Source: §1 · p. 2, §2 · p. 3

  • Method

    The study compares calendar encodings across TSFMs and uses SHAP to assess covariate contributions relative to past target values.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    Encoding choice affects performance across models and data regimes, and the effectiveness and utilization of specific encodings vary by model.

    Source: §7 · p. 11

  • Takeaways & Limitations

    Encoding is a consequential inference-time design choice for covariate-informed TSFMs.

    Source: §1 · p. 2, §2 · p. 3

    Experiments use hourly data, a context length of 8192, and targets influenced by calendar effects; continuous exogenous drivers are outside the study's scope.

    Source: Encodings substantially impact… · p. 10

Abstract

from arXiv · show

Time series foundation models (TSFMs) increasingly offer covariate support, with calendar information being the most common type of categorical covariate in practice, as it is available for every point in time. However, how these features are encoded is largely treated as an implementation detail rather than as an object of study, despite encoding being one of the few design choices left for pretrained models at inference time. With the present study, we demonstrate that the way information is propagated into the model substantially affects forecast accuracy of TSFMs. Therefore, we systematically compare seven calendar encodings, including three conventional target-agnostic representations (ordinal, one-hot, and cyclic) and four target encodings, across four state-of-the-art TSFMs on nine datasets from multiple domains in a zero-shot rolling evaluation. The nine datasets are influenced by social activities, such as weekends and holidays, making calendar information particularly relevant for forecasting. We find that encoding choice affects forecast accuracy across all models, with MASE differences of up to 9.9 % on non-holidays and 83 % on holidays within a single model. Effectiveness and utilization vary substantially across models. Improvements are concentrated around holidays, where calendar information provides signals unavailable from target history, while some model encoding combinations also improve non-holiday forecasts. A SHAP-based attribution analysis locates the contribution of the covariates relative to the autoregressive component and explains why some model-encoding combinations degrade performance. Our findings suggest that current benchmarks for covariate-informed TSFMs can distort model comparisons by imposing fixed encoding strategies.

1 Introduction

TSFMs can forecast without task-specific training, but target history alone may miss effects such as irregular public holidays. This study examines calendar-feature encodings and how models use them.

1 Introduction

  • External factors such as irregular public holidays can be difficult to infer from target history alone.Calendar information has begun to be incorporated into TSFMs, but the effect of its encoding on forecast performance remains largely unexplored.Source: §1 · p. 2
  • The study compares seven calendar encoding strategies across four state-of-the-art TSFMs and nine socially influenced datasets from multiple domains.Source: §1 · p. 2
  • SHAP is used to examine how models use available covariates and the role calendar information plays in forecasting performance.Source: §1 · p. 2

2 Related Work and Foundations

Prior work has explored covariates in TSFMs and calendar-feature encodings, but encoding choices and their effects remain insufficiently examined. Benchmark wrappers also handle categorical covariates differently.

Time Series Foundation Models and Covariates

  • Studies report covariate-related gains in several domains, but t0-beta performance degraded on 11 of 30 fev-tasks with known-future covariates.Those results do not determine whether degradation stems from uninformative covariates or their representation.Source: §2 · p. 2

Covariate Encoding

  • Calendar-feature research includes target-agnostic and target-based encodings, while TSFM studies use varied schemes without systematically comparing encoding itself.Prior holiday-modeling work reported error reductions greater than 80% on holidays and about 10% on ordinary days, but finer grouping was not consistently beneficial.Source: §2 · p. 3
  • Benchmark datasets and wrappers differ in how categorical covariates are stored and encoded, leaving the effect of this variation on TSFM performance unquantified.fev-bench wrappers dispatch on storage dtype, while FETS stores covariates numerically using sine–cosine pairs and binary indicators.Source: §2 · p. 3

3 Methodology

The methodology compares calendar representations and uses SHAP to assess their contributions relative to past target values. Target encodings derive target-scale profiles from context observations.

3.1 Calendar Encodings

  • Calendar encodings use hour, weekday, and holiday information, with conventional representations supplemented by a binary holiday feature and target encodings based on context target means.Source: §3.1 · pp. 3–4, Table 1 · p. 4
  • Target encodings compute a context mean for observations sharing a categorical combination and assign that value to matching context and forecast steps.The encodings are recomputed for each prediction window using context alone, assuming every forecast-window combination occurs in the context.Source: §3.1 · p. 4

3.2 SHAP-based Analysis for Identifying Contributions in Forecast

  • SHAP decomposes forecasts into additive feature-group contributions, interpreted as average forecast changes from adding a group across coalitions.The approach withholds absent groups directly, avoiding background samples that can create unrealistic feature combinations; temporal groups are summed as the past-target contribution.Source: §3.2 · pp. 4–5

4 Experimental Setup

The experiments compare four TSFMs and two baselines across nine hourly datasets in zero-shot rolling evaluation. Forecasts are assessed separately for holiday and non-holiday windows.

4 Experimental Setup

  • Four TSFMs are evaluated alongside a 168-hour seasonal-naive model and a context-recomputed target-encoding baseline.Source: §4 · p. 5
  • Nine datasets from energy, transportation, traffic, and water each provide one year of hourly test data, with holiday information added from geographic locations.Source: §4 · p. 5
  • Zero-shot rolling evaluation uses 8192 context observations, 24-hour forecast horizons and strides, and forecasts starting at midnight; holiday and non-holiday results are reported separately.Source: §4 · p. 5
  • Within each model, encoding is the only varied factor, with other parameters held constant; the study expects model-specific sensitivity and holiday-concentrated gains.It also expects target encodings to be more robust than conventional encodings, which can add up to 32 target-agnostic covariates.Source: §4 · p. 5

5 Results

Across nine datasets, the best encoding varies by model, and results differ between holidays and non-holidays. Target encodings generally lead on holidays, while non-holiday effects depend more strongly on the model.

Overall results across all days

  • TimesFM-3 has the best overall model-level result with cyclic_h,W + binH, followed by Chronos-2 with Th,W + Th,H; no general encoding pattern emerges.Chronos-2, TabPFN-TS-3, and TiRex-2 perform best with target encodings, while TimesFM-3 is comparable across encodings and its best result is not target-based.Source: §5 · pp. 6–7, Table 2 · p. 6, Figure 1 · p. 7

Results on holidays only

  • 26 of 28 model-encoding combinations improve holiday performance; target encodings outperform conventional ones, and TimesFM-3 is best with Th,W (H = Sun).The two exceptions are TiRex-2 with one-hot_h,W + binH and cyclic_h,W + binH.Source: §5 · p. 7, Figure 2 · p. 8

Results on non-holidays only

  • Non-holiday effects vary by model: TabPFN-TS-3 improves across all encodings, while conventional encodings improve MASE by up to 5.8%.Target encodings beat conventional encodings on non-holidays, but Chronos-2, TiRex-2, and TimesFM-3 show marginal gains or deteriorations from covariates.Source: §5 · pp. 7–8, Figure 2 · p. 8

Chronos-2

  • Chronos-2 achieves its best results on all window types with Th,W + Th,H; conventional encodings help on holidays but decrease non-holiday performance.The study attributes this difference to target-scale calendar information supporting autoregressive structure, while calendar-only information can dilute the signal.Source: §5 · p. 7

TabPFN-TS-3

  • TabPFN-TS-3 is the only model that improves under all seven encodings across all three window types.Source: §5 · p. 7

TiRex-2

  • For TiRex-2, only the three joint target encodings improve mean all-day MASE; encodings with multiple covariates have almost no or negative effects.TiRex-2 also shows no substantial encoding-driven improvement on the mta_subway and mta_bus datasets.Source: §5 · p. 8

TimesFM-3

  • TimesFM-3 improves substantially over its no-covariate model on holidays with every encoding, but gains no meaningful non-holiday improvement and target encodings reduce performance there.Source: §5 · p. 8

6 Discussion

Calendar encoding changes forecast performance across models, with the largest within-model differences on holidays. SHAP analyses connect these differences to model-specific use of calendar covariates and motivate treating encoding as an explicit benchmark factor.

Encodings substantially impact performance across all models and data regimes

  • Covariates contribute 20–34% of mean absolute SHAP value on holidays versus at most 14% on non-holidays; most holiday contribution shifts to Th,H, except for TiRex-2.Figure 3 averages SHAP values across datasets for Th,W (H = 7) and Th,W + Th,H.Source: Figure 3 · p. 9
  • Within-model gaps in MASE between the best and worst encoding range from +4.8% (TiRex-2) to +9.9% (TabPFN-TS-3) on non-holidays, and from +44% (TabPFN-TS-3) to +83% (TiRex-2) on holidays.Source: Encodings substantially impact… · p. 9
  • On non-holidays, Chronos-2 relies more on autoregressive structure and changes marginally, while TimesFM-3 weights calendar features more yet performs worse than its non-covariate variant.The discussion attributes TimesFM-3's degradation to covariates diluting a signal the model already captures effectively.Source: Encodings substantially impact… · p. 9
  • On m_dense holidays, Th,H outweighs Th,W for Chronos-2, TabPFN-TS-3, and TimesFM-3, but TiRex-2's near-canceling covariate effects produce an overestimated forecast.The authors suggest differences in handling encodings may stem from architectural and training differences.Source: Encodings substantially impact… · p. 9, Figure 4 · p. 10

6 Discussion

  • Benchmarks using one fixed encoding test only particular model–encoding combinations, and alternative encodings may change observed model rankings.The discussion proposes evaluating multiple encodings or having developers provide model-specific handling or recommendations.Source: Encodings substantially impact… · p. 10

Encodings substantially impact performance across all models and data regimes

  • The study uses hourly datasets and targets affected by calendar effects; it does not assess settings driven mainly by continuous exogenous factors such as temperature.The authors note that contexts exceeding one year and additional time resolutions would further validate the findings.Source: Encodings substantially impact… · p. 10

7 Summary and Outlook

The paper argues that calendar-covariate encoding is under-investigated despite its potential to shape TSFM performance and model comparisons. Its results show encoding effects across models and support treating encoding as an explicit benchmark factor, with adaptive strategies left for future work.

7 Summary and Outlook

  • The paper identifies a gap: covariate encoding is usually treated as an implementation detail, while fixed encodings in benchmarks may conflate model capability with encoding choice.Source: §7 · p. 11
  • Encoding affects performance across models, and the effectiveness and utilization of particular encodings vary by model.Source: §7 · p. 11
  • The authors recommend considering encoding in future multivariate benchmarks because fixing one encoding can influence and potentially distort model rankings.Source: §7 · p. 11
  • Future work could treat encoding as an explicit experimental factor and investigate data-driven approaches such as in-context clustering to select suitable encodings.The proposed approach would use time-series-specific characteristics identified from context.Source: §7 · p. 11

Generative AI Usage Disclosure

The authors disclose uses of generative AI across research, implementation, analysis, and figure and paper preparation, and describe review procedures and exclusions.

Generative AI Usage Disclosure

  • Generative AI tools supported methodology and experiments, method implementation, translation, dataset cleaning and reformatting, and qualitative and thematic analysis.Source: Generative AI Usage Disclosure · p. 12
  • The authors state that AI was not used to propose or refine hypotheses, interpret results, generate synthetic datasets, develop theoretical frameworks, formulate mathematical claims, or assist with proofs.Source: Generative AI Usage Disclosure · p. 12
  • AI tools also assisted with figures, code, information sourcing, readability editing, literature identification, and reference formatting; authors reviewed the work and take responsibility for the final content.The disclosure states that two authors reviewed and tested AI-assisted code and that AI-generated text was checked against the underlying results.Source: Generative AI Usage Disclosure · p. 12

A Appendix

The appendix provides additional methodological and experimental details, followed by dataset-specific results and a second experiment run.

A Appendix

  • Appendix A.1.1 explains and visualizes the encoding strategies and gives further details on the SHAP method.Source: A Appendix · p. 17
  • Appendix A.2 describes the datasets, models, evaluation metrics, and hardware and software used in the study.Source: A Appendix · p. 17
  • Appendix A.3 presents dataset-specific results from the main experiments and results from a second run using context length 2048.Source: A Appendix · p. 17

A.1.1 Encodings

The study compares seven calendar encodings, ranging from conventional numeric representations to target encodings that summarize historical target values for calendar categories. The target encodings differ in how they treat holidays and combine calendar categories.

Ordinal h,W + bin H

  • Ordinal hour and weekday values plus a binary holiday flag produce three covariates.Hours range from 0 to 23 and weekdays from 0 to 6.Source: §A.1.1 · p. 17, Figure 5 · p. 17

One-hot h,W + bin H

Cyclic h,W + bin H

Th,W (H = Sun)

  • Th,W (H = Sun) target-encodes hour and weekday, mapping holidays to Sunday.Sundays and holidays therefore share the same encoding profile.Source: §A.1.1 · p. 19, Figure 8 · p. 19

Th,W (H = 7)

Th,W,H

  • Th,W,H adds a binary holiday category to hour and weekday, creating separate holiday profiles for each weekday.A weekday–holiday combination may occur only a few times or not at all in the context, creating potential data scarcity.Source: §A.1.1 · p. 20, Figure 10 · p. 20

Th,W + Th,H

  • Th,W + Th,H supplies two target encodings, combining hour–weekday and hour–holiday information.Together they contain the same weekday and holiday information as the other strategies.Source: §A.1.1 · p. 21, Figure 11 · p. 21

A.1.2 SHAP Methodology

The appendix details the SHAP procedure, models, evaluation setup, and additional results. In an oracle setting with the target supplied as a covariate, models differ substantially in their ability to use that information.

A.1.2 SHAP Methodology

  • SHAP values measure covariate contributions relative to past target groups, using exact coalition evaluation and masking inactive inputs.Target encodings are computed once from the full context, so their attribution includes the target statistics they summarize; TimesFM-3 uses its internal target mask to avoid leaking masked values.Source: §A.1.2 · p. 21

A.2.1 Models and Pipeline

  • The cited descriptions state that Chronos-2, TabPFN-TS-3, TiRex-2, and TimesFM-3 each use the median forecast as their point forecast.Source: §A.2.1 · p. 22
  • The study compares TSFMs against a 168-hour seasonal-naive baseline and a Th,W (H = Sun) target-encoding baseline recomputed for each forecast window.Source: §A.2.1 · p. 22

A.2.2 Datasets

  • Nine datasets are tested in a zero-shot rolling evaluation, with each test period spanning exactly one year and none included in the models’ pretraining corpora.Higher-frequency TransnetBW and FeederBW data are resampled to hourly resolution.Source: §A.2.2 · p. 23

Mean Absolute Scaled Error

  • MASE scales point-forecast errors by the in-sample mean absolute error of a seasonal-naive forecast, using a 168-hour seasonal period recomputed per context.Source: Mean Absolute Scaled Error · pp. 23–24

Scaled Quantile Loss

  • SQL averages pinball loss over quantiles 0.1–0.9 and scales it by the in-sample seasonal-naive error for each context.Source: Scaled Quantile Loss · p. 24

A.2.4 Hardware and Software

  • Experiments use a single RTX 3090 workstation, with fixed 8192-step contexts, 24-step horizons, and 24-step strides.Source: §A.2.4 · p. 24

A.3.10 Overall Results with Ground Truth as Covariate

  • With the ground-truth target supplied as a known covariate, TabPFN-TS-3 and TimesFM-3 nearly reproduce it, while Chronos-2 and TiRex-2 remain near MASE 0.30.Chronos-2 performs notably worse on holidays; the benefit of holiday covariates depends partly on each model’s ability to use them.Source: §A.3.10 · p. 43, Table 13 · p. 43
Loading 2610.13676v1…