PaperlayerView exact revision on arXiv

Source-linked AI summary

AdaCast: Conditional Parameter Generation for Adaptive Time Series Forecasting

Darahaas Nallagatla, Darryl Cherian Jacob, Pan He

arXiv:2610.12240v1cs.LGcs.AI

TL;DR

Dataset-level adaptation applies the same update to heterogeneous time series, leaving a gap in adapting parameters to individual inputs. AdaCast generates input-specific low-rank updates for a frozen TSFM; experiments report in-domain gains over static LoRA and zero-shot gains over frozen Chronos-Bolt.

  • Problem

    All-in-one TSFM adaptation applies a single dataset-level update across heterogeneous inputs, rather than adapting parameters to each series under distribution shift.

    Source: §1 · p. 1

  • Method

    AdaCast uses an input-conditioned generator to produce low-rank updates for a frozen TSFM backbone for each time series.

    Source: §1 · p. 1, §3.3 · p. 4

  • Results

    Across six benchmarks, AdaCast improves zero-shot forecasting over frozen Chronos-Bolt and in-domain forecasting over static LoRA.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Shuffling generated updates increases average error in both tested settings, supporting a benefit from matching each input with its own update.

    Source: §5 · p. 8

    AdaCast is univariate, and the TimesFM implementation incurs an extra backbone pass at inference.

    Source: §5 · p. 8

Abstract

from arXiv · show

Time-series foundation models (TSFMs) have achieved strong forecasting performance across domains. However, most adaptation methods remain static. Existing all-in-one methods learn a single set of dataset-level parameter updates and apply the same adapted model to every input. As a result, they cannot adapt the model parameters to the temporal patterns, seasonality and dynamics of each input time series. This limits their ability to produce forecasts that are tailored to heterogeneous inputs. To address this limitation, we propose AdaCast, a conditional parameter generation framework for time-series forecasting. AdaCast uses a generator to produce input-specific low-rank parameter updates for a frozen pretrained TSFM. These updates adapt the model to each input during both training and inference. Across six public benchmarks, AdaCast consistently outperforms static adaptation baseline in in-domain forecasting and improves zero-shot generalization to held-out datasets across domains. These results demonstrate that conditional parameter generation provides an effective approach for adaptive forecasting.

1 INTRODUCTION

Static dataset-level updates do not tailor forecasting parameters to heterogeneous inputs or distribution shifts. AdaCast generates input-conditioned updates for a frozen TSFM, and the paper reports strong in-domain and cross-domain forecasting results.

1 INTRODUCTION

  • Existing all-in-one TSFM methods use one dataset-level update across heterogeneous inputs, leaving instance-specific patterns and distribution shifts unaddressed.The introduction frames the gap as a lack of parameter adaptation to each input, not simply limited model capacity.Source: §1 · p. 1

1.1 Contributions

  • AdaCast extends input-conditioned parameter generation to TSFMs, generating updates from each series rather than relying on a single shared adaptation.The paper presents this as a formulation for accommodating heterogeneous inputs and distribution shifts at inference time.Source: §1 · pp. 1–2, §1.1 · p. 2
  • AdaCast generates low-rank updates for each input series and applies them to a frozen TSFM, enabling per-instance adaptation during training and inference.The generator conditions on input-specific representations; the updates are injected into the frozen decoder weights.Source: §1.1 · p. 2, Figure 1 · p. 2
  • AdaCast is evaluated on six benchmarks spanning energy, exchange-rate, and weather forecasting, with reported gains over zero-shot Chronos and static adaptation baselines in-domain and strong zero-shot generalization across domains.The introduction characterizes the experiments as evidence for adaptive forecasting.Source: §1.1 · p. 2

2 RELATED WORK

Prior forecasting approaches range from dataset-specific models to pretrained TSFMs, but common adaptation methods still share fixed updates across inputs. Input-conditioned parameter generation offers a route to adapt weights to individual series.

Time Series Forecasting

  • Traditional forecasting models often perform well in-domain but are trained separately per dataset, limiting zero-shot transfer and requiring data for each new domain.This can be impractical when data are scarce or new series appear frequently.Source: §2 · p. 2

Time Series Foundation Models

  • TSFMs use shared pretrained backbones for zero-shot forecasting, while full fine-tuning or dataset-level LoRA produces fixed updates shared across inputs (Hu et al., 2022).Series within one dataset can differ in trends, seasonalities, and dynamics, so one adapter may not capture each series' patterns.Source: §2 · p. 3

Instance-Specific Conditioning

  • Retrieval-, multimodal-, and covariate-aware methods add instance-specific context, but keep network weights fixed and often depend on external information.The related work identifies conditioning the weights themselves as a complementary route.Source: §2 · p. 3

Conditional Parameter Generation

  • Prior time-series hypernetworks generate forecasting parameters or address distribution shift, but typically condition on channel identities or distribution statistics and target smaller task-specific models.AdaCast addresses the stated TSFM gap by generating input-specific LoRA updates for a frozen backbone using only the input series.Source: §2 · p. 3

3 METHOD

AdaCast keeps a pretrained forecasting backbone frozen and generates input-specific low-rank updates from each series representation. Its generator is trained end-to-end while the backbone parameters remain fixed.

3.1 Problem Definition

  • Given multivariate histories, Chronos-Bolt forecasts each variable independently, using the median of its predicted quantiles for point forecasts.Source: §3.1 · p. 3

3.2 Overview

  • Static adaptation shares one global update across heterogeneous series, whereas AdaCast uses an encoder representation and generator to produce a distinct update for each input.The pretrained backbone remains frozen; only generator parameters are optimized.Source: §3.2 · pp. 3–4

3.3 Conditional Parameter Generation

  • A Transformer generator cross-attends shared learnable parameter tokens to each input, then uses intra- and inter-layer attention to refine them into decoder LoRA A/B matrices.Source: §3.3 · p. 4

3.4 LoRA Injection and Initialization

  • Generated LoRA updates enter the same projection modules across decoder blocks; zero-initialized B heads preserve pretrained forecasts initially, while backbone weights stay frozen.Generated updates are cleared after the current forward pass.Source: §3.4 · p. 4

3.5 End-to-End Training

  • End-to-end training optimizes only generator parameters, using scaled quantile loss over batch samples, variables, and forecast quantiles for H-step predictions.Source: §3.5 · pp. 4–5
  • 1.0% lower average MSE than static LoRA and 2.3% than the frozen backbone; AdaCast achieves the best MSE and MAE on five of six zero-shot datasets.Source: Table 1 · p. 5

4 EXPERIMENTS

Experiments test zero-shot transfer, in-domain conditional adaptation, input–update pairing, decoder and rank sensitivity, and transfer to another backbone. AdaCast improves forecasting against frozen or static references, though outcomes vary by setting and metric.

4.1 Experimental Setup

  • Zero-shot training excludes target datasets; in-domain training uses each target’s training split, with the backbone frozen and updates generated from context at inference.The generator pretraining corpus contains approximately five million windows, each with 512 context observations and 64 target observations.Source: §4.1 · p. 5

4.2 Zero-Shot Forecasting Results

  • MSE 0.2105 and MAE 0.2576 are AdaCast’s lowest average zero-shot errors; it leads both metrics on five datasets and improves both over frozen baselines on all six.Source: §4.2 · p. 6

4.3 Input-specific Conditioning

  • At matched r = 32 and α = 128, AdaCast beats static LoRA and frozen Chronos-BoltB on MSE and MAE across all four ETT datasets; gains also hold on Weather and Exchange.Source: §4.3 · p. 6, Table 2 · p. 6
  • 0.1936→0.1977 in-domain and 0.2099→0.2111 zero-shot average MSE after shuffling; generated adapters also cluster by dataset, unlike close static adapters.Source: §4.3 · p. 7, Table 3 · p. 7, Figure 3 · p. 7

4.4 Contributions of Decoder Segments

  • Disabling updates in any decoder segment worsens both metrics on all four datasets; removing early-segment updates causes the largest degradation, though all tested variants beat the frozen backbone.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 Sensitivity to LoRA Rank

  • Rank 16 underperforms higher ranks on ETTh1 and ETTm1; ranks 32, 64, and 128 differ by at most 0.003 MSE, with no consistently best rank.The default r = 32 matches or nearly matches the lowest MSE on each dataset while requiring fewer generated parameters than r = 64 or r = 128.Source: §4.5 · p. 8, Figure 4 · p. 8

4.6 Extension to a Decoder-Only Backbone

  • 3.8% zero-shot and 8.5% in-domain average MSE reductions on TimesFM 2.5; in-domain also lowers average MAE, while zero-shot MAE slightly rises.Source: Table 5 · p. 8, §4.6 · p. 8

5 CONCLUSION AND DISCUSSION

AdaCast generates input-conditioned low-rank updates for a frozen forecasting backbone without test-time optimization. Across six benchmarks it improves over frozen Chronos-Bolt in zero-shot forecasting and over static LoRA in-domain, while several scope and inference questions remain.

5 CONCLUSION AND DISCUSSION

  • AdaCast improves zero-shot forecasting over frozen Chronos-Bolt and in-domain results over static LoRA, with MSE gains also extending to TimesFM 2.5.The conclusion reports experiments on six benchmarks.Source: §5 · p. 8
  • Shuffling generated updates increases average error in both settings, while generator-depth differences prevent attributing the larger in-domain effect solely to training regime.Source: §5 · p. 8
  • Decoder ablations show greatest sensitivity to removing early-block updates, and the rank study supports r = 32 as a practical choice.Source: §5 · p. 8
  • The model is univariate, early-block sensitivity needs further study, and the TimesFM implementation incurs an extra backbone pass.Future work includes cross-variate conditioning, clarifying decoder depth, strengthening adaptation on unseen domains, and reducing inference overhead.Source: §5 · p. 8

A DATASET STATISTICS

The evaluation covers six publicly available forecasting benchmarks across energy, meteorological, and financial domains. Table 6 reports dataset statistics and split sizes.

A DATASET STATISTICS

B BASELINE METHODS

The baseline section describes several TSFM architectures and their forecasting approaches, and provides cost-table contexts. Its multi-seed evaluation repeats pretraining and evaluation across three seeds.

B BASELINE METHODS

  • The zero-shot baseline set comprises six public TSFMs spanning five architectural families, including autoregressive, patch-based, masked-reconstruction, MLP-Mixer, and decoder-only models.Source: B BASELINE METHODS · p. 11
  • Chronos quantizes scaled inputs into a fixed vocabulary and samples future trajectories autoregressively, whereas Chronos-Bolt uses patch tokenization and directly regresses horizon-wide quantiles.Chronos-Bolt is also the backbone adapted by AdaCast.Source: B BASELINE METHODS · p. 11
  • Table 7 covers training costs on one GPU, and Table 8 covers inference costs on ETTh1 under specified context, horizon, and batch settings.Source: Table 7 · p. 11, Table 8 · p. 11
  • MOMENT uses masked reconstruction on a T5-style encoder and is evaluated with its released forecasting head rather than autoregressive or direct quantile decoding.Source: B BASELINE METHODS · p. 11
  • TTM uses lightweight MLP-based TSMixer blocks without self-attention and is pretrained on public multi-resolution time-series corpora.Source: B BASELINE METHODS · pp. 11–12
  • Moirai supports arbitrary variate counts and sampling frequencies, while TimesFM predicts future-step blocks through a decoder-only transformer.Source: B BASELINE METHODS · p. 12

C.1 Multi-Seed Evaluation

  • The multi-seed evaluation repeats pretraining and evaluation with seeds 1, 43, and 2021 while holding the stated settings fixed; frozen Chronos-Bolt has no seed variation at inference.Source: §C.1 · p. 12

C.2 In-domain Evaluation

AdaCast outperforms static adaptation on both metrics across all six datasets, with average MSE and MAE reductions over the shared adapter. Longer-horizon evaluation extends beyond Chronos-Bolt's native 64-step horizon using rolling forecasts.

C.2 In-domain Evaluation

  • The in-domain comparison uses the same frozen Chronos-Bolt backbone and adapter configuration; the difference is whether LoRA matrices are learned per dataset or generated per input.Source: §C.2 · p. 12
  • 3.4% MSE and 2.0% MAE reductions averaged across benchmarks show AdaCast outperforming a single shared adapter trained on the same target dataset.AdaCast outperforms static adaptation on both metrics across all six datasets; the MSE margin ranges from 1.1% on ETTm2 to 8.3% on ETTm1.Source: §C.2 · p. 12

C.3 Evaluation on Longer Forecast Horizons

  • Longer-horizon tests use T = 512 and H ∈ {64, 96, 192, 336}; beyond the native 64-step horizon, inference uses a rolling-forecast strategy.The rolling strategy is the one used by TS-RAG (Ning et al., 2025).Source: §C.3 · p. 12

C.4 Generator Depth

Generator depth has little effect on four datasets, while Exchange and Weather improve with deeper generators; the sensitivity analysis examines whether perturbation-sensitive decoder blocks are useful locations for AdaCast updates.

C.4 Generator Depth

  • On ETTh1, ETTh2, ETTm1, and ETTm2, the full MSE range across generator depths is at most 0.0009, comparable to seed-to-seed variation.The ablation varies Npg over {1, 4, 8, 16}, holding rank, scaling, learning rate, and pretraining corpus fixed.Source: §C.4 · p. 12
  • Exchange MSE decreases from 0.0654 at Npg = 1 to 0.0591 at Npg = 16, while Weather MAE falls from 0.1891 to 0.1785; gains are largely exhausted by Npg = 8.Npg = 16 doubles generator parameters without further gain, so Npg = 8 is adopted for zero-shot pretraining.Source: §C.4 · pp. 12–13, Table 10 · p. 13

D SPARSE-FEATURE SENSITIVITY AND DECODER-REGION ADAPTATION

  • The sensitivity study trains a separate TopK sparse autoencoder for each ETT dataset and Chronos-Bolt decoder block, then compares feature-removal effects with restricted-region AdaCast performance.The section reports reconstruction quality and intervention effects before relating them to decoder-region adaptation.Source: D SPARSE-FEATURE SENSITIVITY AND… · p. 13, Table 12 · p. 13

D.1 SAE Training Protocol

The appendix measures SAE reconstruction quality, tests high-activation feature removals, and compares their forecast effects against a reconstruction-only control.

D.1 SAE Training Protocol

  • For each of four ETT datasets, frozen Chronos-Bolt activations from T=512, H=64 univariate training windows feed a separate SAE for each of 12 decoder blocks.Each SAE uses 3072 latent coordinates and retains the largest K=64 nonnegative coordinates per hidden state.Source: §D.1 · pp. 13–14, Table 14 · p. 14

D.2 Reconstruction Quality

  • SAEs retain most activation variance: FVE ranges from 0.934 to 0.961 on ETTh1 and from 0.963 to 0.980 on ETTm1.The reconstruction therefore approximates the original hidden state closely for these datasets.Source: §D.2 · pp. 14–15

D.3 Feature Interventions and Sign Statistic

  • At blocks 10–11, all 64 tested high-activation feature removals increase approximate CRPS across four ETT datasets, with interventions evaluated on the chronological training split.At block 9, +frac ranges from 0.844 to 0.938.Source: §D.3 · p. 15

D.4 Intervention Magnitude and Reconstruction-Only Control

  • The reconstruction-adjusted median intervention effect ranges from 0.00586 to 0.00969 at block 10 and from 0.01664 to 0.02193 at block 11.Reconstruction-only controls are substantially smaller than late-block intervention medians; the control-adjusted difference is descriptive, not an additive decomposition.Source: §D.4 · p. 15

D.5 Perturbation Sensitivity versus Adaptation Utility

The most perturbation-sensitive decoder region is not the most useful adaptation region: restricting updates to blocks 9–11 helps over zero-shot Chronos-Bolt but underperforms the best tested restricted region.

D.5 Perturbation Sensitivity versus Adaptation Utility

  • Every tested high-activation feature removal increases approximate CRPS at blocks 10–11 across all four datasets, identifying a perturbation-sensitivity peak there.Figure 6 plots the fraction of Top-64 feature ablations that increase approximate CRPS by decoder block.Source: Figure 6 · p. 16
  • Across all four datasets, restricting AdaCast updates to blocks 9–11 improves over zero-shot Chronos-Bolt but is weaker than the best tested contiguous region in both MSE and MAE.The table compares full AdaCast, the best tested restricted region, and the blocks 9–11 restriction.Source: §D.5 · pp. 15–16, Table 15 · p. 16
  • A plausible explanation is that late-block updates directly affect predicted quantiles, while earlier updates undergo further processing by frozen decoder blocks.The paper suggests high SAE sensitivity may indicate limited tolerance to imperfect updates, rather than adaptation utility.Source: §D.5 · p. 16

Related papers

Loading 2610.12240v1…