PaperlayerView exact revision on arXiv

Source-linked AI summary

Instance-Adaptive Prompts as Context for Time-Series Foundation Models

Zehao Xiao, Shifeng Xie, Lei Zan, Jianfeng Zhang, Lujia Pan, Ievgen Redko, Malik Tiomoko, Keli Zhang

arXiv:2609.34786v1cs.LG

TL;DR

Longer histories can improve forecasting but substantially increase inference cost, and they may be unavailable or exceed a backbone’s context window. PaCTS generates instance-adaptive latent prompts from visible context for frozen TSFMs. Across evaluated settings it improves forecasting, can outperform double-context inference at lower cost, and generalizes better than weight-space adaptation.

  • Problem

    Longer histories can improve TSFM forecasting but increase inference cost, and some series or backbone context windows limit the history available.

    Source: §1 · p. 1, §3 · p. 3

  • Method

    PaCTS prepends continuous instance-adaptive prompts generated from global and segment-level context information to the embeddings of a frozen TSFM.

    Source: §1 · p. 2, §3.1 · p. 3

  • Results

    Across evaluated datasets, architectures, and context lengths, PaCTS consistently improves forecasting; at 4,096 steps it reduces FLOPs by 49.3% versus zero-shot inference at 8,192 steps.

    Source: §4.2 · p. 7, §5 · p. 10

  • Takeaways & Limitations

    Learned input-space prompts provide a compact context surrogate, with favorable accuracy–compute trade-offs and better out-of-distribution transfer than weight-space adaptation.

    Source: §5 · p. 10

    Prompts are trained only under the univariate setting, leaving multivariate prompt training unexplored.

    Source: §5 · p. 10

Abstract

from arXiv · show

Longer histories can improve time-series foundation models (TSFMs), but require substantially higher inference cost. We therefore ask whether contextual information can be provided more efficiently through a compact set of learned token embeddings. We introduce PaCTS, which generates a small set of instance-adaptive latent prompts in the form of continuous embedding tokens conditioned on the visible context. These prompts serve as compact context surrogates for frozen TSFMs. PaCTS constructs them from instance-specific global statistics and further refines them with segment-level temporal information, capturing both global characteristics and local temporal variations. The prompt module is jointly trained and deployed across heterogeneous time series with the frozen backbone. Extensive experiments demonstrate the effectiveness of prompts as context, consistently improving forecasting across context lengths and model architectures. With a shorter input context, PaCTS can outperform the same frozen backbone using double context while requiring substantially less inference computation. Compared with weight-space adaptation methods, PaCTS achieves stronger improvements and better out-of-distribution generalization.

1 INTRODUCTION

Longer histories raise inference cost while often yielding only modest forecasting gains, and they may be unavailable or exceed a backbone’s context window. PaCTS instead supplies frozen TSFMs with learned, instance-adaptive prompts, improving forecasting with shorter context and lower cost than doubling history.

1 INTRODUCTION

  • PaCTS consistently reduces forecasting error across evaluated architectures and context lengths, and can outperform double-context inference at substantially lower inference cost.The prompt parameters amount to less than 0.15% of the pretrained model parameters.Source: Figure 1 · p. 1, §1 · p. 2
  • Longer histories substantially increase inference FLOPs across three TSFMs on GIFT-Eval while yielding only modest reductions in MASE.Figure 1 illustrates this cost–accuracy trade-off.Source: §1 · p. 1
  • More history may be unavailable for short series or exceed the context window supported by a pretrained backbone.The paper asks whether compact context derived from available history can improve forecasting with a frozen TSFM.Source: §1 · p. 2
  • PaCTS supplies frozen TSFMs with continuous, instance-adaptive prompt tokens derived from input statistics and segment-level temporal information.The prompt module is jointly trained across heterogeneous series and reused on new series without further parameter updates.Source: §1 · p. 2

2 RELATED WORK

Prior work adapts time-series foundation models through backbone changes, weight updates, input-space prompts, or related examples in context. PaCTS instead learns compact prompts directly from the target series for shared use across heterogeneous series without dataset-specific tuning.

Transformer-based time-series foundation models

  • Time-series foundation models span varied tokenizations and architectures, while this work studies compact learned context without changing backbone architecture.Examples include value-tokenized Chronos and patch-based models such as MOMENT, Moirai, TimesFM, and Timer.Source: §2 · p. 2

TSFM post-training

  • Post-training methods include full or parameter-efficient weight updates, reinforcement learning, and frozen-backbone corrections; PaCTS trains a shared prompt module in input space.The module is trained across heterogeneous time series and reused without target-specific fine-tuning.Source: §2 · p. 2

Input-space conditioning

  • Unlike prompt work that combines modalities or supplies related examples, PaCTS learns latent prompts from the target series as compact context for frozen TSFMs.Its shared module is used without dataset-specific tuning and aims to provide longer-history benefits at little additional inference cost.Source: §2 · p. 3

3 METHOD

PaCTS adds a compact prompt channel to a frozen TSFM, using shared and instance-adaptive prompt components to represent global and local temporal information. The prompt module is trained across heterogeneous time series and generates prompts at inference without further parameter updates.

3 METHOD

  • PaCTS generates M ≪ L instance-adaptive latent prompt tokens from visible context and prepends them to time-series embeddings, avoiding substantial context extension.The pretrained backbone remains frozen while prompt-generation parameters are optimized.Source: §3 · p. 3

3.1 LATENT PROMPTS AS CONTEXT SURROGATES

  • Latent prompts are learnable continuous embedding vectors prepended to patch embeddings, retaining the complete visible context while adding compact conditioning signals.Because the prompts are added rather than substituted, the time-series tokens remain in the input.Source: §3.1 · p. 3

3.2 INSTANCE-ADAPTIVE PROMPT GENERATION

  • The prompt combines a shared learned component with an input-adaptive component generated from scale-invariant statistics of distributional and temporal structure.A compact latent basis and a small MLP map the statistics to a composition of prompt patterns.Source: §3.2 · pp. 4–5

3.3 SEGMENT-LEVEL PROMPT REFINEMENT

  • Segment-level statistics with relative-position embeddings provide local temporal descriptors that cross-attention uses to refine the global adaptive prompts.This combines instance-level characteristics with localized information without increasing the number of prompt tokens.Source: §3.3 · p. 5

3.4 TRAINING

  • PaCTS is jointly trained across heterogeneous time-series datasets with the pretrained TSFM frozen, and one prompt module generates prompts without further parameter updates.Training masks invalid or padded forecast positions and normalizes each sample’s loss by its valid positions.Source: §3.4 · pp. 5–6

4 EXPERIMENTS

The experiments evaluate a jointly trained prompt module across GIFT-Eval, TIME, and multiple pretrained TSFM architectures. PaCTS improves forecasting over zero-shot and adaptation baselines, including when using shorter contexts.

4.1 EXPERIMENTAL SETUP

  • A single prompt module is jointly trained across GIFT-Eval datasets and evaluated on all test sets, with TIME reserved for untuned generalization evaluation.Chronos-2 is the primary backbone; PatchTST-FM and TimesFM-2.5 are also evaluated, alongside five adaptation baselines on Chronos-2.Source: §4.1 · p. 6

4.2 RESULTS

  • At both 4,096 and 8,192 steps, PaCTS achieves the lowest average MASE and CRPS among the compared Chronos-2 methods.It also outperforms the zero-shot backbone and five adaptation baselines at double context length.Source: §4.2 · p. 6, Table 1 · p. 7
  • At 4,096 steps, PaCTS beats zero-shot Chronos-2 at 8,192 steps while reducing FLOPs by 49.3%, peak GPU memory by 18.4%, and latency by 36.8%.At the same 4,096-step context, prompt generation adds 3.9% FLOPs and 1.4% memory; the prompt module increases model size by 0.128%.Source: §4.2 · pp. 6–7
  • Input conditioning improves both MASE and CRPS over static prompts, and segment-level refinement yields further gains.Static prompts improve CRPS but provide little MASE improvement.Source: §4.2 · p. 7, Table 3 · p. 7
  • On TIME, PaCTS is the only compared method to improve both MASE and CRPS over zero-shot Chronos-2, including when the baseline uses 8,192 steps.The prompt module is applied directly after GIFT-Eval training, without further training or tuning.Source: Figure 4 · p. 8, §4.2 · p. 8
  • Across history-length groups and all three forecast-horizon groups, PaCTS improves both metrics and outperforms double-context zero-shot forecasts.Doubling context helps primarily for longer histories and medium or long horizons, while PaCTS also surpasses adaptation baselines across the horizon groups.Source: §4.2 · pp. 7–8, Figure 3 · p. 8
  • Univariate-trained PaCTS prompts improve both metrics in multivariate evaluation on GIFT-Eval and TIME without additional training or tuning.At 4,096 steps, PaCTS also beats zero-shot Chronos-2 at 8,192 steps on both metrics on both benchmarks.Source: §4.2 · pp. 8–9, Table 4 · p. 9
  • On PatchTST-FM and TimesFM-2.5, shorter-context PaCTS achieves lower MASE and competitive CRPS against zero-shot inference with double context.At 4,096 steps, it uses around 51% and 52% of the GFLOPs, respectively.Source: §4.2 · p. 9, Figure 5 · p. 9

5 CONCLUSION

PaCTS uses learned input-space prompts as context surrogates for frozen time-series foundation models, with gains reported across datasets and architectures. Its scope is limited by univariate prompt training and dependence on the backbone’s context-scaling behavior.

5 CONCLUSION

  • Across GIFT-Eval and TIME, PaCTS provides forecasting gains and favorable accuracy–compute trade-offs, with better out-of-distribution transfer than weight-space adaptation.The approach also extends across multiple TSFM architectures and to multivariate inference.Source: §5 · p. 10
  • PaCTS improves forecasting without explicitly processing substantially longer histories or updating backbone parameters.It uses learned input-space prompts as context surrogates for frozen time-series foundation models.Source: §5 · p. 10
  • Prompts are trained only under the univariate setting, leaving multivariate prompt training unexplored.The ability of prompts to replace additional context also depends on the underlying backbone’s context-scaling behavior.Source: §5 · p. 10

A DATASETS AND EVALUATION PROTOCOL

The evaluation pools prompt training tasks and aggregates results across dataset–frequency–horizon configurations, with separate in-distribution and out-of-distribution benchmarks. Temporal cutoffs protect official test windows, while metric aggregation follows benchmark-specific protocols.

A DATASETS AND EVALUATION PROTOCOL

  • Prompt training pools 48 dataset–frequency tasks, while evaluation aggregates 97 dataset–frequency–horizon configurations across 35 GIFT-Eval datasets.A configuration, rather than a dataset, is the unit of metric aggregation; each trained prompt module is tested without dataset-specific fine-tuning.Source: A DATASETS AND EVALUATION PROTOCOL · p. 14
  • The available series prefix is truncated to avoid overlap with official evaluation windows; training samples use its first ⌊0.89T_avail⌋ observations and validation targets use the remainder.Validation contexts may include earlier training observations, but forecast targets begin after the training boundary; the construction is approximately an 80/10/10 temporal split on long series.Source: A DATASETS AND EVALUATION PROTOCOL · p. 14
  • TIME contains 98 tasks from 50 datasets with no dataset overlap with GIFT-Eval, and evaluates direct transfer of the GIFT-trained prompt module without updates.GIFT-Eval measures in-distribution performance, while TIME evaluates transfer to out-of-distribution datasets.Source: A DATASETS AND EVALUATION PROTOCOL · p. 14
  • GIFT-Eval divides each test-set score by its Seasonal Naive score and takes a geometric mean, while TIME uses MASE and quantile-based CRPS under its official window-level aggregation protocol.GIFT-Eval reports MASE and CRPS; its CRPS is the mean weighted sum quantile loss over Q = {0.1, 0.2, . . . , 0.9}.Source: A DATASETS AND EVALUATION PROTOCOL · p. 14

B STATISTICS OF PROMPT CONSTRUCTION

PaCTS builds prompt descriptors from normalized visible input using ten non-parametric statistics, applying the extractor both globally and to valid segments. Ordered segment descriptors retain local temporal variation, while the statistics are invariant to series location and scale.

B STATISTICS OF PROMPT CONSTRUCTION

  • The prompt generator computes ten statistics from normalized visible input, using the observation mask to exclude missing or padded values.The same extractor is applied to the full context and each valid segment.Source: B STATISTICS OF PROMPT CONSTRUCTION · p. 15
  • The statistics include changes across time and lag-one dependence, and ordered segment descriptors retain local temporal variation.In the segment branch, the configured seasonal period can affect segment granularity.Source: B STATISTICS OF PROMPT CONSTRUCTION · p. 15
  • All statistics are computed non-parametrically and are invariant to the location and scale of the original series.Source: B STATISTICS OF PROMPT CONSTRUCTION · p. 15

C TRAINING AND ADAPTATION BASELINES

PaCTS training uses a shared prompt module with frozen backbones, while adaptation baselines follow the official Chronos-2 fitting pipeline. The comparison reports PaCTS as best in-distribution on GIFT-Eval and the only method improving both metrics over zero-shot inference on TIME.

C TRAINING AND ADAPTATION BASELINES

  • The adaptive generator uses rank r = 4, M = 10 prompt tokens, and up to S = 16 segments; prompts are optimized with AdamW at learning rate 10^-3 for at most 12 epochs.Runs use an effective batch size of 48, validation-loss checkpoint selection, and frozen backbone parameters.Source: C TRAINING AND ADAPTATION BASELINES · p. 15
  • Segment extraction sets reference width W = max(1, ⌊C/S⌋), with adaptive-horizon minimum segment length depending on W, L_min = 48, and the seasonal period or forecast horizon.Actual segment length also depends on available history and cannot exceed observed length; cross-attention uses width 64 and four heads.Source: C TRAINING AND ADAPTATION BASELINES · p. 15
  • Chronos-2 full fine-tuning, LoRA, LayerNorm-only tuning, BitFit, and linear probing use the official fit pipeline, batch size 256, and 5,000 optimizer steps.Validation is performed every 100 steps, with the best validation checkpoint selected.Source: C TRAINING AND ADAPTATION BASELINES · p. 15, Table 6 · p. 16
  • PaCTS achieves the best in-distribution performance on GIFT-Eval and is the only method that improves both metrics over zero-shot inference on TIME.Source: Table 7 · p. 16

D ADDITIONAL EXPERIMENTAL RESULTS

Additional experiments show PaCTS outperforming adaptation baselines and double-context zero-shot inference across history-length and forecast-horizon groups. Results also extend across two additional backbones, with lower inference costs than roughly doubled-context zero-shot comparisons.

Effects of history length

Effects of Forecast horizon

Results across backbones

Inference efficiency

  • 48.57% fewer GFLOPs on PatchTST-FM r1 at 4,096 steps, and 47.72% and 50.95% fewer on TimesFM-2.5 at 4,096 and 8,192 steps, respectively, versus about doubled-context zero-shot.PaCTS adds less than 0.2% parameters per backbone; across these comparisons, MASE improves, CRPS is competitive, and peak memory and observed latency decrease.Source: D ADDITIONAL EXPERIMENTAL RESULTS · pp. 16–17, Table 13 · p. 18, Table 14 · p. 18
Loading 2609.34786v1…