PaperlayerView exact revision on arXiv

Source-linked AI summary

Compositional Spectral Prompts for LLM-based Online Time Series Forecasting

Seungyoon Choi, Hyunchul Kim, Jae-Gil Lee, Chanyoung Park

arXiv:2609.02093v1cs.LG

TL;DR

OTSF methods based on memory buffers struggle with long-term adaptation and unseen patterns. CoSPOT addresses this by guiding a frozen pre-trained LLM with compositional frequency-domain prompts, and experiments report robust performance across shifting online scenarios.

  • Problem

    Existing memory buffer-based OTSF frameworks struggle with long-term adaptation and fail to generalize to unseen patterns.

  • Method

    CoSPOT keeps a pre-trained LLM frozen and composes spectral basis prompts from frequency-domain decompositions of each input time series.

  • Results

    CoSPOT consistently outperforms state-of-the-art OTSF methods across varied online learning scenarios with extremely few online parameter updates.

  • Takeaways & Limitations

    Compositional spectral prompts provide explicit guidance on underlying time-series patterns while combining LLM adaptability with efficient online updates under continuous distribution shifts.

Abstract

from arXiv · show

To address the sequential and evolving nature of time series, the Online Time Series Forecasting (OTSF) task has been extensively studied in multiple domains. Existing research focuses on adapting to non-stationary environments by employing memory buffer-based retrieval strategies. However, we observe that such frameworks struggle with long-term adaptation and fail to generalize to unseen patterns. To this end, we introduce CoSPOT, an LLM-based online time series forecasting framework that leverages a pre-trained LLM as the backbone online forecaster, motivated by its strong few-shot capabilities. For efficient online adaptation, CoSPOT keeps the LLM frozen and employs compositional spectral prompts grounded in frequency-domain bases to guide the model with the overall distribution of the input, thereby substantially reducing the number of parameters updated during the online phase. Specifically, CoSPOT decomposes time series into frequency bases and composes the corresponding spectral basis prompts according to their amplitudes, allowing unseen patterns to be represented as new combinations of learned basis prompts. Our extensive experiments on real-world datasets demonstrate the superiority and practicality of CoSPOT across challenging online scenarios, including extended online phases and cross-dataset settings with substantial distribution shifts. Our code is available at https://github.com/seungyoon-Choi/CoSPOT.

1 Introduction

Online time series forecasting addresses evolving data that static batch methods struggle to accommodate without impractical retraining. CoSPOT responds with an LLM-based framework using frozen-model adaptation and compositional spectral prompts for long-term and unseen-pattern adaptation.

  • Static batch forecasting methods often fail under evolving patterns, while frequent retraining is labor-intensive and impractical.
  • Existing OTSF methods use memory-buffer retrieval to adapt to non-stationary environments.FSNet uses adapters and associative memory, OneNet dynamically ensembles models, and DSOF uses a dual-stream residual framework.
  • Extended online phases expose capacity limits in memory buffers, causing prior approaches' performance to deteriorate as the online-test proportion increases.
  • CoSPOT integrates a pre-trained LLM and keeps it frozen while using trainable prompts to guide adaptation to evolving patterns.The design leverages the LLM's few-shot transferability while avoiding online updates to its large parameter set.
  • CoSPOT decomposes time series into frequency components and compositionally aggregates learnable spectral basis prompts according to those components.The approach uses frequency-domain representations to capture underlying periodic structures and construct prompts for pattern shifts.
  • Across varied online scenarios, CoSPOT is reported to outperform state-of-the-art OTSF methods with extremely few online parameter updates.

2 Related Works

Related work spans online forecasting methods for adapting to evolving streams, LLM-based forecasting, and prompt-based continual learning. These lines motivate CoSPOT's use of transferable LLM representations and small trainable prompts.

  • Online Time Series Forecasting: Prior OTSF methods include calibration, dual-stream updating, and memory-based adaptation for changing time-series patterns.
  • Online Time Series Forecasting: Existing OTSF studies do not explicitly model input-signal patterns and store pattern information, limiting adaptation to unobserved distributions and extended online phases.
  • Time Series Forecasting with LLMs: Recent LLM forecasting methods transfer pretrained knowledge through fine-tuning, reprogramming, or adaptation of selected LLM components.Examples include LLM4TS, GPT4TS, and TimeLLM.
  • Prompt-based Continual Learning: Rehearsal-free continual learning fine-tunes small learnable prompts for each task while leveraging pretrained models' general representations.

3 Preliminaries

OTSF partitions chronological time-series data into training, validation, and online periods, then forecasts future horizons from look-back windows. Its online phase streams observations sequentially and updates the model in real time.

  • A time series dataset is chronologically partitioned into training, validation, and online timestamps according to predefined ratios.
  • Given a look-back window of length L, the forecasting objective predicts the following H time steps from the preceding observations.
  • OTSF uses static batch training first, followed by sequential online streaming with a moving window of size 1 and real-time model updates.

4 Proposed Method: CoSPOT

CoSPOT combines a frozen pre-trained LLM with aligned time-series and textual inputs, while compositional spectral prompts represent evolving patterns through frequency-domain bases. During online adaptation, only the output projection layer is tuned, while the spectral prompt bank remains fixed.

  • Overall Framework: CoSPOT combines a pre-trained LLM, a time-series backbone, textual recent information, and compositional spectral prompting for online forecasting.The LLM remains frozen and serves as a stable knowledge source while the backbone encodes the input sequence.
  • Robust Adaptation via Compositional Spectral Prompting: Compositional spectral prompting represents input patterns using learnable prompts associated with frequency bases.The input time series is decomposed with the DFT, and basis prompts are composed according to frequency amplitudes.
  • Robust Adaptation via Compositional Spectral Prompting: Low-frequency basis prompts are emphasized because high frequencies capture rapidly oscillating periodicities and are often treated as noise.The method therefore does not learn all frequency bases equally for overall pattern representation.
  • Robust Adaptation via Compositional Spectral Prompting: Newly emerging patterns can be represented by recombining learned spectral basis prompts without additional training.The finite prompt bank is intended to preserve memory efficiency while representing unseen online patterns.
  • Overall Framework: The LLM receives the concatenated spectral prompt, embedded text description, and aligned time-series embedding before producing the forecast through an output projection.Special text tokens delimit the heterogeneous modalities in the concatenated input sequence.
  • Overall Framework: During online adaptation, only the output projection layer is tuned, while the LLM, backbone, align module, and spectral prompt bank remain frozen.This design limits online parameter updates while matching the streaming data distribution.

5 Experiments

The experiments use established forecasting benchmarks and compare CoSPOT with deep learning and time-series foundation-model baselines. The supplied experiment passages describe the benchmark setup and baseline categories but do not report quantitative outcomes.

  • Datasets: The evaluation follows prior studies and uses five widely used time-series forecasting benchmarks from various domains.Each dataset is split into training, validation, and testing sets with a 20%, 5%, and 75% ratio.
  • Baselines: The baseline suite includes static forecasting models, online forecasting methods, LLM-based forecasting models, and time-series foundation models.The supplied passages identify representative models including DLinear, PatchTST, iTransformer, TimeMixer, Chronos-2, and TimesFM.
  • Experimental Procedure: Algorithm 1 describes separate training and online phases for CoSPOT.The supplied pseudocode includes alignment, text-description generation, spectral-prompt composition, projection, and online evaluation steps.
  • Experimental Procedure: During the online phase, the pseudocode freezes all parameters except the output projection layer.Each online instance is aligned, described textually, decomposed by DFT, and converted into a compositional spectral prompt before updating and evaluation.

5.1 Overall Performance

Across eight datasets, CoSPOT consistently outperforms existing OTSF, static, foundation-model, and other LLM-based approaches in settings without information leakage. Its robust performance combines pre-trained LLM adaptability with compositional spectral prompts under continuous distribution shifts.

  • Across eight datasets, CoSPOT outperforms existing OTSF, static, time-series foundation, and other LLM-based models.The reported results average three runs, and the comparison includes settings without information leakage.
  • FSNet and OneNet underperform static models without information leakage, suggesting reliance on leaked training data rather than adaptation to newly arriving patterns.
  • Chronos-2 and TimesFM underperform CoSPOT because they lack an explicit mechanism for adapting to evolving target distributions online.
  • CoSPOT combines pre-trained LLM adaptability with compositional spectral prompts that explicitly guide learning of underlying patterns during continuous distribution shifts.

5.2 Ablation Study

Ablation studies show that both compositional spectral prompts and text descriptions improve CoSPOT's online forecasting performance. Spectral prompts capture overall time-series dynamics and support adaptation to unseen patterns through recombination.

  • Compositional spectral prompts improve CoSPOT by capturing overall time-series patterns through learned spectral basis prompts.
  • Nearly a 50% improvement in MSE results from compositionally recombining learned spectral basis prompts to adapt to unseen patterns.
  • Text descriptions improve forecasting by providing the LLM with recent, dynamically changing pattern information.

5.3 Further Analysis

CoSPOT remains effective under extended online phases and cross-dataset distribution shifts, while updating far fewer parameters than baseline OTSF methods. Its spectral prompting also supports unseen-pattern representation without online prompt-bank retraining, with frequency-domain prompting offering a favorable efficiency–performance trade-off.

  • Robustness to the extension of the online phase.: Under the extended 10%/5%/85% train/validation/test split, prior methods degrade because buffers cannot retain recurring patterns and unseen patterns continue to appear.CoSPOT maintains performance comparable to the standard split by using compositional spectral prompts and recent-pattern text descriptions.
  • Robustness to distribution shifts.: CoSPOT outperforms all baselines across eight cross-dataset ETT scenarios with prediction horizon 1.The scenarios use different datasets for training and online phases, inducing distribution shifts between transformers with different temporal patterns.
  • Online Updating Cost.: CoSPOT updates significantly fewer parameters than baseline OTSF models while adapting to new data without forgetting previously learned knowledge.The parameter comparison uses the ETTh2 dataset with prediction horizon 1.
  • Online Updating Cost.: CoSPOT incurs higher initial training cost but achieves superior online-phase performance with comparable overall cost and less parameter tuning.Runtime analysis reports training duration, online duration, and inference latency, with the online phase treated as the cost-sensitive component.
  • Prompt composition analysis.: Unseen online patterns can be represented by recombining spectral basis prompts learned during training, without additional online learning.The comparison contrasts a frozen training-phase prompt bank with an online-trained optimal bank and a randomly generated bank.
  • Prompting strategy.: Continuously updating prompts yields superior performance, whereas frozen frequency-domain prompts preserve performance under distribution shifts with greater efficiency.The comparison covers prediction lengths 1, 24, and 48 and contrasts frequency- and time-domain prompting strategies.

6 Conclusion

CoSPOT is presented as an LLM-based OTSF framework designed for continuous distribution shifts and extended online scenarios. It combines compositional spectral prompting with text descriptions to support rapid adaptation in data-scarce forecasting settings.

  • CoSPOT captures each input’s overall pattern by recombining learnable spectral basis prompts according to its frequency-domain structure.The resulting distribution-aware prompts guide a pre-trained LLM during online forecasting.
  • CoSPOT uses distribution-aware prompts to guide the pre-trained LLM toward continuously shifting online distributions.This design leverages the model’s transferability without requiring updates to the LLM itself.
  • Text descriptions containing recent pattern information enrich limited online data without requiring additional training.
  • CoSPOT demonstrates robust performance across real-world datasets, distribution shifts, and extended online phases.The paper highlights its applicability to real-world online forecasting scenarios.

GenAI Disclosure

The authors disclose limited LLM use for language revision and minor code refactoring and debugging. They state that the core research contributions and interpretations remain original and were reviewed by the authors.

  • LLMs were used to improve grammar, clarity, stylistic variation, and paper length, and for minor code refactoring and debugging.
  • The authors reviewed and validated all AI-assisted revisions and code modifications.
  • The authors state that the core ideas, methodology, experiments, and interpretations are original contributions.
Loading 2609.02093v1…