PaperlayerView exact revision on arXiv

Source-linked AI summary

Model-agnostic Retrieval-Augmented Extended Forecasting for time series

Juan Pablo Villa Serna, Rohan Asthana, Vasileios Belagiannis

arXiv:2608.14054v1cs.LG

TL;DR

When historical context is limited, zero-shot forecasting can struggle and fine-tuning is computationally costly. RAEF addresses this with model-agnostic retrieval and temporal aggregation, achieving 11–16% MASE improvements over baseline foundation models and matching or exceeding fine-tuning on 4/6 datasets.

  • Problem

    Limited historical context challenges zero-shot forecasting, while fine-tuning for domain adaptation incurs substantial computational costs.

  • Method

    RAEF retrieves samples directly in input-space and concatenates them with the input before pretrained foundation-model forecasting.

  • Results

    11–16% MASE improvements over baseline foundation models were achieved, with performance comparable to or exceeding fine-tuning on 4/6 datasets.

  • Takeaways & Limitations

    RAEF provides a practical alternative for time-series domain adaptation in resource-constrained, data-scarce settings.

  • Takeaways & Limitations

    The authors identify information loss from RAF’s averaging and inference overhead from embedding-based retrieval as limitations of RAF.

Abstract

from arXiv · show

Time series forecasting with pretrained foundation models has demonstrated strong zero-shot capabilities. However, achieving optimal performance on time series with short or negligible historical data in domain-specific applications typically requires adaptation via either fine-tuning or RAG. While fine-tuning is effective, it incurs substantial computational costs. This work explores RAG within univariate time series (Retrieval Augmented Generation) as a more efficient alternative, in particular RAF (Retrieval Augmented Forecasting), and introduces RAEF (Retrieval-Augmented Extended Forecasting), a model-agnostic method built upon RAF. RAEF incorporates key refinements to the retrieval and aggregation mechanisms: (1) direct retrieval in input-space rather than embedding-space, reducing inference overhead, and (2) concatenation-based aggregation that preserves temporal structure instead of averaging. Empirical evaluation across multiple benchmark datasets demonstrates that RAEF outperforms RAF in both accuracy and inference overhead. Furthermore, comprehensive comparisons with zero-shot and fine-tuned foundation models show that RAEF achieves competitive or superior performance to fine-tuning while avoiding its computational burden, establishing it as a practical and scalable approach for domain adaptation in time series forecasting.

1. INTRODUCTION

RAEF addresses the difficulty of adapting time-series foundation models when historical context is limited, offering a model-agnostic alternative to computationally costly fine-tuning. It refines RAF through direct input-space retrieval and concatenation-based aggregation to reduce retrieval time while preserving temporal structure.

  • Motivation: Limited historical context can make zero-shot forecasting insufficient, motivating domain adaptation through fine-tuning or retrieval-augmented methods.Fine-tuning updates model parameters but incurs computational costs proportional to model size and dataset scale.
  • Motivation: RAG augments inputs with relevant examples from a prebuilt domain-specific database, enabling extended context and domain-relevant temporal dependencies without retraining.Time-series RAG remains less explored, and several existing methods require custom architectures or domain-specific training.
  • Contribution: RAEF is a model-agnostic forecasting approach that addresses RAF’s limitations through direct input-signal retrieval and concatenation of retrieved samples.These refinements target retrieval efficiency and preservation of temporal structure.
  • Contribution: Retrieval time is reduced by a factor of four across context lengths of 32, 64, and 128 when similarity is computed in input-space rather than embedding-space.The input-space modification reduces retrieval overhead during inference.

2. METHOD

RAEF augments a univariate context with domain-specific training samples before forecasting with a pretrained foundation model. It improves RAF by retrieving directly in input space and aggregating retrieved samples through relevance-aware concatenation that preserves information while controlling sequence length.

  • Method overview: RAEF constructs a vector database from training contexts and futures, retrieves domain-specific samples, and feeds the augmented input to a pretrained foundation model.The context x has length C, the future y has length F, and training samples have length C+F.
  • Input-space retrieval: RAEF omits RAF’s embedding step and performs retrieval directly in input space, reducing inference overhead while preserving retrieval quality.Euclidean distance is directly defined for continuous numerical time-series vectors, avoiding potentially unnecessary embedding layers.
  • Relevance-aware aggregation: RAEF aggregates retrieved samples instead of averaging them, preserving complete retrieved information but increasing the pretrained model’s computational burden on longer sequences.The method introduces a score threshold to balance information retention against computational efficiency.
  • Relevance-aware aggregation: Candidates above the score threshold are concatenated, while lower-relevance residuals are averaged, retaining relevant patterns without discarding all retrieved information.The candidate sequence enriches the input with the most similar samples, whereas residual averaging limits sequence expansion.
  • Implementation: Retrieved samples are concatenated in ascending score order, and if either candidate or residual sets are empty, the augmented input uses only the available set.This preserves similarity-based relevance while handling degenerate retrieval partitions.
  • Alignment removal: RAEF removes RAF’s alignment step so retrieved samples act as past periodic representations, forming a longer semi-periodic sequence for capturing extended temporal patterns.The method assumes high-similarity retrieved samples can naturally extend the input without explicit discontinuity correction.

3. EXPERIMENTS

The experiments evaluate whether RAEF improves forecasting accuracy over baseline foundation models and RAF, surpasses fine-tuning without parameter updates, and benefits from input-space retrieval and structure-preserving aggregation.

  • 3. EXPERIMENTS: The experiments test RAEF against baseline foundation models and RAF for forecasting accuracy, while also assessing performance relative to fine-tuning without parameter updates.They additionally examine the contributions of input-space retrieval and structure-preserving aggregation.

3.1. Experimental Setup

The experiments evaluate RAEF across six diverse time-series benchmarks using MoiraiMoE as the primary backbone, with additional Chronos evaluations for model-agnostic applicability. Performance is measured with MASE at a fixed horizon and multiple context lengths against zero-shot, fine-tuned, and RAF baselines.

  • Datasets and models: Experiments cover six benchmark datasets spanning ET, Power, Traffic, FredMd, ElectricityUCI, and Huawei Cloud, using an 0.8 split and instance normalization.MoiraiMoE is the primary backbone, with additional evaluation on Chronos-T5 and Chronos-Bolt.
  • Evaluation protocol: MASE evaluates forecasting accuracy, with horizon F = 16 and context lengths C ∈{32, 64, 128}.The horizon matches MoiraiMoE’s maximum non-autoregressive length, while vector databases contain 10,000 training samples built with HNSW in Chroma.
  • Baselines and training: Comparisons include Base, Fine-Tuning, RAF, and RAEF, with fine-tuning using specified optimization settings, validation-based early stopping, and up to 500 iterations.RAF uses embedding-space retrieval with alignment and averaging, whereas RAEF is the proposed method.

3.2. Results

RAEF consistently improves forecasting across foundation models and context lengths, often matching or exceeding fine-tuning without parameter updates. Its strongest gains occur with short contexts, while input-space retrieval and concatenation-based aggregation improve efficiency and preserve temporal patterns.

  • Overall results: RAEF improves over the MoiraiMoE baseline by 15.80%, 11.78%, and 11.26% at contexts 32, 64, and 128, respectively.It outperforms fine-tuning in most cases without parameter updates.
  • Overall results: RAEF improves Chronos-T5 by 13.88%, 8.21%, and 13.24% across context lengths, while Chronos-Bolt gains 3.50% at context 128.These results support RAEF’s model-agnostic applicability, including on Huawei Cloud for Chronos-T5.
  • Context-length effects: Average improvement decreases as context length grows, from 15.80% at C = 32 to 11.26% at C = 128, 6% at C = 256, and −0.07% at C = 512.Short contexts benefit most because retrieved samples contribute patterns missing from limited historical information.
  • Comparison with fine-tuning: RAEF matches or exceeds fine-tuning on 4 out of 6 datasets without parameter updates, while fine-tuning’s advantages on Power and Traffic are marginal at 0-1%.This combination of competitive accuracy and zero training overhead supports rapid domain adaptation.
  • Mechanisms: Concatenation-based aggregation preserves individual temporal sequences, enabling selective attention to retrieved patterns instead of diluting them through averaging as in RAF.The threshold-based candidate-residual separation further enhances the retrieval process.
  • Efficiency: Input-space retrieval reduces overhead from 8ms to 2ms per query for C = 32, 64, and 128, with no retrieval-time variation across context lengths.This reduces inference cost while retaining direct access to retrieved input patterns.

3.3. Ablation Studies

The ablation studies show that RAEF’s threshold-based candidate–residual separation and aggregator both improve forecasting accuracy. Performance peaks at normalized distance dt = 1.0, corresponding to one standard deviation from the query sample.

  • 3.3. Ablation Studies: At context 32, RAEF without the aggregator improves over standard RAF, 10.14% vs 7.31%, but remains below full RAEF.The comparison uses MASE improvement (%).
  • 3.3. Ablation Studies: Threshold-based candidate–residual separation adds an average 5.66% improvement by prioritizing high-similarity samples while retaining lower-similarity information through averaging.This mechanism avoids discarding lower-similarity retrieved samples entirely.
  • 3.3. Ablation Studies: Performance peaks at normalized distance dt = 1.0, corresponding to one standard deviation from the query sample.Samples within one standard deviation are described as sharing strong distributional similarity.
  • 3.3. Ablation Studies: Beyond dt = 1.0, retrieved samples no longer provide meaningful individual information and can be safely averaged.The threshold is interpreted using statistical intuition about distributional similarity.
  • 3.3. Ablation Studies: Table 3 shows that the aggregator contributes to RAEF’s improvement in accuracy.The table compares RAEF with and without its aggregator module.

4. CONCLUSIONS

RAEF is a model-agnostic retrieval-augmented framework that improves MASE by 11–16% over baseline foundation models without parameter updates. Across six datasets and three foundation models, it matches or exceeds fine-tuning on four datasets, offering a practical alternative for resource-constrained domain adaptation.

  • RAEF achieves 11–16% MASE improvements over baseline foundation models without parameter updates.The framework is model-agnostic and retrieval-augmented.
  • Across six datasets and three foundation models, RAEF matches or exceeds fine-tuning on 4/6 datasets.These results establish RAEF as a practical alternative for domain adaptation in resource-constrained environments.
Loading 2608.14054v1…