PaperlayerView exact revision on arXiv

Source-linked AI summary

GRAM: Correcting Frozen Time-Series Foundation Models via Graph-Retrieved Amplitude Memory

Xiaoyun Yu, Xiangfei Qiu, Yonggui Huang, Shixiang Tang, Nanqing Dong, Wanli Ouyang, Geguang Pu, Honggang Qi, Jilin Hu, Xi Chen

arXiv:2610.04827v1cs.LG

TL;DR

Existing retrieval correction can reuse futures that include both the frozen model's prediction and non-transferable fluctuation, while recurring model bias offers a more direct correction signal. GRAM uses amplitude-scaled error prototypes and a prototype graph to generate corrections, and improves forecasting across datasets and foundation models.

  • Problem

    Retrieving similar windows' ground-truth futures can transfer the TSFM's already-produced prediction and sample-specific fluctuation, while scale differences and noise make recurring bias difficult to identify.

    Source: §1 · p. 2

  • Method

    GRAM stores amplitude-scaled TSFM errors in retrievable prototypes, models prototype relations, and uses retrieved subgraphs to generate corrections for the frozen forecast.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Across seven datasets and four TSFMs, GRAM improves both MSE and MAE in all 56 dataset–TSFM–metric cells.

    Source: §1 · p. 3, §4.1 · p. 8

  • Takeaways & Limitations

    GRAM can correct forecasts while the TSFM and retrieval encoder remain frozen, a setting relevant when target data are limited or model updates are costly.

    Source: §1 · pp. 1–2

    Evaluation covers seven long-standing benchmarks, four models, four standard horizons, and deterministic point forecasts; probabilistic heads, other model families, and higher-frequency or geographically distributed series remain future work.

    Source: §B.6 · p. 20

Abstract

from arXiv · show

Time-series foundation models (TSFMs) enable zero-shot forecasting through large-scale cross-domain pretraining, while retrieval augmentation further improves their performance by leveraging historical information. However, existing methods typically correct TSFM forecasts using the ground-truth futures of similar historical windows, which contain both predictive components already captured by the foundation model and sample-specific random fluctuation that is difficult to transfer. In contrast, recurring systematic model bias within prediction errors more directly characterizes the failure modes of a frozen TSFM and therefore provides more valuable correction signals. Effectively exploiting such model bias, however, poses two challenges: prediction errors at different numerical levels are difficult to compare due to scale differences, and the recurring bias must be extracted from prediction errors contaminated by random fluctuation. To address these challenges, we propose GRAM, a general retrieval-augmented framework for frozen TSFMs. GRAM first introduces an Amplitude Memory Module (AMM) that scales prediction errors by amplitude and aggregates them into retrievable prototypes. It then employs a Prototype Graph Module (PGM) to model relations among prototypes to aggregate consistent bias information while suppressing random fluctuation. During online forecasting, GRAM retrieves and expands prototypes relevant to the current query and generates per-horizon corrections to refine the original TSFM forecast. Experiments across multiple datasets and foundation models demonstrate consistent forecasting improvements.

1 INTRODUCTION

TSFMs enable zero-shot forecasting but can retain systematic target-domain errors. GRAM builds a memory of recurring amplitude-scaled bias, retrieves related prototypes, and adds per-horizon corrections while keeping the TSFM and retrieval encoder frozen.

1 INTRODUCTION

  • Recent retrieval methods use futures after similar historical windows (Yang et al. (2024); Zhou et al. (2026)), which include both the TSFM's prediction and non-transferable sample-specific variation.The model prediction is already available for the current query, while random fluctuation can interfere with correction.Source: §1 · pp. 1–2
  • For non-stationary series, similar historical patterns can lead to futures that differ in level, amplitude, or trend, motivating retrieval of prediction residuals instead.A single residual still contains random fluctuation, so evidence from multiple related windows must be aggregated to recover recurring bias.Source: §1 · p. 2
  • Residuals require scale-invariant comparison across local numerical levels, and relationships among windows must be modeled because bias can recur across different patterns.Individual residuals also contain random fluctuation that can obscure recurring bias.Source: §1 · p. 2
  • GRAM's AMM scales errors by amplitude and aggregates windows into prototypes, while its PGM relates prototypes; online retrieval expands relevant prototypes into per-horizon corrections.The corrections are added to the frozen TSFM forecast, and the TSFM and retrieval encoder remain frozen throughout.Source: §1 · pp. 2–3
  • GRAM achieves the best forecasting performance across seven public benchmark datasets and four time-series foundation models.The introduction reports the cross-dataset and cross-model result without specifying metric values.Source: §1 · p. 3

2 RELATED WORK

The related work contrasts cross-domain pretrained TSFMs with dataset-trained forecasters and describes retrieval augmentation's use of historical futures. GRAM instead retrieves systematic scaling errors and models relations among bias prototypes.

2.1 TIME-SERIES FORECASTING WITH FOUNDATION MODELS

  • TSFMs pretrained on cross-domain corpora forecast zero-shot on unseen domains, unlike dataset-trained deep models; parameter-efficient adaptation still requires target-specific optimization.GRAM instead corrects systematic scaling errors through an external memory while keeping the TSFM frozen.Source: §2.1 · p. 3

2.2 RETRIEVAL-AUGMENTED FORECASTING

  • Retrieval-augmented forecasters use historical segments or futures as auxiliary evidence, but prior designs generally treat retrieved windows independently and use similarity as the main relevance signal.Examples include TimeRAG, TimeRAF, TS-RAG, and related retrieve-then-fuse designs.Source: §2.2 · p. 3
  • GRAM retrieves TSFM scaling errors and processes prototypes linked by bias and shape relations with a trainable relational message-passing head.The head's output is gated by the evidence behind each retrieved prototype.Source: §2.2 · p. 3

3 METHOD

GRAM addresses zero-shot forecasting with a frozen TSFM and retrieval encoder. Rather than reusing retrieved futures, it stores recurring TSFM error amplitudes in prototypes and reads retrieved subgraphs into corrections to the forecast.

3 METHOD

  • The method predicts the next H steps from a recent context window without target-series labels, using a frozen TSFM and a frozen retrieval encoder.The encoder maps each window to a unit-norm embedding.Source: §3 · p. 3
  • GRAM changes retrieval from historical ground-truth futures to prototypes storing the amplitude of recurring systematic TSFM error.This avoids reusing forecast content already covered by the query's TSFM prediction and sample-specific fluctuation.Source: §3 · p. 3
  • A trainable read-out converts the retrieved prototype subgraph into an error component added to the frozen forecast.The retrieval pipeline itself remains in place while the retrieval target changes to model bias.Source: §3 · p. 3

C. Prototype Graph Module

GRAM builds an offline memory of scale-normalized TSFM errors and prototype relations, then retrieves relevant prototypes online to generate corrections added to the frozen forecast. Graph fusion is the only trainable component.

3.1 OVERVIEW

  • GRAM’s offline memory and online retrieval feed graph fusion, the only trainable component, which outputs an error term added to the frozen forecast.Setting the error term to zero recovers the frozen TSFM exactly.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 OFFLINE MEMORY CONSTRUCTION

  • Offline memory construction runs once per dataset, TSFM, and horizon on the memory-construction segment; online forecasting serves one query without target labels.The offline phase does not see a test label.Source: §3.2 · p. 4, §3.3 · p. 5

3.2.1 PATTERN CLUSTERING

  • Pattern clustering assigns each window embedding to its closest prototype centroid and records each cluster’s member count as support.The centroid is the node identity and retrieval key.Source: §3.2.1 · p. 5

3.2.2 AMPLITUDE MEMORY

  • Amplitude memory aggregates member-window excursion ratios into prototype amplitude and spread vectors, representing recurring mis-scaling and sample-specific variation.The weighted median down-weights windows with near-zero excursions; the amplitude scaling error is αc(h) − 1.Source: §3.2.2 · p. 5

3.2.3 PROTOTYPE GRAPH

  • The prototype graph links nodes by bias similarity between scaling-error curves and shape similarity between centroids, with a support floor for bias edges.Shape neighbors can supply read-out evidence when low-support prototypes have amplitudes near one.Source: §3.2.3 · p. 5

3.3.1 ONLINE RETRIEVAL

  • Online retrieval selects the K nearest prototypes, expands each seed one hop along both relations, and attaches an aggregated answer as an anchor node.The anchor carries the retrieval answer in the same five-tuple format as the prototypes.Source: §3.3.1 · p. 6

3.3.2 GRAPH FUSION

  • Graph fusion uses relational message passing, query-conditioned attention, and a gated per-lead gain to map the retrieved subgraph to an error component.The gate shrinks the trained correction according to evidence and cannot change its sign.Source: §3.3.2 · p. 6

3.3.3 FINAL FORECAST

  • The correction combines fixed retrieval and corpus-prior terms with gated, learned voted scaling, estimating recurring bias in the query’s excursion units.The final forecast adds this error component to the frozen prediction.Source: §3.3.3 · p. 7

4 EXPERIMENTS

GRAM is evaluated as a correction to frozen time-series foundation models across seven benchmarks, four models, and multiple horizons. It improves forecasting broadly, remains useful after LoRA adaptation, and its full configuration outperforms ablations and a less reliable pooled-retrieval alternative.

4 EXPERIMENTS

  • Evaluation uses canonical chronological splits across seven benchmarks and four frozen TSFMs, comparing like-for-like Ori and +GRAM MSE and MAE.Only the Graph Fusion Module is trained, on LOTSA; the TSFMs and retrieval encoder stay frozen, and no evaluation-domain labels are used.Source: §4 · p. 7

4.1 MAIN RESULTS

  • GRAM improves MSE and MAE in all 56 dataset–TSFM–metric cells across four frozen TSFMs and seven datasets.Table 1 averages results over four horizons.Source: Table 1 · p. 8, §4.1 · p. 8
  • After LoRA adaptation, correction improves both metrics on all seven Moirai datasets and six datasets each for Chronos and Sundial; TimesFM changes by less than 0.7%.For fine-tuned Moirai on ETTh1 at H = 720, MSE falls from 1.077 to 0.753, a 30.1% reduction.Source: Table 2 · p. 8, §4.1 · p. 8

4.2 ANALYSIS

  • At H = 96, the complete method is best across all 14 dataset–metric columns, while every tested variant worsens average MSE and MAE.The variants remove a module or replace a stored quantity or the recent level.Source: Table 3 · p. 9, §4.2 · p. 9
  • The deployed gate and fixed weights improve both metrics in 107 of 112 settings, versus 74 for pooled retrieval alone, which can lose up to 8.4% of MSE.The deployed configuration trades peak gain for broader coverage.Source: §4.2 · p. 9

5 CONCLUSION

GRAM retrieves amplitude-scaled errors on recurring patterns to correct frozen TSFM forecasts, making evidence comparable across windows. Across seven benchmarks, four horizons, and four models, it improves both metrics on every dataset under every TSFM, with 7.7–12.2 microseconds added per window.

5 CONCLUSION

  • GRAM retrieves the amplitude of errors on recurring patterns, keeping evidence dimensionless and comparable across windows; it improves both metrics on every dataset under all four TSFMs.The largest reductions are 34.5% in MSE and 16.7% in MAE on the weakest model at the longest horizon.Source: §5 · p. 10
  • The added inference cost is 7.7–12.2 microseconds per window; the paper identifies online memory updates and multivariate bias retrieval as future directions.The reported benchmarks use canonical chronological splits, with memory and calibration portions cut from training and separated by forecast-horizon gaps.Source: §5 · p. 10, §A.1 · p. 15

A.1 DATASETS AND EVALUATION PROTOCOL

  • Each dataset, horizon, and TSFM has a separate memory, while the retrieval encoder and graph fusion module are shared; each channel is forecast independently.The published test portion is not touched before evaluation.Source: §A.1 · p. 15

A.2 METRICS

The appendix specifies the metrics, memory and read-out settings, retrieval and fusion procedure, comparison protocols, diagnostic sampling, and compute costs. The read-out is trained once on an unrelated corpus and frozen for evaluation.

A.2 METRICS

  • MSE and MAE are computed per channel against un-normalised ground truth, then averaged over channels; dataset-level single values average the four horizons.Lower values are better for both metrics.Source: §A.2 · p. 15

A.3 MEMORY CONSTRUCTION AND READ-OUT SETTINGS

  • Offline memory construction is a single pass without optimisation; its principal settings include 128 prototypes, a 24-observation recent level, and a support floor of 8 for bias edges.Online retrieval seeds a subgraph with the three closest prototypes; the read-out uses width 96, two message-passing layers, four attention heads, and dropout 0.2.Source: §A.3 · p. 15

A.4 RETRIEVAL AND THE GRAPH FUSION MODULE

  • The graph fusion module processes a retrieved subgraph through GNN, MHA, and Gate blocks to produce per-horizon error corrections; its head is fitted once on LOTSA and then frozen.The frozen TSFM and retrieval encoder are never updated, and no evaluation-domain labels are used.Source: §A.4 · pp. 15–16, §A.3 · p. 15

A.5 BASELINES AND FINE-TUNING PROTOCOL

  • The baseline compares each frozen TSFM forecast with its GRAM-corrected counterpart on the same dataset, horizon, and metric; LoRA-adapted models are also evaluated.For the fine-tuned setting, the selected adapter and checkpoint are frozen before memory construction.Source: §A.5 · p. 16

A.6 PROTOCOL FOR THE DIAGNOSTICS

  • Diagnostics sample at most 2,000 test windows per setting and thin forecasts to 96 uniformly spaced leads; retrieval alignment is the cosine between implied correction and realised error.The random-pairing baseline permutes retrieved evidence across windows and averages five permutations.Source: §A.6 · p. 16

A.7 COMPUTE

  • A query compares its embedding with the centroids, assembles an average 21-node subgraph, and runs the graph fusion module; the memory stores 128 records per dataset, horizon, and TSFM.The memory contains 128 × H floating-point amplitude values, plus five scalars per record, and is described as negligible next to the frozen model.Source: §A.7 · p. 16

B.1 COMPLETE RESULTS OVER THE 112 SETTINGS

Across the zero-shot settings, GRAM improves most model-dataset-horizon comparisons, with larger gains at longer horizons and for weaker frozen models. It also continues to help after LoRA adaptation on most tested models and datasets.

B.1 COMPLETE RESULTS OVER THE 112 SETTINGS

  • Both metrics improve in 107 of 112 zero-shot settings; the largest reductions are 34.5% of MSE and 16.7% of MAE.The largest MSE reduction is for Moirai on Electricity at H = 720, from 1.476 to 0.966.Source: §B.1 · p. 16, Table 5 · p. 17
  • At H = 96, every illustrated panel removes 25% to 63% of squared error, and the corrected forecast is closer at most forecast steps.The figure shows four frozen models across three datasets, with one test window per dataset.Source: §B.1 · pp. 16–17

B.2 PARAMETER-EFFICIENT ADAPTATION

  • After LoRA adaptation, both metrics improve on all seven datasets under Moirai and six datasets under Chronos and Sundial, averaged across four horizons.The largest reported gain is a 30.1% MSE reduction for fine-tuned Moirai on ETTh1 at H = 720, from 1.077 to 0.753.Source: §B.2 · p. 17

B.3 COMPONENT ABLATIONS

The component ablations show that the complete GRAM method outperforms its tested variants, while the figure illustrates correction behavior on selected test windows.

B.3 COMPONENT ABLATIONS

  • The complete method is best in all 14 dataset–metric columns, and every tested variant is worse on the average of both metrics.Replacing amplitude curves with mean residual curves leaves 0.29% MSE and −0.91% MAE improvement; removing either graph relation lowers both averages.Source: §B.3 · p. 17
  • In the selected H = 96 test windows, correction removes 25% to 63% of squared error across four frozen models and three datasets.The windows are selected to show forecasts that are visibly off before correction and close to the label afterward.Source: Figure 3 · p. 18
  • The two structural variants are close on average, while the shape relation carries a larger share on ETTh2 and ETTm2.The passage relates this to one remaining relation supplying most of the neighborhood when the other is removed.Source: §B.3 · p. 18

B.4 COMPARISON WITH THE STATE-OF-THE-ART RETRIEVAL-AUGMENTED METHOD

GRAM is compared with a retrieval method that reuses similar windows’ futures, then its efficiency and evaluation scope are described. The reported comparisons favor GRAM in most settings, though TS-RAG is ahead on Exchange.

B.4 COMPARISON WITH THE STATE-OF-THE-ART RETRIEVAL-AUGMENTED METHOD

  • GRAM is better than TS-RAG in 24 of 28 settings on MSE and 15 on MAE, despite close dataset-averaged values.The averages are 0.3115 versus 0.3151 MSE and 0.3463 versus 0.3409 MAE for GRAM and TS-RAG, respectively.Source: §B.4 · p. 18, Table 7 · p. 19
  • TS-RAG is ahead on Exchange, the series described as closest to a random walk, where reusing futures is the stronger choice.GRAM transfers a dimensionless ratio, whereas reused futures also carry sample-specific components whose value depends on how much is shared with the query.Source: §B.4 · pp. 18–19

B.5 EFFICIENCY AND MEMORY BUDGET

  • The correction takes 7.7 to 12.2 microseconds per window on an H200, with a read-out below one million parameters.The reported memory contains 128 prototype records per dataset, TSFM, and horizon.Source: §B.5 · p. 19, Table 8 · p. 20

B.6 LIMITATIONS AND FUTURE WORK

  • Evaluation covers seven benchmarks, four foundation models, and four standard horizons, all using deterministic point forecasts.Probabilistic heads, other model families, and higher-frequency or geographically distributed series are left for future work.Source: §B.6 · p. 20

Related papers

Loading 2610.04827v1…