PaperlayerView exact revision on arXiv

Source-linked AI summary

GNA: Granular Neighbor Assembly for Retrieval-Augmented Multivariate Time-Series Forecasting

Vincent Uhse

arXiv:2609.36281v1cs.LGstat.ML

TL;DR

Fixed lookbacks have diminishing returns, and whole-window matches may not suit every variate. GNA combines whole-window and per-variate retrieval with learned trust in retrieved futures. It improves two Transformer backbones across most dataset–horizon settings, with limits on non-stationary series at long horizons.

  • Problem

    Fixed-length lookbacks hide earlier history, while whole-window retrieval can mismatch variates whose best past matches differ.

    Source: §1 · pp. 1–2

  • Method

    GNA assembles whole-window and per-variate neighbors, then learns per-step and per-variate trust against persistence alongside the backbone forecast.

    Source: §1 · p. 2

  • Results

    GNA improves GridTST in 43 and iTransformer in 42 of 48 dataset–horizon settings and gives the lowest MSE on 8 of 12 benchmarks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Both granularities are needed, and gains are largest where the lookback says least.

    Source: §1 · p. 2, §9 · p. 9

    Level drift in retrieved futures limits GNA on non-stationary series at long horizons.

    Source: §8 · p. 8

Abstract

from arXiv · show

Deep forecasters predict from a fixed-length lookback window, and lengthening it gives diminishing returns at a growing cost. Retrieval augmentation instead shows the model how similar past situations continued. Retrieving a whole past window gives every variate the continuation of the same past moment. In multivariate series, however, the best past match differs from variate to variate. We present GNA (Granular Neighbor Assembly), a retrieval layer for forecasting backbones that assembles neighbors at two granularities: whole past windows, which keep the variates coherent, and per-variate neighbors, in which each variate takes its future from its own best-matching past. A learned gate decides, per forecast step and variate, how much to trust these futures against a persistence forecast, next to the backbone's own forecast. Candidates come from an embedding trained to predict each window's future, and retrieval is strictly causal: a past window is used only once its future has been observed. With the same lookback for every model and the same retrieval constants for all datasets, GNA improves two Transformer backbones in 85 of 96 dataset-horizon settings, gives the lowest MSE on 8 of 12 standard benchmarks and beats its backbone in every seed on 10 of them. Both granularities are needed, and neighbors of mismatched queries are worse than none. Retrieval helps most where the lookback says least: the gate shifts trust to retrieved futures further ahead. Where it fails, on hourly non-stationary series at long horizons, the loss is consistent with a drifting level of the retrieved futures.

1 INTRODUCTION

Fixed-length lookbacks miss older situations, while whole-window retrieval can mismatch individual variates. GNA combines coherent whole-window and per-variate neighbors, then learns when to trust their futures.

1 INTRODUCTION

  • Fixed-length lookbacks make earlier history invisible, and extending them brings diminishing returns while attention costs grow with window length.For PatchTST, increasing lookback from 336 to 720 steps changes Traffic MSE from 0.367 to 0.365 and slightly worsens ETTh1 at horizon 96.Source: §1 · p. 1
  • Whole-window retrieval gives every variate the same past continuation, whereas per-variate retrieval can better match each series but loses cross-variate coherence.The model must also learn when retrieved futures are useful across forecast steps.Source: §1 · p. 2
  • GNA assembles whole-window and per-variate neighbors, then gates their futures against persistence and the backbone forecast for each step and variate.Its future-predictive embedding uses training-window futures as privileged information, while retrieval remains causal and the bank grows during deployment without retraining.Source: §1 · p. 2
  • GNA improves GridTST in 43 and iTransformer in 42 of 48 dataset–horizon settings, gives the lowest MSE on 8 of 12 benchmarks, and beats its backbone in all three seeds on 10 of 12.The protocol holds lookback fixed, uses causal retrieval, and shares retrieval constants across datasets.Source: §1 · p. 2

2 RELATED WORK

Prior forecasting models condition on lookback windows, while retrieval methods vary in what they retrieve and how they train the retriever. GNA combines retrieval granularities and trains its embedding to predict futures.

Deep forecasters and context window

  • Deep forecasters condition only on lookback windows; GNA adds information from outside that window and uses GridTST and iTransformer as backbones.The related work also covers linear, MLP, convolutional, and state-space alternatives, as well as instance normalization.Source: §2 · p. 3

Retrieval-augmented forecasting

  • Recent retrieval forecasters use raw-history matches, pretrained embeddings, univariate segments, or global-cycle embeddings, while GNA combines whole-window and per-variate retrieval.GNA trains its retrieval embedding to predict futures and keeps the bank causal as it grows at test time.Source: §2 · p. 3

Learning the retriever

  • Unlike TimeRAF, which trains its retriever to rank candidates by forecaster error, GNA trains its embedding directly to predict each window’s future.Because futures are available only during training, this is learning with privileged information.Source: §2 · p. 3

3 METHOD

GNA adds a retrieval layer while leaving the backbone’s lookback input unchanged, using causally eligible past windows to supply candidate futures. It assembles whole-window and per-variate neighbors, then combines their futures with persistence and the backbone forecast through learned trust.

3.1 PROBLEM SETUP AND CAUSAL ELIGIBILITY

  • Eligible bank windows are used only after their futures have been observed and only when their pasts do not overlap the query’s past.The eligibility rule applies to training, validation, and test queries; at test time, the bank grows as new futures are observed without retraining.Source: §3.1 · p. 3

3.2 RETRIEVING CANDIDATES

  • A retrieval embedding is trained to predict each window’s future, then exact cosine search and context signals rank eligible candidates.The context signals include regime-feature similarity, periodic lags, and calendar match; the candidate list is spread across episodes.Source: §3.2 · pp. 3–4

3.3 GRANULAR NEIGHBOR ASSEMBLY

  • Five global slots preserve one shared past situation, while five per-variate slots let each variate select its own best-matching past from a spaced candidate pool.Per-variate slots are composites that may combine variates from different source windows; spacing keeps them from being near-copies of one episode.Source: §3.3 · p. 4, Figure 2 · p. 4

3.4 FORECASTING WITH LEARNED TRUST

  • A learned gate weights persistence and slot futures per forecast step and variate, alongside the backbone forecast and a neighbor-conditioned backbone term.The gate can favor persistence for near steps and retrieved futures further ahead; the neighbor-conditioned term attends each variate token to its slot-future tokens.Source: §3.4 · p. 5, Figure 3 · p. 5

3.5 TRAINING AND COST

  • The forecaster is trained with the embedder and bank fixed, while exact retrieval has quadratic total search time across all queries.On four timed datasets, online CPU retrieval takes 19–25 ms per query, and training is 14–51% slower per epoch than the backbone alone.Source: §3.5 · p. 5

4 EXPERIMENTAL SETUP

The evaluation tests retrieval on twelve standard benchmarks with a shared lookback of 96 and compares against backbones and other forecasters. Results use standardized-data MSE and MAE averaged over benchmark horizons.

Datasets and questions

  • Experiments address whether retrieval improves forecasts under a same-lookback protocol, which components contribute, and when retrieval helps or fails.The evaluation uses twelve standard benchmarks, including PEMS sensor networks and ETT, ECL, Traffic, Weather, and Solar datasets.Source: §4 · pp. 5–6

Baselines and protocol

  • Every model uses lookback L = 96, with backbone-only, retrieval-based, and 14 additional forecasters among the baselines.Retrieval constants are untuned defaults shared across datasets; the reported GridTST baseline is the authors’ run, and tables report seed 2023.Source: §4 · p. 6

Metrics

  • MSE and MAE are reported on standardized data, averaged over four horizons and the full test set.Each run keeps the checkpoint with the lowest validation MSE over 10 epochs.Source: §4 · p. 6, Table 1 · p. 6

5 RESULTS

GNA improves both Transformer backbones across most dataset-horizon settings and often outperforms its backbone and retrieval-based alternatives. Its gains vary by dataset, with specific failures on hourly ETT datasets and weaker results on ETTm2 and Solar.

Benchmark comparisons

  • On PEMS, GNA is 5–49% below periodic-model baselines on every dataset, and on ECL, Traffic, ETTm1, and Weather it is 0.5–4.9% below the best baseline.GNA is better than GTR on 8 of 12 datasets and better than RAFT on 10 of 12.Source: §5 · p. 7

Dataset-specific results

  • GNA is 2.9% and 2.1% above the best on ETTm2 and Solar, respectively, and is worse than its backbone on ETTh1 and ETTh2.The reported seed-mean gains over GridTST outside PEMS and ETTh are 4–21%.Source: §5 · p. 7

Retrieval versus no retrieval

  • 43 of 48 GridTST and 42 of 48 iTransformer dataset–horizon settings improve with retrieval.With GridTST, GNA has the lowest MSE on 8 of 12 datasets; over three seeds, it beats the plain GridTST in every seed on 10 of 12 datasets.Source: §5 · p. 7

6 ABLATIONS AND SENSITIVITY

Ablations show that GNA depends on relevant retrieved neighbors and benefits from combining global and per-variate slots. The learned embedding, context signals, and slot count add smaller, dataset-dependent gains.

6 ABLATIONS AND SENSITIVITY

  • Mismatched neighbors raise MSE by 6–247% (median 21%), more than removing retrieval (4–142%, median 8%) across all five datasets.The model cannot fully ignore irrelevant neighbors.Source: §6 · p. 7
  • Using only global slots raises MSE by 1.9–8.3% (mean 4.8%), while only per-variate slots raises it by 0.3–15.8% (mean 7.0%).The best per-variate slot is 17–45% closer to the true future than the best global slot on all five datasets.Source: §6 · p. 7
  • Raw-window ranking raises MSE by 0.9–7.4% (mean 3.7%); removing context signals raises it on ETTm1, ETTm2, and Weather, but not PEMS.Without context signals, MSE changes are +4.1%, +12.0%, and +4.5% on ETTm1, ETTm2, and Weather, versus +0.4% on PEMS08 and −1.1% on PEMS04.Source: §6 · p. 7
  • At H = 96, halving slots to K = 5 raises MSE by 4.5% on PEMS08 and 2.1% on ETTm1, while doubling to K = 20 lowers it by only 0.9% and 0.6%.The default K = 10 is near saturation; K = 5 retains most of the gain, with PEMS08 71% below the plain backbone.Source: §6 · p. 7

7 WHEN DOES RETRIEVAL HELP?

Retrieval helps most when the lookback provides less useful context, and the gate increasingly trusts retrieved futures at later forecast steps. Gains diminish as lookback grows, while long-horizon hourly non-stationary series expose a level-drift failure.

7 WHEN DOES RETRIEVAL HELP?

  • Persistence weight falls from 0.29 at steps 1–12 to 0.03 at steps 49–96 on PEMS04, and from 0.42 to 0.08 on Solar.Across datasets, horizons, and both backbones, neighbor weight correlates with larger gains over the backbone (Spearman ρ = 0.71 for GridTST and 0.53 for iTransformer).Source: §7 · p. 7, Figure 5 · p. 8
  • Figure 4 compares each ablation’s change in horizon-averaged test MSE against full GNA; positive values mean worse performance without that component.Filled marks exceed twice the full model’s seed standard deviation; hollow marks fall within that noise.Source: Figure 4 · p. 8
  • Retrieval helps most where the lookback says least; on ETTm1, GNA’s error stays near 0.297–0.310 as lookback changes, while gains over GridTST shrink from 33% at L = 48 to 5% worse at L = 336.At L = 96 the gain is 8%, and at L = 192 it is 1%.Source: §7 · pp. 7–8
  • At H = 720 on ETTh1, GNA is 5.5% better over steps 1–96 but 16.5% worse over steps 337–720, consistent with retrieved-future levels drifting from truth.A hindsight oracle using level-free futures reduces error by 26% on ETTh1 and 36% on ETTh2, while raw futures barely help.Source: §7 · p. 8

8 LIMITATIONS AND FUTURE WORK

GNA’s benefits are bounded by the availability and quality of retrieved futures, and by the evaluation settings used. The authors identify memory use and level drift as practical concerns for future work.

8 LIMITATIONS AND FUTURE WORK

  • GNA requires a history of windows with observed futures, so it cannot help at a cold start; drifting retrieved-future levels limit it on non-stationary series at long horizons.The gate also cannot fully ignore irrelevant neighbors, and retrieval does not improve ETTm1 with L = 336.Source: §8 · p. 8
  • The assembled-slot training cache grows as O(NKD(L + H)), reaching 36 GB on ECL and 106 GB on PEMS07, although the retrieval index is small.Source: §8 · pp. 8–9
  • The main table and ablations use one seed, results mostly fix L = 96, and shared retrieval settings are not tuned per dataset.The study reports three seeds in Appendix E and includes one separate lookback study on ETTm1.Source: §8 · p. 9

9 CONCLUSION

GNA combines whole-window and per-variate neighbors with learned, step- and variate-specific trust, improving two Transformer backbones under a fixed-lookback, causal retrieval protocol. The paper also reports its AI-use practices, public-data ethics assessment, and reproducibility details.

9 CONCLUSION

  • Under a fixed lookback, causal retrieval, and shared retrieval settings, GNA improves two Transformer backbones in most settings across twelve benchmarks and has the lowest MSE on eight.Both granularities are needed, and gains are largest where the lookback says least and on sensor networks.Source: §9 · p. 9

AI USE STATEMENT

  • The authors used an LLM-based coding assistant for implementation, experiments, interpretation, methodology design, figure creation, literature analysis, drafting, and proofreading, and report checking the assisted work.They attribute the central idea and final design to the authors.Source: AI USE STATEMENT · p. 9

ETHICS STATEMENT

  • The paper uses public benchmark datasets with no personal data and reports no specific ethical risks beyond forecasting in general.Source: ETHICS STATEMENT · p. 9

REPRODUCIBILITY STATEMENT

  • The eligibility rule was audited for every retrieval bank; results use fixed seeds, with three seeds reported in Appendix E, and code is promised upon acceptance.Source: REPRODUCIBILITY STATEMENT · p. 9

A METHOD DETAILS

GNA’s retrieval pipeline trains a future-aware embedding, assembles and gates retrieved slots, and evaluates the resulting forecasters under shared settings. The supplied passages describe the implementation, datasets, and results tables, but do not report their comparative values.

A METHOD DETAILS

  • A future-prediction teacher trains the embedding, while a smaller Transformer student is distilled from it for retrieval using only past windows at test time.The student distillation objective combines embedding MSE and cosine distance; the teacher’s first layer alone has 512 LD weights.Source: A METHOD DETAILS · p. 13, Figure 6 · p. 14

B IMPLEMENTATION DETAILS AND HYPERPARAMETERS

  • Shared settings cover the embedder, retrieval, fusion, and assembly, while per-dataset settings are listed separately; the authors report no systematic hyperparameter search.With iTransformer, GNA uses a smaller backbone configuration and fewer total parameters than the plain backbone on every dataset and horizon.Source: B IMPLEMENTATION DETAILS AND… · p. 14, Table 3 · p. 15, Table 4 · p. 15

C DATASETS

  • Datasets use standard chronological splits, and each variate is standardized using the training split’s mean and standard deviation before errors are computed.The dataset table summarizes split sizes.Source: C DATASETS · p. 16, Table 7 · p. 16

D FULL RESULTS

  • The full-results tables compare retrieval and no-retrieval models at lookback 96 and report MSE and MAE, but the supplied passages do not state the outcomes.Per-horizon and broader-baseline comparisons are also identified, without their values.Source: Table 8 · p. 18, Table 9 · p. 19, Table 10 · p. 20

E SEED ROBUSTNESS

Across three seeds, GNA with GridTST is more stable than the plain backbone and beats it in every seed on 10 of 12 datasets. It loses in every seed on ETTh1 and ETTh2.

E SEED ROBUSTNESS

  • 10 of 12 datasets show GNA with GridTST beating the plain backbone in every seed; it loses in every seed on ETTh1 and ETTh2.The seed comparison averages each seed over four horizons.Source: E SEED ROBUSTNESS · p. 21, Table 11 · p. 21
  • 0.3% median seed-to-seed MSE variation for GNA compares with 4% for plain GridTST; the respective maxima are 0.8% and 20%.On PEMS, the seed-2023 plain-backbone run in the main table was its best of three, making that comparison conservative.Source: E SEED ROBUSTNESS · p. 21
  • All but four reported ablation effects exceed twice the full model’s seed standard deviation; the exceptions are Weather without global slots, PEMS08 without rank fusion, and ETTm1 and Weather without future-aware embedding.The four effects within that noise level are marked as hollow markers in Fig. 4.Source: E SEED ROBUSTNESS · p. 21

F ABLATION DETAILS AND QUALITATIVE EXAMPLE

The ablations assess GNA components and retrieved-future quality, while further analyses report computational costs, compare retrieval methods, and diagnose oracle headroom and long-horizon failures.

F ABLATION DETAILS AND QUALITATIVE EXAMPLE

  • The ablation protocol changes one component at a time against the full GridTST-based GNA control; a mismatched-neighbor arm swaps in another query’s neighbor set.Banks were audited for causality, and Table 13 is identified as reporting ablation numbers, which are not included in the supplied passages.Source: F ABLATION DETAILS AND QUALITATIVE… · p. 21, Table 13 · p. 22
  • 17–45% closer on average, the best per-variate slot’s future is to the truth than the best global slot’s, and it is closer on every dataset and horizon.The best per-variate slot improves by 12–48% across dataset-horizon settings; averaging per-variate futures is not always better, with PEMS08 at +11%.Source: F ABLATION DETAILS AND QUALITATIVE… · p. 22, Table 12 · p. 22

G COMPUTATIONAL COST

  • 19–25 ms per query, retrieval takes on the CPU, with exact search identified as the dominant latency stage.Approximate search is noted as a way to make search sublinear in bank size; the implementation also caches assembled slots, a cache 14–90 times larger than the index.Source: G COMPUTATIONAL COST · p. 23, Table 15 · p. 23, Table 16 · p. 24

H COMPARISON WITH RETRIEVAL-BASED FORECASTING METHODS

  • GNA alone among the compared methods combines future-predictive retrieval embeddings with whole-window and per-variate neighbors in one multivariate model.The comparison also states that GNA’s bank grows causally during testing, unlike the fixed or training-data banks described for the other methods.Source: H COMPARISON WITH RETRIEVAL-BASED… · p. 24, Table 17 · p. 24

I ORACLE-CEILING DETAILS

  • At H = 720, ETTh1 is 5.5% better over steps 1–96 but 16.5% worse over steps 337–720; the long-horizon loss is consistent with retrieved futures drifting in level.The same eligible windows with hindsight-selected level-free futures would reduce error by 26% on ETTh1 and 36% on ETTh2, but a fixed shift hurts ECL and Traffic; oracle numbers are optimistic because selection uses the true future.Source: I ORACLE-CEILING DETAILS · p. 25, Table 18 · p. 25, Figure 14 · p. 27
Loading 2609.36281v1…