Source-linked AI summary
TERN: A Delta-rule Memory with a Seasonal Reference and Online Adaptation for Epidemic Forecasting
Shunya Nagashima, Yuta Funayama
TL;DR
Short regional influenza histories and shifting epidemic waves make it difficult to retain useful seasonal patterns while discarding stale phase-specific information. TERN combines phase-conditioned delta-rule memory with a seasonal reference and online adaptation. Across three Cola-GNN benchmarks, it is the best uncontaminated model, and its memory improves RMSE over softmax attention in controlled comparisons.
Problem
Influenza surveillance series offer only a few seasons per region, waves shift in timing and height, and useful information changes across epidemic phases.
Source: §1 · p. 1Method
Results
TERN is the best uncontaminated model on three Cola-GNN benchmarks, and its memory contributes 3–22% RMSE over softmax attention.
Source: §5 · p. 4Takeaways & Limitations
A seasonal naive forecast outperforms every published graph model at long lead times on two datasets.
Source: §4 · p. 3, §5 · p. 4The evaluation is confined to influenza surveillance on three benchmarks.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Weekly influenza surveillance counts guide vaccine distribution and public-health alerts, yet they are hard to forecast. Each region offers only a few seasons, waves shift in timing and height every year, and information that helps while a wave grows misleads after its peak, whereas last season's shape stays informative for a year. Existing epidemic graph models and general forecasters read a short fixed window and treat all past information alike, so they neither exploit earlier seasons nor discard stale associations when the epidemic phase changes. To address these limitations, we propose TERN, a forecaster built around a delta-rule fast-weight memory that decays channel-wise and erases along a learned address under gates driven by local epidemic-phase features, combined with an explicit seasonal reference and online adaptation. On three Cola-GNN influenza benchmarks, TERN outperformed epidemic graph models and general forecasters, matched or exceeded seasonal references, and a controlled comparison confirmed the contribution of the memory itself.
1. INTRODUCTION
Influenza surveillance forecasting must handle short regional histories, shifting epidemic waves, and information whose usefulness changes with epidemic phase. TERN combines phase-conditioned memory with seasonal information and online adaptation, and is evaluated alongside epidemic and general forecasters.
1. INTRODUCTION
- Regions provide only a handful of seasons, while wave timing and height shift and growth-phase information can mislead after the peak.The previous season’s shape can remain useful for a year, so forecasting must selectively forget while retaining information across time scales.Source: §1 · p. 1
- Epidemic graph models and general forecasters use short fixed windows, rarely undergo direct comparison, and do not explicitly control what is forgotten or retained.Recent linear-attention work offers mechanisms including delta rules, gated decay, and separate erase and write operations –[14].Source: §1 · p. 1
- TERN combines an erase-then-delta memory with channel-wise decay and local-phase-conditioned erase and write gates and erase address.An explicit seasonal reference and online adaptation complete the model.Source: §1 · p. 1
- TERN is the best uncontaminated model on all three influenza datasets, and a controlled comparison finds softmax attention has higher RMSE on all three.The comparison holds context length, seasonal reference, and online adaptation fixed; the paper also evaluates epidemic graph models, general forecasters, seasonal references, and zero-shot foundation models under one protocol.Source: §1 · p. 1
2. RELATED WORK
Related work spans epidemic-specific models and general time-series forecasters, alongside linear-attention approaches that control memory updates. The paper positions its evaluation across these communities under a shared epidemic protocol.
Deep learning for epidemic forecasting
- Epidemic forecasting work includes graph-attention models, mechanistic hybrids, and multi-view fusion, while some spatial models fail to outperform naive baselines in recent benchmarks.Cola-GNN learns cross-location attention over an RNN; EpiGNN adds transmission-risk encodings and a region-aware graph learner, while EINNs and CAMul add other forms of structure –, [15] [16].Source: §2 · p. 1
General forecasting and delta-rule attention
- General forecasters use patching, inverted attention, multiscale mixing, linear models, and decomposition-based state-space models; delta-rule methods vary in decay and erase/write design.The paper reruns the first five general forecasters under the epidemic protocol and evaluates delta-rule variants –[14],.Source: §2 · p. 1
3. METHOD
TERN mixes each region’s history with an erase-then-delta memory, then couples regions through attention. Its design adds phase-conditioned memory updates, multiple history regimes, a seasonal reference, and online adaptation.
3. METHOD
- TERN processes each region temporally through normalization, embedding, repeated memory-mixer and MLP blocks, then applies cross-region attention and a linear forecasting head.The cross-region attention can optionally use an adjacency matrix as an attention bias; Figure 1 depicts the full-history forecaster and one memory step.Source: §3 · p. 1, Figure 1 · p. 2
- The memory uses an erase-then-delta fast-weight update: channel-wise decay and an addressed erase precede a gated delta-rule write.The readout uses a query, while the write key and value come from projections of the block input.Source: §3 · p. 2
- The gates read the block input together with phase features for growth, curvature, and relative growth, conditioning the memory update on local epidemic phase.The erase address is factorised and decoupled from the write key, allowing stale associations to be cleared along a separate direction.Source: §3 · p. 2
- Decay uses input-dependent modulation and learned per-channel rates, with time constants initialized log-uniformly from 1 to 20 weeks.The paper also evaluates variants that remove the addressed erase or alter how decay and erasing are parameterized.Source: §3 · p. 2
- TERN is evaluated with a 20-week window and in a full-history regime that reads the entire history causally and forecasts at every step.The window-regime model is refit using training and validation targets after early stopping; full-history training uses one gradient step per epoch on the training sequence.Source: §3 · p. 2
- The full-history model uses a seasonal reference, and forecasts combine it with a learned correction whose shrinkage decreases at longer lead times.Online adaptation uses either one gradient step on observed targets at each new observation or a blend with the seasonal-naive forecast weighted by recent known-target errors; the loss is scale-weighted squared error.Source: §3 · p. 2
4. EXPERIMENTS
Experiments evaluate TERN against epidemic and general forecasters on three Cola-GNN influenza benchmarks. Full-history TERN improves on its window regime, while ablations and qualitative forecasts examine the memory and seasonal components.
4. EXPERIMENTS
- On three Cola-GNN influenza datasets, chronological 50/20/30 splits and 20-week inputs supported forecasts 3, 5, 10, or 15 weeks ahead; RMSE and PCC were pooled across test weeks and regions, averaged over four lead times and five seeds.Comparisons included epidemic-specific models, five general forecasters, and zero-shot Chronos-2 and TimesFM-3 using either the 20-week window or full history.Source: §4 · pp. 2–3
- Full-history TERN reduced RMSE by 27%, 21%, and 3% versus its window regime and was the best uncontaminated entry on all three datasets.It tied the seasonal naive on Japan RMSE (838 vs. 839), exceeded the climatology on US-Regions (698 vs. 727), and matched reported EpiGNN on US-States (199 vs. 200).Source: Table 1 · p. 3, §4 · p. 4
- Seasonal naive beat published GNNs at h=10 and 15 on Japan and US-Regions, yet validation missed this advantage (Japan RMSE 1862 vs. 839 on test); US foundation models were contaminated.The pretraining corpus contained the CDC FluView ILINet and WHO/NREVSS series underlying the two US datasets.Source: §4 · p. 3
- The softmax mixer raised RMSE by 5%, 3%, and 22% (874, 713, 238); erase-then-delta was the best memory rule on all three datasets.The ablation table removes or replaces one component at a time in the full-history regime.Source: Table 2 · p. 3, §4 · p. 4
- TERN placed all four Tokyo and Osaka peaks within one week of observations, while PatchTST and DLinear peaked 3–5 weeks late; learned decay time constants had a 3–4-week median.The plotted time constants stayed within the 1–20-week initialization range.Source: Figure 2 · p. 4, §4 · p. 4
- With the same post-hoc online blend, the best general forecaster reached 864, 697, and 220 RMSE: level with TERN on US-Regions, 3% behind on Japan, and 10% behind on US-States.This indicates that part of the full-history gain reflects context rather than memory.Source: §4 · p. 4
5. CONCLUSION
TERN is the best uncontaminated model on three Cola-GNN benchmarks, and its memory contributes 3–22% RMSE over softmax attention when other components are held fixed. The evaluation is limited to influenza surveillance on three benchmarks.
- TERN is the best uncontaminated model on three Cola-GNN benchmarks, and its memory contributes 3–22% RMSE over softmax attention with everything else held fixed.Source: §5 · p. 4
- At long lead times, a seasonal naive forecast outperforms every published graph model on two datasets.Source: §5 · p. 4
- The evaluation is confined to influenza surveillance on three benchmarks; future work will add probabilistic outputs and cover other diseases and hospitalisation targets.Source: §5 · p. 4