MemoryAthena: Adaptive Routing over Latent and Generated Memories

• The paper asks whether memory must be retrieved as a stored representation, or whether useful representations can instead be reconstructed from memory cues or generated from causal context.Source: §1 · p. 2
• Because generated representations can help, be unnecessary, or harm depending on the input, MemoryAthena retains E as its reference and routes generated candidates as selective interventions.Source: §1 · pp. 2–3
• A lightweight causal head predicts each generated candidate’s E-relative advantage and confidence, admitting it only when both satisfy routing criteria; otherwise, the direct pathway is recovered exactly.Source: §1 · p. 2
• The router is trained without downstream labels by comparing frozen E, GE, and GH pathways under teacher forcing and aggregating future-token likelihood differences into targets.Source: §1 · p. 2
• The three pathways are direct Engram retrieval (E), generation from retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH).Source: §1 · pp. 2–3
• MemoryAthena improves all five QA summary metrics and five of six NLP tasks over the same checkpoint’s direct-memory pathway, with complementary successful predictions across E, GE, and GH.Source: §1 · p. 3

• External retrieval supplies text or datastore examples, while learned memory stores trainable representations; Engram is the direct addressable memory substrate varied in this paper.Source: §2 · p. 3
• Reusable memory separates addressing, storage, and reading, with a target-side reader adapting stored representations for a different backbone.Source: §2 · p. 3
• The setup contrasts direct Engram reading (E), generation conditioned on retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH).Source: §2 · p. 3
• Routing is asymmetric: E is already usable, while GE and GH are candidate modifications whose additional utility, identity, and intervention strength must be determined.Source: §2 · p. 4

3 METHOD

• Training learns addressable memory with a frozen source backbone, adapts memory-side pathways with memory and target backbone fixed, then freezes pathways to train only the router.Source: §3 · p. 4

3.1 Stage1: Direct and Generated Memory Pathways

• E directly reads retrieved Engram memory, GE generates from Engram cues, and GH generates from causal backbone states without consulting the memory table.Source: §3.1 · p. 5

3.2 Stage2: E-Relative Advantage Distillation

• With the backbone and memory pathways frozen, a lightweight head predicts each candidate’s E-relative advantage and confidence from current-state features, using counterfactual future-token supervision.Source: §3.2 · p. 5

3.3 Stage3: E-Anchored Memory Routing

• At inference, GE and GH are candidate corrections admitted by advantage and confidence thresholds; the highest-advantage candidate is boundedly interpolated into E, with exact E fallback on rejection.Source: §3.3 · pp. 5–6

4.1 RQ1: When does generated memory improve a strong direct-memory pathway?

• GE improves WebQA from 33.35 to 35.24, but loses to E on TriviaQA and HotpotQA, while GH is weaker on all five QA metrics.After transfer to Llama, the memory interface reaches a 37.87 average and 36.40 on WebQA; this is not a matched gain over a bare Llama model.Source: §4.1 · pp. 6–7
• 37.65 to 39.28: MemoryAthena raises the five-task QA average over the same-checkpoint E pathway.The improvement spans all five QA metrics, ranging from 0.30 points on HotpotQA to 4.74 on NQ.Source: Table 1 · p. 7, §4.1 · p. 7
• 76.73 to 79.13: MemoryAthena increases the six-task general-NLP average over E using task-specific inference settings.The router uses τ = 0 on five tasks and the more conservative τ = 1 on Yahoo, where it reaches 57.43 versus E at 57.51.Source: §4.1 · p. 7, Table 2 · p. 8

4.2 RQ2: What drives the gains from the memory interface?

• 39.28 to 33.35: Removing the gate lowers the five-task QA average, and replacing the interface with a parameter-matched FFN lowers it to 24.92.The full model also outperforms the no-gate and affine-stitch variants on four of five QA metrics and the parameter-matched FFN on all five.Source: §4.2 · p. 8
• 40.16 versus 39.28: Training the memory system from scratch slightly exceeds the pretrained-memory configuration in this setup.The reported result indicates pretrained memory initialization is not required for these gains.Source: §4.2 · p. 8
• 31.67 versus 39.28: Random routing lowers the five-task average by 7.61 points, showing that pathway availability alone is insufficient.On TriviaQA, F1 falls from 70.68 to 49.18.Source: §4.2 · p. 8

4.3 RQ3: How does MemoryAthena use complementary memory pathways?

• 74.15% on NQ, 76.08% on WebQA, and 76.77% on TruthfulQA comes from GH; CR instead favors GE at 44.60%.These are effective interpolation weights: E retains its (1 − α) component when a generated pathway is admitted; mean interpolation strength ranges from 0.716 to 0.893.Source: Figure 2 · p. 9, §4.3 · p. 9
• 3.04–9.07 points separate the three-source oracle from MemoryAthena across five QA tasks, while the oracle beats the best E-containing two-source oracle on every task.The oracle uses downstream labels to select a source and is therefore an analysis of attainable headroom, not the deployed causal router.Source: §4.3 · p. 9

5 Conclusion

• Selective intervention with generated memory yields gains across QA summary metrics and improvements across the evaluated NLP setting, while oracle headroom points to utility estimation and admission as future work.Generated pathways are treated as candidate corrections to E, not replacements.Source: §5 · p. 10

A Evaluation datasets and protocol

• Evaluation covers five QA benchmarks and six classification tasks, with same-checkpoint comparisons used to isolate inference-rule effects.QA reports open-QA EM and token F1 and the TruthfulQA mean of three multiple-choice metrics; the six-task NLP mean is unweighted.Source: A Evaluation datasets and protocol · p. 13

B Limitations and Future Work

• The routing system still depends on specified memory pathways, features, objectives, and admission hyperparameters, and its performance remains below the gold-label source oracle.The stated future direction is more automated discovery and calibration of candidates without downstream labels.Source: B Limitations and Future Work · p. 13

C.1 Direct and generated pathways

• E directly reads table entries, GE generates from retrieved cues, and GH generates from clean causal backbone states without reading the table.The current GH implementation requires an additional clean-backbone computation.Source: §C.1 · pp. 13–14

C.2 Memory learning and reader adaptation

• Training separates memory learning, interface adaptation, and routing; the final stage freezes pathways and trains the head from counterfactual future-token advantages relative to E.The source adaptor may not be compatible with another backbone, and endpoint language-modeling training does not optimize downstream labels.Source: §C.2 · p. 14, §C.3 · p. 14

C.3 Counterfactual future-advantage supervision

• Future-token advantages are aggregated over multiple horizons for offline supervision, while the deployed router uses causal features without access to future tokens.The head has 534,924 trainable parameters in the audited Mistral runs; its confidence score is not established as a calibrated probability of downstream correctness.Source: §C.3 · pp. 14–15

C.4 E-Anchored Causal Inference

• A candidate is admitted only when predicted advantage and confidence pass their criteria; otherwise, the residual exactly recovers E.The fallback does not ensure every admitted intervention improves downstream performance, because the router can predict incorrectly.Source: §C.4 · p. 15

D.1 Training Configuration

• Each optimization stage uses at most 20M processed input positions, and QA reuses its learned source memory rather than adding another memory-training phase.Router checkpoints are selected using held-out causal-text validation, except the separately reported Yahoo threshold adjustment that uses downstream test performance.Source: §D.1 · pp. 15–16, Table 5 · p. 16

D.2 Architecture Specification

• Reader/generator training contains 167,364,632 trainable parameters for general NLP and coding, while the QA stage contains 167,385,644 and adds auxiliary routing modules for comparison baselines.Those auxiliary modules are not additional memory sources in the deployed MemoryAthena router.Source: §D.2 · p. 16
• Reader/generator training takes approximately 31.70 hours for QA, 16.01 for general NLP, and 16.41 for coding; router training takes approximately 11.19, 10.58, and 10.94 hours, respectively.Processed-token count alone does not represent total compute because routing evaluates counterfactual endpoints and GH adds a clean-backbone pass.Source: §D.2 · p. 16
• Benchmark decontamination is not guaranteed: review corpora may overlap downstream review benchmarks, and CC-News streams share an underlying split rather than document-disjoint partitions.The general-NLP training mixture includes WikiText-103, Amazon Polarity, CC-News, and IMDB.Source: §D.2 · pp. 16–17

E Complete QA Metrics

• The E+GE and E+GH controls are learned two-source fusion systems whose input-dependent mixing weights do not isolate either generated pathway’s contribution.Source: E Complete QA Metrics · p. 17
• TriviaQA and HotpotQA show higher F1 but lower EM for MemoryAthena than same-checkpoint E.NQ and WebQA improve on both metrics.Source: E Complete QA Metrics · p. 17
• Most paired examples remain unchanged, while improved and degraded examples are similar in number; positive mean F1 gains therefore reflect the magnitudes of changes.TruthfulQA is excluded from the paired F1 analysis because it does not use token-overlap F1.Source: E Complete QA Metrics · p. 17

F Source-Oracle Analysis

• The gold-label oracle selects the highest-scoring complete endpoint per example for analysis and is unavailable at inference time.Adding endpoints cannot lower oracle performance, so gains over a two-source oracle indicate usefulness on some examples, not deployable selection ability.Source: F Source-Oracle Analysis · pp. 17–18
• Oracle counts identify the best complete endpoint per example, whereas deployed routing weights measure each pathway’s local contribution across tokens and layers.Ties are assigned to E first, increasing its oracle-winner count.Source: F Source-Oracle Analysis · p. 18

G Ablations and Architectural Controls

• Removing the output gate lowers the five-task average by 5.92 points, a parameter-matched FFN replacement by 14.36 points, and affine stitching by 1.46 points.The ablation metrics are F1 for open-QA tasks and mean multiple-choice score for TruthfulQA.Source: G Ablations and Architectural Controls · p. 18, Figure 6 · p. 20
• Training memory from scratch raises the five-task average by 0.89 points over pretrained memory, improving four QA tasks but slightly decreasing WebQA.Permuting memory keys decreases the five-task average by 0.70 points.Source: G Ablations and Architectural Controls · p. 18

H Threshold Sensitivity

• Yahoo Answers performance improves as the admission threshold becomes more conservative, with the largest changes from τ = 0.3 to 0.9 and relative stability around 0.9–1.0.The results indicate threshold sensitivity and comparatively stable performance in the high-threshold regime.Source: H Threshold Sensitivity · p. 19

I Interpreting Routing Statistics

• The QA checkpoint assigns most interpolation mass to GH, while NLP and coding retain more E mass; GE contributes less in all three settings.The pathway balance varies with training domain and checkpoint.Source: I Interpreting Routing Statistics · p. 19
• Generated-pathway admission rates of 99.97% for QA and 98.07% for NLP coexist with non-trivial E interpolation mass, so admission frequency and interpolation mass are distinct.An admitted generated pathway can still be interpolated with E using α < 1.Source: I Interpreting Routing Statistics · p. 19

J Transfer and secondary evaluations

• Mistral-to-Llama results show target-side adaptation, but a matched advantage over bare Llama cannot be established without its same-scorer baseline.On six NLP tasks, the target router mean is 40.48 versus 38.23 for target E; the missing bare-Llama baseline limits that comparison.Source: J Transfer and secondary evaluations · p. 20
• On HaluEval, gains are mixed across tasks, and classification accuracy does not directly measure hallucinations in free-form generation.The passage also cautions that the summarization gain may reflect response bias and calls for confusion-matrix and class-balance checks.Source: J Transfer and secondary evaluations · p. 20

K Scaling Analysis

• For GPT-2 Small, Medium, and Large on WikiText, routed perplexities are 23.372, 17.291, and 14.545, versus memory-only values of 24.033, 17.797, and 14.955.Across completed runs, routing remains better than memory alone; the reported general-mixture runs show the same pattern.Source: K Scaling Analysis · p. 21, Figure 7 · p. 21
• At 10M, 30M, and 100M training tokens, routed perplexity decreases from 20.222 to 19.889 to 19.462, remaining below memory-only at each budget.Increasing the training budget monotonically reduces perplexity for both systems.Source: K Scaling Analysis · p. 21, Figure 8 · p. 22

L Case Study

• In both case studies, MemoryAthena answers correctly although E, GE, and GH each answer incorrectly, including the year 1907 and the winner HC Davos.The examples illustrate that routing can use complementary information rather than simply selecting the best standalone endpoint.Source: L Case Study · p. 22, Figure 9 · p. 22

M Computational Cost

• In the AMD MI250X BF16 inference microbenchmark with a fixed 338-token input and 16 generated tokens, relative runtime and memory overhead both decrease as the backbone scales.Source: Figure 10 · p. 23

M.1 Training Cost

• The reader/generator and router stages together take approximately 42.89 hours for QA, 26.59 hours for general NLP, and 27.35 hours for coding.These times cover the recorded stages, not the complete lifetime cost of the reusable source memory.Source: §M.1 · p. 23, Table 16 · p. 27

M.2 Inference Microbenchmark

• E-only inference is faster at all four evaluated scales, with its latency advantage declining from 1.88× at Small to 1.40× at XL.The controlled microbenchmark uses an AMD MI250X with BF16, 338 input tokens, and 16 generated tokens.Source: §M.2 · p. 23
• The additional runtime cost comes mainly from evaluating generated-memory pathways and maintaining intermediate states, including a separate clean-state backbone computation for GH.The routing head contains 534,924 parameters; the paper cautions that MemoryAthena is not a zero-overhead inference method.Source: §M.2 · p. 24
• The benchmark measures relative overhead under fixed synthetic-prompt, generation-length, precision, and hardware conditions, not full downstream-task throughput.Task throughput can vary with sequence length, batch size, generation length, routing behavior, and hardware utilization.Source: §M.2 · p. 24

N Language-model diagnostics and incomplete extensions

• For QA, ordinary soft fusion and hard routing have lower validation perplexity than the advantage router, but corpus perplexity alone does not establish better QA performance.The reported perplexities are 7.07310 for ordinary soft fusion, 7.09269 for hard routing, 7.14538 for the advantage router, and 8.57136 for E.Source: N Language-model diagnostics and… · p. 24, Table 17 · p. 27
• On Nemotron-CC-Code, routed perplexity is 2.99563 after 19,998,720 input positions, below the tri-expert arithmetic mean of 3.02853.The individual expert values are 3.10842 for E, 2.99503 for GE, and 2.98369 for GH.Source: N Language-model diagnostics and… · p. 24