PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning to Retrieve Missing Evidence for Long-Term Memory QA

Yi-Xuan Deng, Yi Zhang, Wei Liu, Chao Xue, Shuojin Yang

arXiv:2609.37443v1cs.CLcs.AI

TL;DR

Long-term memory QA must locate evidence scattered across distant turns, sometimes without clues in the question. MERA uses verified evidence to guide retrieval over global memory and trains a lightweight planner on recovered evidence. It achieves strong answer accuracy across two backbones, with its trained 0.6B planner outperforming an untrained 30B planner in the evaluated Qwen3-30B setting.

  • Problem

    Long-term memory QA must find sparse evidence across distant turns even when questions omit entities, events, or temporal cues needed to locate it.

    Source: §1 · p. 1, §3.1 · p. 3

  • Method

    MERA separates a globally searchable atomic-fact index from a question-specific, verifier-gated evidence state and trains a lightweight planner to recover missing evidence.

    Source: §1 · p. 2

  • Results

    MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini; with Qwen3-30B, its trained 0.6B planner exceeds a 30B planner without retrieval-grounded training by 4.10 percentage points on LoCoMo.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The results support evidence-guided retrieval for long-term memory QA and suggest that, under the evaluated settings, training a lightweight planner can outperform increasing planner size without retrieval-grounded training.

    Source: §5 · p. 9

    The study focuses on retrieval and planning algorithms, not product deployment; it does not address real-product privacy or controllable management or deletion of stored memories.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.

1 INTRODUCTION

Long-term memory QA must find evidence scattered across distant turns, including when the question omits clues needed to locate it. MERA uses verified evidence to guide global retrieval and trains a lightweight planner to recover missing evidence.

1 INTRODUCTION

  • Evidence is sparse across distant turns, and questions may omit the entities, events, or temporal cues needed to locate it.Retrieving memories individually relevant to the question may still leave critical evidence undiscovered.Source: §1 · p. 1
  • MERA separates a question-specific evidence state from global search, letting verified entities or temporal relations guide retrieval of evidence not yet represented in the graph.The evidence graph organizes established facts, while retrieval searches a separate global atomic-fact index.Source: §1 · pp. 1–2
  • Planner training rewards newly recovered gold evidence rather than a prescribed query form, and gives no credit for repeated hits.The planner is trained with GRPO while the memory index, evidence model, and answer model remain fixed.Source: §1 · p. 2
  • 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S; the trained 0.6B planner exceeds a 30B planner without retrieval-grounded training by 4.10 and 3.96 percentage points, respectively.These results use Qwen3-30B for evidence processing and answer generation.Source: §1 · p. 2

2 RELATED WORK

Prior memory work focuses mainly on persistent representations or memory operations, while MERA targets question-time retrieval decisions. It also differs from iterative retrieval and graph-based methods through its verifier-gated evidence state and globally searchable index.

2.1 LONG-TERM MEMORY FOR LLM AGENTS

  • MERA shifts the focus from shared persistent memory stores to retrieval decisions for a particular question, using verified facts and prior searches to identify missing information.The memory index remains fixed while the evidence state evolves during answering.Source: §2.1 · p. 3

2.2 LEARNING TO MANAGE AND USE MEMORY

  • MERA trains the query planner at intermediate evidence states using previously missing gold evidence recovered by each executed action, while keeping memory and evidence and answer models fixed.The learned action specifies what to search for next given what earlier rounds established.Source: §2.2 · p. 3

2.3 ITERATIVE RETRIEVAL AND STRUCTURED EVIDENCE

  • MERA also differs from iterative retrieval methods by using an explicit verifier-gated evidence state and rewarding the planner for missing evidence recovered by its actions.This favors queries that complement facts verified in earlier rounds.Source: §2.3 · p. 3
  • Unlike graph-search methods that depend on preconstructed connectivity, MERA builds a transient graph from verified evidence while retrieving from a fixed atomic-fact index.This allows queries to retrieve evidence about entities not yet represented in, or linked to, the current graph.Source: §2.3 · p. 3

3 METHOD

MERA formulates long-term memory QA as sequential evidence acquisition: retrieval actions query persistent memory, and verification updates a question-specific state. A globally searchable index, traceable evidence records, and retrieval-outcome training support that process.

3.1 PROBLEM FORMULATION

  • The task is to acquire enough evidence from a sparse subset of conversation turns for a correct answer within a bounded retrieval budget.The original question may not recover a complete evidence chain in one search because a discovered entity or relation can reveal what to retrieve next.Source: §3.1 · pp. 3–4

3.2 MERA OVERVIEW

  • MERA indexes atomic facts from conversation turns, then uses a lightweight planner, retrieval, aggregation, and verification to update evidence for further search or answering.Multiple sampled queries from the same state are executed against the same memory and rewarded for recovering missing gold evidence; GRPO updates only the planner.Source: §3.2 · p. 4, Figure 2 · p. 4

3.3 TRACEABLE MEMORY AND EVIDENCE STATE

  • Parent-turn fact aggregation gathers all extracted facts from each matched turn with speaker, timestamp, and source identifier before verification, retaining provenance.The evidence state also records verified entities and relations, retrieval anchors, exhausted directions, previous queries, viewed candidates, and accepted evidence.Source: §3.3 · pp. 4–5

3.4 EVIDENCE-STATE-GUIDED ACTIVE RETRIEVAL

  • Later retrieval intents use verified evidence and query history as anchors, while candidate access remains global rather than requiring adjacency to the current graph.The verifier admits candidates that help resolve the question; retrieval can stop when evidence is sufficient, the search is exhausted, or the round budget is reached.Source: §3.4 · p. 5

3.5 RETRIEVAL-GROUNDED POLICY OPTIMIZATION

  • Training compares sampled planner outputs from shared evidence states by the missing gold evidence they recover, with repeated hits receiving no additional credit.The progress term increases reward for partial recovery of remaining evidence later in a trajectory; GRPO optimizes the lightweight planner using relative rewards.Source: §3.5 · pp. 5–6

4 EXPERIMENTS

The experiments evaluate answer accuracy and token costs across two long-term memory QA benchmarks, then examine planner training, multi-round retrieval, and evidence-state ablations. Results show strong accuracy alongside gains from retrieval feedback and later evidence recovery.

4.1 EXPERIMENTAL SETUP

  • The evaluation uses LoCoMo’s 1,986 questions and LongMemEval-S’s 500 questions, with Qwen3-30B and GPT-4o-mini as cache-construction and inference backbones.Answer accuracy is measured by an LLM judge; generative-model costs include offline construction and online answering, but exclude embedding and vector retrieval.Source: §4.1 · p. 6

4.2 MAIN RESULTS

  • MERA-RL has the highest accuracy on both benchmarks with Qwen3-30B, and leads GPT-4o-mini on LongMemEval-S and memory-based systems on LoCoMo.MERA also has lower offline token consumption than most memory systems; its atomic-fact index is reusable across questions.Source: §4.2 · pp. 6–7, Table 1 · p. 7

4.3 DOES RETRIEVAL FEEDBACK IMPROVE A LIGHTWEIGHT PLANNER?

  • The planner comparison uses Qwen3-30B for evidence processing and answering while comparing 0.6B and 30B planners, with answer accuracy as the measure.Source: §4.3 · p. 7

4.3.1 PLANNER SCALE AND POLICY-TRAINING ABLATION

  • 4.10 points on LoCoMo and 3.96 points on LongMemEval-S: trained 0.6B MERA-RL exceeds 30B MERA-Large under shared evidence and answer models.Both gains are in answer accuracy.Source: §4.3 · p. 7, §4.3.1 · p. 8, Table 2 · p. 8

4.4 ONLINE INFERENCE COST

  • With Qwen3-30B, online token consumption is on the order of tens of thousands per question and stays below the graph-navigation baseline on both datasets.The online process performs successive planning and verification over retrieved evidence.Source: §4.4 · p. 8

4.5 HOW DOES MULTI-ROUND RETRIEVAL IMPROVE EVIDENCE COVERAGE?

  • On LoCoMo, MERA-RL recall rises from 0.555 to 0.805 (25.0 points); on LongMemEval-S, from 0.849 to 0.989 (14.0 points).Among questions with a hit, the first gold evidence is recovered at round 1.13 on average.Source: §4.5 · p. 8
  • At least 92% of LoCoMo curves’ post-first-round evidence-recall improvement is realized by round 5.Source: §4.5 · p. 8

4.6 HOW DOES THE EVIDENCE STATE SUPPORT RETRIEVAL AND ANSWERING?

  • Graph-assisted systems outperform flat evidence in all six comparisons, with MERA-RL gaps of 14.41 points on LoCoMo and 16.17 points on LongMemEval-S.MERA-RL's advantage over MERA-Large does not persist with flat evidence.Source: §4.6 · pp. 8–9, Figure 3 · p. 9

5 CONCLUSION

MERA keeps reusable memory globally searchable while using verified, question-specific evidence to guide retrieval and answer generation. The conclusion reports strong benchmark accuracy and a higher accuracy for its trained 0.6B planner than for an untrained 30B planner; the appendix details its memory, evidence, retrieval, data, reward, and optimization procedures.

5 CONCLUSION

  • MERA separates a question-specific, verifier-gated evidence state from global memory and reports strong benchmark accuracy, with its trained 0.6B planner outperforming an untrained 30B planner under evaluated settings.The study uses LoCoMo and LongMemEval-S, whose dialogues are synthetic or de-identified; real-product privacy and memory-management concerns are outside its algorithmic focus.Source: §5 · p. 9, ETHICS STATEMENT · p. 10

A.1 ATOMIC-FACT MEMORY AND PARENT-TURN FACT AGGREGATION

  • Atomic facts provide focused retrieval matches, while aggregated same-turn parent records supply related facts, speaker and timestamp metadata, and provenance for verification and reasoning.Retrieved facts are deduplicated by parent turn and ranked using the highest atomic match score.Source: §A.1 · p. 15

A.2 EVIDENCE-GRAPH CONSTRUCTION AND NORMALIZATION

  • The evidence graph records typed entities and relations, active retrieval anchors, exhausted directions, and parent-turn provenance, with normalization preserving conflicting attributes across time.Turn timestamps are distinguished from event times; relative expressions are resolved against turn time, while vague expressions retain anchored intervals.Source: §A.2 · p. 15

A.3 RETRIEVAL EXECUTION, VERIFICATION, AND TERMINATION

  • The planner forms retrieval intents from gaps in the evidence state, searches the global atomic-fact index beyond graph contents, verifies returned records, and answers from accepted evidence when retrieval ends.Stopping follows evidence sufficiency after minimum exploration, consecutive unproductive rounds, or the round limit; exhausted or capped runs answer only from verified evidence.Source: §A.3 · pp. 15–16

A.4.1 TRAINING DATA CONSTRUCTION

  • Training questions come from LoCoMo's repository-native conversation-and-question generation pipeline and from additional grounded questions generated over LongMemEval training conversations.Table 3 summarizes the resulting training-data construction and filtering.Source: §A.4.1 · p. 16, Table 3 · p. 16

A.4.2 MULTI-ROUND STATE COLLECTION AND FILTERING

  • Training states are collected before planner calls across complete MERA trajectories, covering initial, partial-evidence, and late-stage retrieval.States where all rollouts find no new evidence or all remaining evidence are removed; mixed outcomes are retained to focus training where policy choice can affect retrieval.Source: §A.4.2 · pp. 16–17

A.4.3 INCREMENTAL TURN-LEVEL RETRIEVAL REWARD

  • The turn-level retrieval reward credits newly recovered evidence, not repeated hits, and adds progress weighting for partial recovery later in a trajectory.Informative-state filtering removes cases where policy choice cannot change outcomes; auxiliary keyword-count and response-length penalties shape action form.Source: §A.4.3 · p. 17

A.4.4 POLICY OPTIMIZATION

  • GRPO compares multiple executed planner outputs from the same evidence state, using relative advantages and a clipped policy objective with reference-policy regularization and optional entropy.The setup does not fit a separate value model; informative-state filtering increases the frequency of groups with meaningful reward variation.Source: §A.4.4 · pp. 17–18

A.4.5 SCOPE AND FUTURE DIRECTIONS OF RL TRAINING

The RL setup keeps the memory index, evidence model, and answer model fixed while training a lightweight planner with GRPO and incremental evidence recovery as its main reward. The authors identify matched-budget reward comparisons, component ablations, and cost-aware or trajectory-level objectives as future work.

A.4.5 SCOPE AND FUTURE DIRECTIONS OF RL TRAINING

  • The RL setup trains a lightweight planner with GRPO and incremental evidence recovery as its main reward while keeping the memory index, evidence model, and answer model fixed.The trained-versus-untrained comparison supports the effectiveness of the training setup as a whole.Source: §A.4.5 · p. 18
  • Matched-data and compute-budget comparisons of reward variants, plus separate ablations of auxiliary penalties and state filtering, could identify which components contribute to observed gains.The proposed variants include evidence recall alone, incremental recall without progress weighting, and combinations of retrieval feedback with final-answer correctness.Source: §A.4.5 · p. 18
  • Future work could test cost-aware rewards and trajectory-level optimization to balance evidence acquisition with retrieval cost and jointly train query selection and stopping.These extensions would examine whether better immediate evidence recovery also translates into more efficient and accurate multi-round reasoning.Source: §A.4.5 · p. 18

B.1 DATASETS

The evaluation uses complementary conversational-memory benchmarks, established baselines, and LLM-judged answer accuracy. Implementation and cost reporting specify the inference setup, while runtime analysis finds retrieval-grounded training adds little time relative to using a 30B planner.

B.1 DATASETS

  • LoCoMo contains 1,986 questions across 10 long conversations, while LongMemEval-S contains 500 questions associated with historical sessions.The paper reports LongMemEval-S test evaluations on the same held-out split of 101 questions.Source: §B.1 · p. 18

B.2 COMPARED METHODS

  • Comparisons include full-history and one-shot retrieval systems, iterative retrieval methods, and structured, dynamic, or compressed memory systems, using Qwen3-30B or GPT-4o-mini backbones.The GPT-4o-mini experiments rebuild the cache with that model while holding the planner configuration fixed.Source: §B.2 · p. 19

B.3 METRICS

  • Answer accuracy is the proportion of evaluation questions judged correct by an LLM that reads the question, reference answer, and model answer.Generative-model costs are reported separately for offline construction and online answering; MERA’s persistent-update entries are marked “–” because it performs no separate persistent update.Source: §B.3 · p. 19

B.4 IMPLEMENTATION DETAILS

  • MERA embeds atomic facts and retrieval intents, while adapted IRCoT and GraphReader baselines use specified retrieval budgets, chunking, and call limits.The LongMemEval-S GraphReader chunk size is a resource adjustment in this adaptation.Source: §B.4 · p. 19

C.1 ONLINE INFERENCE COST

  • Online inference costs are reported in thousands of tokens per question and include seed-graph construction, planning, verification, graph updates, and answer generation.The reported tables cover LoCoMo and LongMemEval-S; offline construction, embeddings, and judging are excluded.Source: §C.1 · pp. 19–20, Table 4 · p. 20, Table 5 · p. 20

C.2 RUNTIME STATISTICS

  • Training adds only 5.1 seconds per question, while the retrieval-trained 0.6B planner avoids the inference cost of replacing it with the 30B backbone.MERA-RL retains the same planner architecture and serving setup as MERA-Small.Source: §C.2 · p. 20

C.3 TERMINATION BEHAVIOR

Termination analyses track retrieval rounds, stopping mechanisms, and incomplete recovery of annotated evidence, while later analyses examine retrieval depth, graph structure, and query behavior. The reported diagnostics show that trained retrieval and graph-supported evidence organization can improve retrieval outcomes, with qualifications on what these comparisons establish.

C.3 TERMINATION BEHAVIOR

  • On LoCoMo, MERA-Small averages 3.15 rounds and has incomplete recall on 52.5% of questions, versus 2.20 rounds and 27.2% for MERA-RL.MERA-Small’s verifier-declared convergence accounts for 78.8% of stops; incomplete annotated evidence does not establish that a sufficiency decision was incorrect.Source: §C.3 · p. 20, Figure 4 · p. 21
  • On LongMemEval-S, MERA-RL lowers incomplete recall to 1.1% from MERA-Small’s 37.3%, using 4.16 versus 2.88 rounds; it also has lower incomplete recall and slightly fewer rounds than MERA-Large.MERA-RL stops through retrieval exhaustion in 80.4% of cases, reaches the iteration limit in 8.7%, and converges in 10.9%; these are diagnostics, not a controlled stopping-policy ablation.Source: §C.3 · p. 20, Figure 4 · p. 21

D.1 RETRIEVAL DEPTH AND RETRIEVER COMPATIBILITY

  • At least 92% of each LoCoMo curve’s post-first-round recall improvement is realized by round 5, with later changes no greater than 1.9 points.Evidence gains diminish with retrieval depth, motivating adaptive rather than mechanically maximum-budget termination.Source: §D.1 · p. 21
  • Under dense LoCoMo retrieval, MERA-RL exceeds MERA-Small by 18.2 points after round one and 29.1 points at termination; the earlier keyword diagnostic shows a much smaller gap.MERA-Small gains only 2.3 points after round 3, while MERA-RL gains another 6.1 points; the reported gains depend on planner–retriever compatibility.Source: §D.1 · p. 21

D.2 GRAPH-STRUCTURED VERSUS FLAT EVIDENCE

  • All six graph–flat accuracy gaps favor the graph, ranging from 9.65 to 14.41 points on LoCoMo and from 5.34 to 16.17 points on LongMemEval-S.The flat variant removes the evidence graph and its state-maintenance mechanisms while retaining accepted evidence in a list for planning and answering.Source: §D.2 · p. 22

D.3 CHANGES IN QUERY BEHAVIOR

  • Training leaves both planners’ outputs parseable, increases query length by 0.20 words, modestly raises intent count, and reduces normalized repetition by 1.1 points.Final evidence recall and premature stopping improve in the same direction, but the analysis does not isolate intent count, keyword choice, and stopping behavior.Source: §D.3 · p. 22, Table 7 · p. 22

D.4 A TRAJECTORY FROM THE RL-TRAINED PLANNER

A recorded LongMemEval-S trajectory illustrates how retrieved evidence can establish an entity absent from the question and make it available to subsequent planning. In this example, global retrieval remains able to find relevant evidence outside the current graph, and the run ultimately answers correctly.

D.4 A TRAJECTORY FROM THE RL-TRAINED PLANNER

  • After round 1 establishes Twitter and Facebook comparisons, round 2 queries those verified entities while retaining a broader platform-comparison query.TikTok remains absent from the graph after that round.Source: §D.4 · pp. 22–23
  • Round 3 retrieves TikTok growth evidence even though TikTok is not yet represented in the planner’s graph, after which verification adds the entity and its growth relations.The merged retrieval batch contains evidence of approximately 200 followers gained over three weeks and a rapid increase; the log does not attribute either record to an individual intent.Source: §D.4 · p. 23
  • In round 4, the planner queries TikTok using the newly admitted entity, but verification adds no new accepted evidence IDs.The verifier revisits existing growth records, so a graph-conditioned follow-up does not necessarily yield new evidence.Source: §D.4 · p. 23
  • The six-round run ends through retrieval exhaustion and answers “TikTok,” matching the benchmark reference.The trajectory illustrates graph-conditioned retrieval behavior, but does not isolate the contribution of reinforcement learning or establish an efficiency advantage.Source: §D.4 · p. 23

D.5 PARENT-TURN FACT AGGREGATION

Parent-turn aggregation pairs fine-grained atomic-fact matching with related facts from the same turn for verification and reasoning. The analysis reports paired accuracy drops when aggregation is removed and describes distinct category patterns across LoCoMo and LongMemEval-S.

D.5 PARENT-TURN FACT AGGREGATION

  • Removing parent-turn fact aggregation reduces accuracy by 0.46 points for MERA-RL, 2.77 for MERA-Small, and 4.66 for MERA-Large.All paired comparisons favor aggregation, though the ablation does not test access to original turn wording because both variants present extracted facts.Source: §D.5 · p. 23
  • The aggregation ablation evaluates the packaging interface as a whole, not the separate effects of added fact content, input length, or candidate competition.Without aggregation, sibling facts can occupy separate candidate slots despite a fixed candidate budget.Source: §D.5 · pp. 23–24

E.1 ACCURACY BY QUESTION TYPE

  • LoCoMo accuracy consistently ranks single-hop highest, followed by multi-hop, temporal, open-domain, and adversarial questions.Open-domain scores are less directly attributable to conversational retrieval alone, and adversarial questions test avoiding unsupported answers.Source: §E.1 · p. 24, Figure 5 · p. 24
  • On LongMemEval-S, the largest descriptive MERA-Small-to-MERA-RL difference is 29.5 points for multi-session questions, while temporal accuracy remains 39.3%.The passage attributes the temporal difficulty to normalization, ordering, or arithmetic not being guaranteed by making temporal evidence visible.Source: §E.1 · p. 24

E.2 FAILURE ATTRIBUTION

Failure attribution finds that reasoning errors dominate incorrect answers in both datasets, while retrieval failures are more prominent on LoCoMo. The analysis distinguishes these stage-level patterns from specific downstream causes and cautions that its percentages are diagnostic, not exact causal rates.

E.2 FAILURE ATTRIBUTION

  • Long-term memory QA is approximated as the probability of sufficient evidence multiplied by the probability of a correct answer given sufficient evidence.This separates evidence acquisition from answering with the evidence.Source: §E.2 · p. 24
  • Reasoning errors account for 63.5% of 655 incorrect LoCoMo answers and 87.1% of 31 incorrect LongMemEval-S answers.This is a coarse stage-level diagnosis: evidence was found, but the final answer was still incorrect.Source: §E.2 · p. 24
  • On LongMemEval-S, temporal questions make up 14 of 27 MERA-RL reasoning errors (51.9%), single-session-user questions contribute none, and multi-session questions contribute 7–9 across planner configurations.Many temporal failures involve relative-date calculations; cross-session evidence integration remains difficult despite retrieval improvements.Source: §E.2 · pp. 24–25
  • Retrieval failure accounts for 27.2% of LoCoMo errors versus 3.2% on LongMemEval-S; LoCoMo questions average three to four gold turns, compared with about 1.5 on LongMemEval-S.Premature stopping is uncommon in both runs, at 3.8% and 0.0%, respectively.Source: §E.2 · p. 25
  • The attribution combines deterministic trajectory checks with local 30B-model judgments, so its percentages are bottleneck diagnostics rather than exact causal rates.Reasoning errors need a second-level breakdown by question and error type to identify root causes.Source: §E.2 · pp. 24–25

F RETRIEVAL HYPERPARAMETER SENSITIVITY

LoCoMo accuracy varies with retrieval depth and round budget, with k = 16 producing the highest observed accuracy among the tested settings.

F RETRIEVAL HYPERPARAMETER SENSITIVITY

Related papers

Loading 2609.37443v1…