PaperlayerView exact revision on arXiv

Source-linked AI summary

MemSteer: Turning User Histories into Supervision for Agent Memory Optimization

Jie Huang, Cuiling Lan, Qize Yu, Xiulian Peng, Yan Lu

arXiv:2610.39276v1cs.AI

TL;DR

Personal histories often lack ready-made feedback for adapting memory designs, and creating annotated QA for each history is costly. MemSteer generates evidence-grounded QA from unlabeled histories and uses it to evaluate and revise designs; experiments report competitive performance with Official-QA-guided optimization across ATM-Bench and RHELM.

  • Problem

    Personal records and interactions often lack explicit signals for assessing memory quality, while creating questions, reference answers, and supporting evidence for each new history is costly.

    Source: §1 · p. 1

  • Method

    MemSteer plans how facts across related records support reference answers, generates QA, and checks annotations through source-grounded auditing and blind answer recovery.

    Source: §1 · p. 2

  • Results

    Experiments on ATM-Bench and RHELM show that generated supervision improves memory designs and is competitive with Official-QA guidance.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The generated supervision also benefits two existing optimizers on ATM-Bench and supports continued design optimization on new user histories.

    Source: §1 · p. 2

    Evaluation covers two benchmarks, including synthetic-persona RHELM, so evidence on real, heterogeneous user histories remains limited.

    Source: C LIMITATIONS · p. 20

Abstract

from arXiv · show

An effective agent memory system should adapt to a user's history and task requirements. Human-annotated supervision can guide this adaptation by providing feedback for evaluating and improving memory designs. Yet suitable labels are rarely available for new user histories, and manually constructing QA pairs and supporting evidence is costly. We introduce MemSteer, which turns unlabeled user histories into question-answer (QA) supervision with supporting evidence for memory-design optimization. MemSteer constructs an evidence-grounded answer plan that specifies how facts across related records jointly support a reference answer. The plan guides question generation, while source-grounded auditing and blind answer recovery check the resulting annotations. To study the utility of this supervision, we develop MemSteer-Opt, which combines QA-based design evaluation with evidence-guided failure diagnosis to revise memory store, retrieval, and context construction without updating model parameters. Experiments on ATM-Bench and RHELM show that this supervision is competitive with official-QA-guided optimization. Relative to Base-BM25, MemSteer-Opt improves ATM-Bench Question Score by 13.4 points and RHELM accuracy by 4.1 and 6.6 points without and with external data, respectively. The supervision also benefits two existing optimizers on ATM-Bench, while new-user experiments demonstrate continued design optimization using supervision generated from the target histories.

1 INTRODUCTION

Memory designs must adapt to differing user histories, but personal records often lack explicit performance signals and manually creating QA supervision is costly. MemSteer generates evidence-grounded QA from unlabeled histories and uses it to evaluate and refine memory designs.

1 INTRODUCTION

  • Because user histories differ in content, modalities, and organization, a memory design effective for one user may be less suitable for another.Storage granularity, retrieval strategy, and answer-context construction shape how effectively agents use accumulated history.Source: §1 · p. 1
  • Personal records and interactions often lack explicit memory-quality signals, while creating questions, reference answers, and supporting evidence for each new history is costly.Annotated QA pairs can provide feedback, but the paper asks whether unlabeled histories can guide memory-design optimization.Source: §1 · pp. 1–2
  • MemSteer’s answer plans specify how facts across related records jointly support a reference answer, guiding question generation and source-grounded QA auditing.Blind answer recovery also checks whether the reference answer can be recovered from sampled records.Source: §1 · p. 2
  • MemSteer-Opt uses QA performance and evidence-guided failure diagnosis to revise storage, retrieval, and answer-context construction without updating model parameters.The generated supervision can also support M⋆ and MemEvolve within their own design spaces.Source: §1 · p. 2
  • Experiments on ATM-Bench and RHELM show that generated supervision improves memory designs and is competitive with Official-QA guidance.The introduction also reports benefits for two existing optimizers on ATM-Bench and continued optimization on new user histories.Source: §1 · p. 2

2 RELATED WORK

Prior work optimizes memory designs using task feedback, while synthetic QA has been used for training and evaluation both within and beyond agent memory. MemSteer instead uses QA with source-grounded evidence to guide memory-design optimization.

Memory-design optimization

  • Memory-design optimizers such as ALMA, MemEvolve, M⋆, and CluE use downstream task feedback, but personal records often lack natural success-or-failure signals.The approaches vary from searching code-based designs and memory programs to refining extraction prompts.Source: §2 · p. 2

Synthetic queries and QA for training and evaluation

  • Synthetic QA supports diverse tasks, including retriever training, reading-comprehension training, structured QA synthesis, and corpus-specific evaluation.Prior work also studies selecting synthetic questions for target-domain QA performance rather than relying only on generation or answer-model scores.Source: §2 · p. 3

Synthetic supervision for agent memory

  • Unlike MemBuilder, CMI-Mem, and MemMA, which train memory models or repair stored memories, MemSteer uses QA with source-grounded evidence to optimize memory designs.This provides feedback for optimization without requiring manually annotated QA for the target history.Source: §2 · p. 3

3 METHOD

MemSteer builds QA–evidence supervision from unlabeled histories and uses it to evaluate and revise memory designs. Its pipeline selects related records, plans and verifies evidence-grounded QA, then diagnoses failures to guide design changes.

3.1 OVERVIEW

  • MemSteer represents supervision as history questions, reference answers, and selected source records, using answers to evaluate designs and evidence to diagnose failures.The generated supervision can also be supplied to existing optimizers while they retain their own design spaces and procedures.Source: §3.1 · p. 3

3.2 CONSTRUCTING QA–EVIDENCE SUPERVISION

  • A graph of source records connects shared conversations, adjacent document sections, explicit references, or sufficient lexical overlap to form context-budgeted material bundles.Path-based expansion from a seed samples connected records while preserving source-span references.Source: §3.2 · p. 3
  • An evidence-grounded plan identifies the question, reference answer, supporting facts, source-span citations, and each selected record’s contribution and combination operations.The plan guides generation of a standalone question from the plan and material bundle.Source: §3.2 · pp. 3–4
  • Candidates pass source-grounded verification only when semantic judgments support answer evidence and question-plan consistency, followed by blind recovery of the exact answer with citations covering selected records.Blind recovery uses only the question, answer type, and source records; answers are compared after type-specific normalization.Source: §3.2 · p. 4

3.3 MEMSTEER-OPT: EVIDENCE-GUIDED MEMORY-DESIGN OPTIMIZATION

  • MemSteer-Opt represents memory design through storage, retrieval, and context construction, then evaluates responses against reference answers on a fixed validation set.The answering model, its instruction prompt, and final context budget are held fixed across designs.Source: §3.3 · p. 5
  • Evidence-guided diagnosis classifies incorrect responses as answering failures when evidence is present, storage failures when missing records are unrecoverable, and retrieval failures when they are recoverable.Failure-stratified sampling supplies cases from observed categories to the design proposer.Source: §3.3 · p. 5
  • A proposer uses diagnosed failures to revise storage, retrieval, or context construction, and a revision replaces the current design only when validation accuracy improves.After an accepted revision, the system refreshes training-set failure diagnoses for the next iteration.Source: §3.3 · p. 5
  • Generated questions and answers provide task feedback to existing optimizers, while source evidence can also support feedback when their interfaces accommodate it.Each optimizer retains its own design space.Source: §3.3 · p. 5

4 EXPERIMENTS

The experiments test generated supervision for memory-design optimization, its transfer and adaptation across benchmarks and users, and which supervision and feedback choices matter. MemSteer-Opt improves over Base-BM25 on ATM-Bench and RHELM, while generated supervision is competitive with Official-QA.

4.1 EXPERIMENTAL SETUP

  • Experiments use ATM-Bench’s 1,013-question main set and 31-question Hard set, plus RHELM’s 21,142 records across ten personas; model, answerer, and metrics vary by benchmark.MemSteer-Opt starts from Base-BM25, which retrieves indexed source text and metadata with BM25 within a fixed context budget.Source: §4.1 · p. 6

4.2 CAN GENERATED SUPERVISION GUIDE MEMORY-DESIGN OPTIMIZATION?

  • MemSteer is marginally higher than Official-QA across ATM-Bench’s four metrics, while on RHELM it is 0.1 and 0.8 accuracy points lower without and with external data.M⋆ and MemEvolve also gain 6.0 and 18.6 main-set QS points with MemSteer supervision; MemSteer-Opt is the highest-QS non-oracle system on ATM-Bench and reaches 39.9 RHELM accuracy with external data.Source: §4.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7
  • MemSteer-Opt raises ATM-Bench QS from 54.9 to 68.3 and from 8.2 to 29.1 on Hard; on RHELM, it gains 4.1 and 6.6 accuracy points without and with external data.The ATM-Bench comparison is in Table 1 and the RHELM comparison in Table 2.Source: §4.2 · p. 6, Table 1 · p. 7, Table 2 · p. 7

4.3 CAN OPTIMIZED MEMORY DESIGNS TRANSFER AND ADAPT?

  • On RHELM, the frozen ATM-Bench-optimized design beats Base-BM25 by 4.1 and 5.3 accuracy points without and with external data; target-history continuation improves both held-out users.Direct transfer slightly helps David but reduces Martin’s accuracy; personal continuation improves both, especially Martin.Source: §4.3 · p. 7, Table 3 · p. 7

4.4 WHAT MAKES QA–EVIDENCE SUPERVISION EFFECTIVE?

  • MemSteer outperforms Naive-QA and Ragas-QA in downstream QS and both judges’ QA-quality assessments; verification improves QA quality across all three methods and QS in most settings.Explicit answer planning improves QA quality over direct generation; two-stage plan checks improve it further but have the highest per-question token cost.Source: §4.4 · p. 8, Table 4 · p. 8, Table 5 · p. 8

4.5 DOES EVIDENCE-GUIDED FEEDBACK IMPROVE OPTIMIZATION?

  • Failure attribution yields larger gains than adding gold evidence alone, while diagnosis with stratified sampling achieves the highest downstream QS; random diagnosis sampling has higher evidence recall on Hard.Stratified sampling limits each failure category to up to ten cases so frequent failure types cannot exhaust the feedback budget.Source: §4.5 · p. 9, Figure 3 · p. 9

5 CONCLUSION

MemSteer converts unlabeled user histories into QA–evidence supervision for memory-design optimization. Experiments show generated supervision improves designs, performs competitively with Official-QA guidance, and supports adaptation to new histories.

5 CONCLUSION

  • MemSteer derives reference answers and supporting evidence from records, avoiding the need to manually annotate QA for each new history.Reference answers evaluate candidate designs; source evidence diagnoses unsuccessful responses and guides revisions.Source: §5 · p. 9
  • MemSteer-Opt uses the supervision for optimization, while M⋆ and MemEvolve use it within their own design spaces.Source: §5 · p. 9
  • Experiments on ATM-Bench and RHELM show improved memory designs, performance competitive with Official-QA guidance, and continued adaptation to new user histories.Source: §5 · p. 9

ETHICS STATEMENT

The experiments use released benchmark data and do not collect new personal data or involve human-subjects studies. Processing and generated QA nevertheless raise privacy considerations for deployment on personal histories.

ETHICS STATEMENT

  • Experiments use released benchmark data under its published terms, without collecting new personal data or conducting human-subjects studies.Source: ETHICS STATEMENT · p. 10
  • Source records are processed through third-party LLM APIs, and QA combining facts across records may reveal sensitive information.Source: ETHICS STATEMENT · p. 10
  • Deployment on personal histories should require informed user consent and appropriate privacy safeguards.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The paper identifies the sections and appendices documenting the supervision pipeline, optimizer, and experimental protocol. It also notes that remote LLM APIs can make exact outputs variable.

REPRODUCIBILITY STATEMENT

  • Section 3.2 documents graph-guided material preparation, evidence-grounded answer planning, question generation, and source-grounded verification.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Section 3.3 specifies MemSteer-Opt’s design space, evaluation protocol, evidence-guided diagnosis, and acceptance rule; Section 4.1 describes benchmarks, baselines, iteration budget, and model roles.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Appendix A documents further experimental details and Appendix B reports additional results; remote LLM API nondeterminism and provider updates may vary exact outputs, so model identities are reported.Source: REPRODUCIBILITY STATEMENT · p. 10

LLM USAGE

The paper describes LLM roles, benchmark selection, and reference-system coverage. ATM-Bench and RHELM were selected for their multimodal or evolving-record settings, while appendices include additional reference results.

LLM USAGE

  • LLMs handle planning, question generation, auditing, blind answer recovery, design revisions, fixed answering and judging, and QA filtering and evaluation.The paper names DeepSeek-V4-Flash, GPT-4.1-mini, GPT-4.1, and GPT-5-mini across these roles.Source: LLM USAGE · p. 10

A.1 BENCHMARK SELECTION

  • ATM-Bench and RHELM were prioritized because prior work characterizes LoCoMo and LongMemEval as near-saturated; ATM-Bench combines images, videos, and emails, while RHELM combines evolving dialogues with external text sources.Source: §A.1 · p. 14

A.2 REFERENCE SYSTEMS

  • Table 1 compares DeepSeek-V4-based Memexa, A-Mem, MemPalace, and HippoRAG2 on ATM-Bench.Source: §A.2 · p. 14
  • Table 2 reports RHELM results for RAG, Hybrid, MemGPT, Mem0, and MemU, plus 1M-context inference systems; Hybrid combines dense retrieval with BM25 through reciprocal rank fusion.Source: §A.2 · p. 14
  • Appendix Table 8 adds RAG results with GPT-4.1, Gemini-2.5-Pro, and Claude-Opus-4.5 at k = 20.Source: §A.2 · p. 14
  • Appendix Table 7 includes native Qwen3-VL/GPT-5-mini results for ATM-RAG, Self-RAG, ScrapMem, and MemoryOS.Source: §A.2 · p. 14

A.3 EVALUATION POPULATIONS

The appendices specify evaluation populations, metrics, and feedback inputs, then examine supervision-budget trends and full RHELM results. They also describe the ATM-Bench retest and the scope of the RHELM comparison tables.

A.3 EVALUATION POPULATIONS

  • ATM-Bench evaluation covers all 1,013 official QA items, including items used for Official-QA training and validation; RHELM covers all 1,305 official items.The RHELM setup also includes held-out-persona evaluation and personal continuation experiments using supervision generated from target histories.Source: §A.3 · p. 14

A.4 EVALUATION METRICS

  • ATM-Bench Question Score combines exact match, list-question Jaccard credit, and judge scores; R@10 measures gold-evidence representation among the top 10 retrieved records.Final RHELM evaluation uses mean binary correctness judgments from the official evaluator.Source: §A.4 · pp. 14–15
  • Blind answer recovery uses type-normalized exact agreement, while evidence coverage requires citations to every selected evidence record.Boolean answers are matched exactly; validation accuracy guides revision acceptance and design selection.Source: §A.4 · pp. 14–15

A.5 FEEDBACK INPUTS

  • Feedback variants range from random examples to gold-evidence inputs and failure diagnosis, with stratified sampling organizing diagnostic examples by failure category.Gold evidence means reference source evidence in generated supervision, not human-annotated evidence.Source: §A.5 · p. 15

B.1 ATM-BENCH REFERENCE AND OPTIMIZATION RESULTS

  • MemSteer-Opt’s full QA pipeline was retested with a GPT-5-mini judge, and Table 7 reports this complementary evaluation.Table 7 evaluates DeepSeek-V4-Flash-generated answers with the official GPT-5-mini judge and compares MemSteer-Opt variants against Base-BM25.Source: §B.1 · p. 16, Table 7 · p. 16

B.2 EFFECT OF SUPERVISION BUDGET

  • The largest gains occur between 100 and 200 training questions; afterward, ATM-Bench QS generally rises while R@10 remains largely stable on both benchmarks.ATM-Bench-Hard QS fluctuates slightly, with only 31 test questions.Source: §B.2 · pp. 16, 18, Figure 4 · p. 17

B.3 FULL RHELM RESULTS

  • Table 8 reports RHELM accuracy by question type with and without external data, alongside reference systems that are not controlled comparisons with the GPT-4.1-mini systems in Table 2.Its average is overall accuracy; ATM-Bench-evolved designs may be frozen or further optimized on RHELM with Official-QA or MemSteer supervision.Source: Table 8 · p. 17, §B.3 · p. 18

B.4 PER-PERSONA RESULTS

History-generated supervision raises pooled accuracy across all ten personas and performs close to Official-QA supervision, while gains from a six-user shared design do not transfer in aggregate to held-out users. An example trajectory shows successive retrieval, context-construction, and answer-validation revisions.

B.4 PER-PERSONA RESULTS

  • Table 9 compares shared memory designs in six-user and ten-user settings, with official-QA supervision or history-generated QA–evidence supervision.In the six-user setting, four personas provide training supervision, two provide validation supervision, and four are held out; in the ten-user setting, all personas participate in optimization.Source: §B.4 · p. 18
  • 34.3% pooled accuracy on the six source users, up from 31.5%, does not carry over in aggregate to the four users held out from optimization.The results show heterogeneous transfer and motivate target-user continuation.Source: §B.4 · p. 19
  • 39.9% pooled accuracy with MemSteer exceeds Base-BM25’s 33.3% across all ten personas and is 0.8 points below Official-QA’s 40.7%.MemSteer reserves official QA for evaluation, while Official-QA uses some official questions for training or validation.Source: §B.4 · p. 19

B.5 EXAMPLE ACCEPTED ARCHITECTURE TRAJECTORY

  • The example trajectory progresses from retrieval changes to multi-query context construction and answer validation, ending with four HyDE queries, 120 reranked candidates, 25 records, and a 4,000-token output limit.Figure 5 marks the order of nine accepted revisions, starting from Base-BM25.Source: §B.5 · p. 19, Figure 5 · p. 19

C LIMITATIONS

MemSteer’s supervision and verification rely on LLMs without human certification, and the evaluation covers two benchmarks, including one with synthetic personas.

C LIMITATIONS

  • Shared errors or biases across answer planning, source-grounded auditing, and blind answer recovery can pass verification undetected; recovery checks answer recoverability and evidence citation, not whether each record is necessary.Source: C LIMITATIONS · p. 20
  • Generated questions reflect history distributions rather than users’ actual information needs, and MemSteer-Opt explores a constrained, largely retrieval-centric design space.Source: C LIMITATIONS · p. 20
  • Evaluation on two benchmarks, one comprising synthetic personas, leaves limited evidence about real, heterogeneous user histories.Source: C LIMITATIONS · p. 20

Related papers

Loading 2610.39276v1…