PaperlayerView exact revision on arXiv

Source-linked AI summary

Just-In-Time Agent Memory with Runtime Agentic Research

Bingyu Yan, Chaofan Li, Hongjin Qian, Shuqi Lu, Chaozhuo Li, Zheng Liu

arXiv:2609.34385v1cs.CLcs.AIcs.IRcs.LG

TL;DR

AOT memory systems reduce online serving cost, but request-agnostic preprocessing can discard fine-grained information that later matters. JAM preserves raw histories in a navigable workspace and constructs query-conditioned context through a trained Researcher, achieving stronger task performance than AOT-style systems with a favorable quality–cost trade-off against trained memory approaches.

  • Problem

    AOT systems preprocess histories before a request arrives, reducing serving cost but potentially discarding fine-grained details and cross-session dependencies needed for later requests.

    Source: §1 · p. 1

  • Method

    JAM combines a Memorizer that organizes complete histories in a hierarchical workspace with a Researcher that explores them at runtime, trained using Memory-Gym and cascaded verified-trajectory SFT and Hint-guided GRPO.

    Source: §1 · pp. 1–2

  • Results

    JAM consistently improves task performance over AOT-style memory systems and offers a favorable quality–cost trade-off relative to trained agentic memory approaches.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    Experiments across diverse long-context and memory-intensive benchmarks show that JAM transfers across domains.

    Source: §6 · p. 10

Abstract

from arXiv · show

Memory is critical for AI agents. Many existing agent-memory systems follow an Ahead-of-Time (AOT) design, constructing memory before a specific request arrives. While this reduces online serving cost, such request-agnostic memory construction can discard fine-grained information that later becomes important. To address this limitation, we propose Just-In-Time Agent Memory (JAM), a trainable framework for query-conditioned context construction at runtime. A Memorizer preserves complete raw histories in a hierarchical page-store with compact navigational summaries, while a Researcher iteratively retrieves, inspects, and integrates evidence for each request. To train these memory-use behaviors, we introduce Memory-Gym, an evidence-grounded data synthesis pipeline covering nine task types across six domains, and optimize the Researcher through verified-trajectory supervised fine-tuning followed by Hint-guided Group Relative Policy Optimization. We demonstrate the effectiveness of JAM across a variety of benchmarks on agent memory and long-context processing, where it achieves stronger task performance than AOT-style memory systems while remaining substantially more efficient than prior trained agentic memory approaches. To support reproducibility and future research, we release our anonymized source code at https://github.com/VectorSpaceLab/general-agentic-memory.

1 INTRODUCTION

Request-agnostic memory preprocessing can lose fine-grained details and cross-session dependencies needed for later requests. JAM instead builds query-conditioned context at runtime, with a Memorizer, a Researcher, and a training pipeline designed for memory-use behaviors.

1 INTRODUCTION

  • AOT systems reduce online serving cost through request-agnostic preprocessing, but can discard fine-grained details and cross-session dependencies needed by later requests.This is especially problematic when relevant evidence is dispersed across large, evolving histories.Source: §1 · p. 1
  • JAM separates offline memory organization from runtime, query-conditioned context construction: a Memorizer preserves historical records while a Researcher retrieves, inspects, and integrates relevant information.The Memorizer retains raw histories with lightweight navigational memory; the Researcher assembles context for each request.Source: §1 · pp. 1–2
  • Memory-Gym supplies evidence-grounded training scenarios across memory-use tasks, while verified-trajectory SFT followed by Hint-guided GRPO trains the Researcher’s exploration behaviors.The pipeline covers single-hop localization, multi-hop evidence connection, and multi-session summarization.Source: §1 · p. 2
  • JAM consistently improves task performance over AOT-style memory systems while offering a favorable quality–cost trade-off relative to trained agentic memory approaches.Its contributions combine a Memorizer and Researcher with Memory-Gym and cascaded verified-trajectory SFT and Hint-guided GRPO.Source: §1 · p. 2

2 JUST-IN-TIME AGENT MEMORY

JAM organizes accumulated histories offline in a hierarchical workspace, then constructs request-specific context through iterative runtime exploration. Its formulation treats memory as a compact context that should support downstream performance while limiting context size or serving cost.

2 JUST-IN-TIME AGENT MEMORY

  • JAM pairs a Memorizer that organizes accumulated histories offline with a Researcher that explores the workspace at runtime to construct query-conditioned context.The framework’s objective is a useful, compact context for each request.Source: §2 · p. 2

2.1 PROBLEM FORMULATION

  • The memory problem is to produce compact context from long, temporally ordered histories, since directly feeding the full history can be inefficient and error-prone.The objective balances downstream client-agent performance against context size or serving cost; AOT pre-compresses without the specific request, whereas JIT construction is request-conditioned.Source: §2.1 · p. 3

2.2 MEMORIZER: HIERARCHICAL WORKSPACE CONSTRUCTION

  • The Memorizer preserves raw sessions as source files and uses session memos to organize them into a hierarchical workspace with directory-level navigational summaries.Related sessions are grouped by memo, while READMEs summarize child files and subfolders so the Researcher can navigate to evidence for fine-grained inspection.Source: §2.2 · p. 3

2.3 RESEARCHER: RUNTIME AGENTIC RESEARCH

  • At runtime, the Researcher identifies information needs, issues workspace actions, observes results, and appends actions and observations to its interaction trajectory.Its actions include opening paths, searching candidate paths, and browsing specific paths; it stops after a positive sufficiency assessment or when the action budget is reached, then finalizes context from the accumulated trajectory.Source: §2.3 · pp. 3–4

3 TRAINING JAM WITH MEMORY-GYM

JAM’s Researcher is trained with Memory-Gym and a cascaded workflow that combines verified-trajectory SFT with Hint-guided GRPO. Memory-Gym synthesizes evidence-grounded tasks over accumulated histories, while the training stages initialize and further improve runtime evidence exploration.

3.1 MEMORY-GYM: DATA SYNTHESIS FOR MEMORY AGENTS

  • Memory-Gym addresses limited training coverage of realistic long-history memory scenarios with a scalable data-synthesis pipeline for JAM’s optimization.Each instance contains a synthesized request, reference answer, supporting sessions, and target task type.Source: §3.1 · pp. 4–5

3.1.1 TASK TAXONOMY AND SOURCE DOMAINS

  • Memory-Gym organizes accumulated-history tasks into single-hop localization, multi-hop evidence connection, and multi-session summarization across six heterogeneous domains.The domains are web, books, dialogue, news, academic papers, and long-form documents.Source: §3.1.1 · p. 5

3.1.2 TASK CONSTRUCTION PIPELINE

  • Memory-Gym builds each instance by discovering an anchor, expanding supporting sessions for the task, generating a query-answer pair, and validating the candidate.Expansion gathers local evidence for single-hop tasks, linked sessions for multi-hop tasks, or broader related sessions for summarization.Source: §3.1.2 · p. 5
  • 16,794 of 28,276 Memory-Gym candidates passed validation (59.4%), and 95.0% of 200 audited retained instances met all applicable quality criteria.Validation checks include answer uniqueness, evidence support, and no answer leakage for single- and multi-hop tasks, plus coverage, completeness, and faithfulness for summarization.Source: §3.1.2 · p. 5

3.2 CASCADED OPTIMIZATION OF JAM

  • Verified-trajectory SFT initializes the Researcher’s thinking, tool use, and sufficiency assessment from teacher trajectories filtered for correct answers and high-quality exploration.An LLM judge filters unsupported or shortcut behavior, including shallow search without sufficient inspection.Source: §3.2 · p. 6
  • Hint-guided GRPO continues from the SFT checkpoint, guiding training-time search and tool choices with hints that are removed at inference.The workflow uses source recall as its primary reward to improve locating task-relevant information rather than merely imitating teacher trajectories.Source: §3.2 · p. 6

4 EXPERIMENTS

The experiments compare JAM with training-free, AOT-style, and trained memory baselines on four benchmarks, then assess Memory-Gym supervision, component ablations, test-time scaling, and efficiency.

4.2 MAIN RESULTS

  • JAM remains consistently strong across heterogeneous task types, while several AOT-style systems’ performance degrades on long-document and multi-hop reasoning benchmarks.The evaluation covers conversational memory, interactive memory, long-document reasoning, and multi-hop reasoning over dispersed evidence.Source: §4.2 · p. 7, §4.1 · p. 7, Table 1 · p. 7
  • Despite its Qwen3.5-4B backbone, JAM outperforms 7B–14B trained memory baselines across most settings and remains comparatively strong as HotpotQA context length increases.Results were also stable across inference seeds and persisted under LLM-judge evaluation.Source: §4.2 · pp. 7–8

4.3 EFFECTIVENESS AND TRANSFERABILITY OF MEMORY-GYM

  • Memory-Gym improves over the untrained Researcher on all four benchmarks, while its full task-family mixture outperforms every single-family variant.Among single-family variants, SH performs best on LoCoMo, MH on HotpotQA, and SU on LongMemEval and NarrativeQA.Source: §4.3 · p. 8, Table 2 · p. 8
  • LongCodeQA overall accuracy rises from 54.08% for the untrained Researcher to 65.67% after SFT and 75.54% after SFT+RL, with gains across all three context-length subsets.No LongCodeQA examples were used for training or adaptation; Memory-Gym contains no code repositories or code-specific tasks.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 ABLATION STUDY

  • Hierarchical workspace organization raises LoCoMo F1 from 49.06 to 52.09 versus a flat raw-history store, while reducing latency from 17.11 to 13.81 seconds per query.It also reduces average research rounds from 3.57 to 3.16; removing the Researcher degrades performance more than removing the Memorizer, and removing either retriever also hurts.Source: Table 4 · p. 8, §4.4 · p. 9

4.5 TEST-TIME SCALABILITY AND EFFICIENCY

  • Increasing the action budget from 5 to 20 consistently improves performance with diminishing gains, and increasing retrieved items from 1 to 5 shows a similar trend.Under the 20-round limit, median rounds are 3 on LoCoMo, LongMemEval, and NarrativeQA and 6 on HotpotQA; budget exhaustion stays below 5% across benchmarks.Source: §4.5 · p. 9, Figure 3 · p. 9
  • JAM’s 13.81-second LoCoMo query latency accompanies 52.09 F1; versus MemAgent, latency falls from 58.08 to 13.81 seconds while F1 rises from 46.13 to 52.09.The one-time workspace construction takes 79.53 seconds and is reused across queries; construction is lower than A-MEM and Mem0 but higher than MemoryOS and LightMem.Source: §4.5 · p. 9, Figure 4 · p. 9

5 RELATED WORK

Prior agent-memory work commonly organizes histories ahead of requests, while trained memory agents have often relied on narrow supervision sources that limit scenario diversity.

5.1 AOT-STYLE AGENT MEMORY SYSTEMS

  • AOT systems pre-construct memories from histories, but request-agnostic compression can discard later-relevant details or cross-session dependencies, and simple retrieval or summarization can fall short on iterative context construction.Source: §5.1 · p. 10

5.2 TRAINING AND DATA FOR MEMORY AGENTS

  • Training data for memory agents often comes from constrained multi-hop QA or benchmark-specific splits, limiting coverage of localization, cross-session connection, temporal tracking, and aggregation over long histories.Source: §5.2 · p. 10

6 CONCLUSION

JAM constructs request-specific context at runtime by pairing a Memorizer that organizes accumulated histories with a Researcher that explores them. Memory-Gym draws on six heterogeneous domains, and experiments show improved performance over AOT-style systems, cross-domain transfer, and a favorable quality–cost trade-off.

6 CONCLUSION

  • JAM consistently improves task performance over AOT-style memory systems, transfers across domains, and offers a favorable quality–cost trade-off for runtime memory construction.The conclusion reports results across diverse long-context and memory-intensive benchmarks.Source: §6 · p. 10
  • JAM combines a Memorizer that organizes accumulated histories in a hierarchical workspace with a Researcher that explores it to construct task-relevant context.The training workflow combines verified-trajectory SFT with Hint-guided GRPO.Source: §6 · p. 10

A.1 DATA SOURCES AND CORPUS CONSTRUCTION

  • Memory-Gym spans web, books, dialogue, news, academic papers, and long-form documents, covering factual lookup, narrative reasoning, conversational memory, event evolution, scholarly evidence tracing, and structured document understanding.The source corpora are normalized into memory sessions.Source: §A.1 · p. 13
  • Domain construction uses long FineWeb documents or concatenated Wikipedia articles, full-length books, multi-session dialogues, clustered news, paper sections, and recursively expanded report hierarchies.These procedures preserve varied long-context and session structures across the source domains.Source: §A.1 · pp. 13–14
  • Memory-Gym corpora are kept disjoint from evaluation data, including LoCoMo, by excluding released evaluation instances and generating new LoCoMo-style dialogues independently.The stated non-overlap covers users, conversations, sessions, queries, answers, and supporting evidence.Source: §A.1 · p. 14

A.2 IMPLEMENTATION DETAILS OF THE CONSTRUCTION PIPELINE

Memory-Gym’s construction pipeline uses model-generated, evidence-grounded instances and task-specific validation. Its task designs cover single-session localization, cross-session reasoning, and multi-session aggregation.

A.2 IMPLEMENTATION DETAILS OF THE CONSTRUCTION PIPELINE

  • MiniMax-2.5 generates evidence-grounded question-answer pairs and judges them, while BGE-M3 embeddings support semantic clustering and retrieval.Separate generation and validation prompts are used to reduce the chance that the generation procedure accepts invalid instances.Source: §A.2 · p. 14

A.3 TASK-SPECIFIC CONSTRUCTION AND VALIDATION

  • Task-specific rules target single-session localization, cross-session reasoning, or multi-session aggregation, with instances required to be evidence-supported, leakage-free, and task-consistent.Validation criteria vary with the task’s evidence structure.Source: §A.3 · p. 14
  • Single-hop tasks cover attribute extraction, concept explanation, and condition localization, with checks for evidence support, uniqueness, and valid condition or cause relations.Attribute and concept queries are checked against other sessions for plausible alternatives or competing definitions; condition answers are rejected when other sessions provide conflicting causes or conditions.Source: §A.3 · pp. 14–15
  • Multi-hop tasks cover comparison, provenance tracing, and intersection selection, validating evidence support, unique identification, and the reasoning chain across sessions.Intersection questions combine individually ambiguous constraints whose intersection uniquely identifies a target.Source: §A.3 · p. 15
  • Summarization tasks comprise state evolution, set enumeration, and aggregation, with answers checked for supported state changes, completeness, or faithfulness to collected sessions.Set enumeration checks that no supported items are omitted, while aggregation validation checks alignment between question and answer.Source: §A.3 · p. 15
  • For comparisons among judges, adjudicated human labels are the reference, acceptance is the positive class, and precision, recall, and F1 are reported as percentages.Cohen’s κ is the exception to the table’s percentage units.Source: Table 5 · p. 16

A.4 HUMAN VALIDATION OF MEMORY-GYM

Human audits assess Memory-Gym’s retained instances and automatic filtering using independent task-specific annotations and adjudication. The sampled retained set largely passes quality checks, and judges show strong agreement with adjudicated human labels.

A.4 HUMAN VALIDATION OF MEMORY-GYM

  • Two annotators independently apply task-specific criteria while blinded to automatic decisions and each other’s labels; a third adjudicates disagreements.Quality checks include evidence support, query leakage, and task consistency, with uniqueness or summarization criteria added as applicable.Source: §A.4 · p. 16
  • All three judges achieve over 90% agreement and κ > 0.80 with adjudicated human labels on 200 pre-filter candidates.Judges’ acceptance rates are slightly below the human reference, indicating a moderately conservative tendency on this sample.Source: §A.4 · p. 16
  • 190 of 200 retained instances pass all applicable criteria, a 95.0% pass rate (95% Wilson CI: 91.0%–97.3%).Criterion-level pass rates range from 95.5% to 99.5%.Source: §A.4 · p. 16, Table 6 · p. 17
  • Of 100 sampled rejected candidates, 96 are confirmed correctly rejected after adjudication, while four are judged valid.The 4.0% figure applies to valid instances in this sampled rejected pool, not to all valid candidates rejected by the pipeline.Source: §A.4 · p. 16, Table 7 · p. 17
  • Together, the audits provide human evidence for retained-instance quality and automatic-filtering reliability on the evaluated samples.The audit plan includes pre-filter agreement, a retained-instance quality audit, and error analysis of rejected candidates.Source: §A.4 · pp. 16–17

B.1 IMPLEMENTATION DETAILS

The implementation specifies the Researcher’s training and inference settings, while the LongCodeQA evaluation compares untrained, SFT, and SFT-plus-GRPO checkpoints without code-specific training data. The supplied passages describe evaluation measures and ablation setup but do not report their outcomes.

B.1 IMPLEMENTATION DETAILS

  • The SFT Researcher is trained on verified trajectories generated with Qwen3.5-122B-A10B and filtered with Gemini 3 Flash; a typical run takes about five hours on eight H100 GPUs.SFT uses the standard next-token prediction objective.Source: §B.1 · p. 17
  • At inference, the Researcher uses at most 20 action rounds, merges and deduplicates top-5 BM25 and top-5 BGE-M3 candidates, and uses an untrained Qwen3.5-4B browse and answer backbone.The browse and answer models are not updated during SFT or RL.Source: §B.1 · p. 17
  • Hint-guided GRPO starts from the SFT checkpoint, samples eight trajectories per query, and trains for 200 steps; hints are removed at test time.The typical GRPO run takes about six hours on the same hardware configuration.Source: §B.1 · p. 17
  • The LoCoMo efficiency comparison reports average token consumption per task and overall F1, while the workspace ablation removes memos, hierarchy, and README navigation but keeps raw histories and other settings fixed.The cited captions specify comparison scope and conditions but provide no result values.Source: Figure 5 · p. 18, Table 8 · p. 18

B.2 LONGCODEQA: CROSS-DOMAIN EVALUATION

  • LongCodeQA transfer is evaluated on 233 queries across 64K, 128K, and 256K subsets, with accuracy per subset and an overall score across all queries.Memory-Gym contains no code repositories or code-specific tasks, and LongCodeQA examples are not used for training or adaptation.Source: §B.2 · p. 18

B.3 EFFICIENCY EVALUATION DETAILS

The efficiency, runtime, and variability evaluations show how workspace construction trades one-time cost for lower per-query work, and how the Researcher behaves under its exploration budget. Across the four benchmarks, most queries terminate quickly and results remain stable across inference seeds.

B.3 EFFICIENCY EVALUATION DETAILS

  • Efficiency accounting separates one-time offline workspace construction from per-query online serving, measuring both wall-clock time and token consumption.The 20-round action budget bounds exploration; when it is exhausted without positive sufficiency, JAM still finalizes context from evidence gathered so far.Source: §B.3 · p. 18, §B.4 · p. 19
  • JAM achieves the highest LoCoMo F1 while using substantially fewer tokens per task than MemAgent; one-shot memory systems use fewer tokens but deliver substantially lower answer quality.Offline construction cost is amortized over queries reusing the workspace in the per-task token comparison.Source: §B.3 · p. 18
  • LoCoMo F1 rises from 49.06 to 52.09 as online token use, latency, and average research rounds fall with the JAM Workspace.The workspace costs 36,924.51 tokens and 79.53 seconds to construct once; hierarchical organization reduces online exploration over the same histories.Source: §B.3 · p. 19

B.4 RESEARCHER RUNTIME AND TERMINATION

  • At least 82.80% of queries terminate within five rounds on LoCoMo, LongMemEval, and NarrativeQA, while HotpotQA has 43.83% within five rounds and a P90 of 14.Budget exhaustion ranges from 0% to 4.70%, so the 20-round setting primarily provides exploration headroom rather than fixed inference cost.Source: §B.4 · p. 19

B.5 INFERENCE-TIME VARIABILITY

  • Across the four benchmarks, inference-time sample standard deviations range from 0.31 to 1.34 points and remain below 1.5 points.The evaluation repeats inference with three seeds while holding the trained checkpoint, workspace, prompts, retrieval configuration, and action budget fixed.Source: §B.5 · p. 19

B.6 LLM-AS-A-JUDGE EVALUATION

The paper evaluates answer quality with an LLM judge on three benchmarks, using a shared judging protocol for the compared methods. JAM achieves the highest accuracy among methods assessed under that protocol on all three.

B.6 LLM-AS-A-JUDGE EVALUATION

  • GPT-4o-mini at temperature 0 judges each reference-answer and system-prediction pair as CORRECT or WRONG, with accuracy defined as the proportion judged CORRECT.The evaluation uses the Memory-R1 prompt on LoCoMo and adapts only the task description for NarrativeQA and HotpotQA; LongMemEval is not re-evaluated because its main metric is accuracy.Source: §B.6 · p. 20
  • Memory-R1-PPO and Memory-R1-GRPO results are reproduced from their original paper for reference, not as directly comparable results under this evaluation pipeline.Source: §B.6 · p. 20
  • JAM improves accuracy over the strongest baseline evaluated under the same judging protocol by 12.54, 20.00, and 11.20 percentage points on LoCoMo, NarrativeQA, and HotpotQA, respectively.JAM has the highest judged accuracy among methods evaluated under the shared protocol on all three benchmarks.Source: §B.6 · p. 20

B.7 MEMORIZER BACKBONE SENSITIVITY

The backbone sensitivity test finds no consistent downstream improvement from replacing the default Memorizer, while a NarrativeQA case illustrates the Researcher’s coarse-to-fine evidence search. The prompt specifies the Researcher’s role, tools, interaction loop, and source-grounded answer format.

B.7 MEMORIZER BACKBONE SENSITIVITY

  • Qwen3.5-122B-A10B changes LoCoMo F1 from 52.09 to 51.30, while GPT-5.5 raises it modestly to 52.62; both alternatives cost more to construct.Online latency and average exploration rounds change only slightly, and the default Qwen3.5-4B offers a favorable balance in the evaluated setting.Source: §B.7 · pp. 20–21, Table 12 · p. 21

C CASE STUDY

  • The NarrativeQA workspace has film script and source materials branches, preserving raw evidence while exposing paths for coarse- and fine-grained navigation.The Memorizer constructs this structured workspace from the narrative materials.Source: C CASE STUDY · p. 21
  • The Researcher uses complementary search, open, and browse calls, then focuses on the screenplay to locate supporting evidence and return an answer with source provenance.The trajectory narrows the search space from global workspace inspection to focused browsing.Source: C CASE STUDY · p. 21

D PROMPT

  • The Researcher prompt specifies its role, available tools, think–tool-use loop, and required source-grounded answer format.Source: D PROMPT · p. 21
Loading 2609.34385v1…