PaperlayerView exact revision on arXiv

Source-linked AI summary

SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent

Yuyang Hu, Hongjin Qian, Shuting Wang, Jiongnan Liu, Ziliang Zhao, Jiejun Tan, Zheng Liu, Zhicheng Dou

arXiv:2605.24468v1cs.AI

TL;DR

Long-horizon agents must recover scattered past information as their decision state evolves. SAM uses compact memory cues with preserved trajectory pages for intent-driven recall, and consistently outperforms strong baselines across four benchmarks and diverse backbones.

  • Problem

    Long-horizon agents need state-adaptive access to scattered past information because interaction histories mix thoughts, tool calls, observations, and partial conclusions.

  • Method

    SAM consolidates trajectories into visible memory cues while preserving raw pages for intent-driven recall, optimizing the standalone memory module with expert supervision and reinforcement learning.

  • Results

    SAM consistently outperforms strong context-management baselines across BrowseComp, BrowseComp-ZH, WideSearch, and HLE on diverse agent backbones.

  • Takeaways & Limitations

    Explicit memory modeling provides a simple, general, and effective foundation for improving long-horizon agentic reasoning.

  • Takeaways & Limitations

    The empirical study is limited to web-based and knowledge-intensive benchmarks, leaving generalization to software engineering, embodied environments, and long-form generation untested.

Abstract

from arXiv · show

Long-horizon agentic reasoning requires large language models to act over long interaction histories containing thoughts, tool calls, observations, and partial conclusions. The challenge is not merely that these histories grow long, but that information needed for the current decision may be scattered across distant steps and only become relevant later. Existing approaches address this difficulty by truncating the interaction history, compressing it into shorter surrogates, or retrieving selected parts of it for reuse, but they do not explicitly model how access to past interaction should adapt to the agent's evolving state. We instead cast long-horizon reasoning as a problem of state-adaptive memory. To this end, we propose State-Adaptive Memory~(SAM), a standalone framework that consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall. These cues are not treated as replacements for history; rather, they serve as lightweight handles that allow the agent to reconstruct temporally distant information according to its current needs, without retraining the underlying backbone. We further optimize the memory module through expert-guided supervision and reinforcement learning, aligning it with trajectory-level utility. Across BrowseComp, BrowseComp-ZH, WideSearch, and HLE, SAM consistently outperforms strong baselines over diverse agent backbones. Our results suggest that explicit memory modeling provides a simple and effective foundation for long-horizon agentic reasoning.

1 Introduction

SAM frames long-horizon agentic reasoning as state-adaptive memory: compact cues stay visible while raw trajectory pages remain available for intent-driven recall. It independently optimizes this memory capability and consistently outperforms strong baselines across four long-horizon benchmarks and diverse agent backbones.

  • Motivation: Long-horizon agents must continually gather evidence, track progress, and choose actions from growing histories that interleave thoughts, tool calls, observations, and partial conclusions.The interaction history becomes long and heterogeneous as tasks extend over time.
  • Motivation: Existing context-management strategies discard history, compress earlier steps into summaries, or retrieve selected content, but can fail when relevant information is distant or inadequately preserved.These approaches work best when needed information remains recent or survives compression.
  • Motivation: SAM casts the challenge as state-adaptive memory, maintaining a coherent view of established, resolved, and next-step information as the agent’s decision state changes.The framework treats past information as unequally active rather than uniformly retained.
  • Method: SAM consolidates interaction into compact memory cues visible in context and raw trajectory pages preserved outside the live window for deeper, intent-driven recall.The cues act as lightweight summaries and entry points rather than replacements for the underlying trajectory.
  • Method: SAM optimizes memory as an independent capability whose value depends on future use, later-emerging importance, recoverability, and changing decision states.The memory module is optimized separately from a particular agent backbone.
  • Results: SAM is evaluated on BrowseComp, BrowseComp-ZH, WideSearch, and HLE, consistently outperforming strong baselines across diverse agent backbones.The results support explicit memory modeling as a foundation for long-horizon reasoning.

2 Method

SAM frames long-horizon context management as recovering compact, state-adaptive decision support rather than carrying or replacing the full interaction history. It consolidates trajectories into cue-linked raw pages and reconstructs intent-conditioned support when needed.

  • Problem formulation: Long interaction histories become difficult to use because relevant information is scattered across heterogeneous, temporally distant interactions.The trajectory may include thoughts, tool arguments, tool responses, and partial conclusions.
  • Problem formulation: SAM defines the needed decision state by what has been established, resolved, and remains to be done, without assuming a recent-window solution.The state is conceptual: the goal is to recover support useful for the current decision rather than explicitly estimate a latent state.
  • Page-based episodic consolidation: SAM partitions recent interaction into contiguous token-budgeted pages, generates a compact cue for each page, and stores the raw page externally.Page-based consolidation preserves local temporal coherence while avoiding explicit semantic segmentation.
  • Page-based episodic consolidation: Cues are navigable handles rather than irreversible summaries: they identify candidate pages whose underlying trajectory content remains recoverable.This preserves raw history for later reconstruction instead of flattening it into a single surrogate.
  • Intent-driven episodic recall: Given a recall intent, the agent selects candidate cues, revisits their raw pages, and injects reconstructed decision-relevant content into the active context.Recall is conditioned on the current intent, so the system reconstructs focused support rather than replaying raw history verbatim.
  • Adaptive access and training: SAM is state-adaptive at read time: cue selection, page revisitation, and information reconstruction depend on the agent’s current intent.The memory module is trained first with expert traces and then refined with reinforcement learning.

3 Experiments

SAM is evaluated on four long-horizon benchmarks using two heterogeneous agent backbones under a controlled inference protocol. It is the strongest context-management method across backbones and benchmark cells, with the largest gains on BrowseComp and BrowseComp-ZH.

  • Evaluation benchmarks: Evaluation covers BrowseComp, BrowseComp-ZH, WideSearch, and HLE, spanning web browsing, cross-lingual search, broad exploration, and scientific reasoning.For efficiency, the study randomly samples 200 questions per benchmark.
  • Models: SAM uses proprietary GLM-4.7 and open-source Qwen3.5-35B-A3B agent backbones, while a shared Qwen3.5-9B memory model handles page consolidation and intent-driven recall.Only the memory model is updated during supervised fine-tuning and reinforcement learning.
  • Inference protocol: All context-management methods use a 128K context window, trigger management at 64K, share fixed decoding settings, obey a per-query round cap, and report avg@3.This protocol is intended to reduce sampling variance and control comparisons across methods.
  • Main results: SAM is the strongest context-management method on every backbone, outperforming the best heuristic and no-management baseline on the four-benchmark average.Its lead is largest on BrowseComp and BrowseComp-ZH, where memory demands are greatest and heuristic strategies are weakest.
  • Main results: A single Qwen3.5-9B memory model trained once achieves the best score in every evaluated backbone–benchmark cell across both heterogeneous backbones.The result spans BrowseComp, BrowseComp-ZH, WideSearch, and HLE.
  • Main results: Heuristic baselines change relative ranking across benchmarks, whereas SAM’s gains remain consistent across tasks and backbones.The passage contrasts summary leading on BrowseComp with discard-tool leading on BrowseComp-ZH under GLM.

4 Discussions

The discussion attributes SAM’s gains to both optimization and intent-driven recall, with benefits that persist across longer trajectories and a broad range of memory page sizes. Its advantage weakens only when oversized pages dilute per-page intent signals.

  • Optimization and transfer: Removing either SFT or OAT-GRPO consistently reduces performance across every benchmark, while SAM also transfers to a LoRA-fine-tuned 27B memory backbone.The ablation uses GLM-4.7 as the frozen agent backbone under the §3.2 inference protocol.
  • Read-side design: Summary-only, recency, and raw-content variants all trail full SAM, showing that intent-conditioned per-page recall matters beyond rolling summaries or raw-page access.Recency is barely above the no-memory baseline, and returning raw pages does not close the gap.
  • Trigger-time behavior: SAM tops every trigger-time metric, and its accuracy lift exceeds its confidence lift, indicating more decisive and correct behavior rather than merely increased activity.The comparison measures tool-call count, confidence, and accuracy when context management is triggered.
  • Long-horizon behavior: SAM remains above every baseline in BrowseComp round buckets 21–40, 41–80, and >80, with its advantage over summary-only memory persisting beyond 80 rounds.The gap grows as trajectories lengthen, while summary-only memory begins to lose useful state.
  • Page-size robustness: SAM beats the no-memory baseline at all small-to-medium page sizes on both benchmarks, with the only failure at 128K on BrowseComp.The optimum is 32K–64K; at 128K, pages dilute per-page intent signals.

5 Related Work

Existing long-horizon agent memory work mainly manages active context through learned context-editing actions and lossy surrogates that replace history with compressed representations. These approaches frame context maintenance as policy skill or history summarization, pruning, and folding.

  • Memory and context management for long-horizon agents: Action-space approaches let agents invoke context-editing operations during reasoning, making context maintenance a learned policy skill.The passage cites multiple works in this line, including [14] [18] [19].
  • Memory and context management for long-horizon agents: Lossy-surrogate approaches replace interaction history with summaries produced by prompted or trained folding operators.The passage identifies prompted folding [15] and trained folding operators [29] [41] [25].
  • Memory and context management for long-horizon agents: Other lossy-surrogate methods prune history at inference time or train compact internal representations with reinforcement learning.The passage explicitly mentions inference-time pruning and RL-trained compact internal state.

6 Conclusion

SAM is a standalone memory framework that adapts access to past interaction by combining compact memory cues with preserved raw trajectory pages. It enables intent-driven recovery of information from arbitrary trajectory stages without retraining the backbone, supporting long-horizon agentic reasoning.

  • Framework design: SAM consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall.The framework targets access to the right past information for the agent’s current decision state as trajectories grow.
  • Framework capability: SAM enables agents to recover information from arbitrary trajectory stages without retraining the underlying backbone.This capability follows from combining memory cues with preserved raw trajectory pages.
  • Empirical conclusion: Experiments across multiple long-horizon benchmarks show that explicit memory modeling provides a simple, general, and effective foundation for improving agentic reasoning.The conclusion positions SAM as a step toward modular memory systems for increasingly capable agents.

A Limitations and Broader Impact

SAM’s evaluation is limited in model coverage, task domains, and memory-training configurations, while its reliance on frontier-model supervision adds cost and potential bias. The explicit memory interface may benefit knowledge-intensive applications but could also enable sensitive surveillance, profiling, or harmful coordination.

  • Limitations: SAM’s experiments cover only a limited set of agent backbones and memory configurations.They include a 9B model with full-parameter SFT followed by OAT-GRPO and a 27B model with LoRA-only SFT; larger fully optimized models, mixture-of-experts backbones, and broader families remain unexplored.
  • Limitations: SAM’s empirical evaluation centers on web-based and knowledge-intensive benchmarks rather than software engineering, embodied environments, or long-form generation.Additional evaluation is needed to determine how broadly the current design generalizes.
  • Limitations: Frontier-model expert supervision and committee-based reward approximation impose substantial computational costs and may transmit expert or assessor biases.The paper calls for more efficient and stable alternatives for memory supervision and reward design.
  • Broader Impact: SAM shifts long-horizon context management toward an explicit interface for state-conditioned recovery of earlier interactions, with potential benefits for knowledge-intensive applications.Examples include scientific research assistants, open-domain analysis, and software engineering agents.
  • Broader Impact: Stronger long-horizon memory could also facilitate surveillance, profiling, and long-horizon coordination of harmful tasks.The training pipeline’s reliance on frontier LLMs as expert references may additionally introduce hidden biases into the learned memory.

B Implementation Details · B.1 Inference Protocol

SAM runs as an external memory module alongside the agent, consolidating over-budget trajectory chunks into persistent cues while storing raw pages for later intent-conditioned recall. The inference protocol keeps context-management methods under an identical stack and deploys the backbone and memory components as separate endpoints.

  • B Implementation Details: SAM runs alongside the main agent as an external memory module during interaction.Recent trajectory content accumulates in the agent’s live context.
  • B Implementation Details: When content exceeds a predefined token budget, SAM consolidates the corresponding chunk into a page-level memory cue.The raw chunk is removed from active context after consolidation.
  • B Implementation Details: SAM stores each removed raw page externally while keeping its resulting cue visible through the persistent memory interface.The cue is retained as a lightweight handle rather than replacing the externally stored raw page.
  • B Implementation Details: Each page records a contiguous trajectory segment containing the agent’s tool calls and corresponding tool responses.Pages therefore preserve interaction segments spanning both actions and observations.
  • B Implementation Details: SAM performs consolidation into continuation-oriented summaries and recalls selected pages according to the agent’s current intent.During recall, candidate pages are processed sequentially and relevant information is incrementally integrated into a focused summary.
  • B.1 Inference Protocol: All context-management methods share an identical inference stack, making the memory mechanism the only varying factor.This includes SAM and heuristic baselines.
  • B.1 Inference Protocol: The agent backbone, SAM memory model, and auxiliary model are served through separate OpenAI-compatible endpoints connected by an orchestration script.The orchestration script is released as part of the code.
  • B.1 Inference Protocol: The active context window is 128K tokens.The supplied implementation passage specifies this context-window size for inference.

B.2 Optimization · B.3 Supervised Fine-Tuning Configuration · B.4 Reinforcement-Learning Configuration

SAM’s memory model is trained first with expert-derived supervised traces and then optimized with end-to-end GRPO while the reasoning backbone remains frozen. The configurations specify separate SFT variants, recoverability-oriented rewards, tree-based rollouts, distributed training, and detailed reproducibility procedures.

  • B.2 Optimization: SAM uses expert traces from Claude-4.5-Opus and GPT-5.4 to supervise Qwen3.5-9B, then applies end-to-end GRPO with the agent backbone frozen.The supervised targets cover both consolidation and recall, while reinforcement learning operates in the full agent-environment loop.
  • B.2 Optimization: The reinforcement-learning reward combines final task performance with recoverability estimated from GPT-5.4, GLM-4.7, and DeepSeek-V4-Flash references.GPT-5.4 assesses each recall event using a single 0–10 score rescaled to [0, 1].
  • B.3 Supervised Fine-Tuning Configuration: SFT data comprise paired expert consolidation and recall traces curated from OpenSeeker and OpenResearcher, with 200 examples reserved for fixed evaluation.Training uses the ms-swift trainer and produces two memory-model variants from the same data.
  • B.3 Supervised Fine-Tuning Configuration: Qwen3.5-9B full-parameter SFT runs for 2 epochs on 8 GPUs with effective batch size 64, learning rate 1e−5, and 100K-token sequences.The resulting checkpoint initializes the OAT-GRPO stage and is used for the main results.
  • B.3 Supervised Fine-Tuning Configuration: Qwen3.5-27B is additionally LoRA-fine-tuned for 1.5 epochs on 8 GPUs with effective batch size 64, learning rate 9e−5, and LoRA rank 8.This variant supports results using the larger backbone.
  • B.4 Reinforcement-Learning Configuration: OAT-GRPO trains Qwen3.5-9B from the full-SFT checkpoint using slime with a Megatron-LM backend and a frozen external reasoning service.The reinforcement-learning optimization uses AdamW at learning rate 1e−6, clipped-surrogate ε = 0.2, and zero KL coefficients.
  • B.4 Reinforcement-Learning Configuration: Each rollout iteration samples 6 prompts, expands memory-call trees with branch factor b = 3 to depth 3, and permits at most 8 tool turns per branch.This yields up to 27 leaves per prompt, with one page per memory call and 32K-token page chunks.
  • B.4 Reinforcement-Learning Configuration: The reward weights tree-attributed outcomes at α = 0.3 and oracle-anchored recoverability at 1−α = 0.7, centered by brec = 0.50.Training uses 8 H100-class GPUs with tensor parallelism 2, context parallelism 4, and per-iteration trace dumps for inspection.

C Benchmark Details · D Open-Source Agent-System Baselines · E Case Studies

The paper contextualizes baseline results by distinguishing reported open-source systems from controlled SAM comparisons, then uses matched BrowseComp trajectories to illustrate SAM’s behavior. The baselines differ in whether they rely on context, pinned plans, summaries, workspace reconstruction, or learned folding.

  • D Open-Source Agent-System Baselines: Published open-source baseline numbers are taken directly from original papers rather than reproduced, preserving their reported toolsets, prompts, and decoding configurations.The accompanying descriptions focus on each system’s memory or workspace mechanism relative to vanilla ReAct.
  • D Open-Source Agent-System Baselines: WebThinker uses interleaved browsing and chain-of-thought refinement without a separate memory module, relying on context-window capacity and prompt-level rolling state.
  • D Open-Source Agent-System Baselines: WebSailor anchors multi-hop browsing with explicit subgoal tracking, pinning intermediate plans in context alongside tool-augmented planning.
  • D Open-Source Agent-System Baselines: ReSum replaces raw history with structured summaries of completed sub-trajectories, while IterResearcher reconstructs a workspace each round and retains only active research state.ReSum retrieves only the latest summary; IterResearcher drops completed sub-tasks.
  • D Open-Source Agent-System Baselines: AgentFold learns end-to-end to fold earlier turns into compact summaries and removes the underlying raw trajectory once folding is committed.
  • D Open-Source Agent-System Baselines: SAM’s context-management baselines share its backbone and tool stack and use an identical inference protocol, isolating the memory mechanism’s effect.The variants are w/o CM, discard-tool, recent-k, and summary.
  • E Case Studies: The case studies pair one successful and one failed BrowseComp trajectory from a SAM-equipped Qwen3.5-35B-A3B agent under matched conditions.Both runs use the same backbone, tools, and a 128K context window with a 64K trigger, isolating SAM usage from base-model choice.

E.1 Success: Multi-Constraint Search Closed by Intent-Driven Recall (id=567)

Box E.1 shows SAM closing a seven-constraint author-identification search after 65 rounds by consolidating exploration into two memory pages and later issuing an intent-driven recall. The recall fused new and prior evidence to produce the gold answer, while preserving failed candidates and exposing unresolved uncertainties.

  • Search closure: After 65 rounds, SAM consolidated early exploration into two memory pages and a goal-conditioned recall closed the answer.The run used 4 memory-tool calls; the gold and model answer was Leopold von Ranke.
  • Memory representation: The memory pages stored a failed-candidate ledger with violated constraints and an explicit open blocker, rather than a tool-call dump.This structure preserved why candidates such as Cornelia Funke and the Mann family were rejected after the live context exceeded 64K tokens.
  • Intent-driven recall: Intent-driven recall combined a new Robert Graves lead with sibling-count, von-Ranke-lineage, and Mann-family-rejection evidence.The recall goal targeted Graves’s biography, historian connection, poem, novels, and sequel narrator.
  • Evidence fusion: The recalled evidence matched the poem’s rhyme pairs and connected Robert Graves to Leopold von Ranke through his mother Julia von Ranke.It also returned I, Claudius, its sequel, and Derek Jacobi’s narration and portrayal of Claudius.
  • Memory benefits: SAM’s recall preserved state beyond the 64K boundary, avoided re-litigating rejected candidates, and returned only goal-relevant cross-evidence instead of the entire prefix.The recalled result explicitly left the exact Graves–von Ranke relationship and the poem title, “Fawn” or “Faun,” uncertain.

E.2 Failure: Memory Amplifies a Wrong Frame (id=1058) · F Prompt Templates · G Use of LLMs in Writing

The failure case shows that SAM can faithfully preserve and recall an agent’s wrong frame without correcting upstream anchoring or ignored caveats. The appendix specifies consolidation and recall prompts, and reports that LLMs were used in manuscript preparation only for language polishing.

  • E.2 Failure: Memory Amplifies a Wrong Frame (id=1058): SAM produced a coherent but incorrect answer because it faithfully preserved the agent’s wrong frame.On id=1058, the model answered Home Game; Calcio Storico instead of Untold: Malice at the Palace, with 38 rounds, 1 memory page, and 3 memory-tool calls.
  • E.2 Failure: Memory Amplifies a Wrong Frame (id=1058): Two early anchors—The Darling Buds of May and Home Game—locked subsequent searches and memory calls into a globally wrong frame.The correct chain instead ran through Untold: Malice at the Palace → Brocker Way → Kurt Russell → The Travels of Jaimie McPheeters.
  • E.2 Failure: Memory Amplifies a Wrong Frame (id=1058): The recall query presupposed Home Game and could not introduce Untold because that alternative had never been explored in the run.SAM reorganized evidence around the requested anchors and surfaced a potential contradiction, but did not broaden the search.
  • E.2 Failure: Memory Amplifies a Wrong Frame (id=1058): The failure was upstream rather than a memory hallucination: the agent chose the wrong frame, posed a frame-bound goal, and under-weighted SAM’s uncertainty markers.The authors propose caveat-aware blockers and a frame-broadening recall mode as possible extensions.
  • F.1 Prompt for Memory Consolidation: The consolidation prompt asks the memory manager to produce a concise, factual continuation summary covering goals, established facts, tools, partial conclusions, and open questions.It also prioritizes continuation value, distinguishes confirmed findings from tentative inferences, and preserves relevant entities, dates, URLs, and blockers.
  • F.2 Prompt for Intent-Driven Recall: The intent-driven recall prompt extracts information relevant to a research goal while preserving key prior results, adding new findings, and distinguishing confirmed from incomplete information.Its templates combine the research goal, previous extracted results, and current page into an updated summary.
  • G Use of LLMs in Writing: During manuscript preparation, LLMs were used only for language polishing, including proofreading, grammar correction, and minor rewording of author-written text.The passage excludes their use for research ideas, experiment design, result derivation, or technical content generation.

NeurIPS Paper Checklist

The checklist finds that the paper’s abstract and introduction accurately state its scope and contributions, while noting no formal theoretical results and missing statistical significance reporting. It also documents broader-impact discussion and explains why dedicated safeguards are not required for SAM’s released artifacts.

  • Claims: The abstract and introduction accurately reflect the paper’s scope and contributions, including page-based consolidation, intent-driven recall, OAT-GRPO, and consistent gains over baselines.The checklist identifies context management for long-horizon agents as the scope and names the state-adaptive memory mechanism and training recipe.
  • Theory assumptions and proofs: The paper has no formal theoretical results, so the theory-assumptions-and-proofs checklist item is marked NA.OAT-GRPO is presented as an algorithmic recipe rather than a theorem requiring proof.
  • Experiment statistical significance: Every reported accuracy averages three independent rollouts per query, but the current draft reports neither explicit error bars nor significance tests.The checklist therefore marks the experiment-statistical-significance item No.
  • Societal impacts: Broader impacts and limitations are discussed in Appendix A, including dual-use concerns around stronger long-horizon agent memory and frontier-LLM expert references.The discussion addresses both potential positive and negative societal impacts.
  • Responsible release: SAM does not release pretrained generative models, scraped datasets, or other high-misuse-risk artifacts requiring dedicated safeguards.SAM is described as a context-management module operating over text trajectories on public benchmarks.
Loading 2605.24468v1…