Source-linked AI summary
Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents
Yifan Wu, Lizhu Zhang, Yuhang Zhou, Mingyi Wang, Bo Peng, Serena Li, Xiangjun Fan, Zhuokai Zhao
TL;DR
Long-horizon agents can lose decision-relevant execution state as trajectories expand, leaving requirements, diagnoses, and subgoals unavailable when needed. The paper introduces a proactive memory agent that selectively injects reminders, improving task success across Terminal-Bench 2.0 and τ^2-Bench.
Problem
Long-horizon agents need to preserve execution state, including requirements, environment facts, prior attempts, diagnoses, and open subgoals, as trajectories expand.
Method
A separate memory agent maintains structured execution state and selectively reintroduces grounded reminders into the action loop when they may affect the next decision.
Results
Memory intervention consistently improves task success across Terminal-Bench 2.0 and τ^2-Bench, including 37.6% →45.9% on Terminal-Bench for Claude Sonnet 4.5.
Takeaways & Limitations
Ablations favor maintained memory with selective intervention over passive exposure and general retrieval, while preliminary results show partial transfer from an open-weight memory policy.
Takeaways & Limitations
Prompted memory agents add inference cost and their intervention decisions may be imperfectly calibrated.
Abstract
from arXiv · showhide
In long-horizon tasks, decision-relevant state is often scattered across an expanding trajectory, while the action agent must surface it and act. As trajectories grow, task requirements, environment facts, prior attempts, diagnoses, and open subgoals can be buried in the context window or pushed beyond it, failing to influence decisions when needed. We call this failure mode "behavioral state decay". We study memory as an active intervention mechanism rather than passive retrieval. A separate memory agent runs alongside an unmodified action agent, updating a structured memory bank from the recent trajectory and deciding whether to inject a memory-grounded reminder or remain silent. The module is plug-and-play with frontier action agents and existing agent harnesses. Across Terminal-Bench 2.0 and $τ^2$-Bench, it improves pass@1 for both weaker and stronger action agents, with gains of +8.3 pp on Terminal-Bench and +6.8 pp on $τ^2$-Bench. Ablations show that selective intervention outperforms passive bank exposure, always-on injection, advisor-only guidance, and general retrieval. As an early step toward open-weight memory policies, we train Qwen3.5-27B on SETA using SFT and GRPO, improving validation reward and achieving partial transfer to Terminal-Bench.
1 Introduction
The introduction identifies behavioral state decay as a central failure mode in long-horizon agents and frames effective memory as selective intervention in the action loop, not merely storage or retrieval. It presents a two-phase architecture that maintains structured memory and selectively injects reminders, improving benchmark performance and showing preliminary transfer to an open-weight memory policy.
- Problem: Behavioral state decay occurs when task requirements, environment facts, prior attempts, diagnoses, discoveries, or open subgoals stop reliably influencing the next action.The information may remain in the transcript or context window while no longer controlling behavior.
- Motivation: Effective long-horizon memory is an intervention problem: surfacing too little memory permits repeated mistakes, while surfacing too much adds latency, consumes tokens, and distracts from local progress.The introduction contrasts selective intervention with conventional storage, updating, and retrieval.
- Architecture: The proposed memory agent periodically updates a structured memory bank and decides whether to inject a concise reminder into the next action-agent call or remain silent.It runs alongside an unmodified action agent and can remind it of requirements, environment facts, failed attempts, or diagnoses.
- Results: 37.6% →45.9% and 55.0% →61.8%: Claude Opus 4.6 memory intervention improves Claude Sonnet 4.5 on Terminal-Bench and τ 2-Bench, respectively.Claude Opus 4.6 as the memory agent improves performance across autonomous command-line execution and interactive tool use.
- Results: +2.4 pp and +2.5 pp: Claude Opus 4.6 as the action agent still gains on Terminal-Bench and τ 2-Bench, while ablations favor selective intervention over passive exposure and general retrieval.Qwen3.5-27B memory agents trained with SFT and GRPO also show gains transferring to held-out Terminal-Bench.
2 Related Work
Prior work extends agents with memory, retrieval, context management, reflection, and auxiliary guidance, but long-horizon execution still suffers from behavioral state decay. This work distinguishes itself by treating memory as a selective intervention policy that decides whether remembered execution state should enter the action agent’s context.
- Behavioral state decay: Long-horizon agents can repeat failed actions, lose requirements, or drift from earlier discoveries in realistic multi-step environments.These failures motivate studying memory as a way to maintain decision-relevant execution state during trajectories.
- Memory and retrieval: External-memory research spans non-parametric retrieval, long-context representation caching, reusable skill stores, and persistent user, session, or agent memory.These approaches primarily improve factual access, effective context, experience reuse, or persistent storage and retrieval.
- Memory and retrieval: This work asks when remembered execution state should be reactivated as an intervention, rather than optimizing persistent memory storage or retrieval alone.The distinction is memory reactivation within an ongoing loop, not memory infrastructure by itself.
- Context management: Context-management methods optimize memory editing or curation for downstream outcomes, whereas this work uses a separate memory agent instead of training the action agent to curate its own context.It also differs from optimizing memory construction for question answering or summarizing completed sub-tasks.
- Feedback and guidance: Reflection, episodic memory, search, and advisor-style methods improve agents through feedback, stored reflections, auxiliary reasoning, or second-model guidance.The contribution reframes these concerns around selective intervention during long-horizon execution.
- Selective intervention: The central control problem is whether, when, and how remembered execution state enters context to change the next decision without unnecessary token or latency overhead.This formulation goes beyond deciding only what information to remember or summarize.
3 Method
The method adds a separate memory agent beside an unchanged action agent to maintain structured execution state and selectively reintroduce relevant reminders during long-horizon tasks. It manages a memory bank in two phases—bank updates followed by intervention selection—while allowing prompted or trained memory-agent implementations.
- Architecture: A separate memory agent observes recent trajectory context, updates a structured memory bank, and optionally injects a concise reminder into the unchanged action agent’s next call.The memory agent runs beside the action agent and treats memory as an intervention policy rather than passive context exposure.
- Structured memory: The memory bank preserves execution state that may disappear from action context, including requirements, environment facts, prior attempts, failure diagnoses, discoveries, and open subgoals.Its components are private status, stable knowledge memories, and procedural memories recording attempts and outcomes.
- Two-phase workflow: Each memory step first performs explicit bank edits through predefined calls, then selects either a targeted reminder or the null intervention for the next action decision.Phase 1 may update status, save knowledge, save procedural experience, or delete entries; Phase 2 does not modify the bank.
- Selective intervention: The null intervention is an explicit choice, and the memory agent is encouraged to intervene only when remembered state is likely to affect the next action.Examples include imminent requirement violations, relevant diagnoses, non-repeatable failed attempts, and neglected open subgoals.
- Implementation: The architecture requires no new model training in its main instantiation, where the memory agent can be a prompted model following the two-phase interface.The paper also explores training an open-weight memory agent because prompted agents add inference cost and may be imperfectly calibrated.
4 Experiments and Results
Across Terminal-Bench 2.0 and τ 2-Bench, selective memory intervention improves pass@1 for weaker and stronger action agents. Ablations and qualitative analyses indicate that benefits come from timely, specific reminders grounded in maintained execution state, while training results show partial transfer to Terminal-Bench.
- Main results: +8.3 pp on Terminal-Bench 2.0 and +6.8 pp on τ 2-Bench are the gains for Sonnet 4.5 with memory intervention.Opus 4.6 also improves by +2.4 pp and +2.5 pp, respectively, so gains persist for the stronger action agent.
- Ablations: The full two-phase memory agent achieves the highest macro-average, improving all three τ 2-Bench domains and producing the largest airline gain.Full-bank context trails the full system by 2.8 macro points and 2.6 micro points, while always-inject leads micro-average by 0.3 points within expected run variance.
- Ablations: Selective intervention is necessary beyond persistent retrieval: Mem0 improves average score but does not improve airline over baseline and falls short of the full system on macro-average.The full design combines maintained execution-state memory with a policy that chooses what to say and when to remain silent.
- Mechanisms: Memory helps by reactivating specific execution state when it is about to be ignored, with Terminal-Bench emphasizing debugging continuity and τ 2-Bench emphasizing policy and interaction state.Successful interventions are specific, grounded, and timely, often occurring before state-changing tool calls; remaining failures primarily involve calibration errors.
- Open-weight memory policy: 0.734 SETA reward is achieved after GRPO, up from 0.720 with SFT and 0.693 for the untrained 27B memory agent.On held-out Terminal-Bench 2.0, the SETA-trained memory agent improves the frozen action agent from 37.6% to 41.1% pass@1.
5 Conclusion
The paper frames agent memory as a proactive intervention policy that counters behavioral state decay by preserving decision-relevant execution state and injecting grounded context when needed. Across Terminal-Bench 2.0 and τ 2-Bench, this approach improves task success across different action-agent models.
- Agent memory is presented as a proactive intervention policy rather than passive storage and retrieval.
- Behavioral state decay occurs when execution state that should guide future actions stops influencing behavior.
- The memory agent maintains execution state and injects grounded context when it is likely to affect the next decision.
- Across Terminal-Bench 2.0 and τ 2-Bench, the memory agent consistently improves task success across different action-agent models.