PaperlayerView exact revision on arXiv

Source-linked AI summary

Remember When It Matters: Proactive Memory Agent for Long-Horizon Agents

Yifan Wu, Lizhu Zhang, Yuhang Zhou, Mingyi Wang, Bo Peng, Serena Li, Xiangjun Fan, Zhuokai Zhao

arXiv:2607.08716v1cs.AIcs.CL

TL;DR

As long-horizon trajectories grow, execution state that should guide future actions may no longer remain available in the action context. The paper uses a separate memory agent to maintain that state and selectively reintroduce it. Across Terminal-Bench and τ²-Bench, Claude Sonnet 4.5 improves from 37.6% to 45.9% and from 55.0% to 61.8%, respectively.

  • Problem

    Long-horizon agents need execution state to guide future actions, but that state may no longer remain available in the action context as trajectories grow.

    Source: §3.1 · p. 4

  • Method

    A memory agent maintains execution state outside the action context and selectively reintroduces grounded reminders when they may affect the next decision.

    Source: §3.1 · p. 4, §3.3 · p. 6

  • Results

    Across Terminal-Bench and τ²-Bench, Claude Sonnet 4.5 improves from 37.6% to 45.9% and from 55.0% to 61.8%, respectively.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Training results show that intervention policies can be learned, with SFT teaching memory management and RL improving when remembered state should re-enter the control loop.

    Source: §5 · p. 10

    Prompted memory agents add inference cost, and their intervention decisions may be imperfectly calibrated.

    Source: §3.5 · p. 6

Abstract

from arXiv · show

In long-horizon tasks, decision-relevant state is often scattered across an expanding trajectory, while the action agent must surface it and act. As trajectories grow, task requirements, environment facts, prior attempts, diagnoses, and open subgoals can be buried in the context window or pushed beyond it, failing to influence decisions when needed. We call this failure mode "behavioral state decay". We study memory as an active intervention mechanism rather than passive retrieval. A separate memory agent runs alongside an unmodified action agent, updating a structured memory bank from the recent trajectory and deciding whether to inject a memory-grounded reminder or remain silent. The module is plug-and-play with frontier action agents and existing agent harnesses. Across Terminal-Bench 2.0 and $τ^2$-Bench, it improves pass@1 for both weaker and stronger action agents, with gains of +8.3 pp on Terminal-Bench and +6.8 pp on $τ^2$-Bench. Ablations show that selective intervention outperforms passive bank exposure, always-on injection, advisor-only guidance, and general retrieval. As an early step toward open-weight memory policies, we train Qwen3.5-27B on SETA using SFT and GRPO, improving validation reward and achieving partial transfer to Terminal-Bench.

1 Introduction

The paper frames long-horizon memory as an intervention problem: agents must keep decision-relevant state active, not merely store or retrieve it. It proposes a memory agent that selectively reminds an unmodified action agent, improving benchmark performance and showing preliminary transfer from an open-weight policy.

1 Introduction

  • Behavioral state decay occurs when requirements, environment facts, prior attempts, diagnoses, discoveries, or open subgoals stop reliably shaping later actions, even if they remain in the transcript.Examples include violating an earlier requirement, retrying a failed approach, or treating a previously diagnosed error as new.Source: §1 · p. 1
  • Effective long-horizon memory must decide when to intervene: too little surfacing allows repeated mistakes, while too much adds latency, consumes tokens, and distracts from local progress.This control question differs from summarization, which chooses what to retain but not whether it should affect the next action.Source: §1 · pp. 1–2
  • The proposed memory agent updates a structured bank from recent trajectory at fixed intervals, then injects a concise memory-grounded reminder into the next action-agent call or stays silent.It runs alongside an unmodified action agent and restricts reminders to remembered execution state rather than broad strategic advice.Source: §1 · p. 2
  • 37.6% →45.9% on Terminal-Bench 2.0 and 55.0% →61.8% on τ 2-Bench: Claude Opus 4.6 memory intervention improves Claude Sonnet 4.5.Opus 4.6 as the action agent still gains +2.4 pp and +2.5 pp, respectively; ablations favor proactive intervention over passive exposure and general memory retrieval.Source: §1 · p. 2
  • The authors fine-tune Qwen3.5-27B with SFT and GRPO as an open-weight memory agent and report partial transfer gains on held-out Terminal-Bench.They present this as preliminary evidence that the intervention policy can be learned by an open-weight model.Source: §1 · p. 2

2 Related Work

Prior work equips language agents with memory, context management, reflection, and auxiliary guidance, but this paper frames memory as a selective intervention during long-horizon execution. Its focus is not only what to remember, but whether and when remembered execution state should shape the next action.

2.1 Long-horizon language agents.

  • Long-horizon benchmarks expose failures such as repeated failed actions, forgotten requirements, and drift from earlier discoveries (Merrill et al., 2026; Yao et al., 2024; Chan et al., 2024).The work studies behavioral state decay and memory as a means of maintaining decision-relevant execution state across trajectories.Source: §2.1 · p. 2

2.2 Memory for language-model agents.

  • Existing memory spans retrieval and long-context architectures (Lewis et al., 2020; Wang et al., 2023), agent records, reusable skills (Wang et al., 2024), and persistent production memory.This work complements these systems by asking when remembered execution state should be reactivated as an intervention in an ongoing loop.Source: §2.2 · pp. 2–3

2.3 Learned memory policies and context management.

  • Unlike learned context-management methods, a separate memory agent maintains live execution state and decides whether to intervene, including when silence is preferable.It differs from self-curation, QA-optimized memory construction, and folding completed subtasks into summaries; interventions may prevent repeated failures or reactivate diagnoses and open subgoals, but can also be harmful.Source: §2.3 · p. 3

2.4 Reflection, critics, and advisor models.

  • Unlike reflection, search, or advisor methods (Madaan et al., 2023; Shinn et al., 2023; Yao et al., 2023a; Asawa et al., 2025; Anthropic, 2026), the memory agent gives constrained, state-grounded reminders.Reminders surface execution details such as requirements, environment facts, failed attempts, diagnoses, or open subgoals at risk of becoming inactive, rather than broad strategic advice or general deliberation.Source: §2.4 · p. 3

2.5 Positioning.

  • The paper positions memory as a selective intervention policy that calibrates whether, when, and how execution state enters the action agent’s context.Interventions should affect the next decision without unnecessary token or latency overhead, and must adapt to differing task dynamics, observability, and failure modes.Source: §2.5 · p. 3

3 Method

The method adds a separate memory agent to an unchanged action agent, maintaining execution state and selectively injecting reminders when that state may affect the next decision.

3 Method

  • The memory agent runs beside the unmodified action agent, updates a structured bank from the trajectory, and decides whether to inject a reminder or remain silent.The two-phase architecture is shown in Figure 1.Source: §3 · p. 3, Figure 1 · p. 4

3.1 Problem Setup

  • Execution state includes task requirements, environment facts, prior attempts, diagnoses, discoveries, and open subgoals that may be lost from the action agent’s context.At each memory step, the agent observes the task, a recent trajectory window, and the current bank, then chooses whether to reintroduce remembered state.Source: §3.1 · p. 4

3.4 Triggering the Memory Agent

  • The memory agent runs at the first step and then at a fixed interval, observing the task, recent trajectory, and current bank at each invocation.More selective triggers are possible, but the main implementation uses a fixed schedule to isolate the intervention policy’s effect.Source: §3.1 · p. 4, §3.4 · p. 6

3.2 Memory Bank

  • The memory bank separates private progress and risk tracking from stable knowledge and procedural records of attempts and outcomes.The private status field is not shown to the action agent; knowledge captures facts, while procedural memory records what was tried and what happened.Source: §3.2 · p. 5

3.3 Two-Phase Memory Agent

  • The memory agent first edits the bank through constrained tool calls, then selects a targeted reminder or explicit null intervention without changing the action agent’s base setup.It is encouraged to intervene only when remembered state is likely to affect the next decision, avoiding redundant information and broad strategic advice.Source: §3.3 · pp. 5–6

3.5 Learning Memory Intervention Policies

  • The architecture does not require training a new model; the preliminary open-weight study trains only the memory agent, using supervised fine-tuning followed by reinforcement learning to calibrate intervention timing.Supervised fine-tuning teaches bank operations and reminder-versus-null choices; reinforcement learning targets when remembered state helps and when silence is preferable.Source: §3.5 · p. 6

4 Experiments and Results

Memory intervention improves pass@1 on Terminal-Bench 2.0 and τ 2-Bench for both evaluated action-agent strengths. Ablations, qualitative analysis, and open-weight training results support selective, timely reminders grounded in maintained execution state.

4.1 Benchmarks and Setup

  • The evaluation uses 85 paired Terminal-Bench tasks and 278 τ 2-Bench tasks across airline, retail, and telecom.Claude Sonnet 4.5 and Opus 4.6 are the action agents; the action agent is unchanged between conditions, while the Opus 4.6 memory agent updates from the task, recent k=8 messages, and memory bank before injecting a reminder or staying silent.Source: §4.1 · p. 7

4.2 Main Results

  • +8.3 pp (37.6% →45.9%) for Sonnet 4.5 and +2.4 pp (43.5% →45.9%) for Opus 4.6 on Terminal-Bench; on τ 2-Bench, gains are +6.8 pp (55.0% →61.8%) and +2.5 pp (66.2% →68.7%), respectively.On τ 2-Bench, Sonnet’s airline and retail lifts are +10.0 and +9.7 pp, while telecom is +2.6 pp.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 Ablations and Comparisons

  • The full two-phase system achieves the highest τ 2-Bench macro-average, while passive full-bank context, always-on reminders, advisor-only guidance, and general retrieval are less balanced.Full-bank context trails the full system by 2.8 macro points and 2.6 micro points; always-inject leads micro by 0.3 points within expected run variance, but selective silence is better on macro, especially airline. Mem0 improves the average but not airline over baseline and falls short of the full system on macro-average.Source: §4.3 · pp. 8–9, Table 2 · p. 8

4.4 Qualitative Analysis

  • Qualitative examples show memory reactivating execution state at decision points: debugging requirements and diagnoses in Terminal-Bench, and policies, verified user/tool state, and procedural constraints in τ 2-Bench.Successful reminders often precede state-changing tool calls; remaining failures mainly involve calibration, such as speculative, redundant, or unnecessary reminders.Source: §4.4 · pp. 9–10

4.5 Training Open-Weight Memory Agents

  • On SETA, an untrained 27B memory agent reduces reward from 0.709 to 0.693; SFT raises it to 0.720 and GRPO to 0.734, improving held-out Terminal-Bench pass@1 from 37.6% to 41.1%.The memory agent is Qwen3.5-27B; the frozen action agent is Qwen3.5-122B-A10B.Source: §4.5 · p. 10, Table 4 · p. 10

5 Conclusion

The conclusion frames agent memory as a proactive intervention policy, not passive storage and retrieval, that counters behavioral state decay by maintaining state and injecting grounded context when it may affect the next decision.

5 Conclusion

  • Across Terminal-Bench 2.0 and τ 2-Bench, the memory agent consistently improves task success across different action-agent models.Source: §5 · p. 10
  • The memory agent maintains execution state and injects grounded context when it is likely to affect the next decision.This addresses behavioral state decay, where execution state stops influencing future behavior.Source: §5 · p. 10
  • Training on SETA shows the intervention policy is learnable: SFT teaches memory management, while RL improves when remembered state should re-enter the control loop.Open directions include joint training of memory and action agents, learning invocation timing instead of using a fixed schedule, and determining when verbatim reminders or task-specific abstractions work best.Source: §5 · p. 10

Related papers

Loading 2607.08716v1…