PaperlayerView exact revision on arXiv

Source-linked AI summary

CHIME: Credit-Aware Hierarchical Memory Evolution for Long-Horizon Agentic Planning

Yongshi Ye, Tian Lan, Feihu Jiang, Muyang Ye, Bin Zhu, Qianghuai Jia, Longyue Wang, Zhao Xu, Weihua Luo, Xiaodong Shi

arXiv:2609.02074v1cs.AI

TL;DR

Existing self-evolving memory uses final task outcomes that conflate plan quality with execution, making them biased feedback for planning memories. CHIME attributes outcomes to planning and execution before updating separate memory banks, and consistently outperforms strong baselines across four long-horizon benchmarks.

  • Problem

    Existing self-evolving memory methods use final outcomes that depend on both plan quality and execution, so those outcomes do not faithfully reflect plan quality.

    Source: §3.1 · p. 3

  • Method

    CHIME uses stage-level credit attribution to update separate planning and execution banks for strategic and operational experience, respectively.

    Source: §3.2 · p. 3, §7 · p. 8

  • Results

    Across four long-horizon benchmarks, CHIME consistently outperforms strong training-based and self-evolving memory baselines, exceeding the strongest baseline's eval average by 2.96% on Qwen3.5-Flash and 3.68% on DeepSeek-V4-Flash.

    Source: §5 · p. 6, §7 · p. 8

  • Takeaways & Limitations

    CHIME's results indicate that effective self-evolving memory depends on the quality of accumulated memories, not their quantity.

    Source: §6 · p. 7

Abstract

from arXiv · show

Planning is a central capability that enables agents to decompose complex long-horizon tasks into manageable steps. Test-time search and training-based methods improve planning but incur high inference costs or require expensive training data. Self-evolving memory instead accumulates reusable experience from agent interaction outcomes into an external memory bank, so planning capability keeps improving at inference time without parameter updates. However, existing self-evolving memory methods share an inherent credit assignment problem: they rely on final task outcomes as feedback, but such outcomes conflate plan quality with execution errors and environmental factors, so the accumulated planning experience is often biased and noisy. To address this problem, we propose Credit-Aware Hierarchical Memory Evolution (CHIME), a self-evolving memory framework that maintains a separate planning bank and execution bank and follows an attribute-before-memorize principle: CHIME first attributes each task outcome to the plan, the execution, both, or neither, and then updates only the corresponding memory bank. Extensive experiments on four long-horizon agent benchmarks show that CHIME consistently outperforms state-of-the-art training-based and self-evolving memory baselines. Further analyses reveal several interesting findings. For example, CHIME accumulates effective memory with far fewer items. In addition, the learned memory values faithfully reflect downstream utility: high-quality planning memories are more valuable than execution memories. Finally, the accumulated memory effectively transfers across backbone models. Code will be released at https://github.com/ATH-MaaS/Marco-DeepResearch.

1 Introduction

CHIME addresses noisy planning feedback by attributing task outcomes before updating separate planning and execution memory banks. Across four long-horizon benchmarks, it outperforms training-based and self-evolving memory baselines on two backbones.

1 Introduction

  • Test-time search is costly, planner training requires costly data collection and post-training, and self-evolving memory risks mistaking task outcomes for plan quality.Such credit errors can persist through repeated retrieval, while execution- or environment-level experience can misguide high-level planning.Source: §1 · p. 1
  • CHIME separates planning and execution memories, attributes feedback to planning, execution, both, or neither before writing, and updates only the attributed bank.Its gate reflects on the task, plan, execution, and retrieved memories; bank updates use confidence-weighted feedback and may filter, merge, or insert experience.Source: §1 · p. 2
  • CHIME improves eval average over the strongest baseline by 2.96% and 3.68% on two backbones, respectively.Experiments cover four long-horizon benchmarks and two backbone models.Source: §1 · p. 2
  • CHIME retains 129 memories versus 3,585 for a strong baseline, while planning memories yield larger accuracy gains than execution memories.Planning: 21.7%→50.8%; execution: 23.7%→35.4%.Source: §1 · p. 2
  • Repeated credit attributions agree in up to 97.9% of cases, and accumulated memory transfers effectively across backbones.Generated experience rescues over half of failures; a stronger credit model yields up to 3.12% further gain, and transferred memory beats A-MapReduce’s by up to 4.68%.Source: §1 · p. 2

2 Related Work

Prior work improves agentic planning through test-time search, planner training, and self-evolving memory, but existing approaches judge plans by outcomes that also reflect execution and environmental factors. Self-evolving memory updates an external bank rather than model parameters, while few works target planning specifically.

Self-evolving memory

  • Self-evolving memory treats an external bank as evolvable parameters, keeping model parameters frozen while distilling interaction trajectories into reusable experience or skills.ReasoningBank distills strategies from self-judged successful and failed trajectories, while UMEM jointly optimizes memory extraction and management with reinforcement learning (Ouyang et al., 2026; Ye et al., 2026).Source: §2 · p. 2

Agentic planning

  • Agentic planning decomposes complex objectives into structured steps, helping agents track interdependent subtasks and avoid the lost-in-the-middle problem (Liu et al., 2026).As benchmarks introduce more challenging tasks, improving planning capability has become a central research problem for agent systems (Wong et al., 2025; Lan et al., 2025; Yang et al., 2025).Source: §2 · p. 2

Improving agentic planning

  • Existing planning approaches use test-time search, planner training, or self-evolving memory, but all assess plans through final outcomes that also reflect execution and environmental factors.Test-time methods select among candidate plans; planner training internalizes planning but requires costly data collection and post-training. Self-evolving memory is training-free and continual, though few works focus on planning; A-MapReduce evolves hints for decomposition and aggregation in long-horizon search.Source: §2 · p. 2

3 Methodology of CHIME

Because task outcomes can conflate planning quality with execution, CHIME attributes feedback before updating memory. It separates planning and execution experience and evolves only the bank or banks assigned credit.

3.1 Task Formulation of Self-Evolving Agents

  • The agent’s policy stays frozen while an external memory bank accumulates experience; final outcomes can be biased signals because they depend on both planning and execution.During evaluation, the memory bank is frozen so performance reflects experience accumulated before evaluation.Source: §3.1 · p. 3

3.2 Overview of CHIME

  • CHIME separates strategic planning experience, such as decomposition and dependency modeling, from operational execution experience, such as tool selection and invocation.Each memory item includes an applicable-task key, reusable experience, a value score, and a reuse count.Source: §3.2 · p. 3
  • At each stage, CHIME retrieves by query similarity, adjusts candidate values using reuse counts, then reranks by credited value while bounding its influence so similarity remains dominant.Past-helpful memories are preferred and misleading memories suppressed.Source: §3.2 · p. 3
  • The Credit Attribution Gate uses structured self-reflection to assign outcomes to planning, execution, both, or neither, and credits the responsible stage in successful episodes.It reviews the task, plan, trajectory, outcome, and retrieved memories, assessing plan sufficiency, execution correctness, and external factors.Source: §3.2 · p. 3
  • CHIME rewards memories from stages credited for success, penalizes misleading ones, and otherwise leaves their values unchanged; it merges, inserts, or discards new experience in attributed banks.Value feedback is scaled by attribution confidence, and content updates require sufficient confidence and actionable task-specific experience.Source: §3.2 · p. 4

4 Experimental Setup

CHIME is evaluated on four long-horizon benchmarks with two backbones, using sequential task streams to measure memory accumulation on training tasks and generalization to held-out tasks.

Baselines

Benchmarks

  • Evaluation covers τ 2-bench customer service, VitaBench life-service tasks, BrowseComp-ZH information seeking, and BFCL-v4 function calling.BFCL-v4 uses its Live and long-horizon Agentic groups.Source: §4 · p. 5

Backbones

  • CHIME uses Qwen3.5-Flash (35B total, 3B active) and DeepSeek-V4-Flash (284B total, 13B active), with each backbone serving as planner, executor, and Credit Attribution Gate.The reported backbone citations are Qwen Team (2026a) and DeepSeek-AI et al. (2026).Source: §4 · p. 5

Test-Time Evaluation Protocol

  • The 7:3 stream-order split measures memory accumulation on train tasks and generalization on eval tasks, where accumulated memory is frozen and retrieved without further updates.For multi-domain benchmarks, each domain is truncated to the smallest domain size and split independently; performance is averaged over three random orderings (Avg@3).Source: §4 · pp. 5–6

Implementation Details

  • Evaluation-side LLM roles use Qwen3.6-Flash, while memory retrieval uses Qwen3-Embedding-0.6B.Retrieval and reranking use (N, K) = (20, 3), λ = 3.0, α = 1.0, β = 0.2; memory evolution uses θconf = 0.5, θmerge = 0.88, nmin = 3, θprune = −0.1. Calls use temperature 0.0 except WebAnchor plan generation (0.6).Source: §4 · p. 6

5 Main Experimental Results

CHIME outperforms memory baselines on train accumulation and frozen-bank eval transfer, while ablations show that its separated memory banks, credit attribution, reranking, and evolution each contribute to performance.

Memory Accumulation (Train Split)

  • On train accumulation, CHIME averages 35.02% (Qwen3.5-Flash) and 38.00% (DeepSeek-V4-Flash), beating the strongest baselines by 3.38% and 0.90%.These gains indicate that CHIME’s accumulated memory guides subsequent tasks more effectively than baseline memory.Source: §5 · p. 6

Memory Transfer (Eval Split)

  • With banks frozen, CHIME leads the strongest eval baseline by 2.96% (Qwen3.5-Flash) and 3.68% (DeepSeek-V4-Flash), and A-MapReduce by 5.37% and 3.68%, respectively.The results indicate that attributed memory transfers better than outcome-based memory.Source: §5 · p. 6

Ablation Studies

  • On eval, removing planning, execution, or both memories lowers averages by 3.26%, 4.81%, and 3.13%; sharing banks lowers it by 3.60%.These ablations show that stage-specific experience and explicit bank separation matter, beyond memory accumulation alone.Source: §5 · p. 6
  • On eval, removing attribution, credit reranking, or memory evolution lowers averages by 2.44%, 2.04%, and 3.04%, respectively.Attribution directs feedback to the appropriate stage; reranking and credit-aware evolution determine which memories are reused and retained.Source: §5 · p. 6

6 Analysis

CHIME’s analysis finds that credit-aware memory construction produces higher-quality, more efficient memory and that learned values, gate attributions, stronger credit models, and transferred memories show downstream utility.

RQ1: Does CHIME Accumulate High-Quality Memory?

  • 81.04% of CHIME’s memories passed all three quality criteria, while A-MapReduce reached 36.08% correctness and removing the Gate drove non-redundancy and generality below 31%.The criteria were correctness, non-redundancy, and generality, evaluated by GLM-5.2 on Qwen3.5-Flash train-split memories.Source: §6 · p. 6, Table 3 · p. 6

RQ2: Does Stage-Attributed Evolution Improve Memory Efficiency?

  • With 129 memories versus A-MapReduce’s 3,585, CHIME achieved the highest end-of-stream accuracy while the outcome-based baselines’ accuracy declined.The baselines’ accuracy peaked at the 75% checkpoint; the passage attributes their decline to misleading memories accumulating without attribution.Source: §6 · p. 7

RQ3: Do Learned Memory Values Reflect Downstream Utility?

  • From low to high value, accuracy rose from 21.7% to 50.8% for planning memories and from 23.7% to 35.4% for execution memories.The high-to-low accuracy gains were 29.1% for planning memories versus 11.7% for execution memories; at low value, planning memories scored 21.7% versus 23.7% for execution memories. The findings support using learned values to select reusable experience.Source: §6 · p. 7, Figure 4 · p. 7

RQ4: Is the Credit Attribution Gate Reliable?

  • Gate labels agreed with the majority label in 87.8%–97.9% of cases, and 56.3%–67.6% of attributed failures were rescued when rerun with generated memory.The reliability evaluation covered Qwen3.5-Flash on two benchmarks; stability and rescued rates are reported in Table 4.Source: §6 · p. 7, Table 4 · p. 7

RQ5: Does Stronger Attribution Further Improve Performance?

  • In the reported evaluation, the stronger Qwen3.7-Flash replaces Qwen3.5-Flash for the credit attribution gate; Table 5 reports eval accuracy (%) for a single run, not Avg@3.Source: Table 5 · p. 7

RQ6: Does CHIME’s Memory Transfer Across Backbones?

  • Transferred CHIME memories outperformed A-MapReduce in all four settings by 2.25%–4.68%, matching or exceeding the upper bound on Qwen3.5-Flash and staying within 1.72% on BFCL-v4 for DeepSeek-V4-Flash.The evaluation copied memory from a source backbone’s train split into a target backbone’s frozen eval run.Source: §6 · p. 7, Table 6 · p. 8

7 Conclusion

CHIME separates planning and execution memory updates through stage-level credit attribution, and experiments show consistent gains over strong baselines. Analyses validate its memory quality, evolution efficiency, utility, attribution reliability, scaling, and transfer.

  • Across four long-horizon agent benchmarks, CHIME consistently outperforms strong training-based and self-evolving memory baselines.Source: §7 · p. 8
  • CHIME applies an attribute-before-memorize principle, using stage-level credit attribution to separate and update planning and execution memory banks.Source: §7 · p. 8
  • Analyses validate CHIME across six dimensions: memory quality, evolution efficiency, memory value utility, attribution-gate reliability, credit-model scaling, and cross-backbone transfer.The results suggest reliable memory evolution depends on attributing each experience to the decision stage where it can provide effective guidance.Source: §7 · p. 8

A Implementation Details

CHIME keeps the backbone frozen and evolves external memory from interaction experience, while evaluations use shared benchmark conditions and specified model roles.

A Implementation Details

  • CHIME freezes the backbone throughout, updates only its planning and execution memory banks during accumulation, and freezes those banks during transfer.Updates use interaction trajectories and environment feedback, not reference answers.Source: A Implementation Details · p. 11
  • All methods share the benchmark setup and evaluation-side models, use each baseline’s best-performing reported configuration, and average results over three random instance orderings (Avg@3).Source: A Implementation Details · p. 11
  • CHIME is evaluated with Qwen3.5-Flash and Deepseek-V4-Flash, while Qwen3.6-Flash handles evaluation-side LLM roles and Qwen3-Embedding-0.6B serves as retriever.Within each setting, the same backbone handles planning, execution, and self-reflection in the Credit Attribution Gate.Source: A Implementation Details · p. 11

B Failure Analysis

CHIME has two practical boundaries: its benefits depend on the backbone agent’s ability to follow memory guidance and execute actions, and its memories transfer best across tasks with similar operating conditions.

Agent Capability

  • Rerunning failed episodes with generated memory resolves 56.3–67.6% of attributed failures, but CHIME remains bounded by the agent’s ability to follow guidance and execute actions.CHIME augments memory guidance without changing the backbone, so useful memories may not improve outcomes when the agent lacks the required capabilities.Source: B Failure Analysis · p. 11

Memory Transferability

  • Memory transfers across backbones when tasks remain within the same benchmark and share tools, valid actions, and success criteria.Tasks within a benchmark generally share the conditions under which a memory applies.Source: B Failure Analysis · p. 11
  • Transfer across benchmarks is weaker because semantic similarity does not ensure that operating conditions remain unchanged.CHIME’s memory is most transferable across tasks with similar scenarios and operating conditions; transfer across different benchmarks remains limited.Source: B Failure Analysis · p. 11

C Case Studies

Three case studies show CHIME’s memories supporting status-aware planning and precise tool execution, while its trajectory-level gate routes a failed episode to the planner when the reusable error lies in planning.

Planner-side decomposition

  • Retrieved planner memories encode status-conditioned retail protocols, enabling correct action choices for delivered-item returns, pending-order edits, and cancelled-order tracking before tool use.The baselines produce plausible but verifier-failing summaries.Source: C Case Studies · p. 11, Figure 5 · p. 12

Executor-side precision

  • Executor memories preserve account identities, compute net credits after missing fees, and bind exact per-account amounts to tool arguments when the high-level plan is already correct.The memories are written at the granularity of the eventual action arguments.Source: C Case Studies · p. 11, Figure 6 · p. 14

Layer-aware attribution

  • For a failed delivery episode, the gate identifies a strict “before” constraint error, writes a planner memory, and leaves executor memory untouched because execution followed the plan.This layer-specific routing keeps memories more specific and reduces cross-layer contamination.Source: C Case Studies · p. 11, Figure 7 · p. 15

Related papers

Loading 2609.02074v1…