Source-linked AI summary
CHIME: Credit-Aware Hierarchical Memory Evolution for Long-Horizon Agentic Planning
Yongshi Ye, Tian Lan, Feihu Jiang, Muyang Ye, Bin Zhu, Qianghuai Jia, Longyue Wang, Zhao Xu, Weihua Luo, Xiaodong Shi
TL;DR
Existing self-evolving memory methods use final outcomes to update memory, but those outcomes conflate plan quality with execution and environmental factors. CHIME attributes feedback to planning, execution, both, or neither before updating separate memory banks, and consistently outperforms strong baselines across four long-horizon benchmarks while transferring effectively across backbones.
Problem
Existing self-evolving memory methods rely on final outcomes that conflate plan quality with execution errors and environmental factors, producing biased and noisy planning experience.
Method
CHIME uses an attribute-before-memorize process with separate planning and execution banks, stage-specific retrieval and value reranking, and a credit attribution gate that assigns feedback before memory updates.
Results
Across four long-horizon benchmarks and two backbones, CHIME consistently outperforms training-based and self-evolving memory baselines, improving eval average over the strongest baseline by 2.96% and 3.68%.
Takeaways & Limitations
CHIME’s results indicate that effective memory evolution depends on attributing experience to the decision stage where it can provide effective guidance.
Abstract
from arXiv · showhide
Planning is a central capability that enables agents to decompose complex long-horizon tasks into manageable steps. Test-time search and training-based methods improve planning but incur high inference costs or require expensive training data. Self-evolving memory instead accumulates reusable experience from agent interaction outcomes into an external memory bank, so planning capability keeps improving at inference time without parameter updates. However, existing self-evolving memory methods share an inherent credit assignment problem: they rely on final task outcomes as feedback, but such outcomes conflate plan quality with execution errors and environmental factors, so the accumulated planning experience is often biased and noisy. To address this problem, we propose Credit-Aware Hierarchical Memory Evolution (CHIME), a self-evolving memory framework that maintains a separate planning bank and execution bank and follows an attribute-before-memorize principle: CHIME first attributes each task outcome to the plan, the execution, both, or neither, and then updates only the corresponding memory bank. Extensive experiments on four long-horizon agent benchmarks show that CHIME consistently outperforms state-of-the-art training-based and self-evolving memory baselines. Further analyses reveal several interesting findings. For example, CHIME accumulates effective memory with far fewer items. In addition, the learned memory values faithfully reflect downstream utility: high-quality planning memories are more valuable than execution memories. Finally, the accumulated memory effectively transfers across backbone models. Code will be released at https://github.com/ATH-MaaS/Marco-DeepResearch.
1 Introduction
Long-horizon planning requires coordinating interdependent steps and constraints, but existing approaches either incur high inference or training costs or update memory from outcomes that conflate planning and execution quality. CHIME addresses this with credit attribution before separately updating planning and execution memory.
- Long-horizon agents must coordinate interdependent decisions while maintaining task constraints across execution.
- Test-time search improves performance by exploring candidate plans but incurs high inference costs and does not retain reusable experience.
- Planner training internalizes planning capability but requires costly trajectory collection and post-training.
- CHIME replaces outcome-based memory updates with an attribute-before-memorize process using separate planning and execution memory banks.
- Across four benchmarks and two backbones, CHIME improves eval average over the strongest baseline by 2.96% and 3.68%, respectively.
- CHIME retains 129 memories versus 3,585 for the strong baseline while achieving the highest accuracy, and planning memories provide more than twice the accuracy gain of execution memories.Planning memories improve accuracy from 21.7%→50.8%, compared with 23.7%→35.4% for execution memories.
2 Related Work
Prior work improves agentic planning through test-time search, planner training, or self-evolving memory. These approaches trade off inference cost, training cost, continual updatability, and the extent to which memory is specifically evolved for planning.
- Self-evolving memory provides a training-free, continual alternative by updating an external memory bank while keeping model parameters frozen.
- Agentic planning decomposes complex objectives into structured steps and coordinates their execution across interdependent subtasks.
- Test-time search explores candidate plans or trajectories and selects among them using value estimates or environmental feedback.
- Planner model training internalizes planning capability into parameters but requires costly trajectory collection and post-training for continual updates.
- Few self-evolving memory methods evolve memory specifically for agentic planning, with A-MapReduce as a representative approach that evolves structured hints from past executions.
3 Methodology of CHIME
CHIME separates planning and execution experience, attributes outcome credit before memorization, and updates only the responsible memory bank. Stage-specific retrieval, credit-aware reranking, and memory evolution then support continual planning and execution improvement.
- Hierarchical Memory Bank: CHIME maintains separate planning and execution banks, storing strategic experience in the former and operational experience in the latter.Planning memories cover decomposition, dependencies, and constraints; execution memories cover tool selection and invocation.
- Hierarchical Memory Bank: Each memory item contains an applicability key, reusable experience, a reuse-calibrated value score, and a reuse count.The applicability key supports retrieval and merging, while value guides reranking and pruning.
- Episode Workflow: The planner retrieves planning memories from the task, the executor retrieves execution memories from the task and plan, and execution produces a trajectory and binary outcome.The episode then proceeds through credit attribution and memory-bank transition.
- Hierarchical Memory Retrieval: Stage-specific retrieval first selects Top-N candidates by similarity, then reranks them by credited value to retrieve Top-K memories.A reuse-based smoothing factor shrinks rarely reused values toward zero, preserving similarity-only ranking when evidence is limited.
- Credit Attribution: The Credit Attribution Gate reviews the task, plan, trajectory, outcome, and retrieved memories to classify credit as planning, execution, both, or neither.It assesses plan sufficiency, execution correctness, external factors, attribution confidence, and misleading retrieved memories.
- Memory Evolution: Credit-Aware Memory Evolution rewards credited memories, penalizes misleading ones, and filters, merges, inserts, or discards stage-specific experience.Updates apply only to attributed stages, with confidence controlling feedback and acceptance thresholds controlling content evolution.
4 Experimental Setup
The evaluation compares CHIME with increasingly close planning and memory baselines on four long-horizon benchmarks using two backbone models. Performance is measured on sequential train and evaluation streams with repeated random orderings and fixed retrieval and evolution settings.
- Baselines: CHIME is compared with No-Plan, WebAnchor, TodoEvolve, and A-MapReduce, with all methods using the same runtimes, tools, and verifiers.The baselines span no persistent planning, test-time search, planner training, and outcome-based memory evolution.
- Benchmarks: The benchmarks cover multi-turn customer service, life-service tasks, long-horizon information seeking, and function calling.They are τ 2-bench, VitaBench, BrowseComp-ZH, and BFCL-v4.
- Backbones: Evaluation uses Qwen3.5-Flash and DeepSeek-V4-Flash as backbone models, with the same backbone serving the planner, executor, and Credit Attribution Gate.The reported backbone configurations are 35B total with 3B active and 284B total with 13B active, respectively.
- Test-Time Evaluation Protocol: Each benchmark is processed as a sequential stream split 7:3 into train and evaluation portions, with multi-domain data truncated and split independently for equal domain contribution.The train split measures accumulation, while the evaluation split tests performance after accumulation.
- Reporting: Results are averaged over three random instance orderings using Avg@3, and averages are computed across benchmarks within each split.Table 1 reports the main results and Table 2 reports ablations under the same ordering protocol.
- Implementation Details: Retrieval uses (N, K) = (20, 3), λ = 3.0, α = 1.0, and β = 0.2, while memory evolution uses θconf = 0.5, θmerge = 0.88, nmin = 3, and θprune = −0.1.Evaluation-side LLM calls use temperature 0.0 except where otherwise specified for WebAnchor.
5 Main Experimental Results
CHIME achieves the strongest main-task performance on both train accumulation and eval transfer, while its ablations show that hierarchical memory and credit-aware evolution are central to these gains.
- Memory Accumulation: CHIME achieves the highest average on both backbones, reaching 35.02% on Qwen3.5-Flash and 38.00% on DeepSeek-V4-Flash.It outperforms the strongest baseline by 3.38% and 0.90%, respectively.
- Memory Transfer: CHIME outperforms the strongest baseline on eval average by 2.96% on Qwen3.5-Flash and 3.68% on DeepSeek-V4-Flash.Against A-MapReduce, the gains are 5.37% and 3.68%, respectively.
- Ablation Studies: Removing planning memory, execution memory, or both reduces eval average by 3.26%, 4.81%, and 3.13%, while sharing one bank reduces it by 3.60%.These ablations support stage-specific experience and explicit bank separation.
- Ablation Studies: Removing the Credit Attribution Gate, credit reranking, or memory evolution reduces eval average by 2.44%, 2.04%, and 3.04%, respectively.Attribution determines which stage receives feedback, while reranking and evolution determine which memories are reused and retained.
6 Analysis
Analyses show that CHIME produces high-quality, compact memories whose learned values track downstream utility, whose gate assignments are repeatable, and whose guidance transfers across backbones.
- Memory Quality: 81.04% of CHIME memories pass correctness, non-redundancy, and generality checks, the highest rate among compared methods.Without the Gate, non-redundancy and generality fall below 31%; A-MapReduce reaches 36.08% correctness.
- Evolution Efficiency: At the end of accumulation, CHIME retains 129 memories versus 3,585 for A-MapReduce while achieving the highest accuracy.Outcome-based counterparts keep growing their banks, and their accuracy declines at full accumulation.
- Memory Value Utility: From low to high value, accuracy rises from 21.7% to 50.8% for planning memories and from 23.7% to 35.4% for execution memories.The monotonic pattern supports learned values as indicators of downstream memory utility.
- Gate Reliability: Planning and execution labels reach 87.8%–97.9% agreement, while 56.3%–67.6% of failures are rescued across the two benchmarks.Stability measures agreement across repeated samples; Rescued measures whether attributed failures disappear when rerun with generated memory.
- Credit-Model Scaling: Replacing the attribution model with Qwen3.7-Flash yields consistent eval gains from 2.25% to 3.12% across three benchmarks.The experiment uses a single run rather than Avg@3.
- Cross-Backbone Transfer: Transferred CHIME memory outperforms A-MapReduce memory by 2.25%–4.68% across all four settings and stays within 1.72% of the upper bound on BFCL-v4 for DeepSeek-V4-Flash.It matches or exceeds the upper bound on the Qwen3.5-Flash target model.
7 Conclusion
CHIME uses stage-level credit attribution to separate and update planning and execution memories, and its results indicate that effective memory evolution depends on attributing experience to the decision stage where it can guide future behavior.
- Conclusion: CHIME separates planning and execution memory banks and updates them through stage-level credit attribution.The framework is based on the attribute-before-memorize principle.
- Conclusion: Across four long-horizon agent benchmarks, CHIME consistently outperforms strong training-based and self-evolving memory baselines.The analyses cover memory quality, efficiency, value utility, gate reliability, credit-model scaling, and transfer.
- Conclusion: Effective memory evolution depends on attributing each experience to the decision stage where it can provide effective guidance.
A Implementation Details
CHIME keeps the backbone fixed while updating external planning and execution memory banks from interaction data and environment feedback under a consistent evaluation setup.
- Implementation Details: The agent backbone remains frozen while only the external planning and execution memory banks are updated during accumulation and frozen during transfer.
- Implementation Details: Memory updates use interaction trajectories and environment feedback rather than reference answers.
- Implementation Details: Experiments use Qwen3.5-Flash and Deepseek-V4-Flash backbones, Qwen3.6-Flash for evaluation-side roles, and Qwen3-Embedding-0.6B for retrieval.
B Failure Analysis
CHIME’s benefits are constrained by the agent’s underlying capabilities and by how closely deployment conditions match those in the benchmark. Memory transfer is stronger across similar tasks and weaker across different benchmarks.
- Agent Capability: 56.3–67.6% of attributed failures are resolved when failed episodes are rerun with generated memory, leaving some failures unresolved.CHIME’s benefit remains bounded by whether the agent can follow memory guidance and execute required actions.
- Memory Transferability: Memory transfers across backbones when the benchmark remains unchanged, because tasks share tools, valid actions, and success criteria.The transfer result depends on preserving the operating conditions under which the memory applies.
- Memory Transferability: Transfer across different benchmarks is weaker because semantic similarity does not ensure matching tools, valid actions, or success criteria.CHIME is most transferable across tasks with similar scenarios and operating conditions.
C Case Studies
The case studies show CHIME using separate planner and executor memories for distinct failure modes, while the attribution gate routes reusable errors to the responsible layer. These examples connect memory granularity and credit assignment to successful tool execution and reduced cross-layer contamination.
- Planner-side decomposition: Planner memories decompose similar-looking retail requests by object status before tool execution.They distinguish delivered, pending, and cancelled orders so each object receives the correct action family.
- Executor-side precision: Executor memory addresses precise arguments when the high-level plan is already correct.It preserves account identities, computes net credits, and binds exact amounts to credit-tool calls.
- Layer-aware attribution: The gate routes a failed delivery episode to the planner layer when execution follows the plan but treats a strict “before” constraint as equality.It writes a planner memory about strict deadline buffers and leaves the executor bank untouched.
- Planner-side decomposition: 1.0 is achieved by Ours on τ 2 Bench, versus 0.0 for AMapReduce-Mem and NoPlan.This case concerns multi-status retail request decomposition.
- Experimental settings: Table 7 reports implementation parameters for the main experiments, with benchmark-specific settings shared across methods.The table provides experimental configuration rather than a case-study outcome.
- Executor-side precision: 1.0 is achieved by Ours on τ 2 Bench, versus 0.0 for AMapReduce-Mem and NoPlan.This case concerns account-level net-credit execution.