PaperlayerView exact revision on arXiv

Source-linked AI summary

StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management

Ruizhe Zhang, Xinke Jiang, Zhibang Yang, Zhixin Zhang, Jiaran Gao, Yuzhen Xiao, Tao Feng, Yue Fang, Yuxuan Liu, Ruiqing Li, Hongbin Lai, Huheng Huang, Xu Chu, Junfeng Zhao, Yasha Wang

arXiv:2601.05890v2cs.AI

TL;DR

Centralized multi-agent systems can struggle with long-horizon coordination because central agents lack effective task and cross-task memory management. StackPlanner introduces hierarchical coordination with active task-memory control, structured experience retrieval, and reinforcement learning; experiments report state-of-the-art results across evaluated benchmarks, while extended human interaction and experience-memory cold starts remain underexplored.

  • Problem

    Centralized multi-agent systems can be overwhelmed by long reasoning chains and expanding information, with limited memory management contributing to poor cold-start performance and cross-task generalization.

  • Method

    StackPlanner decouples coordinator planning from specialized execution and jointly learns active task-memory revision, structured experience retrieval, and coordination through reinforcement learning.

  • Results

    StackPlanner achieves state-of-the-art results across multi-hop QA and agentic benchmarks, with reported F1 scores of 16.48%, 7.71%, and 16.23% using Qwen2.5-3B on MuSiQue, GAIA, and FRAMES.

  • Takeaways & Limitations

    Active task and experience memory support reliable long-horizon multi-agent collaboration across the evaluated deep-search and agent-system benchmarks.

  • Takeaways & Limitations

    Evaluation covers short follow-up human interactions, leaving long interaction chains, ambiguous preference updates, repeated goal shifts, and experience-memory cold starts underexplored.

Abstract

from arXiv · show

Multi-agent systems based on large language models, particularly centralized architectures, have recently shown strong potential for complex and knowledge-intensive tasks. However, central agents often suffer from unstable long-horizon collaboration due to the lack of memory management, leading to context bloat, error accumulation, and poor cross-task generalization. To address both task-level memory inefficiency and the inability to reuse coordination experience, we propose StackPlanner, a hierarchical multi-agent framework with explicit memory control. StackPlanner addresses these challenges by decoupling high-level coordination from subtask execution with active task-level memory control, and by learning to retrieve and exploit reusable coordination experience via structured experience memory and reinforcement learning. Experiments on multiple deep-search and agent system benchmarks demonstrate the effectiveness of our approach in enabling reliable long-horizon multi-agent collaboration.

1 Introduction

LLM-based multi-agent systems support complex, long-horizon tasks, but centralized systems struggle when a single coordinator must manage expanding information and reasoning chains. StackPlanner addresses these memory-related challenges by separating coordination from execution and treating task and experience memory as controllable components.

  • Motivation: Centralized multi-agent systems place planning, task allocation, information integration, and decision-making on one central agent.This burden becomes more difficult as task scale and complexity increase.
  • Motivation: Long-horizon collaboration can overwhelm the central agent, degrading performance through limited task-level and cross-task memory management.The paper identifies context overload and weak memory management as central sources of this difficulty.
  • Motivation: Limited training for long-horizon cross-agent reasoning contributes to poor cold-start performance and limited cross-task generalization.
  • Approach: StackPlanner constructs a hierarchical system centered on a coordinator that explicitly manages task memory and experience memory.
  • Approach: The framework decouples high-level coordinator decisions from specialized sub-agent execution while unifying memory management, experience retrieval, and coordination actions under reinforcement learning.

2 Related Work

Prior multi-agent research explores centralized, decentralized, debate-based, and structured coordination, while long-horizon agents increasingly combine search, planning, retrieval, and learning. StackPlanner focuses on making evolving coordinator memory an explicit object of revision and retrieval rather than a passive context buffer.

  • LLM-based multi-agent coordination: Multi-agent systems use specialized roles and structured communication, while decentralized and debate-based designs increase diversity but can create redundant communication and inconsistent global state.
  • Long-horizon agentic search and learning: Long-horizon agentic systems combine search, planning, retrieval, and reinforcement learning, with persistent coordination state supporting extended tasks.
  • Memory and retrieval: Existing memory approaches commonly use summarization, heuristic truncation, retrieval-based reconstruction, reflection, dialogue personalization, or cross-task reuse.
  • Memory and retrieval: StackPlanner models task-memory revision and experience retrieval as first-class coordinator actions jointly learned with planning and delegation.

3 Methodology

StackPlanner uses a hierarchical coordinator with explicit task and experience memory, active REVISE operations, retrieval, human-feedback integration, and reinforcement-learning optimization. The design alternates planning, delegation, retrieval, reasoning, and memory operations while preserving task-oriented state.

  • Hierarchical coordination: A central coordinator performs high-level planning, delegation, and memory operations, while specialized sub-agents execute concrete tasks.The framework maintains separate coordinator task memory and structured experience memory.
  • Coordinator actions: The coordinator action space is A = {PLAN, DELEGATE, REVISE, FINISH}.PLAN selects the next coordination step, DELEGATE assigns scoped subtasks, REVISE manages memory, and FINISH terminates completed tasks.
  • Task memory: REVISE actively condenses verbose memory and prunes unproductive or erroneous exploration while retaining concise failure causes.These operations reduce redundancy and support correction of earlier coordination errors.
  • Experience memory: Structured experience memory stores user profiles, semantic knowledge, and procedural SOPs for retrieval across tasks.Retrieved entries are summarized and injected into task memory to inform coordination and mitigate cold-start issues.
  • Reinforcement learning: The coordinator is trained with reinforcement learning over dynamically updated search results and memory states in an interleaved retrieval–reasoning–memory execution process.GRPO supplies group-relative advantages without a learned value function, while trajectory-level rewards remain the dominant optimization signal.
  • Human-in-the-loop coordination: Human feedback enters as a prioritized memory update, allowing the coordinator to continue from preserved task state and invoke sub-agents only for affected parts.

4 Experiment

STACKPLANNER is evaluated across answer-oriented deep-search, open-ended research, multi-turn refinement, memory ablations, and interactive decision-making benchmarks. Results show strong benchmark performance, report quality, targeted refinement, complementary memory contributions, and gains in interactive environments.

  • Answer-Oriented Deep Search: STACKPLANNER achieves state-of-the-art results across multi-hop QA and agentic benchmarks, including strong out-of-distribution generalization.On MuSiQue, GAIA, and FRAMES, it reports F1 scores of 16.48%, 7.71%, and 16.23% with Qwen2.5-3B, and 22.01%, 9.45%, and 19.44% with Qwen2.5-7B.
  • Open-Ended Deep Research: 42.55 Overall and 14.82 Effective Citation Rate with Qwen3-32B, while DeepSeek-V3.2 raises these scores to 47.47 and 32.75, respectively.STACKPLANNER attains the best score on every RACE dimension with Qwen3-32B; WebWeaver has higher Citation Accuracy under DeepSeek-V3.2.
  • Multi-Turn Refinement: 71.11 total is achieved on 100 follow-up instances through task-memory reuse and targeted revision of report portions affected by follow-up requests.STACKPLANNER obtains the best scores on instruction following, evidence grounding, logical consistency, and completeness.
  • Memory Ablation: Removing both task and experience memory produces the largest F1 degradation across all four benchmarks, with drops of 15.80%, 9.05%, 5.24%, and 9.90%.Task memory preserves current execution state, while experience memory transfers reusable procedures from prior tasks.
  • Experience-Retrieval Ablation: 52.48 average is achieved by the full system, compared with 48.20 without reinforcement learning and 46.06 without reinforcement learning plus experience retrieval.On GPQA, performance rises from 11.8 to 17.6 and then to 23.8 across these variants.
  • Interactive Generalization: STACKPLANNER improves ALFWorld Avg. from 36.5 to 51.1 and WebShop Score from 14.7 to 27.8, with WebShop Acc. increasing from 0.8 to 2.3.The evaluation covers interactive environments requiring persistent goals, observations, and action history across multiple steps.

5 Conclusion and Future Work

STACKPLANNER is a hierarchical centralized multi-agent framework that treats memory as an explicit coordination control target. It combines decoupled coordination, active task-memory management, reusable experience memory, and joint reinforcement-learning optimization to support long-horizon collaboration.

  • STACKPLANNER uses hierarchical centralized coordination, with memory treated as an explicit control target.
  • The framework combines decoupled coordination with active task memory management and reusable experience memory.
  • Joint reinforcement learning optimizes high-level coordination and memory control.
  • Future work: The authors identify more expressive yet compact task memory and latent inter-agent communication as future directions.These directions target decision robustness under longer horizons, reduced coordination overhead, and faster multi-agent inference.

Limitations

The evaluation covers only short follow-up interactions, while extended human-in-the-loop settings and experience-memory cold starts remain underexplored.

  • STACKPLANNER’s human-in-the-loop evaluation focuses on short follow-up scenarios with localized user intents.Long interaction chains, ambiguous preference updates, and repeated shifts in user goals remain underexplored.
  • Experience memory remains vulnerable to cold-start issues when insufficient prior experience limits early effectiveness.Simulated users can partially mitigate this issue, but initialized experiences may generalize poorly to real or diverse user behaviors.
  • Future work should extend task memory to track evolving conversational dependencies more robustly.

Ethical considerations

The study uses publicly available benchmark datasets and avoids personally identifiable information and human or animal subjects.

  • All experiments use publicly available benchmarks, including 2WikiMultiHopQA, MuSiQue, GAIA, and FRAMES.The experiments comply with the datasets’ respective licenses and usage terms.
  • The study uses no personally identifiable information and involves no human or animal subjects.
  • Training uses a curated multi-hop question-answering dataset constructed from 2WikiMultiHopQA training splits.Instances requiring no external retrieval or only a single trivial retrieval step are filtered out.

A.2 Testing set.

The testing set spans multi-hop question answering, agentic reasoning, deep research, multi-turn refinement, and interactive decision-making benchmarks. These evaluations measure evidence aggregation, long-horizon reasoning, report quality, refinement consistency, and grounded interaction.

  • Multi-Hop QA Benchmarks: The test suite includes multi-hop QA datasets requiring reasoning across multiple documents and dependent reasoning steps.2WikiMultiHopQA contains 192,606 question–answer pairs, while MuSiQue contains 19,938 training, 2,417 development, and 2,459 test examples.
  • Agentic Benchmarks: Agentic benchmarks evaluate real-world questions involving multi-step reasoning, web interaction, multimodal input handling, factual accuracy, retrieval, and multi-source reasoning.GAIA contributes 127 text-only validation questions, and FRAMES contains 824 multi-hop questions.
  • Deep Research Benchmark: DeepResearch Bench evaluates open-ended research report generation across 100 expert-level tasks in 22 domains.Its protocol considers writing quality and factual grounding, with tasks evenly split between English and Chinese.
  • Multi-Turn Refinement Extension: The multi-turn extension tests whether agents revise reports according to follow-up instructions while preserving unaffected content.Revised reports receive five 0–20 scores for instruction compliance, evidence, logical consistency, content preservation, and expression; Tot. sums these dimensions.
  • Interactive Decision-Making Benchmarks: ALFWorld and WebShop test long-horizon state tracking, action selection, and external-environment interaction.ALFWorld uses multi-step household tasks, while WebShop requires search, product inspection, comparison, configuration, and constraint preservation.

B Baseline Implementation Details

The section groups baselines into four reasoning and coordination paradigms, then describes STACKPLANNER’s coordinator actions and prompt-driven memory operations. It also presents case studies illustrating active memory revision during medical QA and deep research.

  • Baseline paradigms: The baselines span Naive, Single-Agent, Multi-Agent, and Agentic-RL paradigms with distinct reasoning and coordination strategies.Naive methods use parametric knowledge or fixed retrieval; Single-Agent methods alternate reasoning and tools; Multi-Agent methods coordinate interacting agents; Agentic-RL methods learn multi-step decisions.
  • Naive: Naive baselines include Base, which uses only parametric knowledge, and FS-RAG, which retrieves evidence independently for each input sentence.
  • Single-Agent: Single-Agent baselines include ReAct’s interleaved reasoning and tool use and IRCoT’s alternating retrieval and chain-of-thought reasoning.
  • Multi-Agent: Multi-Agent baselines coordinate interacting agents through centralized or automated orchestration, including DAG-based MacNet, centralized RL-planner OWL, and MCTS-based AFlow.
  • Coordinator prompts: The coordinator supports PLAN, DELEGATE, SUMMARIZE, REFLECT, and FINISH actions, using task memory to determine coordination and revise stored context.
  • Case studies: Case studies show active memory revision narrowing noisy medical retrieval and retaining higher-quality task-oriented information during deep research and report synthesis.

E Ablation Study on Memory Components

The ablation study evaluates task and experience memory separately and jointly, while case analyses show how memory artifacts can bias coordinated decisions. Removing memory components degrades performance, especially when experience memory is absent.

  • Ablation results: Removing task memory consistently degrades performance across benchmarks, indicating its importance for maintaining intermediate reasoning structure.
  • Ablation results: Removing experience memory causes larger performance drops, suggesting that it is critical for cross-task generalization.
  • Ablation results: Removing both memory components produces significant deterioration, attributing observed gains to explicit memory modeling and control.
  • Case studies: Representative medical cases cover noisy user profiles, SOP overfitting, and retrieval or recall errors as memory-related failure modes.
  • Case studies: The case analyses recommend validating memory, structuring stored information, and using conservative reasoning under uncertainty.

F.3 Case 3: Retrieval and Recall Errors

The retrieval-and-recall case shows how incomplete patient-specific memory combined with generalized knowledge can produce an overly strong recommendation. The proposed safeguard is to verify provenance and uncertainty before deciding.

  • Failure mode: Incomplete patient-specific memory and generalized knowledge are combined to produce a strong affirmative recommendation about amoxicillin.
  • Safeguard: The recommended response is to verify allergy reaction type and timing, then provide conservative guidance when uncertainty remains.
  • Analysis: The stored patient-specific memory lacks essential details and provenance, while the system fails to validate completeness.
  • Conclusion: The broader conclusion is that memory errors can systematically bias decisions, requiring explicit validation, structured storage, and conservative reasoning.

G System-Level Efficiency and Overhead Analysis

The section evaluates efficiency, memory overhead, and preliminary generalization. STACKPLANNER reduces interaction costs through structured memory control while bounding retrieved context and maintaining comparable code-repair performance with a smaller model.

  • Efficiency: STACKPLANNER reduces token usage, response latency, and interaction steps compared with the ReAct baseline, despite adding REVISE operations.
  • Efficiency: Structured memory control produces more compact interaction trajectories by reducing redundant reasoning and unnecessary tool usage during training.
  • Memory overhead: Personal information memory uses a fixed budget below 512 tokens, while SOP and semantic memories are retrieved selectively from a Faiss vector database.
  • Memory overhead: Top-k retrieval keeps prompt length bounded as memory grows, with average retrieval latency within 800 ms and not a major system bottleneck.
  • Generalization: On SWE-bench-verified, STACKPLANNER achieves performance comparable to a significantly larger baseline despite using a 7B model.
  • Workflow context: The broader report-generation material describes multi-agent workflows that decompose tasks into collection, analysis, writing, formatting, and integrated output.
Loading 2601.05890v2…