Source-linked AI summary
Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in LLM Multi-Agent Systems
Shanglin Wu, Yuyang Luo, Yueqing Liang, Kaiwen Shi, Yanfang Ye, Ali Payani, Kai Shu
TL;DR
Existing work has largely treated team-size scaling and lifelong learning separately, leaving their interaction under realistic coordination and cost constraints unclear. This paper introduces LLMA-Mem and a joint scaling perspective that integrates structured memory across multi-agent teams. Across three MultiAgentBench environments, LLMA-Mem improves long-horizon performance while reducing token cost, and smaller teams can outperform larger ones when memory supports experience reuse.
Problem
Prior research largely studies team-size scaling and lifelong learning in isolation, leaving their interaction in multi-agent systems underexplored.
Method
The paper proposes LLMA-Mem, which integrates episodic, procedural, and transactive memory under flexible topologies for lifelong multi-agent learning.
Results
Across coding, research, and database environments on MultiAgentBench, LLMA-Mem improves long-horizon performance while reducing token cost by 9.4% to 71.7% relative to competing memory baselines.
Takeaways & Limitations
The interaction between team size and lifelong learning is non-monotonic: smaller teams can outperform larger teams when memory supports stable experience accumulation and reuse.
Takeaways & Limitations
The explored team scale reaches only seven agents, and evaluation covers only coding, research, and database environments in MultiAgentBench.
Abstract
from arXiv · showhide
Large language model (LLM) multi-agent systems can scale along two distinct dimensions: by increasing the number of agents and by improving through accumulated experience over time. Although prior work has studied these dimensions separately, their interaction under realistic cost constraints remains unclear. In this paper, we introduce a conceptual scaling view of multi-agent systems that jointly considers team size and lifelong learning ability, and we study how memory design shares this landscape. To this end, we propose \textbf{LLMA-Mem}, a lifelong memory framework for LLM multi-agent systems under flexible memory topologies. We evaluate LLMA-Mem on \textsc{MultiAgentBench} across coding, research, and database environments. Empirically, LLMA-Mem consistently improves long-horizon performance over baselines while reducing cost. Our analysis further reveals a non-monotonic scaling landscape: larger teams do not always produce better long-term performance, and smaller teams can outperform larger ones when memory better supports the reuse of experience. These findings position memory design as a practical path for scaling multi-agent systems more effectively and more efficiently over time.
1. Introduction
The introduction frames team size and lifelong learning as interacting scaling dimensions in LLM multi-agent systems. It proposes LLMA-Mem to improve long-horizon performance and efficiency through structured memory, while showing that scaling outcomes are non-monotonic.
- Prior research studies team-size scaling and lifelong learning largely in isolation, leaving their interaction in multi-agent systems underexplored.
- LLM multi-agent systems should be analyzed in a joint scaling space defined by team size and lifelong learning ability.
- LLMA-Mem combines episodic, procedural, and transactive memory under flexible topologies to support retention, transfer, and team coordination.
- MultiAgentBench evaluates sequential multi-agent task completion and inter-agent communication across multi-step environments.
- Larger teams do not always achieve better long-term performance because coordination overhead and fragmented information can weaken long-horizon gains.
- LLMA-Mem improves long-horizon performance while reducing token cost relative to existing memory baselines.The introduction reports a 71.7% token-cost reduction relative to competing memory baselines.
2. Related Works
Related work examines scalable collaboration, lifelong learning, and persistent memory for LLM agents. These strands motivate studying memory mechanisms that support continual improvement and cross-task reuse.
- Research on collaboration networks finds that increasing the number of agents can produce systematic gains with a saturating scaling pattern.
- Lifelong-learning research models continual improvement through the interaction of perception, memory, and action, often using external memory mechanisms.
- Memory systems such as Generative Agents and MemGPT give LLM agents persistent experience stores and mechanisms for reflection, retrieval, or hierarchical memory management.
3. Method
LLMA-Mem is a lifelong memory framework that separates task experiences, reusable procedures, and team capability knowledge while supporting multiple memory topologies. Its lifecycle retrieves relevant memories, updates experience and coordination statistics, and consolidates successful episodes into reusable procedures.
- Memory architecture: LLMA-Mem separates episodic, procedural, and transactive memory for complementary lifelong-learning roles.Episodic memory preserves task-level experiences, procedural memory abstracts reusable strategies, and transactive memory models agent capabilities and team coordination.
- Memory architecture: Episodic memory preserves full task trajectories as experiential material for later abstraction and case-based reasoning.Stored records include task context, team composition, actions, outcomes, environmental context, and extracted lessons.
- Memory architecture: Procedural memory stores generalized strategies abstracted from episodic experiences to improve retrieval efficiency and transfer across future tasks.Procedures track successful and failed applications, update times, and supporting source episodes; repeated successful patterns become explicit action templates.
- Memory architecture: Transactive memory represents who knows what and which team configurations work for which tasks, supporting decomposition, role assignment, and team formation.It models agent profiles and team patterns using accumulated competence and collaboration information.
- Memory topology configurations: The local, shared, and hybrid topologies control how memories are distributed and accessed across agents.Local stores preserve role-specific memories, shared stores enable collective access, and hybrid stores keep episodic histories local while sharing procedures and team statistics.
- Memory lifecycle: The memory lifecycle comprises retrieval, update, and consolidation phases.Retrieval queries procedural memory first and falls back to episodic memory; consolidation groups similar episodes, identifies recurring successful patterns, and abstracts candidate procedures.
4. Experiments
The experiments evaluate LLMA-Mem across models, environments, memory baselines, long-horizon learning, cost, team size, and memory-design ablations. LLMA-Mem generally improves sustained performance while reducing token usage, but team-size and topology effects are non-monotonic.
- Experimental Setup: LLMA-Mem is evaluated across MultiAgentBench environments using closed- and open-source models, with no-memory, MARBLE, and A-Mem as baselines.The evaluation covers coding, research, and database settings and uses identical inference settings across models.
- Main Results: Task Score results are best or tied-best for LLMA-Mem in most model–environment pairs, while Communication Score improvements vary by model and environment.Communication gains include DeepSeek-v3.2 on Research (+7.17) and Qwen3-32B-Instruct on Database (+10.11).
- Cost Analysis: Token usage decreases by 9.4% to 71.7% relative to competing memory baselines, partly because procedural consolidation reduces input-context overhead.The resulting efficiency–effectiveness trade-off avoids repeatedly replaying raw trajectories while preserving useful long-term knowledge.
- Influence of Team Size: Team-size scaling is non-monotonic: a 3-agent Qwen3-32B-Instruct team can outperform a 5-agent team, while increasing team size consistently raises computational cost.Qwen3-32B-Instruct at three agents is reported as a local cost–performance optimum among evaluated settings.
- Memory Ablations: Under controlled ablations, local memory topology performs best across all three metrics, and consolidation interval N=5 outperforms both more frequent and less frequent alternatives.The study attributes the topology result to role-specific memory formation and the interval result to a balance between evidence accumulation and update speed.
5. Conclusion
The paper frames multi-agent scaling jointly across team size and time, proposing LLMA-Mem to convert experience into reusable knowledge under coordination and cost constraints. Across benchmark environments, it improves long-horizon performance while reducing cost, and shows that larger teams are not always better.
- LLMA-Mem integrates episodic, procedural, and transactive memory under flexible topologies for lifelong multi-agent learning.The decomposition distinguishes components supporting lifelong learning from those required for long-horizon collaboration.
- LLMA-Mem improves long-horizon performance across coding, research, and database environments while reducing token cost by (↓9.4%) to (↓71.7%) versus competing memory baselines.
- Larger teams do not always achieve better long-term performance because communication overhead and information fragmentation can weaken long-term gains.
- Smaller teams can outperform larger ones when memory better supports stable accumulation and reuse of experience.
6. Limitations
The study’s evidence is bounded by limited team sizes, three benchmark environments, and no direct assessment of memory quality.
- Team-size experiments vary teams only up to seven agents, limiting conclusions about larger collectives with potentially stronger bottlenecks, specialization demands, and memory interference.
- Evaluation covers only coding, research, and database environments, excluding lifelong scenarios such as web search, embodied control, and personalization.
- The study measures overall task and communication performance without explicitly assessing memory redundancy, staleness, or retrieval-error robustness.
A. Details of LLMA-Mem Design
LLMA-Mem uses three memory types and persists memories across tasks through configurable topologies, with procedural memory distilling reusable strategies from prior trajectories.
- The implementation combines episodic, procedural, and transactive memory, retrieves the top-3 memory items by default, and uses local topology with consolidation interval N=5 in the main comparison.
- Local topology gives each agent private episodic, procedural, and transactive stores, whereas shared topology uses one common store and hybrid topology shares procedural memory and coordination statistics.
- Procedural memory is a compact, structured, reusable strategy distilled from multiple task trajectories rather than raw interaction logs.
- Procedural memories encode workflow patterns, success and failure counts, timestamps, source episodes, tool-use guidance, and failure categories to preempt.
B. Details of MultiAgentBench Implementation
Experiments use MultiAgentBench’s graph-coordinated coding, research, and database environments with standardized scoring, controlled team-size comparisons, and fixed generation settings.
- The benchmark evaluates coding, research, and database environments with a maximum of three iterations and graph-based communication structures.
- The team-size study uses 16 research tasks supporting at least seven agents and compares team sizes {1, 3, 5, 7} under the same evaluation pipeline.
- Task outcomes are scored by an evaluation model using task score and communication score, with Claude-Sonnet-4.5 serving as the unified evaluator across model backbones.
- Benchmark task content, agent profiles, and environment rules remain unchanged while experiments vary the memory mechanism or team size.
C. Experiments Implementation Details
The experiments compare memory-disabled and memory-enabled multi-agent baselines while holding the underlying workflow and benchmark conditions constant. LLMA-Mem is evaluated against conventional shared memory and associative memory configurations.
- W.o. Memory disables memory usage while retaining the same task definitions, agent profiles, and graph communication structure.
- MARBLE provides the original MultiAgentBench memory-enabled setup with its default shared-memory mechanism.
- A-Mem uses an evolving associative memory structure with official configuration settings and retrieval top-3.The configuration uses Titan-textembeddings-v2, a maximum memory context of 5, and link threshold 0.72.
D. Prompt and Evaluation Notes
The appendix documents how LLMA-Mem constructs action prompts, inserts episodic and procedural memory, extracts lessons, and consolidates successful experiences into reusable procedures.
- D.2. Agent Action Prompt: At action time, LLMA-Mem combines agent information, optional reasoning instructions, retrieved memory, the current task, and other-agent descriptions in one user prompt.
- D.2. Agent Action Prompt: The agent action template presents the agent identity and profile, reasoning prompt, past experience, and available interacting agents.
- D.3. Inserted Memory Context: Retrieved memory is inserted in type-specific blocks for episodic experiences and procedural strategies.
- D.3. Inserted Memory Context: Procedural memory entries include a procedure title, success rate, and strategy content.
- D.4. Lesson Extraction Prompt: After each task, LLMA-Mem extracts one to three actionable lessons from the task experience and returns them as a JSON array.The extraction prompt uses task context, actions, outcome, and a success-or-failure focus, emphasizing concrete tool calls and strategies.
- D.4. Lesson Extraction Prompt: The lesson-extraction instructions prioritize concrete actions and specific next steps over vague advice.
- D.5. Procedural Consolidation Prompt: Every N tasks, LLMA-Mem consolidates successful episodic experiences into generalized, reusable procedures and skills.The consolidation prompt supplies task descriptions, lessons, and outcomes, then requests a titled strategy in JSON format.
- D.5. Procedural Consolidation Prompt: Procedure generalization is instructed to produce actionable strategies for similar future situations while avoiding vague advice.
D.1. Inherited Benchmark Task Prompt
LLMA-Mem inherits the benchmark’s task prompts and adds memory context without rewriting the downstream task description.
- Coding, research, and database experiments use task prompts inherited directly from the released MultiAgentBench configurations.Each task supplies its objective, environment specification, output format, and agent profiles.
- LLMA-Mem prepends memory context while preserving the benchmark’s downstream task description.
- The benchmark-provided task description, agent profile, environment rules, and output requirements define the task prompt components.