Why Do Multi-Agent LLM Systems Fail?

Mert Cemri; Melissa Z. Pan; Shuyi Yang; Lakshya A. Agrawal; Bhavya Chopra; Rishabh Tiwari; Kurt Keutzer; Aditya Parameswaran; Dan Klein; Kannan Ramchandran; Matei Zaharia; Joseph E. Gonzalez; Ion Stoica

Source-linked AI summary

Why Do Multi-Agent LLM Systems Fail?

Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica

arXiv:2503.13657v3Mar 17, 2025cs.AI

TL;DR

MAS often provide minimal gains and exhibit substantial failure rates, motivating a principled account of why they fail. The paper constructs MAST-Data and MAST through empirical analysis and expert validation, then scales annotation with an LLM pipeline. The resulting resources support systematic failure diagnosis and indicate that reliable MAS may require structural redesign beyond isolated fixes.

Problem
MAS often show minimal performance gains over single-agent frameworks or best-of-N sampling, while robust-system construction lacks clear consensus.
Method
The authors analyze over 150 traces to build and validate MAST, assemble 1642 traces from 7 MAS frameworks, and develop an LLM-as-a-Judge pipeline for scalable annotation.
Results
+9.4% increase in overall task success rate follows a MAST-guided workflow adjustment giving ChatDev’s CEO agent final say.
Takeaways & Limitations
MAST-Data, MAST, and the LLM annotator provide a standardized empirical basis for diagnosing MAS failures and guiding more robust system development.
Takeaways & Limitations
The authors acknowledge that MAST is not exhaustive and focus on failures addressable through system design, coordination, and verification improvements.

Abstract

from arXiv · show

Despite enthusiasm for Multi-Agent LLM Systems (MAS), their performance gains on popular benchmarks are often minimal. This gap highlights a critical need for a principled understanding of why MAS fail. Addressing this question requires systematic identification and analysis of failure patterns. We introduce MAST-Data, a comprehensive dataset of 1600+ annotated traces collected across 7 popular MAS frameworks. MAST-Data is the first multi-agent system dataset to outline the failure dynamics in MAS for guiding the development of better future systems. To enable systematic classification of failures for MAST-Data, we build the first Multi-Agent System Failure Taxonomy (MAST). We develop MAST through rigorous analysis of 150 traces, guided closely by expert human annotators and validated by high inter-annotator agreement (kappa = 0.88). This process identifies 14 unique modes, clustered into 3 categories: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification. To enable scalable annotation, we develop an LLM-as-a-Judge pipeline with high agreement with human annotations. We leverage MAST and MAST-Data to analyze failure patterns across models (GPT4, Claude 3, Qwen2.5, CodeLlama) and tasks (coding, math, general agent), demonstrating improvement headrooms from better MAS design. Our analysis provides insights revealing that identified failures require more sophisticated solutions, highlighting a clear roadmap for future research. We publicly release our comprehensive dataset (MAST-Data), the MAST, and our LLM annotator to facilitate widespread research and development in MAS.

1 Introduction

MAS coordinate multiple LLM agents for decomposition, parallelization, context isolation, specialized ensembling, and diverse reasoning, yet often deliver minimal gains and high failure rates. This paper introduces datasets, a taxonomy, and scalable annotation tools to systematically diagnose these failures and guide more reliable system design.

Motivation: MAS are collections of interacting agents orchestrated for collective intelligence across software engineering, drug discovery, scientific simulation, and general-purpose tasks.The paper defines agents through prompt specifications, conversation traces, and environmental actions such as tool use.
Motivation: 41% to 86.7% failure rate is observed across 7 state-of-the-art open-source MAS, alongside minimal gains over single-agent frameworks and best-of-N sampling.The paper reports no clear consensus on how to build robust and reliable MAS.
Contributions: 1642 annotated execution traces from 7 MAS frameworks cover coding, math problem-solving, and general-agent functionalities.The authors also release a smaller human-annotated dataset from expert agreement studies.
Contributions: MAST is an empirically grounded taxonomy developed from close analysis of over 150 traces to standardize MAS failure identification and classification.The construction addresses root-cause verification difficulties and the absence of standardized failure definitions.
Contributions: +9.4% increase in overall task success rate follows a workflow adjustment giving the CEO agent final say in ChatDev.The authors caution that robust reliability generally requires more than isolated fixes and may require fundamental MAS redesigns.
Contributions: MAST-Data, MAST, and the LLM annotator are released as resources for diagnosing failure dynamics and developing more robust MAS.The paper frames these resources as a basis for concrete research problems in MAS design and reliability.

2 Related Work

Prior work addresses specific agentic-system challenges, evaluation, debugging, and design principles, but systematic characterization of autonomous MAS failures remains limited. MAST-Data and MAST target this gap with an empirical dataset and taxonomy focused specifically on MAS failure patterns.

Scope of prior work: Existing work studies workflow memory, programming-agent flows, state control, and broader MAS challenges, but does not provide this paper’s comprehensive empirical focus on MAS failures.The cited efforts either target specific capabilities or offer high-level surveys.
Design principles: Single-agent design guidance emphasizes modularity and avoiding excessive complexity, while this work extends systematic failure analysis to multi-agent systems.The paper collects and analyzes a large corpus of MAS failure instances to provide structured understanding.
Evaluation and debugging: AgentEval, AGDebugger, and Who&When-related work address evaluation, message-history debugging, or specialized failure summarization rather than the paper’s taxonomy-centered MAS failure dataset.These systems contribute complementary tools or datasets with narrower goals.

3 The Multi-Agent Systems Dataset

MAST-Data is constructed through grounded analysis, taxonomy development, expert agreement studies, and scalable LLM annotation. The resulting resource contains 1642 traces from 7 MAS frameworks across coding, math, and general-agent tasks.

Dataset construction: 1642 annotated execution traces from 7 popular MAS frameworks form MAST-Data across coding, math problem-solving, and general-agent functionalities.The dataset combines frameworks from initial studies, generalization validation, and Manus.
Dataset construction: MAS failure annotation is difficult because convoluted agent interactions compound individual model behaviors with overall system design, while standardized definitions are absent.Consequently, root-cause analysis requires understanding system dynamics rather than simple error detection.
Empirical identification: The methodology begins with 150 traces from five MAS frameworks, closely examined by six human experts using theoretical sampling and grounded-theory techniques.The process seeks diverse and generalizable failure patterns rather than artifacts of one system.
Taxonomy development: MAST supplies clear, empirically grounded failure labels for consistent annotation across different MAS.The taxonomy is designed as a common framework for understanding and standardizing observed failure patterns.
Taxonomy validation: κ = 0.88 is achieved in final inter-annotator agreement rounds after iteratively refining MAST definitions with three expert annotators.Three rounds required about 10 hours of disagreement resolution, excluding annotation time.
Scalable annotation: 94% accuracy and Cohen’s Kappa of 0.77 are achieved by the LLM annotator against expert human annotations.The pipeline uses traces, MAST definitions, and few-shot examples to scale failure labeling.
Generalization validation: Out-of-domain validation on OpenManus and Magentic-One with MMLU and GAIA yields Cohen’s Kappa of 0.79 in an additional human agreement round.These systems and benchmarks were not part of the initial MAST development.

4 The Multi-Agent System Failure Taxonomy

MAST is an empirically grounded taxonomy for diagnosing MAS failures across system design, inter-agent coordination, and task verification. It identifies 14 failure modes and shows that failures arise from design and interaction problems as well as verification weaknesses.

MAST provides a structured framework for defining, understanding, and annotating common MAS failures.
MAST identifies 14 fine-grained failure modes across three overarching categories and maps them to pre-execution, execution, and post-execution stages.
System Design Issues: System design failures include disobeying task or role specifications, repeating steps, losing context, and failing to recognize task completion.
Inter-Agent Misalignment: Inter-agent misalignment includes conversation resets, missing clarification, task derailment, information withholding, ignored input, and reasoning-action mismatches.
Task Verification: Task-verification failures include premature termination, incomplete verification, and incorrect verification, with these modes accounting for 6.20%, 8.20%, and 9.10% respectively.Explicit verifiers generally show fewer total failures, but verification alone does not guarantee high MAS success.

5 Towards better Multi-Agent LLM Systems

MAST-Data reveals that MAS failure profiles vary by architecture, model, and task, so improvement requires system-specific diagnosis rather than a universal fix. Case studies show that targeted workflow and verification changes can improve success, while substantial reliability gains remain difficult.

Failure distributions differ markedly across MAS, reflecting distinct architectural characteristics and design philosophies.Figure 4 illustrates system-specific failure profiles rather than performance comparisons because systems may use different tasks and benchmarks.
AppWorld frequently exhibits premature termination, while OpenManus and HyperAgent show prominent step-repetition problems.
Within MetaGPT programming tasks, GPT-4o shows 39% fewer System Design Issues failures than Claude 3.7 Sonnet.
On ProgramDev, MetaGPT has 60-68% less failure in FC1 and FC2 but 1.56x more FC3 failure than ChatDev.
MAST enables quantitative failure-profile analysis and guides targeted MAS improvements through workflow and prompt interventions.
With the same underlying model, intervention case studies achieve maximum improvements of 15.6%, but not all failure modes are resolved and task completion remains low.The authors conclude that high reliability may require combinatorial changes spanning agent organization and model improvements.

6 Conclusion

The paper presents MAST-Data and MAST as public resources for systematically studying why MAS fail. Together, they provide annotated evidence, a structured failure vocabulary, and a basis for diagnosing and improving multi-agent systems.

The study introduces over 1600 annotated execution traces from 7 popular MAS frameworks to outline MAS failure dynamics.
MAST is built through Grounded Theory analysis of 150 traces and identifies 14 distinct failure modes across 3 categories.Its definitions were validated with strong inter-annotator agreement, and an LLM annotator supports scalable annotation.
MAST-Data and MAST provide empirical grounding and a foundational framework for future MAS research.
System Design Issues: The taxonomy describes system-design failures involving architecture, conversation management, task specifications, and agent roles.
Inter-Agent Misalignment: Inter-agent failures involve ineffective communication, poor collaboration, conflicting behavior, and derailment from the initial task.

A.3 FC3. Task Verification

Task-verification failures arise when systems terminate prematurely or inadequately check the correctness, completeness, and reliability of outputs. MAST distinguishes three verification failure modes.

Task-verification failures include premature termination and insufficient mechanisms for guaranteeing accurate, complete, and reliable outcomes.
MAST identifies three task-verification failure modes.
The modes are premature termination, no or incomplete verification, and incorrect verification.

B Details of Multi-Agent Systems Evaluated

The study evaluates seven open-source MAS frameworks across coding, math, and general-agent settings, while excluding closed-source systems from primary trace analysis when their internals are inaccessible.

Figure 5 reports failure rates for six MAS using GPT-4o and Claude-3.7-Sonnet, but the benchmarks differ and are not directly comparable.
Seven open-source frameworks are evaluated, with their architectures and purposes documented for comparison.
MetaGPT simulates a software company whose specialized agents collaboratively solve natural-language programming tasks using role-specific standard operating procedures.
ChatDev organizes software development into design, coding, and testing phases with role-based agents and multi-turn communicative dehallucination.
HyperAgent uses Planner, Navigator, Code Editor, and Executor agents with specialized tools, standardized messages, and subtask queues.
Closed-source systems such as Manus are excluded from primary MAST-Data analysis because undisclosed models and inaccessible execution traces prevent reliable fine-grained annotation.

E MAS Failure Modes Correlation

MAST’s failure categories show limited overlap, supporting distinct categories, while specific modes with similar symptoms can still challenge automated root-cause annotation.

0.17-0.32 correlations between categories suggest that MAST captures distinct aspects of MAS failures with limited overlap.
Similar surface behaviors can arise from different root causes, including memory management or agent coordination.
Specific failure modes show correlations up to 0.63, which may cause automated evaluators to conflate distinct root causes.

F Understanding Failures: The Impact of Different LLMs and Agent Architectures

Failure patterns vary with both the underlying LLM and MAS architecture, while interventions reduce failures and topology changes outperform prompt changes in the reported case studies.

LLM choice: GPT-4o produces fewer FC1 and FC2 failures than Claude 3.7 Sonnet within MetaGPT, while both models retain many FC3 failures.
MAS architecture: MetaGPT has fewer FC1 and FC2 failures than ChatDev with GPT-4o, whereas ChatDev has fewer verification issues.
Interventions: Tactical interventions modify prompts, agent topology, or conversation management to target specific failure modes.
Structural strategies: Weak verification mechanisms are identified as significant contributors to MAS failures, motivating more involved structural solutions.
Case-study boundary: Prompt refinement and clear role definitions can reduce failures, but their effectiveness varies with the underlying LLM.
Interventions: ChatDev’s redesigned topology changes the workflow from a DAG to a cyclic graph and terminates after CTO confirmation or an iteration cutoff.
Interventions: Both prompt and topology interventions decrease observed failure modes, with topology-based changes reported as more effective for AG2 and ChatDev.

I Analysis on Multi-Agent Systems with Open-Source Models

Open-source models differ substantially in robustness: Qwen2.5-Coder-32B-Instruct has fewer failures than CodeLlama-7b-Instruct-hf, yet both trail the leading closed-source models analyzed.

Qwen2.5-Coder-32B-Instruct is substantially more robust than CodeLlama-7b-Instruct-hf, exhibiting far fewer failures overall.
Both open-source models produce failures more frequently than GPT-4o and Claude-3 in the paper’s analysis.
Table 6 groups failure-mode occurrences across 400 traces by Qwen versus CodeLlama and ChatDev versus MetaGPT.

J.1 How Indicative are Different Failure Modes of Actual Success?

Failure modes occur in both successful and unsuccessful MAS runs, but their frequencies and apparent consequences differ across failure types and benchmarks. The analysis also examines annotation costs and the roles and verification procedures used in the example systems.

Failure modes and task success: Successful runs still contain failures, although unsuccessful traces have a higher overall failure frequency.The analysis reports that a greater number of failures signals a higher chance of final task failure.
Failure modes and task success: Failures involving termination conditions and information withholding appear almost exclusively in failed runs.The paper identifies 1.5 Unaware of Termination Conditions and 2.4 Information Withholding as failures likely to derail tasks.
Failure modes and task success: Verification failures occur frequently even in successful runs, revealing flaws that do not necessarily prevent task completion.The cited examples are 3.2 No or Incomplete Verification and 3.3 Incorrect Verification.
Benchmark variation: More challenging benchmarks produce higher failure rates, while failure profiles also vary across MMLU, GSM, and OlympiadBench.AG2 and GPT-4o are fixed while the benchmark changes; GSM has fewer Inter-Agent Misalignment and Specification failures than MMLU and Olympiad.
Annotation cost: The LLM-as-a-Judge pipeline costs $1.8 on average per MAS, with cost depending strongly on trace length.Table 9 reports costs normalized by the number of traces collected for each MAS framework.
Agent workflow: The example agent workflow assigns separate problem-solving, code-execution, and verification roles, with the verifier requiring both proposed solutions before deciding.The verifier is instructed to avoid assumptions about variables and to suggest the next speaker when a solution is missing.

L.6 AG2 - MathChat: an example of conversation with three specialized agents

The AG2 MathChat example coordinates specialized agents to solve a word problem: one reasons, one executes Python, and one verifies the result. The agents converge on the answer that Sara must mow the lawn 17 times.

Problem setup: Sara’s total purchase cost is $320, while savings and babysitting provide $150 before mowing.The calculation combines a $120 jacket, two $100 shoe pairs, $100 saved, and $50 earned from babysitting.
Solution: Sara must mow the lawn 17 times to cover the remaining $170 at $10 per mowing.The problem solver computes $170 / $10 = 17, and the code executor independently produces 17.
Verification: The Code Executor reports successful execution with output 17.The verifier records exitcode: 0, indicating that the submitted program executed successfully.
Verification: The verifier accepts the solution after both paths agree and presents the final result as SOLUTION_FOUND 17.The verifier states that the code result supports the problem solver’s proposal.
Agent roles: ChatDev’s specialized roles include Programmer, Code Reviewer, Software Test Engineer, CEO, and CTO, each with distinct software-development responsibilities.The role prompts emphasize coding, review, testing, organizational decisions, and technology infrastructure respectively.

N.1 Example for FM-1.3 Step repetition, FM-2.1 Conversation reset, FM-1.2 Disobey role specification

The examples show MAS failures as repeated reasoning, lost or ignored context, role and task deviations, and incorrect verification across coding and math tasks.

FM-1.3 Step repetition and FM-2.1 Conversation reset: HyperAgent repeated the same matplotlib diagnosis, while another agent pursued unrelated steps and ultimately waited for the user.The repeated diagnosis concerns the Line3D error; the trace also records failure to communicate a proposed solution to the Planner.
FM-1.2 Disobey role specification: Agents answered different questions despite reaching correct intermediate solutions, including a math solution followed by an unrelated final answer.One trace explicitly says the agents solved another problem; another reports finding the solution correctly before answering a different problem.
FM-1.1 Disobey task specification: A system used a password as an access token while retrieving account credentials, illustrating a task-specification failure in tool use.The trace records both the mistaken token choice and the credential-retrieval action.
FM-2.4 Information withholding: Other traces show architectural and communication failures: an editor claimed an edit that was not made, and HyperAgent agents failed to share or correctly interpret information.The HyperAgent example includes Navigator-to-Planner noncommunication and the Planner misreading an example output as passed tests.