Source-linked AI summary
SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
Rui Yang, Junjie Xu, Zhengyu Liu, Neil Fendley, Yang Hong, Ziyang Li, Yinzhi Cao
TL;DR
Safe agents can fail together when information, state, decisions, and authority cross principal boundaries, while observed failures do not by themselves establish a genuinely multi-agent effect. This SoK analyzes 197 works through an execution-centered A →I →R framework, organizes defenses around a five-part contract, and audits evaluation practice. It concludes that MAS security analysis should trace attacks end to end, test path closure with appropriate counterfactuals, and evaluate system-level effects using comparable, diagnostic evidence.
Problem
Existing surveys leave unclear how attacks move through multi-agent systems, making it difficult to distinguish interaction-dependent effects from failures that would occur without interaction.
Method
The paper treats complete multi-agent execution as its unit of analysis, tracing adversarial influence from adversary position through interaction interfaces to system-level risk.
Results
Across 197 works, the synthesis organizes MAS security through configurations, six interfaces, four adversary positions, seven risk families, eight attack paths, defenses, and evaluation evidence.
Takeaways & Limitations
Security analysis should examine how influence crosses principal boundaries, what state and authority remain reachable, whether defenses close the path, and whether evaluations verify the resulting system-level effect.
Takeaways & Limitations
Attack Success Rate definitions vary across benchmarks, so reported values cannot be directly compared, ranked, or aggregated unless their evaluation units and success conditions match.
Abstract
from arXiv · showhide
Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation. Together, these findings motivate an interaction-aware view of MAS security: trace attacks end to end, test whether defenses close those paths, and evaluate system-level effects with appropriate counterfactuals.
1 Introduction
Multi-agent systems can turn locally safe agents into system-level failures through cross-principal interactions. The paper addresses this with an execution-centered framework that traces adversarial influence from entry through interaction to risk, then connects defenses and evaluation to the same path.
- Motivation: Locally safe agents can fail together when messages, shared state, aggregation, or delegation propagate influence across principals.Benign fragments, truthful reports, and attacker-controlled content can become harmful through composition and authority transfer.
- Motivation: Existing surveys catalog components or multi-agent concerns, but leave unclear where attacks enter, which interfaces they cross, and what system-level failure they cause.Without this execution view, a failure observed in a multi-agent setting may be mistaken for evidence that interaction caused it.
- Framework: The A →I →R framework traces adversarial influence from four adversary positions through six interaction interfaces to seven system-level risk families.Eight configuration dimensions condition executions, while eight non-exclusive attack paths summarize recurring execution patterns.
- Defenses: Defenses are organized around the path targeted, observable information, intervention point and timing, trust boundary, and possible recovery.The framework maps control primitives such as authority enforcement, interaction-structure control, detection, state governance, and recovery onto attack paths.
- Evaluation: The paper audits 44 evaluation and benchmark works, identifying open problems in interaction-specific comparisons, metric comparability, benchmark portability, and open-system evaluation.The evaluation agenda emphasizes determining whether interaction leaves a failure unchanged, amplifies it, creates it through composition, or defines the protected property itself.
2 Overview
The SoK defines an MAS by interaction among separately addressable principals rather than by model count, and distinguishes interaction-dependent security effects from inherited ones. It positions its contribution as an execution-based synthesis across prior work and a corpus of 197 studies.
- 2.1 Scope: An MAS contains at least two separately addressable principals whose outputs can affect one another through messages, shared state, collective decisions, or delegated actions.Systems may share an LLM backend; pipelines remain out of scope when all steps stay within one principal.
- 2.1 Scope: Interaction effects are classified as inherited, interaction-amplified, composition-induced, or structurally multi-agent.The categories distinguish unchanged failures from altered likelihood, reach, persistence, privilege, impact, or properties defined by relations among principals.
- 2.2 Related Work: Table 1 compares prior multi-agent security syntheses across analytical dimensions using systematic, partial, and not-systematically-treated coverage markers.These markers describe analytical coverage rather than ranking the cited works’ quality.
- 2.2 Related Work: The paper organizes prior syntheses around cross-principal execution rather than primarily around components, attack names, design patterns, or governance concerns.The same model connects adversarial entry, interaction interfaces, protected effects, defensive intervention, assumptions, and supporting evidence.
3 System Model and Interaction Surface
The system model separates execution conditions from security-relevant transitions. Configuration dimensions shape an attack’s reach, persistence, and impact, while interaction interfaces describe how information, state, authority, decisions, and controls cross boundaries.
- Configuration: Configuration dimensions C1–C8 describe how an MAS is arranged and condition the reach, persistence, and impact of an execution.Multiple dimensions may change together, so comparisons should isolate one dimension when possible and report coupled changes otherwise.
- Configuration: Communication topology specifies which principals can communicate directly and which paths connect them.Common arrangements include complete, chain, ring, tree, and star topologies.
- Configuration: Principal composition captures participating roles and independence, including homogeneous or heterogeneous models and generalist or specialist roles.Shared models, prompts, or evidence sources can affect the execution context.
- Interaction Interfaces: Interaction interfaces I1–I6 mark transitions through which inputs, messages, state, authority, collective decisions, and security controls move.I6 covers detection and intervention during the execution.
- Interaction Interfaces: Defenses can monitor, block, isolate, revoke, roll back, or recover, but only within the information they observe and the execution components they control.Prevention requires intervention before the protected effect becomes irreversible.
4 Attack Landscape
The attack landscape traces how adversarial influence enters multi-agent systems, crosses interaction boundaries, and produces system-level security failures. It emphasizes recurring propagation, manipulation, confidentiality, delegation, and collective-decision mechanisms, while warning that observed paths do not by themselves establish interaction-dependent causation.
- Threat model: MAS threat models distinguish four adversary positions: external, user-level, member, and infrastructure adversaries.These positions differ by where control begins and what system components the adversary can affect.
- System-level risks: Figure 2 shows covert coordination: protocol-valid messages between compromised principals can make apparent agreement non-independent evidence.This illustrates how strategically dependent principals can undermine collective conclusions without visibly invalid messages.
- System-level risks: Seven risk families capture consequences including containment failure, loss of independence, integrity failure, confidentiality breach, and availability or resource-isolation failures.The same risk may arise through different interfaces, and one execution may trigger several risks.
- Attack paths: The corpus identifies eight recurring attack paths connecting adversary positions to risks through one or more interaction interfaces.Paths are recurring mechanisms rather than mutually exclusive categories, so one execution may contain several paths.
- Attack paths: Propagation and persistence can spread a local compromise through communication, retrieval, generated content, or agent state, with topology affecting reach and shared state affecting persistence.These effects may continue after the original source disappears.
- Attack paths: Collective manipulation can produce strong agreement despite compromised or non-independent evidence, while delegated influence can reach privileged tools through trusted principals.Figure 3 notes that three malicious agents among seven violate the classical f < n/3 condition, and control-flow hijacking exemplifies an A1 →I1 →I2 →I4 path to authorization failure.
- Cross-cutting findings: Execution traces show where harm travels, but counterfactual comparisons are needed to determine whether interaction caused or amplified the failure.The protected object may include independence, topology, private context, or delegated authority, even when the final answer appears correct.
5 Defense Landscape
The defense landscape is organized around a five-part contract and four control primitives that describe how defenses target, observe, interrupt, and recover from multi-agent attack paths. Comparing these defenses reveals persistent gaps in end-to-end path closure, observation and intervention adequacy, mediation, and recovery.
- Defense contract: The five-part defense contract covers path target, observation, intervention, trust boundary, and recovery.It provides a common basis for comparing how protections are obtained and bounded across multi-agent executions.
- Control primitives: Four control primitives organize defenses by the main object they control, while adaptive defense allocation addresses where limited capacity should be concentrated.The primitives are not mutually exclusive, and one system may combine several.
- D1 Identity, Policy & Authority Enforcement: D1 controls identity, policy, and authority at admission and authorization boundaries, using identity, provenance, intent, and scope.Representative work extends these controls to delegation chains, identity-aware routing, context scoping, governance, and execution-time restrictions on downstream invocations.
- D2 Trust, Communication & Aggregation: D2 changes communication, contributor influence, or aggregation, but reconfiguring trust or topology does not remove manipulated evidence already committed to persistent state or collective decisions.Its guarantees depend on assumptions about identity, membership, fault bounds, observability, and sometimes synchrony.
- D3 Detection, Attribution & Containment: D3 detects harmful influence and intervenes across propagation, routing, and collective manipulation, including mechanisms that reconstruct flows or intercept predicted state changes before message delivery.Its evidence spans communication structure, message and response signals, peer history, anomaly scores, and cross-principal propagation.
- Cross-cutting findings: Path closure remains unresolved because blocking one transition does not establish end-to-end protection when another route remains reachable.Defense effectiveness also depends on intervention timing and observation, while recovery requires tracing descendants, revoking inherited authority, repairing state, and verifying restoration.
6 Benchmarks and Evaluation
The audit finds broad but uneven evaluation coverage across MAS designs and security questions. Four persistent gaps concern isolating interaction effects, making metrics comparable and diagnostic, enabling benchmark reuse, and evaluating open-system operation.
- Evaluation coverage: 44 works evaluate MAS attacks, defenses, or security properties across varied system designs and threat models.Forty-three were reviewed through paper PDFs and one through a Zenodo software archive.
- Evaluation coverage: 42 works evaluate an attack or failure, including 16 that also evaluate a defense.CalBench and MAGPIE evaluate privacy without an explicit attacker.
- Observation and artifacts: 20 evaluations use full traces, 20 use principal-level messages or actions, two use episode outcomes, and two use final outputs.Thirty-one works identify code, data, logs, test tools, or project pages, including 26 with exact URLs.
- Isolating multi-agent effects: Interaction-specific comparisons remain limited: six of 21 works compare MAS execution with a single-principal baseline, while 12 of 36 relation-focused studies vary the relation under study.Matched single-principal comparisons distinguish inherited from interaction-amplified effects; relation-focused comparisons are needed when no meaningful single-principal counterpart exists.
- Comparable and diagnostic metrics: ASR values are not directly comparable across benchmarks unless their evaluation units and success conditions match.An end-to-end ASR also cannot distinguish an attack stopped at entry from one spreading across several agents before being blocked.
- Comparable and diagnostic metrics: Full execution traces do not necessarily provide causal provenance: Agent-BOM is the only one of 20 full-trace records coded as reporting an explicit dependency or provenance graph.Traces can show events across principals without showing how those events depend on one another.
- Configurable and extensible benchmarks: 39 works compare multiple conditions, but such within-benchmark comparisons do not establish configurability, extensibility, or portability across systems.ACIARena, RiskLab, and A2ASecBench expose extension points, but differing formats would require adapters for component transfer.
- Open-system evaluation: 42 works use fixed membership, while two allow limited runtime spawning under one administrator, leaving changing membership and cross-domain trust under-evaluated.Current benchmarks largely begin after trust relationships are established, providing little evidence about entry, changing trust, or access removal.
7 Conclusion
The SoK frames MAS security around end-to-end execution across configuration, interaction interfaces, attack paths, defenses, and evaluation. It emphasizes how adversarial influence crosses principal boundaries, what remains reachable, whether defenses close paths, and whether evaluations verify system-level effects.
- 7 Conclusion: The execution-centered view connects system configuration, interaction paths, security effects, defenses, and evaluation across 197 works.The analysis organizes MAS security around how adversarial influence crosses principal boundaries and what state and authority remain reachable.
Ethical Considerations
The paper synthesizes publicly available research without human-subject studies, private user data, or experiments against live or third-party systems. Because the topic is dual use, it emphasizes security-oriented analysis.
- Ethical Considerations: The study uses published papers, public artifacts, advisories, and bibliographic metadata rather than human-subject data or live-system experiments.The authors identify the organization of attack mechanisms and weaknesses as inherently dual use.
Open Science
The literature corpus and associated artifacts will be released publicly with the final version of the paper.
- Open Science: The corpus and associated artifacts are planned for public release with the final paper.
A Evaluation Audit Details
Table 4 maps 44 security-evaluation works across attack and defense roles, observation depth, comparison design, membership setting, artifact availability, reported system information, and A-I-R taxonomy assignments.
- Audit scope: 44 works are classified by adversary position, configuration dimensions, interaction interfaces, attack paths, and system-level risks.The audit records A1-A4, C1-C8, I1-I6, P1-P8, and R1-R7 assignments for each work.
- Audit dimensions: Role distinguishes attack or failure evaluation, defense evaluation, and security-property evaluation without an explicit attacker.The table codes these roles as A, D, and S.
- Audit dimensions: Observation depth ranges from final outputs and episode outcomes to principal or message events and full execution traces.Obs. records the deepest data consumed by each evaluation.
- Audit dimensions: Comparison design records attack or component controls, defense or component controls, matched single-principal comparisons, relation-focused MAS comparisons, or no comparison.These categories are coded AC, DC, SP, Rel, and none.
- Audit dimensions: The audit also records fixed membership or limited runtime spawning, evaluated-artifact availability, and whether principal count, model information, and communication topology are reported.Artifact status distinguishes exact URLs, claimed artifacts, upstream-only URLs, and no identified artifact.