Source-linked AI summary
G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systems
Shilong Wang, Guibin Zhang, Miao Yu, Guancheng Wan, Fanci Meng, Chongye Guo, Kun Wang, Yang Wang
TL;DR
MAS improve complex task-solving through collaboration but introduce vulnerabilities to adversarial attacks and information propagation. G-Safeguard uses a multi-agent utterance graph, GNN-based anomaly detection, and topological intervention for remediation. Experiments report effective defense across attack strategies, LLM backbones, topologies, and MAS scales, while the method cannot preemptively prevent initial compromise.
Problem
Multi-agent interactions create additional security risks beyond single-agent vulnerabilities, while existing defenses often overlook MAS topology and transferability across configurations.
Method
G-Safeguard analyzes multi-agent utterance graphs with an edge-featured GNN to identify high-risk agents and applies topological intervention to limit malicious information propagation.
Results
G-Safeguard reduces attack success across prompt injection, tool, and memory attacks, generalizes across LLMs and topologies, and scales to large MAS.
Takeaways & Limitations
The framework provides topology-aware detection and remediation that can transfer from small-scale training to larger MAS and integrate with mainstream MAS pipelines.
Takeaways & Limitations
G-Safeguard cannot preemptively prevent compromise because it analyzes communication data after some nodes may already be compromised.
Abstract
from arXiv · showhide
Large Language Model (LLM)-based Multi-agent Systems (MAS) have demonstrated remarkable capabilities in various complex tasks, ranging from collaborative problem-solving to autonomous decision-making. However, as these systems become increasingly integrated into critical applications, their vulnerability to adversarial attacks, misinformation propagation, and unintended behaviors have raised significant concerns. To address this challenge, we introduce G-Safeguard, a topology-guided security lens and treatment for robust LLM-MAS, which leverages graph neural networks to detect anomalies on the multi-agent utterance graph and employ topological intervention for attack remediation. Extensive experiments demonstrate that G-Safeguard: (I) exhibits significant effectiveness under various attack strategies, recovering over 40% of the performance for prompt injection; (II) is highly adaptable to diverse LLM backbones and large-scale MAS; (III) can seamlessly combine with mainstream MAS with security guarantees. The code is available at https://github.com/wslong20/G-safeguard.
1 Introduction
LLM-based multi-agent systems extend individual-agent capabilities through interaction, but those interactions introduce additional security risks and make topology-aware detection and remediation necessary. G-Safeguard addresses these risks with graph-based anomaly detection and topological intervention, showing effectiveness across attacks, models, and system scales.
- Motivation: Multi-agent interactions enhance collective intelligence and complex task-solving but can transmit toxicity, misinformation, or malicious behavior across the system.An attacked agent may spread harmful information, potentially paralyzing the MAS and triggering collective malicious behavior.
- Security challenges: MAS security is harder than single-agent defense because it requires both locating anomalies and mitigating their propagated impact.Existing defenses often target specific units while overlooking communication structure and remediation.
- Security challenges: Topology-aware detection must incorporate neighboring agents’ relationships and behaviors to address MAS-specific anomalies.The interconnected structure of MAS makes topology a critical component of anomaly diagnosis.
- Security challenges: Custom-designed defenses hinder inductive transferability across the many agent, external-unit, and configuration combinations found in MAS.This transferability challenge applies especially to large-scale MAS applications.
- G-Safeguard: G-Safeguard constructs a multi-agent utterance graph, detects high-risk agents with an edge-featured GNN, and applies topological interventions to disrupt harmful propagation.Security assessments occur at the conclusion of each dialogue round in a multi-turn MAS.
- Evaluation: 12.50% ∼33.23% of infections in chain structures and 10% ∼38.52% in star structures were blocked on MMLU, while ASR fell across multiple attack types and large-scale settings.Prompt-injection ASR reductions were 21.38% on CSQA, 22.01% on MMLU, and 19.50% ∼39.23% in large-scale MAS; tool and memory attacks fell 12.67% and 16.27%.
2 Preliminary
The paper models MAS as interacting agent graphs whose execution proceeds through ordered, iterative dialogue rounds and aggregation. It formalizes attacks and defense as identifying compromised agents and remediating their impact.
- Multi-agent system: A MAS is represented as a graph G = (V, E), with agents as nodes and connectivity encoded by edges.Each agent has an underlying LLM, role, memory, and external-tool repertoire.
- Execution logic: Agents execute in an order determined by an orchestration function, with each agent waiting for its in-neighborhood to complete.The ordering function maps the graph to an activation sequence σ.
- Execution logic: Each agent output combines global system instructions with the query and insights from preceding agents.Outputs may be rationales, intermediate results, or direct solutions.
- Execution logic: Across K dialogue rounds, an aggregation function synthesizes agent outputs into the system’s final answer.The system produces the final solution a(K) after iterative interaction.
- MAS attack: MAS attacks target prompts, memories, or tools, potentially distorting outputs into bias, misinformation, or operational failures.The attacked system is denoted ˜G, and attacked agents form Vatk ⊆ V.
- Defense: Defense separates attack detection from remediation: the detector identifies Vatk from observable behavior, and the remediator reduces compromised agents’ negative impact.Remediation is evaluated through differences in utility, safety, cost, and related properties.
3 Methodology
G-Safeguard monitors each dialogue round by representing the multi-agent system as an utterance graph, detecting risky agents with an edge-featured GNN, and pruning their outgoing edges to suppress harmful propagation.
- 3.1 Multi-agent Utterance Graph: G-Safeguard collects prior agent outputs and topology to construct a multi-agent utterance graph for round-by-round security monitoring.The graph encodes agent-wise discourse dynamics and inter-agent connectivity.
- 3.1 Multi-agent Utterance Graph: Each node represents an agent using an embedding of its historical records, while each edge represents interaction history between two agents.Text embeddings map utterances into fixed-dimensional vectors, and edge histories are fused into fixed-dimensional representations.
- 3.1 Multi-agent Utterance Graph: A learnable permutation-invariant fusion function distills variable-length interaction histories into fixed-dimensional edge representations, supporting time-varying topologies.The number of interaction occurrences K need not equal the dialogue round t.
- 3.2 Graph-based Attack Detection: Attack detection is formulated as node classification on the utterance graph, with an L-layer GNN propagating structural and semantic dependencies through neighboring agents.The graph layers aggregate edge-aware neighborhood information and combine it with each agent’s own representation.
- 3.3 Edge Pruning for Remediation: At each interaction round, G-Safeguard identifies high-risk agents and intervenes by excising their outgoing edges to suppress adversarial message propagation.Alternative remediation can filter compromised content or issue precautionary alerts, depending on user requirements.
- 3.2 Graph-based Attack Detection: Unlike individual-agent safeguards operating only at input-output boundaries, G-Safeguard perceives inter-agent information flow for topology-aware attack detection.A probabilistic classifier assigns each agent an attack probability after message propagation.
4 Experiment
Experiments evaluate G-Safeguard across attack types, datasets, topologies, LLMs, agent scales, and multi-role MAS. Results indicate reduced attack success, transferability across models and structures, scalability to larger systems, and attacker recognition above 80% in CAMEL-based settings.
- Experiment Setup: Experiments test direct prompt, tool, and memory attacks across CSQA, MMLU, GSM8K, InjecAgent, and PoisonRAG-derived settings.The evaluation varies attack methods, topologies, LLMs, communication rounds, and agent counts.
- Effectiveness of G-Safeguard: After three dialogue rounds, ASR decreases by about 18.01% and 20.01% in low-connectivity topologies on CSQA and MMLU, respectively.In high-connectivity settings, the corresponding average decreases are about 24.74% and 24.90%.
- Effectiveness of G-Safeguard: Training on GPT-4o-mini communication data generalizes to other LLMs and unseen topologies, with most recognition results exceeding 75% and LLaMA-3.1-70b ASR reductions of 3.49% ∼35.97%.Recognition accuracy is higher for LLaMA-3.1-70b and Claude-3.5-haiku than for GPT-4o-mini under the memory attack in the reported comparison.
- Real-world Application: In a multi-role CAMEL-based MAS, attacker recognition accuracy exceeds 80% on both CSQA and MMLU across systems built with various LLMs.The experiment measures attacker recognition in a multi-role setting rather than only homogeneous agent configurations.
5 Related Work
Related work distinguishes single-agent safety from multi-agent safety and models MAS interactions as graphs. Collaboration improves task-solving but also creates risks of toxicity transmission across agents.
- Agent Safety: Agent safety research covers both single-agent and multi-agent settings, with agents’ roles, memory, and tool invocation introducing distinctive vulnerabilities.Attacks can target tools or memory, motivating security measures for agent parameters and external entities.
- Agent Safety: MAS collaboration enhances task-solving but can transmit toxicity, allowing an attacked agent to spread harmful behavior across the system.The related work describes possible collective malicious behavior and system-wide paralysis from propagated toxicity.
- Multi-agent as Graphs: Graph-based MAS research uses predefined, hierarchical, or optimized topologies to facilitate communication, collaboration, efficiency, and performance.Examples include ChatEval, AutoGen, DyLAN, GPTSwarm, and AgentPrune.
- Multi-agent as Graphs: NetSafe studies topology-based misinformation and bias propagation in MAS under attacks across different system topologies.
6 Conclusion
G-Safeguard targets anomaly detection and security protection for individual modules within MAS. Experiments across configurations and attacks show defensive transfer across larger systems and different base LLMs.
- 6 Conclusion: G-Safeguard addresses anomaly detection and security protection for individual modules within multi-agent systems.
- 6 Conclusion: The framework trains on small-scale MAS and transfers defensive mechanisms to larger-scale MAS architectures.
- 6 Conclusion: Experiments across tree, chain, and graph configurations and prompt injection and memory attacks demonstrate defense and transfer across different base LLMs.
Limitation
G-Safeguard mitigates attacks within compromised MAS but does not preemptively prevent compromise. Its communication-data analysis primarily limits further malicious-information dissemination after detection.
- Limitation: G-Safeguard cannot preemptively prevent an MAS from being compromised before detection.Because it analyzes communication data, some nodes may already be compromised when an attacker is identified.
- Limitation: Its primary function is to curtail further dissemination of malicious information within an attacked MAS.The paper identifies proactive attack prevention as future work.
A Additional results of G-Safeguard
Additional results are reported for CSQA, MMLU, GSM8K, InjecAgent, and PoisonRAG benchmarks in Appendix A.
- A Additional results of G-Safeguard: Detailed G-Safeguard results are provided for the CSQA, MMLU, GSM8K, InjecAgent, and PoisonRAG benchmarks.
B.1 LLM-based Agent Safety
LLM-agent safety research distinguishes single-agent from multi-agent safety. Agent roles, memory, and tool invocation expand functionality while introducing additional vulnerability risks.
- Agent safety research is broadly divided into single-agent safety and multi-agent safety.
- Compared with standalone LLMs, agents establish distinct roles and incorporate memory and tool invocation to enhance functionality.
- These agent characteristics also introduce increased risks of vulnerability.
B.2 Multi-agent as Graphs
Multi-agent interactions can be modeled from a graph perspective, motivating topology-oriented approaches to agent communication and coordination. The section also documents attack-prompt examples and ASR measurements across datasets, conversation rounds, and LLM backbones.
- B.2 Multi-agent as Graphs: Interactions between multiple agents can be naturally modeled from a graph perspective.
- B.2 Multi-agent as Graphs: Prior systems use graph structures, predefined topologies, or optimized communication distributions to support complex interactions and cooperation.
- B.2 Multi-agent as Graphs: Figures present prompts for normal and attacker nodes under prompt-injection, tool, and memory attacks across the evaluated datasets.
- B.2 Multi-agent as Graphs: Tables report ASR after each conversation round for MAS built with different LLMs on CSQA, MMLU, GSM8K, InjecAgent, and PoisonRAG.
- B.2 Multi-agent as Graphs: ASR measures the proportion of agents exhibiting malicious or incorrect behaviors.
- B.2 Multi-agent as Graphs: Under memory attacks, figures show prompts for normal and attacker nodes on the PoisonRAG dataset.