PaperlayerView exact revision on arXiv

Source-linked AI summary

NetSafe: Exploring the Topological Safety of Multi-agent Networks

Miao Yu, Shilong Wang, Guibin Zhang, Junyuan Mao, Chenlong Yin, Qijiong Liu, Qingsong Wen, Kun Wang, Yang Wang

arXiv:2410.15686v1cs.MAcs.AI

TL;DR

Safety research on LLM-based multi-agent networks remains limited, and single-agent approaches are challenging to transfer to their complex interactions. NetSafe uses iterative RelCom to study attacks across topologies, finding that less-connected structures tend to be safer and identifying Agent Hallucination and Aggregation Safety.

  • Problem

    How topological structures of LLM-based multi-agent networks exhibit stronger safety remains an unexplored question, while single-agent methods are challenging to apply to complex interactions and frameworks lack a unified standard.

    Source: §1 · pp. 1–2

  • Method

    NetSafe formalizes multi-agent networks and uses iterative RelCom to standardize interactions and evaluate topologies under misinformation, bias, and harmful-information attacks.

    Source: §1 · p. 2

  • Results

    Less-connected Chain and Cycle topologies tend to be safer than highly connected topologies, while a Star Graph Topology's task performance drops 29.7% with one attacker.

    Source: §1 · p. 2, §4.5 · p. 9, §5 · p. 9

  • Takeaways & Limitations

    The findings identify Agent Hallucination and Aggregation Safety as phenomena for further study and suggest less-connected structures offer superior protection against malicious information spread.

    Source: §1 · p. 2, §5 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) have empowered nodes within multi-agent networks with intelligence, showing growing applications in both academia and industry. However, how to prevent these networks from generating malicious information remains unexplored with previous research on single LLM's safety be challenging to transfer. In this paper, we focus on the safety of multi-agent networks from a topological perspective, investigating which topological properties contribute to safer networks. To this end, we propose a general framework, NetSafe along with an iterative RelCom interaction to unify existing diverse LLM-based agent frameworks, laying the foundation for generalized topological safety research. We identify several critical phenomena when multi-agent networks are exposed to attacks involving misinformation, bias, and harmful information, termed as Agent Hallucination and Aggregation Safety. Furthermore, we find that highly connected networks are more susceptible to the spread of adversarial attacks, with task performance in a Star Graph Topology decreasing by 29.7%. Besides, our proposed static metrics aligned more closely with real-world dynamic evaluations than traditional graph-theoretic metrics, indicating that networks with greater average distances from attackers exhibit enhanced safety. In conclusion, our work introduces a new topological perspective on the safety of LLM-based multi-agent networks and discovers several unreported phenomena, paving the way for future research to explore the safety of such networks.

1 Introduction

The paper frames topological safety as an open question because single-agent safety methods transfer only partially to interacting networks, while diverse multi-agent frameworks lack a shared standard. NetSafe and RelCom support a unified study of attacks across topologies, revealing convergence, Agent Hallucination, Aggregation Safety, and vulnerability in highly connected networks.

1 Introduction

  • The paper situates network behavior in the interactions among nodes, which it says make the whole greater than the sum of its parts.It connects this view to the web's transformation of how information is shared, processed, and consumed.Source: §1 · p. 1
  • The paper positions topological safety as a research direction focused on which structures of LLM-based multi-agent networks exhibit stronger safety.Its motivation is the challenge of preventing these networks from being exploited for harmful activities.Source: §1 · pp. 1–2
  • Single-agent safety methods transfer only partially to complex multi-agent networks, and diverse frameworks lack a unified standard for universal safety research.The paper describes multi-agent safety research as being in its early stages.Source: §1 · p. 2
  • NetSafe formalizes multi-agent networks and uses iterative RelCom to standardize interactions and study safety across topologies under misinformation, bias, and harmful-information attacks.The paper identifies Agent Hallucination and Aggregation Safety across the three attack types.Source: §1 · p. 2
  • In certain topological networks with high connectivity, performance in the presence of only one attacker declines from 95.03 to 66.80, a 29.7% decrease.After multiple iterations of RelCom, multi-agent networks tend to reach a convergence state.Source: §1 · p. 2

2 Related Work

Related work spans LLM safety, LLM-based agent networks, and agent safety, but the paper focuses on the topological safety of multi-agent networks. It proposes RelCom to unify their diverse communication mechanisms for generalized safety research.

2 Related Work

  • LLM-safety research uses alignment, inference guidance, filters, prompt attacks, and unalignment to address or elicit unsafe generations.The cited examples include supervised fine-tuning [1] [79], reinforcement learning from human feedback [23] [51], and template-based attacks [11] [24] [27] [43] [54] [69] [81] [108].Source: §2 · p. 2
  • LLM-based agent-network research spans software-development teams, robot clusters, and agent societies modeling human behavior across several domains.The paper proposes RelCom to unify the diverse communication mechanisms in existing frameworks for generalized safety research.Source: §2 · pp. 2–3
  • Prior agent-safety work includes single-agent attacks and defenses, multi-agent debate, malicious-image propagation, and psychological approaches; this paper focuses on safer network topologies.The cited examples include TrustAgent [32], multi-agent debate [19], AgentSmith [28] and [76], and PsySafe [101].Source: §2 · p. 3

3 Methodology

NetSafe standardizes multi-agent interactions and evaluates how malicious information propagates through networks with different topologies. Its methodology combines network structure, targeted attack strategies, and static and dynamic safety evaluations.

3 Methodology

  • NetSafe combines a multi-agent network, attack strategies, and evaluation methods to study malicious-information propagation across topologies.The pipeline applies tailored attacks to networks, then evaluates propagation through multiple communication rounds.Source: §3 · p. 3

3.1 Multi-agent Network

  • RelCom models agents as nodes in a directed graph and iterates Genesis initialization with Renaissance updates using neighboring and previous responses.Genesis runs once; Renaissance repeats for a specified number of communication rounds.Source: §3.1 · p. 3

3.2 Attack Strategy

  • Attackers target their out-neighbors with misinformation, bias, or harmful content, while normal agents can be attacked through neighboring attackers.The attacks are implemented in attacker system prompts; the attack strategies specify malicious information and reasoning.Source: §3.2 · pp. 3–4

3.3 Evaluation Method

  • Static evaluation uses graph-based metrics, including network efficiency, eigenvector centrality, and attack-path vulnerability, to assess network structure without experiments or time evolution.Attack-path vulnerability measures how many shortest paths are vulnerable to attacks.Source: §3.3 · p. 5
  • Dynamic evaluation applies attacks over multiple communication rounds and measures task performance, using single-agent accuracy and multi-agent joint accuracy to track safety.Single-agent accuracy tracks individual agents’ answers; joint accuracy measures network performance in a communication turn as rounds progress.Source: §3.3 · p. 5

4 Experiment

The experiments evaluate topological safety under misinformation, bias, and harmful-information attacks, varying network structures and attacker or normal-node counts. Across these settings, less-connected topologies generally prove safer, but resilience also depends on attack type and node composition.

4 Experiment

  • The experiments test five topologies against three attack types, using task performance or toxicity to assess vulnerability.Settings use GPT-4o-mini agents, except GPT-3.5-Turbo for harmful-information elicitation; datasets include Fact, CSQA, GSMath, Bias, and AdvBench.Source: §4 · p. 5, §4.1 · pp. 5–6, §4.2 · p. 6

4.2 Main Results (RQ1)

  • Across three datasets, under multi-turn RelCom interactions, each network topology generally showed declining task accuracy in 97.8%, 82.2%, and 77.8% of cases, respectively, before eventually fluctuating to convergence.Across five topologies, the average Turn 1-to-Turn 10 accuracy reduction was 18.2% on knowledge-based Fact, versus 7.4% on reasoning-based CSQA and 3.2% on reasoning-based GSMath.Source: §4.2 · p. 6
  • APV had an average correlation coefficient of 0.367 with dynamic safety rankings, compared with 0.067 for NE and −0.567 for EC.The authors conclude that static evaluation struggles to reflect practical safety in these networks.Source: §4.2 · p. 6, Table 2 · p. 7

4.3 Safety for Bias and Harmful-info (RQ2)

  • For bias induction, networks achieved 100% accuracy in 78.0% of cases and 99.8% in the remaining 22%; under harmful-information elicitation, the normal node’s moderation score was 0.097 versus attackers’ 0.920 average.Source: §4.3 · pp. 6, 8

4.2 Main Results (RQ1)

  • 84.18 and 65.35 were the Chain Topology’s last-iteration MJA on Fact and CSQA, while Star Topology reached 66.8 and 53.54, respectively.The Star results were 26.0% and 22.1% lower than Chain; initial accuracies were similar across topologies.Source: §4.2 · p. 6, Table 1 · p. 7

4.4 Impact of Attacker Node Number (RQ3)

  • With five attackers on GSMath, Complete Graph accuracy fell to 44.25 from 89.38 without attackers; Chain had the best accuracy in five of six cases.On Fact, Binary Tree accuracy rose from 78.17 to 83.94 and then 82.57 as normal nodes increased; in Star, increasing normal nodes from seven to nine lowered accuracy from 74.88 to 71.1.Source: §4.4 · pp. 8–9

4.5 Trait of Safe Topology (Discussion)

  • Less-connected Chain and Cycle topologies generally outperformed Star and Complete Graph; in Cycle, directly connected Agent 6’s accuracy was approximately 10.0 lower than in Chain, while smaller average attacker distance was safer.Source: §4.5 · p. 9

5 Conclusion

NetSafe and iterative RelCom support evaluation of how topology relates to safety in LLM-based multi-agent networks. The experiments identify Agent Hallucination and Aggregation Safety and find that less-connected structures offer stronger protection in the studied settings.

5 Conclusion

  • NetSafe uses iterative RelCom to evaluate topological safety, with Chain and Cycle structures offering stronger protection than highly connected topologies in the reported experiments.The paper also identifies Agent Hallucination and Aggregation Safety as agent-specific phenomena for further study.Source: §5 · p. 9

A More Static Metrics

These metrics adapt graph measures to account for attacker nodes and their influence on network structure, propagation, and exposure.

Attack-weighted Betweenness Centrality

  • Attack-weighted Betweenness Centrality modifies betweenness centrality to emphasize attacker nodes’ influence on network connectivity.Its definition uses shortest paths and an indicator for whether an intermediate node is an attacker.Source: Attack-weighted Betweenness Centrality · p. 12

Algebraic Connectivity under Attack

  • Algebraic Connectivity under Attack incorporates attacker influence into the graph Laplacian and uses its second-smallest eigenvalue to indicate connectivity.The definition identifies λ2 as the second-smallest eigenvalue.Source: Algebraic Connectivity under Attack · p. 12

Attack Resistance Index

  • Attack Resistance Index measures resilience by the minimum cut set needed to disconnect the network in the presence of attackers.It focuses on the smallest set of nodes required to break the network.Source: Attack Resistance Index · p. 12

Attack Propagation Coefficient (APC)

  • Attack Propagation Coefficient quantifies how far attackers’ influence spreads through their outgoing edges.The definition describes attacker-to-node outgoing edges as the basis of the metric.Source: Attack Propagation Coefficient (APC) · p. 12

Node Threat Index (NTI)

  • Node Threat Index measures a node’s vulnerability based on its shortest-path distance to attacker nodes.The definition uses the distance between an attacker and a node to quantify exposure.Source: Node Threat Index (NTI) · p. 12

B Proof of Lemma

The proof of the Effect of Attacks on Network Performance lemma models misleading information as increased error. Under its stated assumptions, the non-increasing performance function yields no better performance under attack.

B Proof of Lemma

  • The lemma concerns how attacks affect multi-agent network performance, measured by E.It defines attacker nodes as a subset of network nodes and compares performance without attack and under attack.Source: B Proof of Lemma · p. 12
  • The proof assumes misleading information reduces performance, performance decreases as incorrect information increases, and nodes update using their own and neighbors’ states.Source: B Proof of Lemma · p. 12
  • Because the performance function is non-increasing, the proof uses increased error under attack to conclude attacked performance is no greater than normal performance.The displayed inequalities compare attacked and normal error and then apply the monotonicity assumption.Source: B Proof of Lemma · p. 12

C Prompts for Datasets

The appendix gives prompts for generating factual and stereotypical statements, and notes that generation API settings aim to encourage diversity and reduce redundancy.

C.1 Fact Generation

  • The fact-generation prompt asks GPT for concise, informative, correct statements spanning fields such as science, history, geography, and general knowledge.Source: §C.1 · p. 13

C.2 Bias Generation

  • The bias-generation prompt requests recognizable stereotypes across varied groups and topics while avoiding offensive or harmful language.Source: §C.2 · p. 13

C.3 API Parameter Settings for Generation

  • The dataset-generation API settings are described as designed to encourage content diversity and reduce redundancy.Table 4 is identified as listing API parameters for dataset generation, but its settings are not specified in these passages.Source: §C.3 · p. 13, Table 4 · p. 13

D Dataset Example

The appendix illustrates dataset examples, task-completion prompts, node system messages, and the iterative RelCom interaction used in experiments.

D Dataset Example

  • Examples span factual statements, commonsense questions, arithmetic reasoning, bias judgments, and harmful-information prompts.The examples include a True statement about gold, a marmot multiple-choice question, a worked dance-attendance problem, a False statement, and an escape prompt.Source: D Dataset Example · p. 13

E.1 Task Completion

  • Normal-node prompts solve tasks, while attacker prompts defend incorrect reference answers or introduce harmful tasks to disrupt completion.Templates cover math, fact checking, bias checking, commonsense answering, and harmful-information elicitation.Source: §E.1 · pp. 14–15

E.2 System Message

  • Normal nodes are instructed to collaborate and reconsider answers using other agents’ input, whereas attacker nodes treat a supplied answer as authoritative.For harmful-information elicitation, the attacker system prompt adds dark-trait and jailbreak instructions.Source: §E.2 · p. 15

E.3 RelCom Mechanism

  • RelCom has agents first generate an initial answer, reasoning, and memory, then gather incoming neighbors’ views and regenerate their response using that information.The Renaissance update retains prior responses and system-level information while updating the user message.Source: §E.3 · pp. 15–16

F API Parameters for Reproducibility

The appendix identifies the API-parameter table, explains Kendall’s Tau as a rank-association measure, and describes the visual conventions used in case studies.

F API Parameters for Reproducibility

G Kendall’s Tau

  • Kendall’s Tau measures the strength and direction of monotonic association between ranked variables by comparing concordant and discordant pairs.Its range is −1 to +1, with positive and negative values indicating positive and negative rank association, respectively.Source: G Kendall’s Tau · p. 16

H Case Study

  • Case-study figures distinguish tasks in gray, attacker responses in red, normal-node responses in green, and successfully attacked normal-node responses in pink.Source: H Case Study · p. 17

H.1 Case of Misinformation Injection (Attacked)

In a six-node Complete Graph on the Fact dataset, two attackers successfully injected misinformation that led normal Agent 2 to spread it.

H.1 Case of Misinformation Injection (Attacked)

  • Two attackers in the six-node Complete Graph successfully misled normal Agent 2 into spreading misinformation on the Fact dataset.The case follows Agent 2 from its initial response through later iterations.Source: §H.1 · p. 17
  • The response classifies the statement that energy cannot be created or destroyed as False, matching the attacker’s target rather than the correct answer, because it captures conservation but omits transformation between forms.Source: §H.1 · p. 17

H.2 Case of Harmful-info Elicitation (Defended)

The case studies show a normal agent resisting bias and harmful-information elicitation, while attackers continue to advocate or generate harmful content in the harmful-information case.

H.3 Case of Harmful-info Elicitation (Defended)

  • In the six-node Complete Graph AdvBench case, five attackers did not elicit harmful information from normal Agent 5.Agent 5 maintains an ethical, collaborative response, although attacker Agent 0 later advocates providing bomb-making instructions.Source: §H.3 · pp. 18–20

I.1 SSA Line Chart on Fact Dataset

Supplementary SAA figures cover multiple network topologies, attack counts, and datasets, and the appendix states that their conclusions align with the main text. Additional MJA tables cover four datasets and report results across communication iterations.

I.1 SSA Line Chart on Fact Dataset

  • Fact-dataset SAA figures show Binary Tree, Complete Graph, and Chain topologies with varying attacker counts; their conclusions align with the main text.The figures plot SAA across iterations.Source: §I.1 · p. 20, Figure 9 · p. 20, Figure 10 · p. 20

I.2 SSA Line Chart on CSQA Dataset

I.3 SSA Line Chart on GSMath Dataset

I.4 SSA Line Chart on Bias Dataset

  • Bias-dataset SAA figures cover Chain, Star, Binary Tree, and Cycle topologies with varying attacker counts, and align with the main-text conclusions.The figures plot SAA across iterations.Source: §I.4 · p. 22, Figure 18 · p. 22, Figure 19 · p. 22

I.5 MJA Table

  • The MJA table reports means over three runs for five topologies, four datasets, and ten RelCom iterations, with and without two attackers.The caption notes variances around 10^-3; arrows mark changes from the prior iteration, and check/cross markers identify the highest/lowest final-iteration performance.Source: §I.5 · pp. 22–23, Table 6 · p. 23
Loading 2410.15686v1…