AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management

1 Introduction

• The paper identifies insufficient safeguards and a lack of hierarchical control over information flow as security gaps in LLM-based multi-agent systems.These gaps are associated with risks including unauthorized access and exposure of sensitive data to external attackers.Source: §1 · p. 1
• AgentSafe categorizes information flow by safety rankings and restricts sensitive data to authorized agents through filtering.ThreatSieve authenticates information sources and evaluates communication security rankings to direct information to the appropriate receiving-agent sub-memory.Source: §1 · p. 2
• HierarCache stores historical information in relationship-based hierarchical drawers, with access permissions that keep information flow controllable, traceable, and manageable.Its Junk Memory mechanism uses instruction-based comparisons to filter potentially irrelevant information as junk, supporting efficient memory utilization.Source: §1 · p. 2
• The evaluation covers topology-based attacks exploiting agent relationships and authorization hierarchies, and memory-based attacks manipulating stored data through misinformation or identity deception.The introduction reports defense against all attack types considered.Source: §1 · p. 2
• 85.93% defense success at turn 5 versus 50.32% for the baseline in topology-based attacks; AgentSafe maintains 82.50% at turn 50.In memory-based attacks, CSR stays above 0.65 after 10 rounds while the baseline drops below 0.4; scaling results report CSR between 0.68 and 0.85.Source: §1 · p. 2

Related applications

• Multi-agent applications include AI teaching and urban planning, as well as debate frameworks for reasoning and workflows using role specialization (Xu et al., 2024; Zhou et al., 2024; Du et al., 2023; Liang et al., 2023; Li et al., 2023a; Hong et al., 2023).Source: §2 · p. 3

Attacks in single LLM

• Single-LLM attacks include red teaming, templated-based bypasses, and neural prompt-to-prompt methods (Ganguli et al., 2022; Ding et al., 2023; Shah et al., 2023).Source: §2 · p. 3

Memory Attacks in MAS

• Research on memory attacks in multi-agent systems is limited; Chen et al. (2024) first explored memory poisoning, while this work claims the first hierarchical defense framework.The paper presents AgentSafe as a scalable solution for secure information exchange in real-world applications.Source: §2 · p. 3

3.1.1 Multi-agent System as a Graph

• Agent communication is modeled as a directed graph whose nodes are agents and whose edges are directed communication links.A directed spanning tree exists when root V0 has a directed path to every other agent.Source: §3.1.1 · p. 3

3.1.2 Memories of Agent

• Each agent’s memory Mi is a tuple of storage units Si and a task-conditioned update function ϕi, with each unit assigned a security level.The levels represent degrees of importance or sensitivity.Source: §3.1.2 · p. 3

3.2.1 Overview

• AgentSafe aims to route information only to the appropriate agent subset and minimize the overall attack success rate across agents and time.Access to information at level i is restricted to agents whose permission level is at least i.Source: §3.2.1 · pp. 3–4

3.2.2 Defense Mechanisms

• ThreatSieve controls communication by permission level and evaluates message legitimacy by extracting and verifying sender identity.Communication is allowed when the sender’s permission level is at least the receiver’s; identity checks use API calls and a program that compares received information with instructions.Source: §3.2.2 · p. 4
• Table 1 compares defense success rates across attack methods and datasets over 10 interaction turns.Its stated setting is seven agents, including one attacker, on the RIOH and WCEI datasets.Source: Table 1 · p. 5
• HierarCache stores messages at their security level only when sender authority and message validity pass; otherwise, messages go to junk memory.Validity is checked against natural-language criteria using semantic similarity and a predefined threshold.Source: §3.2.2 · p. 5
• Periodic detection inspects stored information and isolates false information to maintain the integrity of hierarchical storage.The reflection process uses a language model and a prompt designed to encourage reflection on the information.Source: §3.2.2 · pp. 5–6

4.1 Experimental Setups

• RIOH and WCEI address prior datasets’ lack of interpersonal relationships and multilevel privacy information; evaluation uses five named LLMs and Defense Rate, CSI, and CSR.The models are Llama 3.21, GPT-3.5-Turbo2, GPT-4o3, GPT-4o-mini4, and GPT-45.Source: §4.1 · p. 6

4.2 Defense and Ablation Results (RQ1)

• At turn 5, TBA defense success is 80.67% with AgentSafe versus 34.24% baseline; at turn 50, it remains 55.20%, above the baseline’s peak.Both frameworks decline over time, but AgentSafe maintains higher defense success rates across 5 to 50 rounds.Source: §4.2 · pp. 6–7
• With many communication turns, AgentSafe reduces token consumption by 60% for topology-based attacks and 75% for memory-based attacks.Figure 3 compares token consumption with and without AgentSafe for the two attack types.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 Multi-round Attacks Analysis (RQ2)

• With AgentSafe, LLaMA 3.2 TBA similarity after round five is 0.44 versus 0.56 baseline; after 10 MBA rounds, similarity is 0.65–0.75 versus 0.4 without it.The reported comparisons cover topology-based and memory-based attacks across LLM environments.Source: Figure 2 · p. 6, §4.3 · p. 7, Figure 4 · p. 7

4.4 System Complexity on AgentSafe (RQ3)

• Across agent counts, CSR stays 0.68–0.85; across information-complexity levels, it stays 0.67–0.82 for both attack types.Figure 5 reports agent-count effects on the left and information-complexity effects on the right.Source: §4.4 · p. 8, Figure 5 · p. 8

4.5 Topological Structures Analysis (RQ4)

• Across topologies, at nine agents in the Cycle topology, CSI is 45.82 with AgentSafe versus 77.67 without; in Chain, unprotected CSI rises from 54.46 at four agents to 70.42 at ten.Table 2 compares CSI with and without AgentSafe across topologies and agent counts.Source: §4.5 · p. 8, Table 2 · p. 8

• AgentSafe is introduced to enhance LLM-based multi-agent systems’ robustness against various attacks.Source: §5 · p. 8
• AgentSafe maintains high performance as memory information complexity and the number of agents vary.Source: §5 · pp. 8–9
• The authors identify secure, scalable multi-agent systems as a potential real-world application for AgentSafe.Source: §5 · p. 9

6 Limitations

• Evaluation uses simulated datasets and controlled environments; performance in dynamic real-world settings with unpredictable users and ambiguous security boundaries remains unvalidated.The authors call for field trials and real-world deployments to assess practical effectiveness.Source: §6 · p. 9

A Dynamic Workflow

• The workflow initializes agents’ security levels and memory, then runs multi-round interactions with hierarchical information exchange and attack detection, blocking, and logging.Source: A Dynamic Workflow · p. 12

B Components

• The component model represents agents, memory, communication, and tasks; memory stores information in security-level units, while communication checks authorization and the recipient’s access limit.Memory updates depend on the agent’s task; a recipient may receive a memory segment only when authorized and its security level is within the recipient’s maximum authorized level.Source: B Components · pp. 12–13

C Datasets

• RIOH models social interactions across Family, Friend, Colleague, and Stranger privacy levels, while WCEI models corporate information flow across four workplace security categories.The datasets are designed to evaluate secure communication protocols in social and corporate settings, respectively.Source: C Datasets · pp. 13–14

D Attacks in MAS

• The attack taxonomy distinguishes agent attacks, including topology-based acquisition and authorization mixup, from memory attacks targeting stored information through interference or identity manipulation.Source: D Attacks in MAS · p. 15, §D.1 · p. 15, §D.2 · pp. 15–16

D.1 Agent Attacks

• Topology-based acquisition exploits an intermediary to seek sensitive information, while authorization mixup combines topics of different sensitivity to confuse access control.Source: §D.1 · p. 15

D.2 Memory Attacks

• Memory interference injects false or irrelevant information over multiple rounds and assesses its effect on accurate outputs; identity manipulation seeks to make a target confuse an attacker with a trusted agent.Source: §D.2 · pp. 15–16

• For topology-based attacks, the targeted agent receives adversarial tasks correlated with other agents’ private information, and LLMs or human judges assess whether outputs reveal it.Source: §D.3 · p. 16
• A lower CSR indicates less association between topology-attack outputs and private information, and therefore a more successful defense.Source: §D.3 · p. 16
• For memory-based attacks, agents face sustained false or irrelevant information before normal tasks; higher CSR indicates outputs closer to expected normal results and a more successful defense.Source: §D.3 · p. 17

E Computational Overhead Analysis

• The total-cost analysis defines communication cost, dialogue rounds, information cost, original input I, filtered content I′, and HierarCache’s detection function D.These parameters frame the resource-consumption analysis.Source: E Computational Overhead Analysis · p. 17

E.1 Cost of ThreatSieve and HierarCache

• ThreatSieve’s hierarchical judgment and HierarCache’s validation contribute to the computational cost before information is stored in memory.The supplied passages introduce these cost components but do not provide their formulas.Source: §E.1 · p. 17

E.2 Cost of Detection

• Periodic detection checks stored information across security layers and junk memory at each detection time, with the total detection cost accumulated over T operations.The layers include ε1 through εN and εjunk, which holds irrelevant or harmful information.Source: §E.2 · p. 17

E.3 Computational Overhead Reduction by AgentSafe

• Filtering irrelevant or harmful information and processing only historical data within accepted security levels are identified as sources of computational-cost savings.The passage denotes the information’s security level at time t as kt.Source: §E.3 · p. 18

E.4 Impact of Frequent Attacks and Efficient Memory Usage

• During frequent attacks, junk memory grows, but detection intervals shorter than the dialogue run can reduce overhead; mostly regular, lower-security tasks yield less junk data and lower costs.The net cost balances attack-related junk-memory growth against efficiency from security-level-based memory use.Source: §E.4 · p. 18

F Case Study of Junk Memory

• In the case study, the system flags Alice’s attempt to extract sensitive information, stores the inquiry in junk memory, and avoids disclosing meaningful details about Michael.William’s response does not reveal Michael’s involvement in the team-conflict discussions.Source: F Case Study of Junk Memory · p. 18

G Ablation Study on the Contributions of ThreatSieve and HierarCache to AgentSafe’s Defense Performance

• The study evaluates the individual contributions of ThreatSieve and HierarCache by measuring how component removal affects defense effectiveness.The authors state that isolating component effects may help guide future optimization.Source: G Ablation Study on the Contributions… · p. 19

G.1 Results

• The results are presented in tables comparing defense strategies under topology-based and memory-based attacks.Table 3 covers topology-based attacks across communication turns; Table 4 covers memory-based attacks across communication turns.Source: Table 3 · p. 19, Table 4 · p. 19, §G.1 · p. 19

G.2 Analysis

• ThreatSieve alone improves on the baseline, particularly under topology-based attacks, but remains below the full AgentSafe system.The supplied passage gives no table cell values.Source: §G.2 · p. 19
• HierarCache alone improves on the baseline, especially under memory-based attacks, but does not match the combined system.The supplied passage gives no table cell values.Source: §G.2 · p. 19
• AgentSafe with both components has the highest defense effectiveness among the compared configurations, which the authors characterize as complementary component contributions.The passage describes the combined system as synergistic and both components as essential for optimal performance.Source: §G.2 · pp. 19–20

H Validation for Periodic Detection Mechanism

• Under memory-based attacks, the periodic-detection evaluation compares defense rates with and without the mechanism, and the authors report higher rates when it is active.Table 5 is captioned as a comparison of defense rate with and without periodic detection.Source: Table 5 · p. 20, H Validation for Periodic Detection… · p. 20

H.1 Results

• Defense effectiveness against jailbreak attacks improves when periodic detection is included.The reported defense rate is consistently higher with the mechanism active.Source: §H.1 · p. 20
• Periodic detection assesses stored information’s validity and moves invalid or junk information to separate memory.The passage describes this as maintaining the integrity of stored information.Source: §H.1 · p. 20
• The stated purpose is to ensure that only reliable, valid data is used for decision-making, improving defense robustness against attacks.This consequence is presented alongside the periodic assessment and separation of junk information.Source: §H.1 · p. 20