PaperlayerView exact revision on arXiv

Source-linked AI summary

Firewalls to Secure Dynamic LLM Agentic Networks

Sahar Abdelnabi, Amr Gomaa, Eugene Bagdasarian, Per Ola Kristensson, Reza Shokri

arXiv:2502.01822v7cs.CRcs.CY

TL;DR

Agent-to-agent communication creates privacy and security risks when assistants disclose personal data or process manipulative external messages. The paper addresses both risks with dual firewalls that project outgoing data and incoming messages onto task-specific contexts. Across 864 attacks, the architecture reduces privacy attack success rates by up to 90% and security attack success rates to under 4% while preserving or improving task utility.

  • Problem

    Agent-to-agent communication exposes delegated assistants to unnecessary personal-data disclosure and manipulation through external messages crossing trust boundaries.

    Source: §1 · p. 2, §3 · p. 4

  • Method

    A dual-firewall architecture converts incoming messages into verified structured protocols and abstracts outgoing personal data to task-appropriate granularity.

    Source: §1 · p. 3, §4.1.1 · p. 5, §7 · p. 16

  • Results

    Privacy attack success rates decrease by up to 90% and security attack success rates fall below 4% across 864 attacks while task utility is preserved or improved.

    Source: §1 · p. 3, §7 · p. 16

  • Takeaways & Limitations

    Projecting both communication directions onto task context provides structural protection while retaining dynamic task interaction.

    Source: §1 · p. 3, §7 · p. 16

    The threat model excludes poisoned knowledge bases and fabricated valid protocol values, while field-level abstraction may still allow re-identification through attribute combinations.

    Source: §6 · pp. 14–15

Abstract

from arXiv · show

The emergence of agent-to-agent communication protocols mirrors the early internet: powerful connectivity with minimal security infrastructure. When AI agents communicate on behalf of users, every message crosses a trust boundary where the user's personal data and the external agent's unconstrained language each present distinct risks. We address both through a dual-firewall architecture grounded in a unifying principle: each task defines a context, and both sides of the communication carry information far exceeding what that context requires. Our firewalls act as projections onto the task context, allowing only contextually appropriate content to cross each boundary. The Language Converter Firewall projects incoming messages onto a closed, domain-specific, structured protocol; an external agent's message is converted to validated fields while persuasive framing, urgency tactics, and embedded instructions are structurally eliminated through deterministic verification. This replaces the asymmetric challenge of resisting every possible manipulation with the structural guarantee that manipulation has no channel through which to arrive. The Data Abstraction Firewall projects outgoing information onto the granularity appropriate for the task, rather than applying binary disclose-or-redact filtering, as previous airgapping solutions did. Both firewalls operate in a trusted environment isolated from external input, applying domain-specific rules learned automatically from demonstrations. Across 864 attacks spanning three domains on the ConVerse benchmark, our architecture reduces privacy attack success rates (e.g., from 84% to 10% for GPT-5) and security attacks (from 60% to 3%), while maintaining or even improving task completion quality.

1 Introduction

Agent-to-agent systems create trust-boundary risks because external messages can manipulate assistants while outgoing responses can expose unnecessary personal data. The paper proposes dual firewalls that project communication onto task-specific contexts and evaluates them across broad attacks.

Motivation

  • Agent-to-agent communication creates trust-boundary risks involving both external manipulation and unnecessary disclosure of user data.These risks arise because agents communicate, negotiate, and collaborate on users’ behalf across open networks.Source: §1 · p. 2

Context projection

  • The dual-firewall architecture projects external messages and personal data onto the information appropriate for each task context.The Language Converter constrains incoming content, while the Data Abstraction Firewall controls outgoing disclosure.Source: §1 · p. 3

Language Converter Firewall

  • Incoming natural language is converted into validated structured fields, eliminating persuasive framing, urgency tactics, and embedded instructions from the assistant’s input.The structured protocol uses deterministic verification and removes content without a permitted representation.Source: §1 · p. 3

Data Abstraction Firewall

  • The Data Abstraction Firewall transforms personal information to task-appropriate granularity rather than relying on binary disclosure or redaction.Examples include replacing a home address with a broader region and reducing specific ages to categories.Source: §1 · p. 3

Rule learning

  • Domain-specific firewall rules are learned from demonstrations and can be incrementally refined as new contexts emerge.The two firewalls derive rules from different interaction corpora suited to their respective functions.Source: §1 · p. 3

Evaluation

  • Across 864 ConVerse attacks in three domains, the architecture reduces privacy attack success rates by 80–90% and security attack success rates to under 4% while preserving or improving task utility.The evaluation covers four LLMs and 12 user personas.Source: §1 · p. 3

2 Preliminaries and Related Work

Prior work frames agent privacy around cryptographic or data-minimization mechanisms and agent security around defenses against prompt injection. LLM-based agent networks instead combine oversharing with adaptive, contextually plausible manipulation during dialogue.

Privacy

  • LLM agents introduce privacy leakage during task execution because they may overshare without adversarial prompting.This differs from traditional privacy concerns focused on training-data memorization and membership inference.Source: §2 · p. 4

Security

  • Prompt-injection defenses commonly assume a trusted user and untrusted retrieved data, but agent-to-agent dialogue lacks that clean boundary.External agents can embed attacks in contextually plausible questions and persuasive framing.Source: §2 · p. 4

Privacy

  • Existing data-minimization approaches restrict access to task-relevant data, whereas this paper argues that private data may need to be shared at an appropriate granularity.The paper positions its system-level defense as complementary to model-level privacy enhancements.Source: §2 · p. 4

Benchmarks

  • Existing benchmarks cover privacy leakage and tool-based prompt injection, while ConVerse evaluates contextual safety in agent-to-agent dialogues.The paper uses ConVerse as the benchmark most closely aligned with its setting.Source: §2 · p. 4

3 Problem Setup and Threat Model

The paper models a personal assistant that must exchange information and perform actions through multi-turn dialogue with potentially cooperative, self-interested, or adversarial service-provider agents. The threat model distinguishes inappropriate disclosure from manipulation of behavior.

Setting

  • The setting involves a personal assistant with access to a user knowledge base and tools that interacts with external service-provider agents.Tasks include travel booking, housing search, and insurance quotes.Source: §3 · p. 4
  • The assistant must determine what personal information to share and how to interpret external messages during a multi-turn natural-language dialogue.The formal dialogue is D = (m1, m2, . . . , mn), with each message represented as natural-language text.Source: §3 · p. 4

Privacy threats

  • Privacy threats involve extracting information unnecessary for the task or inappropriate for the context, even when the dialogue appears legitimate.Some disclosure remains necessary, such as providing dates and location to book a hotel.Source: §3 · p. 4

Security threats

  • Security threats include selective framing, artificial urgency, misleading claims, preference overrides, and inducing actions beyond the current task.These attacks target the assistant’s behavior rather than merely its stored data.Source: §3 · p. 5

4 Dual-Firewall Architecture

The architecture uses two complementary, task-specific projections: incoming messages are converted into a closed verified protocol, while outgoing personal data is abstracted before reaching the assistant. Both components rely on learned domain rules and isolated enforcement contexts.

4.1 Language Converter Firewall

  • The Language Converter Firewall replaces open-ended natural-language input with a closed structured protocol whose fields are deterministically verified.This design removes arbitrary persuasive or instructional text from the channel rather than attempting to detect every manipulation.Source: §4.1 · p. 5, §4.1.4 · p. 6

4.1.1 Designing Task Protocols

  • The task protocol defines permitted keys, valid values, and type constraints for domain-specific communication.Travel examples include destination_name, price_per_night, and star_rating.Source: §4.1.1 · p. 5

4.1.2 Conversion and Verification Pipeline

  • The conversion pipeline performs LLM extraction, deterministic verification, and string anonymization before sanitized content reaches the assistant.The verifier checks enumerations, primitive types, ranges, and composite formats; free-form strings receive anonymous identifiers.Source: §4.1.2 · pp. 5–6

4.1.3 Learning the Structured Language

  • The structured language is learned from benign demonstrations that reveal legitimate fields, values, and validation formats for each domain.Benign-only learning captures legitimate variability without adding attack-derived protocol keys.Source: §4.1.3 · p. 6

4.1.4 Security Guarantees

  • Closed vocabulary, constrained values, type safety, string isolation, and deterministic verification provide structural security guarantees.These guarantees hold because arbitrary natural language is eliminated before the assistant processes the input.Source: §4.1.4 · p. 6

4.2.1 Architectural Placement

  • The Data Abstraction Firewall operates between the personal knowledge base and assistant, receiving raw data and rules but neither the query nor external messages.This input-side placement prevents adversary-influenced context from shaping abstraction.Source: §4.2.1 · p. 7, §4.2.4 · p. 8

4.2.2 Abstraction Process

  • Abstraction transforms, filters, or passes information at task-appropriate granularity because binary pre-filtering cannot reliably preserve both privacy and utility.Private data may sometimes be needed to infer preferences, while only an abstracted form should be disclosed.Source: §4.2.2 · p. 7

4.2.3 Rule Learning from Demonstrations

  • Data-abstraction rules are learned contrastively from benign and attack conversations and encode what to block, abstract, or allow for each domain.Unlike the language converter’s benign-only vocabulary learning, contrastive data rules distinguish appropriate from inappropriate disclosure.Source: §4.2.3 · pp. 7–8

5 Experimental Evaluation

Across ConVerse’s three domains, the dual-firewall architecture sharply reduces privacy and security attacks while preserving utility. Ablations, generalization tests, baseline comparisons, and failure analyses show complementary protections, graceful degradation, and remaining calibration and preference-alignment limits.

5.3.1 Privacy Attack Mitigation

  • 84.68% to 10.20%: GPT-5’s all-domain privacy ASR falls substantially with firewalls, with comparable reductions across Claude Sonnet 4 and Gemini models.Travel Planning results likewise reduce GPT-5 privacy ASR from 88.51% to 7.77%, bringing models to roughly 7–9% ASR.Source: §5.3.1 · pp. 9–10, §5.3 · p. 9

5.3.2 Security Attack Mitigation

  • 60.39% to 3.42%: GPT-5’s all-domain security ASR drops below 4% across models as incoming language is converted into structured protocol fields.On Travel Planning, GPT-5 falls from 55.32% to 3.26%, while the passage attributes the gain to removing persuasive and manipulation vectors.Source: §5.3.2 · p. 10, Table 2 · p. 10

5.3.3 Utility Preservation

  • Utility is preserved or improved: GPT-5 plan quality rises from 8.07 to 8.42 on privacy scenarios and from 7.71 to 8.27 on security scenarios.Coverage also improves for Claude Sonnet 4 and Gemini 2.5 Flash, consistent with cleaner, more focused inputs.Source: §5.3.3 · p. 10

5.4 Ablations

  • The two components provide complementary privacy defenses: Data Abstraction alone reduces ASR to 29.33%, Language Conversion alone to 20.67%, and both to 7.77%.For security, Language Conversion is primary, reducing ASR to 1.09%, while Data Abstraction alone reduces it to 30.77%.Source: §5.4.1 · p. 11, §5.4.2 · p. 11, Table 3 · p. 11

5.5 Generalization Across Personas

  • Held-out personas achieve comparable protection: GPT-5 privacy ASR increases only from 6.36% to 9.09%, while plan quality remains comparable or higher.The results suggest learned rules capture domain norms rather than persona-specific details.Source: §5.5 · p. 12, Table 5 · p. 12

5.6 Sensitivity of Rule Learning to Demonstration Data

  • With reduced demonstrations, ASR never exceeds 8.3% on 24 attacks versus 83.3% privacy and 50.0% security ASR without firewalls, though held-out evaluation and human refinement remain necessary.The authors report graceful degradation and recommend iterative rule maintenance in deployment.Source: §5.6 · p. 12, Table 6 · p. 12

5.7 Comparison to Binary Data Minimization Baselines

  • 5.5% versus 67.3% overall privacy ASR and 4.5% versus 27.3% security ASR: granularity-aware abstraction outperforms binary filtering.For related-and-useful information, abstraction reduces privacy ASR from 76.2% to 23.8% because it discloses task-appropriate resolution rather than passing data in full.Source: §5.7 · p. 13, Table 7 · p. 13

5.8.1 Taxonomy of Residual Failures

  • Residual failures mainly involve overly fine privacy granularity or field overlap, while security failures include valid but misaligned insurance upselling and assistant-side errors.These failures do not defeat architectural isolation; they expose rule-coverage and preference-alignment limits.Source: §5.8.1 · pp. 13–14

6 Discussion and Limitations

The architecture’s protection is broad but bounded by explicit threat-model, privacy, domain, learning, efficiency, and topology limitations. Its broader projection principle may extend beyond pairwise agent channels, but several deployment questions remain open.

Threat-model boundaries

  • The threat model excludes poisoned knowledge bases and false protocol-field values, so realistic deployments need complementary defenses for retrieval contamination and data fabrication.Trusted user knowledge bases are assumed, while fabricated values can pass type and range checks; reference APIs and plausibility checks are proposed as extensions.Source: §6 · pp. 14–15

Privacy limitations

  • Field-wise abstraction can still permit re-identification through intersections because the current mechanism does not formally model joint information across disclosed attributes.Combination-aware abstraction and closed-vocabulary limits may mitigate, but do not resolve, this concern.Source: §6 · p. 15

Scalability to unstructured domains

  • Structured language conversion is strongest in enumerable domains; creative or unstructured tasks require more string-typed fields, making anonymization central and potentially reducing utility.The boundary between viable structured protocols and less structured domains remains an open question.Source: §6 · p. 15

Learning rules in practice

  • Rule learning degrades gracefully with less data but lacks completeness guarantees, and demonstrations are noisy proxies for users’ intended privacy preferences.The paper recommends held-out validation, human review, incremental feedback, and domain-specific rules; personalization and automated refinement remain future work.Source: §6 · p. 15

Computational efficiency

  • Each firewall adds one short-context LLM call per turn, while deterministic verification remains sub-millisecond and could be moved to smaller fine-tuned models.Neither firewall receives dialogue history, so added context does not grow with conversation length.Source: §6 · p. 15

Beyond pairwise channels

  • The architecture currently secures a single assistant–service channel, leaving composition of contextual-integrity norms across richer multi-agent topologies unresolved.The projection principle may nevertheless generalize to tool outputs, retrieved documents, and API responses.Source: §6 · pp. 15–16

7 Conclusion

The paper presents dual firewalls that structurally constrain incoming and outgoing agent communication to the task context. Across 864 attacks, the approach sharply reduces privacy and security attack success while preserving or improving utility.

Conclusion

  • Across 864 attacks, privacy attack success rates fell by up to 90% and security attack success rates stayed under 4% while task utility was preserved or improved.The guarantees arise from architectural constraints rather than detection heuristics, with domain-specific rules learned from demonstrations.Source: §7 · p. 16

A Architecture Diagram

The architecture separates incoming language conversion from outgoing data abstraction, using a trusted, domain-specific structured protocol and deterministic enforcement. The conversion firewall is specified through verification, anonymization, and learned rule construction.

A Architecture Diagram

  • Incoming messages pass through a Language Converter Firewall, while outgoing user data passes through an isolated Data Abstraction Firewall governed by contextual rules.The incoming path uses learned schemas and deterministic checks; the outgoing path filters, abstracts, or passes information according to task context.Source: Figure 2 · p. 19

B.1 Deterministic Verifier

  • The deterministic verifier removes unknown keys and invalid values, validates typed fields, casts accepted values, and parses valid formats.String fields are anonymized rather than passed directly.Source: §B.1 · p. 20

B.2 String Anonymization

  • String anonymization assigns stable category-specific opaque identifiers and maintains reverse mappings for outgoing de-anonymization.Repeated values reuse their existing identifier, while responses can restore the original names before transmission.Source: §B.2 · p. 21

B.3 Language Specification Learning

  • The structured language is learned from benign conversations by proposing information keys and value constraints, then optionally undergoing human review.The specification favors enumerated values and restricts string fields to non-enumerable proper nouns; overly restrictive or permissive rules can be refined with examples or constraints.Source: §B.3 · p. 21

C Structured Language and Rule Specifications Details

The structured-language examples show how travel messages are represented, verified, anonymized, and reduced to neutral fields. The end-to-end example removes social engineering and embedded instructions while preserving legitimate booking information.

C Structured Language and Rule Specifications Details

  • The travel specification organizes communication into constrained value categories and a domain-specific schema for permissible fields.The supplied tables provide examples of value categories, travel-language fields, and abstraction rules.Source: Table 8 · p. 22, Table 9 · p. 22, Table 10 · p. 22

D End-to-End Example of Language Conversion

  • A travel agent’s natural-language message mixes legitimate hotel and date information with requests for employer details, budget information, and persuasive framing.The LLM conversion represents both valid content and manipulative elements as candidate JSON fields.Source: D End-to-End Example of Language… · pp. 22–23
  • The verifier processes each candidate field, after which hotel names are anonymized as hotel_1 and hotel_2.The verification results and anonymized output are presented in the example’s referenced tables.Source: D End-to-End Example of Language… · p. 23, Table 11 · p. 23, Table 12 · p. 24
  • The final output drops the employer-information request and persuasive savings framing, retains a neutral budget-confirmation flag, and anonymizes hotel names to block embedded instructions.The example specifically removes the social-engineering request and preserves only legitimate structured content.Source: D End-to-End Example of Language… · p. 23
  • The assistant’s response can use an anonymized hotel identifier, which the outgoing de-anonymization step restores to the original hotel name.This preserves usable responses without exposing the original name during internal processing.Source: D End-to-End Example of Language… · p. 23

E Data Abstraction Firewall Details

The Data Abstraction Firewall learns domain-specific abstraction rules from paired benign and attack conversations, then applies them to raw knowledge-base data before it reaches the assistant. Its isolated process sees only the raw data and learned rules, producing transformed data that preserves task utility while controlling sensitive disclosure.

E Data Abstraction Firewall Details

E.1 Data Abstraction Rule Learning

  • Paired benign and attack conversations are contrasted to generate abstraction rules that permit necessary sharing, block sensitive exposure, and generalize to similar domain scenarios.The learned rules are returned as a domain-specific policy R.Source: §E.1 · p. 25

E.2 Abstraction Process

  • The abstraction process applies learned rules R to raw data x and returns transformed data x̃ before it reaches the assistant.The firewall prompt instructs the model to redact or abstract sensitive information while preserving utility for the external agent’s task.Source: §E.2 · p. 25

F Examples of Data Abstraction

The travel-planning example illustrates how the firewall retains task-relevant aggregate information while excluding unrelated personal identity details. The accompanying table presents this transformation for a travel query.

F Examples of Data Abstraction

  • The firewall can preserve “four travelers, two adults and two children” while excluding “Carlos Silva, managing director at TechCorp.” from the assistant’s context.The excluded identity detail therefore cannot be revealed in the assistant’s response to the external travel agent.Source: F Examples of Data Abstraction · p. 25
  • Table 13 presents an example of data abstraction for a travel-planning query.The example concerns a family vacation and a personal knowledge base containing detailed family information.Source: F Examples of Data Abstraction · p. 25, Table 13 · p. 26

G ConVerse Benchmark Details

ConVerse evaluates assistant behavior in multi-turn interactions among an assistant, a simulated user environment, and an adversarial external agent. It measures privacy leakage, security compromise, and preservation of task utility using final plans and interaction traces.

G ConVerse Benchmark Details

  • ConVerse coordinates three agents while the assistant queries user data, invokes tools, and communicates with an external service-provider agent pursuing both legitimate and adversarial objectives.The user environment is LLM-simulated and contains personal data and tools.Source: G ConVerse Benchmark Details · p. 27
  • During iterative planning, the external agent interleaves task-relevant responses with adversarial probes, and the simulation ends when the assistant produces a completion phrase and final JSON plan.Responses are parsed into environment queries, external messages, tool invocations, and the final plan.Source: G ConVerse Benchmark Details · p. 27
  • Privacy ASR measures targeted disclosure, security ASR measures preference overrides, unauthorized actions, or denial of service, and utility measures final-plan coverage and quality.An LLM judge compares responses and plans with attack specifications and plan-quality annotations.Source: G ConVerse Benchmark Details · p. 27

H Results on All Domains

The all-domain evaluation reports privacy and security attack success rates across Travel Planning, Insurance, and Real Estate, while qualitative examples show how the firewalls distinguish appropriate information from private or manipulative content. The security example specifically illustrates removal of persuasive manipulation through structured fields.

H Results on All Domains

I.1 Privacy Attack Examples

I.2 Security Attack Example

  • The security example shows the Language Converter Firewall stripping urgency tactics and emotional appeals by enforcing structured numeric and enumerated fields.The example concerns preference manipulation through persuasive language.Source: Table 18 · p. 32
Loading 2502.01822v7…