PaperlayerView exact revision on arXiv

Source-linked AI summary

Maris: A Formally Verifiable Privacy Policy Enforcement Paradigm for Multi-Agent Collaboration Systems

Jian Cui, Zichuan Li, Chi Wang, Luyi Xing, Xiaojing Liao

arXiv:2505.04799v5cs.CR

TL;DR

Multi-agent collaboration systems need privacy controls across agent communication and interactions with external environments, but existing frameworks lack fine-grained flow controls and message monitoring. Maris integrates typed actions and policy enforcement into development frameworks, and its evaluation reports zero attack success under passive and proactive scenarios while preserving task success.

  • Problem

    Existing MACS frameworks lack fine-grained information-flow controls and message-flow monitoring, complicating privacy-policy enforcement across agent and environment interactions.

    Source: §1 · pp. 1–2

  • Method

    Maris integrates typed, schema-defined agent actions and privacy-policy enforcement into development frameworks without altering original application logic.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    Maris reduces attack success rate to 0% under passive and proactive scenarios while preserving task success rate across three MACS task suites.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Maris provides a framework-level approach to privacy compliance that can be integrated without altering original MACS application logic.

    Source: §1 · p. 2

    The study focuses on GDPR; health-data regulations and other sector-specific requirements are beyond its scope.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

Multi-agent collaboration systems (MACS), powered by large language models (LLMs), solve complex problems efficiently by leveraging each agent's specialization and communication between agents. However, the inherent exchange of information between agents and their interaction with external environments, such as LLM, tools, and users, inevitably introduces significant risks of sensitive data leakage, including vulnerabilities to attacks such as eavesdropping and prompt injection. Existing MACS lack fine-grained data protection controls, making it challenging to manage sensitive information securely. In this paper, we take the first step to mitigate the MACS's data leakage threat through a privacy-enhanced MACS development paradigm, Maris. Maris enables rigorous message flow control within MACS by embedding reference monitors into key multi-agent conversation components. We implemented Maris as an integral part of widely-adopted open-source multi-agent development frameworks, AutoGen and LangChain. To evaluate its effectiveness, we develop a Privacy Assessment Framework that emulates MACS under different threat scenarios. Our evaluation shows that Maris effectively mitigated sensitive data leakage threats across three different task suites while maintaining a high task success rate.

1 Introduction

Maris addresses the challenge of enforcing fine-grained privacy policies across multi-agent communication and external interactions. It combines formal policy enforcement with framework-level integration and is evaluated across three task suites.

1 Introduction

  • Existing safeguards lack robust formal guarantees, while MACS privacy policies require temporal constraints and fine-grained field-level flow control.The challenges include tracking information across agent-agent and agent-environment boundaries and adapting policies to evolving conversational state and tool outputs.Source: §1 · pp. 1–2
  • Existing development frameworks lack fine-grained information-flow controls and message-flow monitoring, while integrating safeguards into application logic can be disruptive.The paper frames its research question around integrating formally guaranteed privacy techniques while retaining seamless deployment and compatibility.Source: §1 · p. 2
  • Maris integrates a formally verifiable privacy-compliance framework into agent development frameworks without requiring changes to original application logic.Its architecture uses MFOTL policies and runtime reference-monitor and policy-enforcer components to check interactions across agents and external environments.Source: §1 · p. 2
  • Maris reduces attack success rate to 0% under passive and proactive scenarios while preserving task success rate across three MACS task suites.It introduces an average enforcement delay overhead of 4.87%.Source: §1 · p. 2
  • The paper contributes an implemented and evaluated framework and an automatic privacy assessment framework spanning healthcare, supply-chain optimization, and travel-agent task suites.The contributions also include incorporating Maris into AG2 and releasing code and a dataset.Source: §1 · p. 2

2 Background

The background describes the frameworks and communication patterns relevant to Maris, its threat model, and the study’s scope. The study focuses on policy-based protection and GDPR-related requirements.

2.1 Multi-Agent Development Framework

  • Multi-agent development frameworks support agent coordination and interactions with LLMs, tools, and people through static or dynamic conversation modes.The study demonstrates its design on AG2 and LangGraph; it also reports implementation on A2A elsewhere in the paper.Source: §2.1 · p. 3

2.2 Threat model

  • The threat model covers intentional or accidental privacy-policy violations through agent-agent and agent-environment communication, including malicious agents, users, tools, and workflow errors.It assumes attackers do not require system-level compromise of the host operating system.Source: §2.2 · pp. 3–4

2.3 Scope of Study

  • The study focuses on GDPR and policy-based data protection; health-data and other sector-specific obligations are out of scope.LLM-assisted policy generation is supported, but automated policy generation for usability and scalability is not a main focus.Source: §2.3 · p. 4

3 Design

Maris combines typed agent actions, MFOTL privacy policies, and runtime enforcement to check actions before execution. Its design targets formal policy guarantees and deployment without disrupting original application logic.

3.1 Design Goal and Principles

  • Maris aims for formally verifiable privacy enforcement, machine-enforceable GDPR-aligned policies, and deployment that preserves original application logic and supports static and dynamic conversations.The design goals include contextual expressiveness and information-flow-level policy definitions.Source: §3.1 · p. 4

3.2 Design Overview

  • The Agent Action Generator derives typed action specifications and signature files from system prompts and tool descriptions, while runtime enforcement intercepts actions and checks them against deployed policies.Permitted actions execute and are logged; disallowed actions receive enforcement before execution.Source: §3.2 · p. 4

3.3 Agent Action Specifications

  • Maris restricts agents to structured, schema-defined messages and tool calls, enabling structured logging, precise action capture, type checking, and a reduced natural-language prompt-injection attack surface.Action specifications are parsed into MFOTL signatures; the generator creates message schemas from task descriptions and agent roles, while tool definitions come from agent definitions.Source: §3.3 · p. 5

3.4 MACS Privacy Policy Representation

  • MFOTL policies express information-flow restrictions and time-bounded workflow requirements, such as requiring a prerequisite action within the previous 3600 seconds.The policy schema associates an MFOTL formula with a block, warn, or mask action, with optional fields to redact; LLM-assisted generation produced 100% syntactic validity and 100% violation detection, while valid-workflow results were 100% for PR#1 and 90% for PR#2.Source: §3.4 · p. 6

3.5 Runtime Policy Enforcement

  • At runtime, the Reference Monitor intercepts actions, the Event Parser converts them into MFOTL predicates, and the Policy Enforcer checks proposed actions with MonPoly against policies and action history.The monitor blocks, masks, or warns according to the policy; temporal checks use the full history, and allowed actions are recorded.Source: §3.5 · pp. 6–7

4 Implementation

Maris implements runtime policy enforcement in AG2 by extending conversation and agent components, and also provides implementations for LangGraph and A2A. Its implementation validates and parses actions before policy checks, then blocks, masks, or allows them.

Nuts and bolts: AG2 deployment

  • During dynamic group-chat broadcasts, each recipient’s message is checked separately, allowing the same information to be delivered to one agent and blocked from another.Successfully delivered messages are recorded in a shared event buffer for later verification.Source: §4 · p. 7
  • FormalConversableAgent validates structured agent outputs against action and message schemas, providing error feedback and retrying invalid responses up to a configurable maximum.Tool calls are checked before execution, and tool results are validated against their return schemas before agents share them through predefined message schemas.Source: §4 · p. 7

Agent Action Generator

  • The Action Generator produces message schemas and MonPoly event signatures, generating inter-agent schemas from task context and agent roles.Tool definitions and return schemas come from registered tool specifications in code.Source: §4 · p. 7

System building

  • A reference monitor validates proposed actions and sends them for policy enforcement, which can block, mask, or allow them.The Event Parser converts actions into typed MFOTL predicates; the Policy Enforcer checks them against policy constraints using MonPoly.Source: §4 · pp. 7–8

Discussion: LangGraph and A2A Implementation

  • Maris extends LangGraph’s StateGraph with schema validation and policy enforcement, reusing the Policy Enforcer for tools and monitored inter-agent messages.FormalNode validates node outputs, FormalToolNode checks tool calls, and EdgeMonitor logs inter-agent events.Source: §4 · p. 8
  • The A2A implementation wraps the SDK with monitors that validate incoming messages before execution and outgoing messages before network calls.The prototype extends AG2’s GroupChatManager and ConversableAgent, and was implemented on AG2 version 0.10.2.Source: §4 · pp. 7–8

5 Evaluation

Maris was evaluated across three MACS task suites using emulated privacy threats and policy requirements. It reduced reported policy violations to 0% in the evaluated enforcement scenarios, while Travel Agent bookings retained a 100% task success rate.

5.1 Environment Setup

  • The framework emulates MACS tasks, privacy requirements and settings, insider/external threats including prompt injections, and time-bound policy scenarios, then evaluates privacy violations with PVR and system utility with TSR.Source: §5.1 · pp. 8–9

5.2 Task Suite #1: HospitalGPT

  • HospitalGPT policies masked patient PII from the Supervisor and Epidemiologist while preserving the Outreach Agent’s access; Maris detected freshness violations and blocked outdated outreach.The freshness rule requires patient data queried within the preceding 3600 seconds; task success is not applicable when the violating action is blocked.Source: §5.2 · p. 10

5.3 Task Suite #2: OptiGuide

  • In OptiGuide, Maris checked for supplier consent and blocked data access after revocation, achieving a 0% policy violation rate.The evaluation injected consent events followed by revocations before testing data retrieval.Source: §5.3 · p. 11

5.4 Task Suite #3: Travel Agent

  • Across HospitalGPT, OptiGuide, and Travel Agent, Maris reduced reported policy violations to 0% in the evaluated enforcement scenarios.For Travel Agent, bookings retained a 100% task success rate with enforcement.Source: §5.2 · p. 10, §5.3 · p. 11, §5.4 · p. 12
  • In Travel Agent, Maris restricted unnecessary sharing between flight and hotel agents and blocked analytics use without consent, with bookings still completing.The reported task success rate was 100%; booking completion was unaffected because bookings typically precede analytics processing.Source: §5.4 · p. 12

6 Performance Overhead

Maris adds enforcement latency, measured through component timing and end-to-end comparisons with vanilla MACS. Across the reported task suites, enforcement overhead varies, with policy checking dominating component costs.

6 Performance Overhead

  • 946ms per query (4.9% of wall-clock time) is the average total enforcement overhead across 10 runs per use case.Policy checking is the dominant overhead component; schema validation and action parsing each incur less than 1ms.Source: §6 · p. 12
  • OptiGuide incurs 1.1%, HospitalGPT 5.0%, and TravelAgent 9.3% overhead in the end-to-end comparison.The comparison is between vanilla MACS and MACS with policy enforcement.Source: §6 · p. 13, Figure 8 · p. 13
  • Maris adds an average response delay of 1.13s compared to vanilla AG2, primarily due to extended prompts for fixed message schemas.The end-to-end comparison uses 10 queries per task suite with identical LLM configurations; the evaluation avoids violations that could leave tasks unfinished.Source: §6 · pp. 12–13

7 Discussion

Maris is implemented in AG2, LangGraph, and A2A, and the paper describes how its reference monitor could be integrated into other frameworks. The proposed integrations use framework-specific conversation hooks or handlers.

7 Discussion

  • Maris was implemented on AG2, LangGraph, and A2A to demonstrate design generalizability.AG2 was selected for its support for dynamic and static conversation modes, agent-to-agent and agent-to-environment communication, and diverse applications.Source: §7 · p. 13
  • CrewAI integration could use hooks such as before_kickoff, while llama-index could use BaseCallbackHandler to attach the reference monitor.The paper says these integrations primarily involve modifying the frameworks’ conversation handler modules while reusing the policy schema and enforcer.Source: §7 · p. 13

8 Related Work

Prior agent defenses address threats such as harmful instructions, data exfiltration, and prompt injection, but the cited approaches are limited in their MACS coverage and deployment characteristics. Maris is presented as supporting non-intrusive, formally verifiable, fine-grained, and time-bounded control in MACS.

8 Related Work

  • Prior defenses include guardrails, isolation of tools, fixed control-flow code, structured planning, and supplementary LLM agents.The related work also describes policy-decoupled approaches, including ShieldAgent’s probabilistic representation of privacy-sensitive actions and LLM-based translation of outputs.Source: §8 · p. 13
  • The cited approaches focus on single-agent settings, overlook inter-agent threats, lack fine-grained and time-bounded control, and can require disruptive deployment effort.Source: §8 · p. 13
  • Maris is presented as enabling non-intrusive, provable, fine-grained information control and expressive GDPR-compliant policy enforcement within MACS.Source: §8 · p. 13

9 Conclusion

The paper presents Maris as a formally verifiable privacy compliance framework for MACS and reports reduced attack success with preserved task success and 4.87% average enforcement overhead.

9 Conclusion

  • Maris uses MFOTL for policy specification and runtime enforcement, providing provable privacy-compliance guarantees while integrating without altering application logic.Source: §9 · p. 13
  • Across three real-world MACS task suites, Maris reduces attack success rate to 0% under passive and proactive scenarios while preserving task success rate.Source: §9 · p. 13
  • Maris introduces 4.87% average enforcement overhead across the evaluated task suites.Source: §9 · p. 13

Ethical Considerations

The evaluation uses synthetic or publicly available data across three use cases to support ethical compliance. Synthetic user profiles are used only for evaluation.

Ethical Considerations

  • The three use cases use synthetic or publicly available data to ensure ethical compliance.HospitalGPT uses a synthetic FHIR R4 dataset; OptiGuide and TravelAgent use publicly available records.Source: Ethical Considerations · p. 14
  • HospitalGPT uses synthetic FHIR R4 data, while OptiGuide and TravelAgent use publicly available supplier, roaster, hotel, and flight information.The OptiGuide information comes from code samples.Source: Ethical Considerations · p. 14
  • Synthetic user profiles generated using an LLM are used for evaluation only.Source: Ethical Considerations · p. 14

Open Science

The paper releases implementation, assessment, use-case, dataset, and reproduction artifacts. Its policy-generation prompt takes a natural-language requirement and supporting formal-policy context and requests an MFOTL formula.

Open Science

  • The repository releases Maris with AG2 integration, the assessment framework, three use cases, policy specifications, evaluation datasets, and reproduction scripts.A devcontainer configuration is also provided for reproducible environment setup.Source: Open Science · p. 14

A Prompt for automated policy generation

B Prompt Injection Strategies

The evaluation describes four prompt-injection strategies adapted from AgentDojo. It also reports LLM-assisted policy-generation trials on HospitalGPT policies.

B Prompt Injection Strategies

C Privacy Policy of Task Suites

The task-suite policies specify data masking, consent and timing requirements, and limits on inter-agent information flows. The syntax reference describes temporal policy checks and their runtime enforcement, and the appendix shows Maris with HospitalGPT in AG2.

C.1 Privacy Policy for HospitalGPT

  • HospitalGPT policies mask phone, first-name, and ID fields for non-essential recipients and block outreach unless a patient query occurred within the previous hour.Source: §C.1 · p. 17

C.2 Privacy Policy for OptiGuide

  • OptiGuide policies block supplier-data access unless consent has occurred and has not been revoked.Source: §C.2 · p. 17

C.3 Privacy Policy for Travel Agent

  • TravelAgent policies block flight-booking instructions from reaching the Hotel Agent, hotel-booking instructions from reaching the Flight Agent, and flight-booking results from reaching the Hotel Agent.Source: §C.3 · pp. 17–18
  • TravelAgent policies restrict customer PII use for analytics unless the customer approves and limit agents to information needed for assigned tasks.Source: §C.3 · pp. 17–18

D MFOTL Syntax Reference

  • MFOTL extends first-order logic with metric-time temporal operators, and the examples use negation for information-flow constraints and ONCE intervals for time-bounded prerequisites.One example requires a database query within the previous 3600 seconds before an outreach message.Source: D MFOTL Syntax Reference · p. 18
  • Maris runs MonPoly over streaming event logs; empty output indicates compliance, while non-empty output indicates a violation and triggers the configured enforcement action.Source: D MFOTL Syntax Reference · p. 18

E Maris Usage in AG2

Loading 2505.04799v5…