PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy Preference Controls for Agentic Systems

Shreya Kochar, Peeranat Sugsavanvit, Vidur Gupta, Joseph A. Calandrino, Deepak Kumar, Brett Frischmann, Sebastian Zimmeck

arXiv:2610.29929v2cs.CYcs.HC

TL;DR

As AI agents act autonomously on behalf of users, existing privacy preference controls face a setting in which agents can access sensitive data and may overshare or participate in tracking. The paper develops a taxonomy from laws, empirical studies, and an audit, then explores implementation through eight open-source prototypes; it systematizes enforceable preferences and argues that legal and technical work must proceed together.

  • Problem

    Online tracking and autonomous agents’ access to sensitive data raise the question of what privacy controls users should be able to express and have respected in agentic systems.

    Source: §1 · p. 1

  • Method

    The authors develop a taxonomy informed by laws, empirical opt-out studies, and an audit of AI products, then analyze its application through eight agentic prototypes.

    Source: §1 · p. 2

  • Results

    The paper systematizes enforceable privacy preferences for agentic systems and identifies the need for integrated legal and technical effort to shape their protections.

    Source: §7 · p. 15

  • Takeaways & Limitations

    Protecting privacy in agentic systems requires developing privacy-preserving technology while mapping existing legal concepts onto agentic-system components.

    Source: §7 · p. 15

    The taxonomy addresses future data processing and privacy controls under privacy and data protection laws, but excludes deletion rights and harms caused by AI outputs.

    Source: §4.7 · p. 11

Abstract

from arXiv · show

In response to vast and widespread online tracking, regulators and technologists alike have worked to develop laws and tools that allow people to express their privacy preferences. These measures seek to offer users greater control over the sale and sharing of personal information, putting agency in their hands to protect their privacy. However, the web is undergoing a seismic shift, away from strictly user-driven behavior towards agentic behavior, where AI agents take autonomous actions on behalf of users. In this study, we ask: What should privacy preference controls look like in the agentic web and agentic systems more generally? To answer this question, we develop a taxonomy of enforceable privacy preferences for agentic systems. Our taxonomy is informed both by laws and empirical studies of existing privacy tools as well as an audit of 30 leading AI assistants and agent platforms. To explore how our taxonomy can be implemented, we design and develop eight open-source prototypes that mirror real-world agentic applications and demonstrate how major aspects of our taxonomy can be structurally integrated in agentic protocols, such as the Model Control Protocol and Agent2Agent protocol. Ultimately, our study seeks to provide the groundwork for ensuring end-user privacy and autonomy is not lost in the newly arriving agentic world.

1 Introduction

Privacy controls designed for user-driven web activity face new challenges as agents act autonomously, access sensitive data, and may overshare without direct user control. The study develops a taxonomy of enforceable preferences and examines their implementation through an audit and open-source prototypes.

1 Introduction

  • Online tracking has prompted privacy laws and technical tools intended to let users opt out and ease the burden of privacy self-management.The introduction frames agentic systems as a shift away from user-driven interactions.Source: §1 · p. 1
  • Agents may need privileged access to sensitive data, yet may overshare or use it without direct user control.The introduction also notes that AI systems are participating in third-party tracking and that companies have begun integrating ads.Source: §1 · pp. 1–2
  • The authors develop a taxonomy of expressible agentic privacy preferences informed by existing laws, empirical opt-out studies, and an audit of AI products from 22 companies.They implement eight agentic prototypes to analyze where and when the taxonomy can be applied to enforce preferences.Source: §1 · p. 2
  • The study asks what privacy preferences people should be able to express and have respected, and what technological and legal challenges affect their effectiveness.It also asks how to structurally integrate controls into agentic systems interacting with the web and online ad ecosystem.Source: §1 · p. 2
  • The paper contributes an empirical audit of companies’ privacy controls and supported agentic protocols, plus open-source prototypes for MCP and A2A.The taxonomy is intended for implementation in agentic protocols and is informed by the audit.Source: §1 · p. 2

2 Background and Related Work

Existing web privacy controls address tracking, but agentic systems introduce autonomous, multi-hop data flows that can evade users’ awareness and control. The section motivates protocol-level preferences that support data minimization and contextual boundaries.

2.1 Notice and Choice

  • Notice and choice offers little meaningful negotiation, while agentic systems can disclose sensitive data to unknown downstream parties without user awareness or supervision.Agents may also over-disclose when manipulated.Source: §2.1 · p. 3

2.2 Privacy Preference Signals

  • GPC builds on lessons from P3P and Do Not Track by combining technical support with legal and regulatory backing.P3P lacked enforcement and DNT relied on self-regulation; both were not widely adopted.Source: §2.2 · p. 3

2.3 Web Advertising and Tracking

  • Tracking mechanisms now operate inside conversational AI services, and agentic delegation can expose user data across multiple agents and systems.Figure 1b depicts a booking flow in which a user’s name and dates reach the travel agent, hotel agent, and booking system.Source: §2.3 · pp. 3–4, Figure 1 · p. 4

2.4 Privacy in Agentic Protocols

  • Because users may know only their orchestrator, downstream delegation creates an accountability vacuum; the paper argues controls should be built into communication protocols.MCP connects agents to tools and data, while A2A supports agent-to-agent interaction.Source: §2.4 · p. 5

2.5 Data Minimization and CI

  • Agentic systems offer a path from data maximization toward minimizing information to what a user-authorized task requires, rather than company-defined purposes.Agents make minimization harder because requests are expressed semantically; challenges include determining task-required data, limiting context contamination, and avoiding tool over-privilege. Contextual integrity frames appropriate flows by sender, recipient, subject, information type, and transmission principle.Source: §2.5 · pp. 5–6
  • Prior approaches constrain agent data flows through task-specific minimization, structured firewalls, scoped delegation, and MCP interceptors.Interceptors can observe, filter, or modify traffic.Source: §2.5 · p. 6

3 Methodology

The methodology derives and revises a privacy-preference taxonomy using prior legal, empirical, and technical work, an audit of AI products, and prototype implementations. The audit also records which agentic protocols companies use or support.

3 Methodology

  • Taxonomy development began with legal, regulatory, and technical sources, then incorporated an audit of AI products and prototype implementations for agentic scenarios.The categories drew on GPC and privacy laws, opt-out studies, privacy taxonomies, contextual integrity, social boundaries, autonomy harms, and human-choice engineering.Source: §3 · p. 6, §3.1 · p. 6

3.2 Empirical Audit

  • The audit covered 30 public-facing products from 18 of 22 selected companies, identified using technology-company, AI-company, and chatbot rankings.Four companies had no audited product.Source: §3.2 · p. 6
  • Authors reviewed company documentation for controls that let users, administrators, or developers stop or limit data practices or AI behavior, then used repeated LLM categorization and author review.The process yielded 32 candidate controls; authors placed 24 within the initial taxonomy’s scope and added human review as a subcategory.Source: §3.2 · pp. 6–7
  • 16 of 22 audited companies use MCP and 5 use A2A in their own products; including partner platforms and third-party packages, 18 support MCP and 12 support A2A.Three A2A support cases relied on minimal evidence; MCP is dominant for agent-tool connections, while A2A is the only agent-to-agent protocol used by more than one company.Source: §3.2 · p. 7

3.3 Taxonomy Category Revision

  • The authors compared audited controls with the initial taxonomy, revised categories as needed, and checked that each control mapped to at least one subcategory.The audit captures documented controls, not whether they work as described; login restrictions, enterprise contracts, geography, and retrieval omissions may also limit coverage.Source: §3.3 · p. 7, §3.2 · p. 7

4 Taxonomy

The taxonomy organizes enforceable privacy preferences around agent presence, collection, data use, persistence, and autonomy, grounded in data minimization and user-specified task boundaries. Its audit found that most identified settings map to the taxonomy, while also exposing weak or undocumented controls and scope limits.

4.1 Category A: Presence

  • Agents may not be integrated without genuine consent or run in the background without affirmative activation; users can set these preferences independently.The activation preference applies to ambient or background presence, not foreground, on-demand use.Source: §4.1 · p. 8

4.2 Category B: Collection

  • Collection is limited to task-relevant data: submitted inputs, user-generated metadata, and delegated-action records may be collected or retained only as necessary for the task.The taxonomy distinguishes data knowingly submitted by the user from data unknowingly produced, and defaults to deletion once retention is no longer necessary.Source: §4.2 · p. 8

4.3 Category C: Secondary Use, Repurposing, Sharing, and Human Review

  • Collected data may be used for the requested task; secondary uses, model development, further sharing, and human review require the relevant user preference, subject to legally required review.These preferences address distinct uses and can be asserted independently.Source: §4.3 · p. 9

4.4 Category D: Persistence

  • Persistence preferences distinguish ending an agent’s presence after a task, barring unprompted activity, and preventing expansion beyond the original scope; the restrictions form a hierarchy.Asserting D1 implicitly includes D2 and D3, while asserting D2 implicitly includes D3.Source: §4.4 · p. 10

4.5 Category E: Autonomy

  • By default, an agent may not start tasks for the user; users can permit autonomous decisions by tiers, while unresolved decisions must be surfaced or declined.An agent cannot express preferences on the user’s behalf.Source: §4.5 · p. 10

4.6 Audit and Taxonomy Analysis

  • 567 of 788 audited settings and actions (72%) fit a taxonomy subcategory, 21 (3%) concern retention duration, and 200 (25%) fall outside its scope.Figure 2 displays product shares by subcategory; denominators vary where subcategories do not apply to every product.Source: §4.6 · p. 10, Figure 2 · p. 11
  • Only 14 of 29 products offer sale-or-sharing opt-outs, 7 of 29 offer analytics or product-improvement opt-outs, and 6 of 30 offer human-review opt-outs.Some existing controls are also less restrictive than the taxonomy, and only 40% of settings and actions document their default state.Source: §4.6 · pp. 10–11

4.7 Scope and Limitations

  • The taxonomy covers future data processing, not past processing or harms from AI outputs; legal, security, and user-initiated disclosure exceptions also constrain controls.Of 200 out-of-scope settings, 73 are deletion controls addressing past processing.Source: §4.7 · pp. 11–12

5 Agentic Prototypes

Eight prototypes mirror common agentic applications and use MCP and A2A to explore preference propagation and enforcement. They illustrate how controls can affect data flows and agent actions, while showing that enforceability depends on context.

5 Agentic Prototypes

  • The travel-planning prototype attaches preferences to each A2A message and MCP call, blocking profile and log writes and history reads while returning the same itinerary.Without preferences, the pipeline saves the trip and logs the interaction; with them, it retains nothing after the conversation.Source: §5 · p. 12
  • The medical-question prototype lets users separately turn off advertising, analytics, and model-training uses while preserving the same answer.Without preferences, the conversation goes to all three secondary uses; with preferences, all three are turned off.Source: §5 · p. 12
  • The productivity prototype checks tool categories against versioned consent, leaving newly added tools off until the user approves or declines integration.The check also covers background tracking and capabilities beyond the user’s original agreement.Source: §5 · p. 12
  • In product research, preferences reach publishers but do not materially change the provider’s own log; in web search, inference writes are dropped and logged while results remain the same.The web-search prototype still computes inferences but does not retain them when the user opts out.Source: §5 · p. 13
  • The email-revision prototype separately blocks retention of submitted drafts, editing telemetry, and inferences while preserving the polished email.The telemetry includes edits and pauses that the user may not know they are producing.Source: §5 · p. 13
  • The general-questions prototype distinguishes discarding sessions, keeping a user-visible archive without recall, and retaining transcripts without building a user model.With preferences set, the first session is discarded and not recalled, while same-session context remains available.Source: §5 · p. 13
  • In travel booking, users set action-specific autonomy preferences; unspecified actions require approval, and the agent declines them if the user cannot answer.Without preferences, four of six actions run without approval under the platform’s default tiers.Source: §5 · p. 13
  • Across the prototypes, preference propagation is feasible and often preserves utility, but enforcement and verification vary by context; existing protocols lack dedicated preference fields.The authors use the meta field in MCP and A2A, and identify compliance verification as future work.Source: §5 · pp. 12, 14

6 Analysis and Discussion

Agentic systems require privacy controls that protect personal autonomy across the agent lifecycle, while legal and technical boundaries remain difficult to align. The authors argue that protocol-level integration is feasible, but standardization and usability remain challenges.

6.1 RQ1: What privacy preference controls should people be able to express and have respected in agentic systems?

  • Agentic systems need controls spanning presence, data collection and use, post-task persistence, and authorization of autonomous decisions, balanced against choice fatigue.The authors call for user studies to reduce cognitive burden when controls are standardized.Source: §6.1 · p. 14

6.2 RQ2: What are the technological and legal challenges that need to be considered and resolved for privacy preference controls to be effective in agentic systems?

  • Task-bounded data minimization requires processing only what a user-authorized task needs, with departures requiring demonstrably informed consent.Establishing clear task boundaries remains challenging.Source: §6.2 · p. 15
  • Technical session isolation may not align with psychological or social context, and end-to-end signal verification across opaque, multi-hop delegation chains remains a challenge.The authors also identify shifting legal baselines from purpose-based maximization to task-bounded minimization as a challenge.Source: §6.2 · p. 15

6.3 RQ3: How can we structurally integrate privacy preference controls into agentic systems, especially, as such systems interact with the web and online ad ecosystem?

  • Privacy controls can be integrated into agentic protocols through mandatory preference-checking interceptors and standardized header signals at delegation points.The authors emphasize enforcement before agents interact with web endpoints or ad networks.Source: §6.3 · p. 15

7 Conclusion

The conclusion calls for integrated legal and technical work to protect privacy and autonomy as the agentic ecosystem develops.

7 Conclusion

  • Protecting privacy and personal autonomy in the agentic ecosystem requires coordinated legal work and privacy-preserving technical infrastructure.Future work includes mapping legal concepts such as controllers and processors onto agentic-system components.Source: §7 · p. 15

A Audit of Privacy Preference Controls

The audit catalogs privacy controls across 30 AI products and maps them to the paper’s taxonomy, while distinguishing controls from legal mechanisms and out-of-scope items.

A Audit of Privacy Preference Controls

  • Auditors examined product settings, help pages, and policies in addition to a representative source page listed for each product.The table reports companies and products in the audit sample and counts verified settings and actions.Source: A Audit of Privacy Preference Controls · p. 21, Table 2 · p. 22
  • 18 of 30 products offered control over which connectors and third parties receive data, while 21 of 28 offered limits on which apps, sites, files, and networks an agent can reach.Products counts apply only where a control type applies; Controls counts verified settings and actions.Source: A Audit of Privacy Preference Controls · p. 21, Table 1 · p. 21
  • The audit covered settings ranging from turning AI features on or off to requiring approval before agent actions, with the latter found in 19 of 28 applicable products.The table maps these opt-out types to taxonomy subcategories.Source: A Audit of Privacy Preference Controls · p. 21
  • Deletion was found in 27 of 30 products but was out of scope for the taxonomy; requesting privacy rights was treated as a legal implementation mechanism.T25 is an application of the taxonomy, while T27 is a legal mechanism to implement parts of it.Source: A Audit of Privacy Preference Controls · p. 21, Table 1 · p. 21

B Audit of Protocols

The protocol audit catalogs which agentic protocols major technology and AI companies use and support, drawing on company documentation and related materials.

B Audit of Protocols

  • Tables 3 and 4 distinguish protocols companies use internally from those they support.Each company entry draws on documentation, specifications, code repositories, and partner platforms, alongside a representative documentation page.Source: B Audit of Protocols · p. 22, Table 3 · p. 23, Table 4 · p. 23
  • For Apple, the audit lists MCP internally and MCP and A2A as supported; for Microsoft, it lists MCP, A2A, and WebMCP among protocols used internally.The tables cover the ten largest technology companies and a specified cohort of AI companies and chatbot developers.Source: B Audit of Protocols · p. 22, Table 3 · p. 23, Table 4 · p. 23

C Prototypes

The taxonomy organizes privacy preferences by what they govern and maps categories to prototypes. It covers agent presence, data collection, secondary use, sharing, and human review.

C Prototypes

  • Table 5 maps taxonomy categories to the prototypes that demonstrate them, which can be run as real-world examples.The table summarizes categories, subcategories, and the scenarios they govern in the agentic web.Source: C Prototypes · p. 23, Table 5 · p. 24
  • Presence preferences govern whether AI can be integrated without genuine consent or run ambiently without affirmative choice.Collection preferences cover knowingly submitted input beyond task needs, passive metadata, delegated-action records, and retention after the task.Source: C Prototypes · p. 23
  • Secondary-use preferences cover repurposing data, targeting, inference, commercial and analytical use, and training, fine-tuning, improving, or evaluating models.Sharing preferences govern how far data travels through connected systems or agents, while human-review preferences govern provider personnel access to user data and agent records.Source: C Prototypes · p. 23

D Implementation Details

The implementation examples recommend checks tailored to the desired privacy outcome and distinguish data retention from controls on agent presence, initiation, and scope. They also identify operational checks for current preferences, delegated-activity logs, and enforcement points.

D Implementation Details

  • Table 6’s recommendations assume an established preference and permitted task scope; retention follows a task-based rule with explicit choices and legally required exceptions.The rule does not provide a retrospective deletion right; Category D instead governs post-task presence, unprompted initiation, and scope expansion.Source: D Implementation Details · p. 24
  • Preventing storage does not necessarily prevent collection or inference, so the recommended check depends on the required outcome.Figure 3 places checks in application code, an optional gateway, and cooperating recipient services; direct unrestricted network routes bypass the dispatcher, and independent recipient enforcement depends on its operator.Source: Table 6 · p. 24, Figure 3 · p. 25
  • A queued training export must check the current preference when admitted; holding it during a preference-store outage preserves the restriction but delays completion.Operation-only access logs still describe delegated activity and must meet the chosen retention limit; Figure 4 calls for new approval after a mismatch and duplicate-safe retry or status checking after uncertain execution.Source: D Implementation Details · p. 25

D.1 Privacy at Intermediaries

Intermediaries can access prompts and tool data, creating additional locations for retention or theft beyond the final provider. The paper recommends accounting for intermediary practices when selecting routes, while noting that provider settings were not tested.

D.1 Privacy at Intermediaries

  • Each added router or intermediary gives another operator access to prompts and tool data and another place where information may be retained or stolen.A final provider’s no-retention promise does not cover a relay’s own logs, and the authors did not test whether providers honor their settings.Source: §D.1 · p. 25
  • Among 428 paid and free routers, Liu et al. found nine injecting malicious code and 17 accessing researcher-owned cloud credentials.Ji et al.’s measurements across 39 reseller endpoints revealed shared access to upstream prompt caches, evidence of hidden dependencies rather than verified supplier relationships.Source: §D.1 · pp. 25–26
  • For consequential actions, the executor checks the final request against approval made outside the model and the current preference; mismatches require new approval.After an uncertain result, retry with duplicate-execution protection or check the original operation’s status.Source: Figure 4 · p. 26
  • The authors recommend excluding routes whose intermediaries’ declared data practices conflict with user preferences, even if that limits services or delays a task.Checking only the final provider would leave intermediate storage, reuse, and disclosure unchecked.Source: §D.1 · p. 26

E Prototype Scenarios

The prototypes demonstrate privacy controls across scenarios involving consent, tracking, collection, inference, tool calls, and purpose-specific opt-outs. They also expose limits in signal propagation and show how collection controls can address distinct data sources.

E Prototype Scenarios

  • The scenarios cover delegated travel and medical data, consent updates that add tools, multi-publisher research, inferred sensitive attributes, and behavioral traces during email editing.The examples include itinerary sharing, medical data used for analytics, training, and targeting, and searches from which health, financial, and employment details are inferred.Source: E Prototype Scenarios · p. 26
  • Purpose-specific opt-outs can be applied at the pipeline’s gating stage.The supplied publication record lists receipt on 20 October 2026 and revision and acceptance as TBD.Source: E Prototype Scenarios · pp. 26–27
  • Consent-scoped tools receive no pipeline data by default when added after consent, while tool-level blocking carries opt-out signals with MCP calls.Specified tools are blocked from storing and handling data.Source: E Prototype Scenarios · p. 26
  • Forwarding an opt-out signal lets honoring publishers stop tracking, but the intermediary still logs queries and a dropped signal prevents downstream enforcement.Recording that the signal was set can itself turn the opt-out into tracked data.Source: E Prototype Scenarios · p. 26
  • Opting out of inference blocks derived attributes from profile writes while searches continue; collection controls can also limit input logs, behavioral traces, and inferences.The collection prototype labels inferences by source, and the scenario table summarizes which privacy preferences each prototype demonstrates.Source: E Prototype Scenarios · p. 26, Table 7 · p. 27

Related papers

Loading 2610.29929v2…