Source-linked AI summary
AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents
Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, Maya Pavlova, Ruslan Salakhutdinov, Kamalika Chaudhuri
TL;DR
Autonomous agents need access to personal information, creating a need to determine whether they use sensitive data only when necessary. The paper introduces AgentDAM, an end-to-end benchmark that evaluates privacy during realistic web navigation and finds frequent data-minimization failures, while privacy-aware prompting reduces leakage with limited task-performance impact.
Problem
Privacy measurement for agents using sensitive information at inference time is not well understood, because probing language models does not assess privacy during concrete multi-step web actions.
Method
AgentDAM evaluates web agents in realistic, isolated WebArena and VisualWebArena environments by measuring task utility and irrelevant private-information leakage in agent trajectories.
Results
Current agents built on GPT, Llama, and Claude exhibit varying degrees of privacy leakage, while privacy-aware prompting reduces leakage with minimal impact on task performance.
Takeaways & Limitations
End-to-end measurement is needed because directly probing language models can overestimate privacy in action, and current agents still require stronger privacy-awareness.
Takeaways & Limitations
The benchmark currently uses fictional entities and scenarios within simulated VISUALWEBARENA and WEBARENA environments, and should be expanded to more tasks, applications, and agentic scenarios.
Abstract
from arXiv · showhide
Autonomous AI agents that can follow instructions and perform complex multi-step tasks have tremendous potential to boost human productivity. However, to perform many of these tasks, the agents need access to personal information from their users, raising the question of whether they are capable of using it appropriately. In this work, we introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of ``data minimization''. For the purposes of our benchmark, data minimization means that the agent uses a piece of potentially sensitive information only if it is ``necessary'' to complete a particular task. Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents. We use AgentDAM to evaluate how well AI agents built on top of GPT-4, Llama-3 and Claude can limit processing of potentially private information, and show that they are prone to inadvertent use of unnecessary sensitive information. We also propose a prompting-based defense that reduces information leakage, and demonstrate that our end-to-end benchmarking provides a more realistic measure than probing LLMs about privacy. Our results highlight that further research is needed to develop AI agents that can prioritize data minimization at inference time.
1 Introduction
Autonomous agents need personal information to complete tasks, but current methods do not adequately measure whether they use sensitive data only when necessary. AgentDAM provides an end-to-end benchmark for evaluating privacy leakage during realistic web interactions and finds that agents often use irrelevant sensitive information.
- Autonomous agents may access personal information such as credit-card numbers, emails, and chats while performing tasks, raising questions about appropriate use and privacy.
- Existing privacy evaluations often probe language models, but practical agents must manage personal information while executing complex, multi-step web tasks.
- Data minimization means using potentially sensitive information only when it is required for the target task.
- AgentDAM is an end-to-end benchmark built on realistic, isolated WebArena and VisualWebArena environments to evaluate data minimization in action.It covers Reddit, GitLab, and Shopping environments and constructs tasks involving relevant and irrelevant private information.
- The benchmark measures both task utility and irrelevant private-information leakage, using an LLM-based judge to analyze agent trajectories.
- Current web-navigation agents often fail to follow data minimization even in straightforward benign settings, while privacy-aware prompting can reduce leakage with minimal task-performance impact.Direct privacy probing can overestimate privacy, whereas end-to-end evaluation exposes task-irrelevant leakage across agents built on GPT, Llama, and Claude.
2 Related Work
Prior work studies agent privacy through language-model probing, memorization, simulated environments, and adversarial leakage, but AgentDAM evaluates privacy during realistic multimodal web-agent execution.
- Web-navigation agents scaffold language models with website representations and browser backends to execute user-specified tasks.
- Inference-time privacy research has commonly asked language models whether revealing sensitive information is socially appropriate, rather than observing agents acting.
- AgentDAM differs from emulated text-only environments by evaluating multimodal agent trajectories on targeted tasks in realistic web interactions.
- Language-model privacy research has primarily focused on training-data memorization, including membership inference and targeted extraction.
- Other agent-security work examines prompt injection, adversarial defenses, or exfiltration attacks involving malicious parties, unlike AgentDAM’s benign setting.
3 AGENTDAM: Privacy Benchmark for Web Agents
AgentDAM benchmarks privacy leakage during realistic, long-horizon web-agent interactions by combining task execution with evaluation of unnecessary sensitive-information use. It constructs tasks and synthetic private data across web environments, then measures leakage in agent actions.
- Benchmark setup: AgentDAM evaluates web agents in realistic, isolated, controllable environments rather than on the open web.The benchmark builds on WEBARENA and VISUALWEBARENA to support realism and reproducibility.
- Agentic setup: Each interaction supplies user instructions, private user data, and a webpage representation to an agent that selects browser actions.Representations include accessibility trees and screenshots, with Set-of-Marks prompting available for multimodal models.
- Agentic setup: The agent-environment loop repeats across long-horizon tasks as actions change webpage state and produce new observations.The environment applies a deterministic transition from the current state and action to the next state.
- Task design: The benchmark contains 246 tasks whose synthetic user_data mixes instruction-relevant information with irrelevant potentially sensitive information.Tasks span Reddit, e-commerce, and GitLab environments, with eight task types across these applications.
- Task design: Human annotators create instructions and Data Seeds containing plots plus irrelevant sensitive facts, which are expanded into realistic user_data.The dataset-generation pipeline uses prompting LLMs to produce long-form private texts such as conversations or email threads.
- Evaluation: Privacy evaluation examines every agent action for sensitive-data leakage, using keyword matching when sufficient and contextual judgment for generic concepts.An LLM-based judge analyzes trajectories, while the benchmark also measures task performance as utility.
- Initial finding: 12% to 46% leakage rates were observed for most agentic models using default VISUALWEBARENA scaffolding.The benchmark then motivates testing simple mitigation strategies to reduce these rates.
4 Experiments
The experiments compare privacy probing with end-to-end agent evaluation, measure current agents across website representations and mitigation settings, and analyze where leakage occurs. Results show that probing can overestimate privacy, baseline privacy varies substantially across models, and prompting improves privacy with a utility trade-off.
- The evaluation asks whether privacy probing measures data minimization in action, how current agents perform, and whether privacy-aware system prompts help.
- 4.2 Is Probing LLMs sufficient to assess agents’ performance in action?: Table 3 compares privacy scores from probing LLMs with scores from end-to-end agents running in the web environment.Higher privacy scores indicate better privacy according to the table caption.
- 4.2 Is Probing LLMs sufficient to assess agents’ performance in action?: Probing often overestimates privacy awareness and cannot measure the utility/privacy trade-off because no agent runs during evaluation.The result is reversed for Llama models, which are safer in the agentic environment.
- 4.3 Main Evaluation: Multimodal inputs generally slightly improve task performance while having little effect on privacy awareness.Table 4 organizes results by website representation and privacy mitigation.
- 4.3 Main Evaluation: GPT models achieve privacy performance of 25% to 46%, whereas Llama and Claude show around 90% privacy awareness without mitigation.Llama and Claude are not the strongest models in utility despite their higher privacy awareness.
- Mitigation: Reasoning-enhanced system prompts raise privacy performance for all models but remain below 94% and degrade task performance through false refusals.The observed utility degradation mostly involves false denial-of-service, such as refusing to comment on a post.
- 4.4 Analyzing privacy leakages: Leakages occur often in long-text tasks, especially Reddit posts, contact forms, and Reddit or GitLab comments, and can reflect confusion among account identifiers.These tasks require more information from the agent than shorter shopping searches.
5 Discussions and Conclusion
The benchmark reveals that current web-navigation agents often leak privacy-sensitive information and that existing mitigations do not achieve perfect privacy. The authors call for stronger strategies that improve data minimization while preserving task performance.
- Current agents, including GPT-4-, Llama-3-, and Claude-based systems, exhibit varying degrees of privacy leakage.
- Directly probing LLMs about privacy overestimates privacy in action, motivating end-to-end data-minimization measurements.
- None of the evaluated mitigations achieves a perfect privacy score.
- Future work should develop stronger mitigation strategies that enhance privacy while maintaining task performance.
- The benchmark should expand to more tasks, web applications, and agentic scenarios beyond web interactions.
A Details of the agentic environment
The benchmark reuses VISUALWEBARENA components for web-agent interaction, while Claude computer use operates through its own browser environment. It also covers task types across hosted GitLab, shopping, and Reddit applications.
- VISUALWEBARENA supplies the accessibility-tree and screenshot-with-Set-of-Marks website representations.
- VISUALWEBARENA also provides the backend that parses agent actions and executes browser commands.
- Claude-3.5-Sonnet with computer use instead runs a functional browser and desktop applications inside a Docker container, interacting through screenshots and an API.
- Table 7 organizes available task types across GitLab, shopping, and Reddit, with example data seeds describing required user data and sensitive information not to leak.
B.1 Example data seeds
Each task is represented by a high-level description of required user data together with irrelevant sensitive information that the agent should not use. These paired seeds are expanded into concrete user data by an LLM.
- PLOT specifies the high-level user data required to complete an agentic task.
- SENSITIVE_DATA lists irrelevant private information that the agent should not use.
- The paired PLOT and SENSITIVE_DATA seed prompts an LLM to generate the actual user_data.
B.2 Regarding our dataset size
The dataset uses repeated LLM sampling to create diverse user-data variations while controlling benchmark cost and size. Two samples per seed were selected because the privacy-leakage curve stabilized there, yielding 246 examples.
- Sampling each data seed once produces high variance and lower performance, while the privacy-leakage curve stabilizes at two repetitions.
- The final dataset contains 2 × 123 = 246 examples.
- The dataset is based on privacy-relevant tasks from the VISUALWEBARENA and WEBARENA frameworks.
- Benchmark size is constrained by computational expense; running Claude-cua on the full benchmark takes approximately 36 hours and costs around $1200.
- The current dataset is presented as a starting point that can be expanded dynamically with new websites and cases.
D Detailed evaluation results
The evaluation reports complete privacy-leakage counts across models and website representations, while separately examining whether models leak on the same instances. Models generally fail on different samples, although models from the same family correlate more strongly.
- Table 8 reports privacy leakage as raw counts for each website, where fewer leaks indicate better privacy.The table evaluates five state-of-the-art models across accessibility-tree and Set-of-Marks representations, plus Claude computer use.
- Pearson correlations between model leakage vectors are generally low, indicating that different agentic models tend to fail on different samples.The correlation analysis uses vision-capable models with the accessibility-tree and Set-of-Marks representation.
- Models from the same family, such as GPT models, generally show higher privacy-leakage correlation with one another.
E.1 Example 1
The first example shows an agent creating a GitLab issue for a frontend bug while exposing private notes from the user’s previous employment. The issue description therefore includes sensitive background information unrelated to the requested bug-fix plan.
- The user asks the agent to create a GitLab issue based on relevant frontend bug-fix notes from Notion.
- The agent reveals notes from the user’s previous project at Zenithal Analytics in the GitLab issue description.The leaked material includes a security feature and names of former collaborators.
- The example demonstrates leakage of private employment information during an otherwise ordinary issue-creation task.
E.2 Example 2
The second example shows an agent drafting an anonymous Reddit post about a family health and relationship concern while exposing identifying and employment information. The output includes names, an employer, and other irrelevant sensitive details.
- The user requests an anonymous Reddit post seeking advice about their son’s health and the couple’s relationship.
- The example illustrates privacy leakage in a long-form text-generation task that explicitly requests anonymity.
- The agent includes names, employment information, and other irrelevant sensitive details in the Reddit post.The source conversation identifies Jennifer, Mark, Ethan, and Freecity Insurance.
NeurIPS Paper Checklist
The checklist records that the paper’s claims and limitations are addressed, while noting that it has no theoretical results and relies on code and setup details for reproducibility. Experimental results are averaged over repeated runs without reported error bars.
- Claims and limitations: The checklist states that the abstract and introduction accurately reflect the paper’s main claims, contributions, and scope.
- Claims and limitations: The paper discusses its limitations in Section 5, consistent with the checklist response.
- Theory: The paper provides no theoretical results, so assumptions-and-proofs evaluation is marked not applicable.
- Reproducibility and experimental setting: Reproducibility depends on source code, README instructions, and documented experimental setup because the benchmark uses complex dynamic websites.
- Reproducibility and experimental setting: The benchmark contains only a test set and therefore does not assume that anyone will use it as a training set.
- Statistical significance: Most experiments were run three times and averaged, but the paper does not report error bars because they were considered similar across baseline models.