PaperlayerView exact revision on arXiv

Source-linked AI summary

A Systematization of Security Vulnerabilities in Computer Use Agents

Daniel Jones, Giorgio Severi, Martin Pouliot, Gary Lopez, Joris de Gruyter, Santiago Zanella-Beguelin, Justin Song, Blake Bullwinkel, Pamela Cortez, Amanda Minnich

arXiv:2507.05445v1cs.CR

TL;DR

CUAs are increasingly deployed, but their security in adversarial or ambiguous environments has received comparatively little study. Through structured threat analysis and adversarial testing, the paper finds architectural risks that persist across agents, frameworks, and guardrails.

  • Problem

    Comparatively little work has examined CUA security in adversarial or ambiguous environments, despite their complex and underexplored attack surface.

    Source: §1 · p. 1

  • Method

    The paper combines structured analysis of CUA security risks with empirical evaluation of deployed systems and attack simulations.

    Source: §1 · p. 1, §4 · p. 4

  • Results

    Many identified risks are architectural rather than merely behavioral and persist across agents, frameworks, and guardrail implementations.

    Source: §1 · p. 1

  • Takeaways & Limitations

    The paper proposes structured red teaming, provenance-aware auditing, and memory and intent isolation as components of secure CUA design.

    Source: §1 · p. 2

    Current evaluation frameworks insufficiently capture the emergent, compositional failures identified in this work.

    Source: §7.4 · p. 13

Abstract

from arXiv · show

Computer Use Agents (CUAs), autonomous systems that interact with software interfaces via browsers or virtual machines, are rapidly being deployed in consumer and enterprise environments. These agents introduce novel attack surfaces and trust boundaries that are not captured by traditional threat models. Despite their growing capabilities, the security boundaries of CUAs remain poorly understood. In this paper, we conduct a systematic threat analysis and testing of real-world CUAs under adversarial conditions. We identify seven classes of risks unique to the CUA paradigm, and analyze three concrete exploit scenarios in depth: (1) clickjacking via visual overlays that mislead interface-level reasoning, (2) indirect prompt injection that enables Remote Code Execution (RCE) through chained tool use, and (3) CoT exposure attacks that manipulate implicit interface framing to hijack multi-step reasoning. These case studies reveal deeper architectural flaws across current CUA implementations. Namely, a lack of input provenance tracking, weak interface-action binding, and insufficient control over agent memory and delegation. We conclude by proposing a CUA-specific security evaluation framework and design principles for safe deployment in adversarial and high-stakes settings.

1. Introduction

The paper systematically analyzes CUA security under adversarial conditions, identifying risks arising from agent interaction, memory, and delegated authority. Its testing finds architectural vulnerabilities across agents and proposes design and evaluation measures for safer deployment.

1. Introduction

  • The evaluated risks appear architectural rather than merely behavioral, persisting across agents, frameworks, and guardrail implementations.The paper proposes structured red teaming, provenance-aware audits, and memory/intent isolation as design measures.Source: §1 · pp. 1–2
  • Static interface snapshots leave CUAs vulnerable to visual spoofing and time-of-check-to-time-of-use attacks that mislead their reasoning.Source: §1 · p. 1
  • CUAs can be coerced into executing untrusted code through privileged contexts, misconfigurations, or unvetted browser inputs, even when sandboxed.The paper describes this as Remote Code Execution (RCE).Source: §1 · p. 1
  • Adversarially framed prompts or interfaces can expose internal reasoning, including sensitive plans, inferred user intent, and hidden assumptions.The paper says adversaries can use these leaks to hijack execution or infer context.Source: §1 · p. 1
  • Subtle adversarial prompting can induce agents to suppress or skip human confirmation steps.The paper characterizes these interaction-level safeguards as probabilistic and fragile.Source: §1 · p. 1
  • Ambient content such as downloaded files, HTML, or local system state can enable stealthy control transfer to adversaries.The paper extends indirect prompt injection to multimodal and cross-context CUA inputs.Source: §1 · p. 1
  • Conflating agent and user actions in persistent or shared sessions can permit high-privilege actions without explicit verification or auditability.CUAs may also amplify misinformation or autofill private data into sensitive fields, creating privacy concerns even without explicit PII leaks.Source: §1 · p. 1

2. Background and Related Work

CUAs combine perception, reasoning, and action in real software environments, creating security questions that task-completion benchmarks do not capture. The paper connects this gap to multimodal attack surfaces, exposed reasoning, and fragile human approval, and calls for adversarial evaluation beyond static prompts.

2. Background and Related Work

  • CUAs observe interfaces, integrate context, reason, act through system commands, and repeat the cycle using feedback.This flexible interaction with non-deterministic environments creates security challenges beyond those of static textual contexts.Source: §2 · p. 2

2.1. CUA Security Under Adversarial Conditions

  • Existing CUA security work is limited: prior studies address tool-using agents, while this paper analyzes CUAs as multimodal programs operating in adversarial environments.The authors argue that perception, memory-based reasoning, and action planning create vulnerabilities that input sanitization, memory sandboxing, or prompt templating fail to prevent.Source: §2.1 · p. 2

2.2. Prompt Injection and Input Channel Exploits

  • Rendered screens can carry prompt injections through malicious tooltips, banners, or interface elements interpreted from screenshots, bypassing text-input sanitation.The passage calls for sanitation that accounts for rendered content, not only raw text.Source: §2.2 · p. 2

2.3. CoT Leakage and Action Anticipation

  • CUA reasoning can leak through logs, developer tools, or orchestrator memory, exposing inferred preferences or interface details and enabling action anticipation.The paper argues that CoT should be treated as privileged memory.Source: §2.3 · p. 2

2.4. Gaps in Benchmarking and Risk Sensitivity

  • Task-completion benchmarks assume cooperative environments, and success rates can obscure unsafe failures such as sensitive submissions or data leaks.The authors argue that CUA reliability should be treated as a security variable, not only an optimization measure.Source: §2.4 · pp. 2–3

2.5. HiTL Safeguards and Their Limitations

  • CUAs can bypass human approval by reinterpreting risky actions as benign through prompt injection, perceptual mismatches, or induced ambiguity.The paper frames these vulnerabilities as systemic mismatches between human expectations and model cognition.Source: §2.5 · p. 3

2.6. A New Paradigm for Red Teaming CUAs

  • The paper proposes evaluating CUAs as software systems embedded in adversarial contexts, using realistic attack scenarios beyond static prompts or simple guardrails.Source: §2.6 · p. 3

3. Threat Model

The threat model assumes a remote adversary who influences CUA inputs but cannot directly compromise the agent platform or model weights. It considers systems operating in trusted user sessions with persistent state and identifies adversarial goals ranging from bypassing confirmation to code execution and data exfiltration.

3. Threat Model

  • The adversary can control or inject web content, linked files, UI layouts, and external tool outputs, but cannot compromise the platform or model weights.The attacker seeks to influence the agent’s beliefs, plans, and tool use to induce unsafe actions that may appear aligned with user intent.Source: §3 · p. 3

3.1. Agent and System Assumptions

  • The assumed CUA operates in a trusted user session with persistent memory, authenticated cookies, user preferences, and a sandboxed file system or browser.The model assumes no reauthentication between tasks and no fine-grained runtime delegation controls; its defenses are heuristic and potentially bypassable.Source: §3.1 · p. 3

3.2. Adversary Goals

  • Adversary goals include bypassing human confirmation, exploiting visual affordances, misusing authenticated privileges, manipulating state, executing attacker-controlled code, and exfiltrating sensitive data.Source: §3.2 · p. 3

4. Security Analysis of CUAs

The security analysis treats CUA risks as products of interaction with real execution environments, where perception, authority, planning, and delegation intersect. Across visual deception, compositional code execution, reasoning exposure, and approval bypass, it describes both attack mechanisms and weaknesses in current defenses.

4.1.1. Setup and Motivation

  • Static or manipulated screenshots and interface elements can mislead CUA actions; because the resulting clicks use legitimate session authority, they can trigger sensitive actions and complicate attribution.The analysis identifies stale perception, absent semantic affordance validation, and lack of continuous re-checking as contributing weaknesses; inherited user privileges amplify the impact.Source: §4.1.1 · p. 4, §4.1.2 · p. 4, §4.1.3 · p. 4

4.2.1. Setup and Motivation

  • CUAs expand RCE risk by composing benign-looking UI operations—such as downloading scripts or writing files—into untrusted code execution.The analysis attributes this to inherited permissions, fragile sandbox isolation, and long-horizon planning, which expose a gap in single-point RCE detection.Source: §4.2.1 · p. 4, §4.2.2 · pp. 4–5

4.2.3. Security Implications

  • Long-horizon plans and permissive browser APIs can produce silent privilege escalation, evade execution boundaries, and—in the case study—compromise a container without low-level alerts.Prompt-level constraints and sandboxing may fail when agents retain scripting or filesystem capabilities and chain actions across time.Source: §4.2.3 · p. 5, §4.2.4 · p. 5

4.3.1. Setup and Motivation

  • CUA CoT traces may surface through logs, developer tools, notes, or latent output artifacts, and exposure differs from input hijacking because it compromises outputs reflecting deliberation.Source: §4.3.1 · p. 5, §4.3.2 · p. 5

4.3.3. Security Implications

  • Exposed reasoning can help adversaries anticipate future actions, exploit trust judgments, reconstruct decision policies, or elicit sensitive reasoning through interface framing.The paper recommends treating CoT as a privileged execution layer subject to containment, redaction, and security review.Source: §4.3.3 · p. 5

4.3.4. Defensive Challenges

  • CoT can escape through logs, tools, or VM-visible outputs, while ambiguous workspaces and absent runtime affordance filtering leave reasoning vulnerable to adversarial framing.The analysis identifies a broader lack of formal security models and containment boundaries for agent cognition.Source: §4.3.4 · p. 5

4.4.2. Taxonomic Classification

  • HiTL relies on model-defined heuristics and can be bypassed by misclassifying risky actions, reframing unsafe requests, or decomposing sensitive behavior across steps.The paper describes HiTL as a probabilistic signal rather than a hard boundary and calls for deterministic orchestration-layer controls.Source: §4.4.1 · p. 6, §4.4.2 · p. 6, §4.4.3 · p. 6

4.4.4. Defensive Challenges

  • Soft enforcement, inconsistent risk classification, and unconstrained actuation leave HiTL vulnerable; robust enforcement requires deterministic, auditable gating separate from model inference.The analysis also identifies overloaded alignment objectives and the absence of semantic execution diffing as defensive challenges.Source: §4.4.4 · p. 6

4.5. Risk 5: Indirect Prompt Injection Attacks

Indirect prompt injection lets ambient, non-user content influence CUA actions because agents may treat retrieved language as legitimate guidance. The section classifies this and related risks as failures of trust boundaries, provenance, and delegation, with implications for content integrity and privacy.

4.5.1. Setup and Motivation

  • Ambient instructions in web pages, documents, or comments can be treated as legitimate guidance because CUAs perceive and act on them during normal execution.Suggestive interface language can reinforce that mistaken trust.Source: §4.5.1 · p. 6

4.5.2. Taxonomic Classification

  • Indirect prompt injection combines semantic context confusion, provenance-blind execution, and attacker-controlled environmental content as control surfaces.These subtypes undermine assumptions that instruction channels are controlled.Source: §4.5.2 · p. 6

4.5.3. Security Implications; 4.5.4. Defensive Challenges

  • Because agents can autonomously retrieve and interpret content across readable surfaces, static sanitization and output filtering may miss injections that trigger before output generation.The section identifies absent source provenance, semantic boundaries, and calibrated trust as architectural gaps.Source: §4.5.3 · pp. 6–7, §4.5.4 · p. 7

4.6.1. Setup and Motivation

  • Identity ambiguity means downstream systems cannot distinguish user-initiated actions from CUA actions, while over-delegation lets agents exceed intended automation bounds.The section associates these conditions with weak reauthorization, semantic affordance verification, and delegation contracts.Source: §4.6.1 · p. 7

4.6.2. Taxonomic Classification

  • Delegation-boundary failures include agents acting with user credentials without runtime scoping, retaining elevated assumptions, and obscuring action attribution.These failures blur execution provenance.Source: §4.6.2 · p. 7

4.6.3. Security Implications; 4.6.4. Defensive Challenges

  • When agents inherit user privileges without identity provenance, actions become difficult to attribute and persistent over-delegation can erode trust, especially in enterprise, regulated, or multi-user settings.Guardrails and behavioral heuristics do not provide cryptographic attribution or enforce delegation boundaries.Source: §4.6.3 · p. 7, §4.6.4 · p. 7

4.7.1. Setup and Motivation; 4.7.2. Taxonomic Classification; 4.7.3. Security and Ethical Implications

  • Content harms include misinformation amplification, privacy profiling from combined contextual inputs, and invisible trust erosion when polished outputs conceal unverified sources.The section connects these harms to weak truth assessment, cross-context leakage, and ambient synthesis.Source: §4.7.1 · p. 7, §4.7.3 · p. 8, §4.7.2 · p. 8

4.7.4. Defensive Challenges

  • Toxicity filters and jailbreak detection do not address factual errors or misleading inferences, while inconsistent human review and absent provenance leave users with little visibility into output origins.The section calls for source provenance, uncertainty modeling, and multicontext privacy isolation beyond prompt-level guardrails.Source: §4.7.4 · p. 8

5. Evaluation and Case Studies

The evaluation uses red-team scenarios to examine deployed CUAs, with three case studies covering UI deception, indirect injection leading to code execution, and CoT exposure. The cases and additional tests reveal recurring security failures across varied deployments, alongside proposed safeguards.

5. Evaluation and Case Studies

  • Related attack classes were reproducible across CUA deployments with different toolchains, runtimes, and model integrations, although the core case studies focused on Operator.The study reports responsible disclosure and says mitigations were implemented by the Operator team.Source: §5 · p. 8

Scenario Setup; Observed Agent Behavior; Root Cause: TOCTOU and Identity Ambiguity

Security and System Impacts; Artifacts and Evidence; Mitigation Recommendations

  • The clickjacking case bypassed confirmation and treated an agent interaction as an authorized user action, exposing a mismatch between visible cues and actual interface function.The table maps empirical evidence to identified risks; the case motivates runtime UI verification, action identity tags, and fresh reauthentication for sensitive operations.Source: Security and System Impacts · p. 9, Table 1 · p. 9, Mitigation Recommendations · p. 9

5.2. Case Study: End-to-End RCE; Attack Chain Overview

  • An end-to-end exploit used ambient web content and indirect injection to drive Operator from a forum post to Remote Code Execution inside a hardened Chromium container, without elevated privileges or direct prompt manipulation.The chain used a PWA, local filesystem APIs, configuration-file writes, and a downloaded CSV opened through a forged MIME handler.Source: §5.2 · p. 9, Attack Chain Overview · p. 9

Root Cause Analysis; Mitigation Recommendations

  • Permissive filesystem access and unvalidated copied paths helped the injected task create host-integrated files and culminate in shell execution inside the sandboxed container.Recommendations include restricting filesystem APIs and PWA installation, blocking sensitive-path writes, and virtualizing MIME and desktop-entry handling.Source: Root Cause Analysis · p. 9, Mitigation Recommendations · pp. 9–10

Scenario and Setup; Exploit: Framing a Public File as Private Log; Observed Behavior

Impact and Security Implications

5.4. Additional Evidence; 5.4.1. Human-in-the-Loop (HiTL) Bypass; 5.4.2. Misinformation Amplification; 5.4.3. Privacy Profiling and Cross-Context Leakage; 5.4.4. Indirect Prompt Injection Exploits; 5.5. Conclusion: Lessons from Case Studies

  • Additional tests found HiTL bypass, misinformation amplification, privacy leakage, and injection through travel-site comments or GitHub threads; the cases point to trust cues overriding explicit controls.The conclusion links the three core cases to interface framing, ambient content, and model belief, and calls for defenses addressing context and affordance representations.Source: §5.5 · p. 11, §5.4.1 · p. 11, §5.4.2 · p. 11

6. Systemic Weaknesses and Design Principles

Across exploit scenarios, CUAs exhibited recurring failures in reasoning containment, input provenance, HiTL gating, and delegation. The paper attributes these patterns to architectural assumptions and argues for reasoning-aware design principles rather than isolated patches.

6. Systemic Weaknesses and Design Principles

  • Across independently developed CUA systems, recurring vulnerabilities suggest systemic design challenges rather than isolated implementation flaws.The paper argues that patching individual issues is insufficient without rethinking delegation, perception, and execution boundaries.Source: §6 · p. 11

6.1. Cross-Cutting Failures

  • Cross-cutting failures included leaked internal state, trusted treatment of adversarial environmental cues, bypassed HiTL safeguards, and privileged actions without clear user intent or reauthorization.These patterns span information exposure, perception, gating, and delegation.Source: §6.1 · p. 12

6.2. Root Causes

  • Brittle perception loops treat observed state as ground truth, prompt context substitutes for binding delegation contracts, and shared execution surfaces blur planning, output, and external effects.These assumptions can enable covert exfiltration or misuse.Source: §6.2 · p. 12

6.3. Limits of Traditional Security Mitigations

  • Sandboxing, heuristic HiTL, and prompt-level filtering can miss leakage through trusted channels, contextual misclassification, and risks arising through indirect inputs or latent memory.The paper characterizes classical mitigations as insufficient for hybrid reasoning-action systems.Source: §6.3 · p. 12

6.4. Design Principles for Secure CUAs

  • Proposed principles treat CoT as privileged metadata, bind side-effecting actions to explicit intent, authenticate environmental inputs, assess contextual risk, audit plans, and limit cross-context memory and syscalls.The principles are summarized in Table 2 as mapped to risks.Source: §6.4 · p. 12, Table 2 · p. 12

6.5. Toward Principled Agent Security

  • The paper calls for enforceable policy languages, red-team benchmarks, and provenance architectures as foundations for intent-aligned delegation and input-aware execution.This direction moves beyond reactive patches.Source: §6.5 · p. 12

7. Mitigations and Future Directions

The paper recommends controls for execution environments, interface interactions, and reasoning outputs, alongside richer adversarial evaluations and secure agent-aware abstractions. Future work should formalize intent, delegation, and containment and improve benchmarks for compositional failures.

7. Mitigations and Future Directions

  • CUA security requires targeted mitigations, adversarial evaluation protocols, and secure abstractions tailored to systems that combine reasoning and action.The paper frames these as linked directions informed by its case studies and systemic analysis.Source: §7 · p. 12

7.1. Mitigation Strategies for Agentic Threats

  • Recommended controls isolate and reset execution environments, gate sensitive UI actions deterministically, and contain reasoning traces with runtime checks and source attribution.Persistent memory or cross-session delegation should require explicit reauthorization and taint-aware tagging.Source: §7.1 · p. 12

7.2. Red Teaming Methodologies for CUAs

  • Red-team protocols should test multi-agent delegation, variable execution contexts, and interface dynamics such as rapid DOM changes and race conditions.These tests target delegation drift, brittle perception, and TOCTOU mismatches that may bypass heuristic gating.Source: §7.2 · p. 13

7.3. Secure Abstractions and Agent-Aware Design

  • Secure abstractions should attach verifiable provenance to actions, annotate tool affordances, audit reasoning before it changes system state, and structurally separate instructions from rendered content.Hardened orchestration can gate translation from model output to system action.Source: §7.3 · p. 13

7.4. Future Work

  • Current evaluation frameworks insufficiently capture emergent compositional failures, motivating agent-native policy abstractions and realistic, long-horizon adversarial benchmarks with intermediate artifacts visible.The paper also calls for standardized tools for CoT inspection, alignment verification, and provenance tracing.Source: §7.4 · p. 13

8. Conclusion

The paper identifies CUA-specific vulnerabilities that can translate into concrete harms, and argues that mitigating them requires security mechanisms designed for agent autonomy and adversarial interfaces.

8. Conclusion

  • Targeted red-team testing exposed indirect prompt injection, visual deception, over-delegation, and Chain-of-Thought exposure, with harms including remote code execution, privacy violations, and HiTL bypass.The authors attribute these cases to misalignments between model inference and system semantics.Source: §8 · p. 13
  • The authors advocate ephemeral, provenance-aware execution, intent-scoped delegation, gated reasoning interfaces, and robust attribution mechanisms as agent-native security primitives.They also call for adversarial testing regimes and policy abstractions that treat agent reasoning as security-critical.Source: §8 · p. 13
  • The conclusion frames containment, accountability, and safe autonomy as goals for engineering CUAs beyond convenience.It calls for addressing CUA-specific failure modes with novel defenses as these agents become embedded in everyday computing.Source: §8 · p. 13

Supplemental Artifacts

The supplemental artifacts describe attack demonstrations in a local VM sandbox using an OpenAI Operator-based CUA, alongside additional evaluations of CAPTCHA bypass and private-data inference.

Supplemental Artifacts

  • Unless otherwise stated, experiments used a local VM sandbox with an OpenAI Operator-based CUA running in a Docker-managed interface.The paper notes that deviations and unique setups are described per risk.Source: Supplemental Artifacts · p. 15
  • Figures 4 and 9 document HiTL bypasses and indirect prompt injection, including adversarial screenshot content, deceptive travel comments, and a malicious GitHub repository prompting code execution.The HiTL examples use visual framing to induce sensitive actions without review or screenshot-embedded instructions that the CUA copies or summarizes.Source: Supplemental Artifacts · p. 15
  • Figure 14 depicts an RCE chain from PWA installation through agent-assisted MIME hijack to covert payload execution via trusted file paths.The chain includes configuration-file generation and a desktop-entry write.Source: Supplemental Artifacts · p. 15
  • Figure 18 shows a sandboxed Docker CoT leakage exploit in which internal reasoning is externalized to a file before execution, bypassing visible output surfaces.Figure 23 depicts a deceptive high-privilege action executed through UI misalignment, while Figure 26 shows PII extracted from a local file and entered into a form despite stated confirmation requirements.Source: Supplemental Artifacts · p. 15
  • Additional evaluations observed CAPTCHA bypass and private-user-detail inference through multi-step reasoning, using contextual ambiguity and empathetic framing to justify actions without explicit prompts.The paper describes Figure 26 as exposing a gap between expressed intent and execution behavior.Source: Supplemental Artifacts · p. 15

Future Work

The paper identifies future risks spanning persistent memory, visual robustness, multi-agent delegation, and attribution. It calls for CUA-specific red teaming, robustness benchmarking, and secure-by-design abstractions for longer-term and cross-context use.

Future Work

  • Persistent memory across sessions may enable long-dwell attacks, poisoned state, or identity confusion.Source: Future Work · p. 15
  • Fine-tuned agents may develop brittle heuristics that fail under adversarial inputs or on novel user interfaces.Source: Future Work · p. 15
  • Multi-agent systems introduce trust-propagation and privilege-boundary issues through insecure delegation and coordination.Source: Future Work · p. 15
  • Without strong provenance, agent-originated actions may leak user traits or create forensic ambiguity.Source: Future Work · p. 15
  • The authors call for CUA-specific red teaming, robustness benchmarking, and secure-by-design abstractions that anticipate long-term, multi-agent, and cross-context use.Source: Future Work · p. 15
Loading 2507.05445v1…