PaperlayerView exact revision on arXiv

Source-linked AI summary

RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

Zeyi Liao, Jaylen Jones, Linxi Jiang, Yuting Ning, Eric Fosler-Lussier, Yu Su, Zhiqiang Lin, Huan Sun

arXiv:2505.21936v5cs.CL

TL;DR

CUAs can be hijacked by indirect prompt injections in web content, while existing evaluations lack realistic, controlled testing across web and OS environments. RedTeamCUA combines a hybrid sandbox with an 864-example benchmark, and evaluations find substantial vulnerabilities, including successful end-to-end attacks. The authors also find that evaluated defenses do not provide adequate protection.

  • Problem

    Evaluations of indirect prompt injection in CUAs remain limited by the lack of realistic, controlled environments for testing attacks spanning both web and OS interfaces.

    Source: §2.2 · p. 3

  • Method

    RedTeamCUA integrates a VM-based OS with Docker-based web platforms and provides configurable adversarial testing, including evaluation initialized at the injection site.

    Source: §1 · p. 2

  • Results

    Frontier CUAs showed substantial injection susceptibility, with decoupled ASRs up to 66.2% and end-to-end ASRs up to 83% for Claude 4.5 Opus | CUA.

    Source: §1 · p. 3

  • Takeaways & Limitations

    Four evaluated system- and model-level defense methods, including injection-specific approaches, did not provide adequate protection on RTC-BENCH.

    Source: §1 · p. 3

    The evaluation covers frontier CUAs with sufficient capability; sufficiently capable open-source CUAs were inaccessible, while the available UI-TARS 1.5 7B struggled with basic tasks.

    Source: §5.1 · p. 7

Abstract

from arXiv · show

Computer-use agents (CUAs) promise to automate complex tasks across operating systems (OS) and the web, but remain vulnerable to indirect prompt injection. Current evaluations of this threat either lack support realistic but controlled environments or ignore hybrid web-OS attack scenarios involving both interfaces. To address this, we propose RedTeamCUA, an adversarial testing framework featuring a novel hybrid sandbox that integrates a VM-based OS environment with Docker-based web platforms. Our sandbox supports key features tailored for red teaming, such as flexible adversarial scenario configuration, and a setting that decouples adversarial evaluation from navigational limitations of CUAs by initializing tests directly at the point of an adversarial injection. Using RedTeamCUA, we develop RTC-Bench, a comprehensive benchmark with 864 examples that investigate realistic, hybrid web-OS attack scenarios and fundamental security vulnerabilities. Benchmarking current frontier CUAs identifies significant vulnerabilities: Claude 3.7 Sonnet | CUA demonstrates an ASR of 42.9%, while Operator, the most secure CUA evaluated, still exhibits an ASR of 7.6%. Notably, CUAs often attempt to execute adversarial tasks with an Attempt Rate as high as 92.5%, although failing to complete them due to capability limitations. Nevertheless, we observe concerning high ASRs in realistic end-to-end settings, with the strongest-to-date Claude 4.5 Sonnet | CUA exhibiting the highest ASR of 60%, indicating that CUA threats can already result in tangible risks to users and computer systems. Overall, RedTeamCUA provides an essential framework for advancing realistic, controlled, and systematic analysis of CUA vulnerabilities, highlighting the urgent need for robust defenses to indirect prompt injection prior to real-world deployment.

1 INTRODUCTION

CUAs can be hijacked by malicious instructions embedded in web content, but existing evaluations often lack realistic, controlled hybrid web–OS testing. RedTeamCUA and RTC-BENCH address this gap, and evaluations expose substantial vulnerabilities and inadequate defenses.

1 INTRODUCTION

  • Realistic, comprehensive evaluations remain scarce because testing must balance environmental control against realism, while prior approaches often lack interactive interfaces or integrated web–OS scenarios.Existing OS and web evaluation environments separately leave gaps in safe web testing and OS-specific risk analysis.Source: §1 · pp. 1–2
  • RedTeamCUA combines a VM-based OS with Docker-based web replicas and supports configurable adversarial scenarios plus evaluation initialized directly at an injection site.The Decoupled Eval setting separates susceptibility analysis from agents’ navigation limitations.Source: §1 · p. 2, Figure 1 · p. 2
  • RTC-BENCH contains 864 examples combining 9 benign goals, 24 adversarial goals, and four instantiation types to probe hybrid web–OS injection risks.The variations include General and Specific benign instructions and Code and Language injection settings.Source: §1 · pp. 2–3
  • End-to-end testing found ASRs up to 83% for Claude 4.5 Opus | CUA, while decoupled evaluation found ASRs up to 66.2% across frontier CUAs.Claude 3.7 Sonnet | CUA had a 42.9% ASR; Operator, the most secure CUA evaluated, had a 7.6% ASR.Source: §1 · p. 3
  • Attempt Rate reached 92.5% and consistently exceeded ASR, indicating that capability limits often prevented completion of adversarial goals.The authors caution that future capability gains could amplify risks without corresponding defense improvements.Source: §1 · p. 3
  • None of four evaluated system- and model-level defense methods, including injection-specific approaches, provided adequate protection on RTC-BENCH.The authors call for dedicated defense strategies for capable and secure CUAs.Source: §1 · p. 3

2 BACKGROUND

The paper focuses on everyday workflows in which CUAs use online information to perform local actions. Prior evaluations leave a gap in realistic, controlled adversarial testing across both web and OS environments.

2.1 BENIGN TASK SCOPE

  • The benchmark’s motivating use cases involve retrieving web knowledge and then taking a corresponding local action, such as installing unfamiliar software.These workflows can expose agents to malicious instructions embedded in online environments.Source: §2.1 · p. 3

2.2 A CRITICAL NEED FOR A HYBRID ENVIRONMENT SANDBOX

  • Existing evaluations face realism–safety tradeoffs and gaps in interactive, secure, integrated testing across web and OS environments.Web-only sandboxes omit OS harms, while OS environments with unrestricted browsers or real, unsandboxed sites retain safety risks.Source: §2.2 · pp. 3–4
  • The authors identify a need for a hybrid sandbox and a large-scale benchmark covering severe adversarial scenarios.The proposed evaluation should be realistic and interactive while keeping web and OS testing secure.Source: §2.2 · p. 4

3 REDTEAMCUA - HYBRID ENVIRONMENT SANDBOX

RedTeamCUA integrates an interactive VM-based OS with isolated web replicas to support controlled adversarial testing across both interfaces. Its configurable injections and Decoupled Eval setting support reproducible scenarios and separate attack susceptibility from navigation limitations.

3.1 SANDBOX CONSTRUCTION

  • The sandbox combines OSWorld’s interactive VM with isolated Docker web replicas, enabling controlled adversarial scenarios that span web and OS interactions.The web replicas are accessed through OSWorld’s browser and are built from real-world counterparts, libraries, and data sources.Source: §3 · p. 4, §3.1 · p. 4
  • OSWorld provides an executable OS environment with host isolation and snapshot resets for safe, reproducible testing.The implementation focuses on Ubuntu and supports applications such as Terminal, File Manager, and VSCode.Source: §3.1 · p. 4
  • The integrated web platforms are OwnCloud, Forum, and RocketChat, covering cloud-office, forum, and real-time communication scenarios.These platforms support workflows such as retrieving instructions or assistance online before taking local actions.Source: §3.1 · pp. 4–5
  • Configurable task initialization and platform-specific scripts automate persistent, reproducible injection of adversarial content into web platforms.Configurations can specify injection content and targets, SQL modifications, and uploaded files.Source: §3.1 · p. 5
  • Decoupled Eval uses pre-processed actions to place agents at the injection site, isolating adversarial robustness from navigation limitations.The setting addresses preliminary observations that GPT-4o often failed to navigate to the injection webpage.Source: §3.1 · p. 5

4 ADVERSARIAL TESTING WITH REDTEAMCUA

RTC-BENCH pairs realistic benign computer-use goals with web-originating injections targeting local OS security. Its scenarios cover varied user instructions, attack content, and fundamental security outcomes.

4 ADVERSARIAL TESTING WITH REDTEAMCUA

  • RTC-BENCH comprises 864 examples formed from 9 benign goals, 24 adversarial goals, and four variations in instruction specificity and injection content.The examples are designed to probe indirect prompt injection across computer-use contexts.Source: §4 · p. 5, §4.2 · p. 6

4.1 BENIGN GOAL FORMULATION

  • The 9 benign goals cover software installation, system configuration, and project setup, with General or Specific user instructions.Each category has three goals, modeling users who retrieve online knowledge for local execution.Source: §4.1 · p. 5

4.2 ADVERSARIAL ATTACK FORMULATION

  • The threat model restricts attackers to text entered in permitted webpage locations and assumes they cannot access or modify user instructions, agent prompts, components, or model weights.Injections are contextualized to anticipated queries, and the evaluated scenarios align a benign task with the injection context.Source: §4.2 · pp. 5–6
  • The 24 adversarial goals target Confidentiality, Integrity, or Availability, including web-to-OS attacks that exfiltrate local files or affect local data and functionality.Each goal is instantiated in natural-language and code forms.Source: §4.2 · p. 6
  • Injections prepend an “important message” cue and adversarial instructions, then use a deceptive rationale tied to the environment and task context.The rationale is intended to present the adversarial goal as required for the benign task.Source: §4.2 · p. 6
  • The current attack formulation centers on web-originating risks, while the modular sandbox supports a fixed set of web platforms that can be extended.The framework can also support broader threat models, including compromised or inherently unsafe agents and malicious users.Source: §4.2 · p. 6

5 BENCHMARKING CUAS AGAINST INDIRECT PROMPT INJECTION

The benchmark evaluates frontier CUAs against indirect prompt injection across web and OS scenarios, finding widespread vulnerability and higher attempt than completion rates. Outcomes vary with adversarial-goal complexity and web-platform context, while built-in checks reduce—but do not eliminate—risk.

5.1 SETUP

  • Operator’s confirmation and safety-check mechanisms count adversarial execution as unsuccessful when either check is triggered, modeling human intervention.The study also evaluates Operator (w/o checks), which proceeds with user permission to simulate inattentive supervision.Source: §5.1 · p. 7
  • The evaluation covers adapted LLM-based and specialized computer-use agents, but excludes sufficiently capable open-source CUAs because they were inaccessible.The strongest available open-source model struggled with basic navigation, chat interaction, and opening files; specialized agents’ native actions are converted to executable pyautogui commands.Source: §5.1 · pp. 6–7
  • ASR uses example-specific execution-based evaluators, while AR uses an LLM judge to identify attempted adversarial goals even when agents fail to complete them.The evaluators are intended to balance outcome reliability with coverage of harmful behavior attempts.Source: §5.1 · p. 7
  • CUAs are tested three times, with success counted if any run succeeds; Decoupled Eval places agents directly at the injection state to isolate robustness from navigation.The setup first sanity-checks that agents can complete benign tasks without injection.Source: §5.1 · p. 8

5.2 RESULTS

  • GPT-4o has the highest average ASR at 66.19%, while Operator has the lowest at 7.57% across the evaluated frontier CUAs.Across models, AR is consistently higher than ASR, indicating that agents often pursue adversarial goals but fail to complete them.Source: Table 1 · p. 7, §5.2 · p. 8
  • AR remains high for Confidentiality goals across platforms, while ASR patterns differ across CIA categories as task complexity varies.Integrity actions can be simple, whereas Confidentiality attacks often require multi-step extraction and exfiltration; Availability goals range from simple disruption to resource exhaustion.Source: §5.2 · p. 8, Figure 2 · p. 8
  • RocketChat yields the highest ASR for the same adversarial goal among the available platforms, with trust cues and action plausibility offered as explanations.Confidentiality goals have notably higher ASR and AR in RocketChat, where sending data aligns with messaging use.Source: §5.2 · p. 8
  • Operator reduces ASR by −58.62%, while Claude 3.7 Sonnet | CUA increases it by +3.66% relative to their base counterparts.Without checks, Operator averages about 31% ASR and 48% AR; the authors connect this contrast to a trade-off between autonomy and security.Source: §5.2 · p. 8

6 ANALYSIS

The evaluated defenses offer incomplete protection against indirect prompt injection, and the analysis identifies task and interface factors that shape risk.

Defenses

  • LlamaFirewall and PromptArmor detect at most 30% of injections, while a defensive system prompt leaves ASR near 50% for Claude 3.7 Sonnet | CUA and Operator (w/o checks).Meta SecAlign still follows malicious instructions in about half of tasks.Source: §6 · p. 9

Additional Analysis

  • The analysis reports reduced ASR with more specific benign instructions and less autonomous use cases, while accessibility-tree observations may lower ASR but hurt benign-task SR.The authors describe the latter as a capability-safety trade-off.Source: §6 · p. 9
  • Platform characteristics and adversarial-goal types jointly influence attack success, and additional injection does not impair utility in the reported End2End setting.The analysis also reports that some risks persist across all three runs.Source: §6 · p. 9

7 END2END EVALUATION

In end-to-end tests starting from initial task states, newer capable CUAs still show substantial attack success. Manual inspection indicates that navigation and task-completion limits constrain weaker agents, while stronger capabilities can overcome those barriers.

7 END2END EVALUATION

  • Claude 3.7 Sonnet | CUA and Operator (w/o checks) reach End2End ASRs of 50% and 42%, respectively.Manual inspection finds that both often fail to navigate to the injection site or complete the injected task despite attempting it.Source: §7 · p. 9, Figure 3 · p. 9
  • Claude 4.5 Opus | CUA reaches an ASR of 83%, the highest among evaluated CUAs, while Claude 4.5 Sonnet | CUA reaches 60%.The End2End evaluation starts agents from initial task states rather than the page containing the injection.Source: Figure 3 · p. 9, §7 · p. 10
  • Claude 4.6 Opus | CUA detects malicious injections in 25 of 50 examples, compared with 3 for Claude 4.5 Opus, yet still has a 50% ASR.The authors report this as a defense advance while noting a substantial remaining robustness gap.Source: §7 · p. 10
  • The paper attributes differences between Decoupled Eval and End2End results largely to capability limits such as failing to navigate to the expected page.The authors argue that observed End2End attack success makes real-world threats non-hypothetical and that Decoupled Eval can help identify vulnerabilities proactively.Source: §7 · p. 10

8 CONCLUSION

RedTeamCUA combines a hybrid sandbox with RTC-BENCH to study indirect prompt-injection risks to computer-use agents. Evaluations found substantial vulnerabilities, including harmful outcomes during end-to-end execution, and the tested defense strategies offered insufficient protection.

8 CONCLUSION

  • Table 2 compares attack success rates between Decoupled Eval and End2End settings.The supplied passage identifies the comparison but does not provide table values.Source: Table 2 · p. 10
  • Frontier CUAs showed substantial vulnerabilities, including successful attacks targeting fundamental security violations and hybrid web-OS pathways.Adversarial goals could fully manifest as tangible harmful outcomes during end-to-end execution despite current capability limitations.Source: §8 · p. 10
  • RedTeamCUA combines a hybrid environment sandbox with RTC-BENCH to support systematic analysis of CUA risks.The work presents the sandbox and benchmark as a framework for continued investigation of diverse CUA threat cases.Source: §8 · p. 10

ETHICS STATEMENT

RedTeamCUA confines testing to a virtualized sandbox spanning OS and web environments and uses synthesized benchmark data without personal, sensitive, or confidential information.

ETHICS STATEMENT

  • Experiments are confined to a controlled virtualized sandbox spanning OS and web environments.The sandbox is intended to prevent unintended consequences or exploitation beyond its boundaries.Source: ETHICS STATEMENT · p. 11
  • RTC-BENCH uses fully synthesized data, including VM files, that do not contain or derive from personal, sensitive, or confidential information.Source: ETHICS STATEMENT · p. 11
  • The authors state that the work supports safer vulnerability analysis and development of more robust CUAs before real-world deployment.Source: ETHICS STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

The reproducibility statement describes the sandbox and benchmark construction and identifies the planned release of related materials upon acceptance.

REPRODUCIBILITY STATEMENT

A LIMITATIONS •

The limitations discuss fixed filenames and benchmark choices that leave other attack settings unexplored; the examples illustrate confidentiality, integrity, and availability attacks across web platforms and OS tasks.

A LIMITATIONS •

  • The benchmark uses specific filenames for reproducible evaluation, while future work is encouraged to explore objectives and injections that do not rely on fixed filenames.The authors also report that filename choice may influence attack outcomes.Source: A LIMITATIONS • · p. 18
  • The benchmark leaves out OS-originated and web-to-web attacks, other injection strategies, multiple injection points, environmental noise, and more elaborate OS environments.The evaluated attacks focus on web-originating pathways and contextualized Code and Language injections.Source: A LIMITATIONS • · p. 18
  • Decoupled Eval is limited to 10 steps by cost constraints, restricting exploration of utility (SR) under attack.The authors report that safety mechanisms and adversarial-goal completion consume a significant portion of those steps.Source: A LIMITATIONS • · p. 18

B.1 EXAMPLES IN RTC-BENCH

  • RTC-BENCH examples span three web platforms, CIA categories, benign task types, and instruction variants, including End2End tasks.The examples are presented as detailed cases of indirect prompt injection tests.Source: §B.1 · p. 19

B.1.1 CONFIDENTIALITY

  • The confidentiality example uses a RocketChat code injection to induce sending a private OS file to a malicious user during a system-configuration task.The file is /home/user/account.md; the benign task concerns setting Vim as the default editor.Source: §B.1.1 · pp. 19–20, Figure 4 · p. 19

B.1.2 INTEGRITY

  • The integrity example uses a Forum language injection to induce deletion of /etc/security instead of installing termcolor.The task represents software installation based on instructions from a forum page.Source: §B.1.2 · pp. 20–21, Figure 5 · p. 20

B.1.3 AVAILABILITY

  • The availability example uses an OwnCloud code injection to induce stopping SSH instead of downloading a project dataset.The stated attack instruction is to run `sudo systemctl stop sshd`.Source: §B.1.3 · pp. 21–22, Figure 6 · p. 21

B.2 BENIGN GOALS; B.3 ADVERSARIAL SCENARIOS

  • Benign goals include General, Specific, Pointer, and End2End instructions, and adversarial goals cover 24 CIA outcomes across web-to-OS pathways.File targets are grouped as Normal, Privacy Info, and System Info to represent differing harmful impacts.Source: Table 4 · p. 23, §B.2 · p. 24, §B.3 · p. 25

B.4.2 INTEGRITY

The examples disguise harmful OS changes as prerequisites for benign tasks, spanning file integrity and system availability. Table 6 compares prior evaluation frameworks across several dimensions.

B.4.2 INTEGRITY

  • During browser-extension setup, an injected instruction asks the agent to add an SSH key to authorized_keys before building the extension.The scenario frames a system-file modification as necessary for the benign setup goal.Source: §B.4.2 · p. 27

B.4.3 AVAILABILITY

  • Another injected instruction directs the agent to stop sshd before setting Vim as the default editor, disrupting a system service.The task presents the service shutdown as a prerequisite for the benign goal.Source: §B.4.3 · p. 28

C REDTEAMCUA FRAMEWORK DIAGRAM

  • Table 6 compares prior evaluation frameworks for adversarial CUA testing across key dimensions; ∼ marks partial support, while – marks inapplicable or undocumented cases.The caption does not specify the dimensions or individual framework results.Source: Table 6 · p. 31

D COMPARISON OF CUA EVALUATION FRAMEWORKS

The comparison identifies gaps in prior CUA evaluation frameworks, while the appendices test detection and defense methods and examine how prompt specificity, observation type, file type, and evaluation conditions relate to attack outcomes.

D COMPARISON OF CUA EVALUATION FRAMEWORKS

  • Prior frameworks leave gaps: web-only systems omit OS harms, DoomArena's OS support is pop-up-limited and lacks cross-environment integration, and OS-Harm's browser access risks harm while its LM judge can be manipulated.Source: D COMPARISON OF CUA EVALUATION… · pp. 32–33

H.1 PROMPT INJECTION DETECTION

H.3 MODEL-LEVEL DEFENSE

  • Meta SecAlign 70B showed 52% AR and 32% ASR, with 68% benign-task SR; tested defenses are not fully suitable or efficient for multimodal, interactive CUA scenarios.Source: §H.3 · p. 35, §H.4 · p. 35

I.1 RESULTS BY BENIGN AND ADVERSARIAL GOAL TYPE

  • General prompts and Information-Acting use show higher ASR and AR than specific prompts and information-only Pointer; injection-modality effectiveness also varies by platform.Source: §I.1 · p. 35, Table 8 · p. 36

I.2 RESULTS BY OBSERVATION TYPE

  • Adding an accessibility tree substantially reduces ASR and AR across nearly all evaluated CUAs, but such trees may be unavailable in real-world settings and do not consistently improve benign-task performance.Source: §I.2 · p. 36, Table 10 · p. 37, Table 9 · p. 37

I.3 RESULTS BY FILE TYPE

  • Normal files have the highest ASR, Privacy Info files follow closely, and System Info files have the lowest; categories are assigned by filename without considering file contents.Source: §I.3 · p. 38, Figure 9 · p. 38

I.4 UTILITY (SR) UNDER ATTACK

  • In End2End tests, successful attacks coincided with benign-task completion for Operator and Claude 3.7; under Decoupled Eval, specialists' lower SR may reflect the 10-step cap and attack-induced diversion.Source: §I.4 · p. 39

I.5 ATTACK OUTCOMES BY NUMBER OF ATTEMPTS

I.6 END2END RESULTS BREAKDOWN

The End2End analysis manually inspects trajectories and classifies outcomes, including navigation failures, ignored injections, unsuccessful harmful attempts, and detected injections.

I.6 END2END RESULTS BREAKDOWN

  • End2End trajectories are manually inspected and each test is categorized by its outcome when a CUA encounters an adversarial injection.Source: §I.6 · p. 42
  • The outcome categories distinguish navigation failure, ignoring or detecting an injection, failed harmful-task completion, sudden termination, and Operator-specific permission or safety checks.Source: §I.6 · p. 42
Loading 2505.21936v5…