PaperlayerView exact revision on arXiv

Source-linked AI summary

CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

Hanna Foerster, Tom Blanchard, Kristina Nikolić, Ilia Shumailov, Cheng Zhang, Robert Mullins, Nicolas Papernot, Florian Tramèr, Yiren Zhao

arXiv:2601.09923v3cs.AI

TL;DR

CUAs need untrusted screen observations to act, but exposing those observations to a planner enables prompt injection and conflicts with architectural isolation. The paper adapts Dual-LLM isolation through NOVA’s single-shot branching plans and quarantined perception, retaining utility on OSWorld while identifying Branch Steering as a residual vulnerability.

  • Problem

    CUAs depend on continuous, untrusted UI observations, creating a tension between the visual feedback needed for action and the planner isolation needed to prevent prompt injection.

  • Method

    CaMeL-NOVA uses a trusted planner to emit a complete branching plan upfront, while quarantined perception resolves runtime UI values and verification outcomes within that plan.

  • Results

    The system improves performance by up to 19% for small open-source models and retains up to 57% of larger closed-source model performance on OSWorld.

  • Takeaways & Limitations

    Single-shot planning provides control flow integrity against arbitrary instruction injection while supporting dynamic CUA workflows.

  • Takeaways & Limitations

    Dual-LLM isolation remains vulnerable to Branch Steering, and redundancy defenses leave residual vulnerabilities because attackers can manipulate perception-driven data flows.

Abstract

from arXiv · show

AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guarantees by strictly separating trusted task planning from untrusted environment observations. However, applying this design to Computer Use Agents (CUAs), which automate tasks by viewing screens and executing actions, presents a fundamental challenge. Current agents require continuous observation of UI state to determine each action, which conflicts with the isolation required for security. We resolve this tension by demonstrating that UI workflows, while dynamic, are structurally predictable. Single-shot planning, where a trusted planner emits upfront a complete branching plan covering all anticipated runtime states, provides control flow integrity guarantees against arbitrary instruction injections. We introduce NOVA (Navigating via Observation, Verification, and Action) to make this viable in the combinatorially large UI state space, where the plan can invoke a perception model to resolve runtime values such as UI coordinates. We evaluate our design on OSWorld, and retain up to 57% of the performance of frontier models while improving performance for smaller open-source models by up to 19%, demonstrating that rigorous security and utility can coexist in CUAs. Although upfront planning prevents instruction injections, we show that additional measures are needed to defend against \textbf{Branch Steering} attacks, where adversaries deceive the perception model into routing execution down attacker-preferred branches of the plan, such as redirecting the agent to a malicious website.

1 Introduction

CUAs expose a security challenge because dynamic screen interaction requires untrusted observations, while isolated planning requires the planner to remain blind to the environment. CaMeL-NOVA addresses this tension with complete branching plans and quarantined perception, preserving utility while exposing Branch Steering as a residual threat.

  • Motivation: CUAs use ambiguous, context-dependent actions whose meaning depends on UI state, creating attack surfaces for injected instructions and adversarial visual content.Unlike typed APIs, actions such as click(x, y) can trigger materially different outcomes depending on the target coordinates.
  • Motivation: Dual-LLM isolation separates a trusted planner from quarantined perception, preventing the planner from observing untrusted environment content.The planner emits a complete plan upfront, while the perception model processes observations within that plan.
  • CaMeL-NOVA: NOVA uses verify_hypothesis to let quarantined observations select among planner-written branches without exposing observations to the planner.The methodology supports runtime state handling while maintaining control flow integrity.
  • Plan-Complexity Gap: CUA plans require an order of magnitude more tool calls and branches than AgentDojo plans, with roughly half of plan nodes serving as untaken fallback paths.This structural complexity motivates specialized planning for CUAs rather than directly applying typed-API agent designs.
  • Results and Residual Threats: Branch Steering manipulates visual cues to route execution through dangerous but valid paths already present in the plan.This differs from arbitrary command injection, which the architecture prevents by restricting attackers to pre-written paths.

2 Background

Prior CUA attacks exploit injected instructions or manipulated pixels, while existing defenses rely mainly on fragile recognition, monitoring, or sandboxing. Dual-LLM systems provide strong control-flow guarantees, but their protection does not extend to data flows in data-dependent CUA workflows.

  • CUA Attacks: CUA attacks use malicious instructions or optimized pixels in interfaces to obtain arbitrary control over task execution.Reported attack vectors include adversarial pop-ups, malicious image patches, fine-print injections, and redirection through trusted platforms.
  • CUA Defenses: Existing CUA defenses rely on pattern recognition, action monitoring, or HTTP-level sandboxing and are vulnerable, brittle, or difficult to adapt to broad CUA action spaces.These approaches include few-shot attack recognition, rule-based auditing, and browser-oriented policies.
  • Dual-LLM Security: Dual-LLM architectures separate trusted planning from untrusted perception and enforce which actions may execute, but face limitations in data-dependent workflows.CaMeL plans upfront, whereas Fides generates actions iteratively while redacting tool outputs from the planner.
  • Security Properties: Control flow integrity fixes function-call structure, ordering, and conditional logic, whereas data-flow security protects call arguments and branch-driving values.Dual-LLM guarantees the former because the planner never observes the environment, but not the latter.
  • Branch Steering: Branch Steering manipulates environmental observations to trigger a valid but malicious conditional branch in a pre-approved plan.The attack targets environment-querying functions such as verify_hypothesis and find.

3 Methodology

CaMeL-NOVA adapts Dual-LLM isolation to CUAs by compiling complete plans whose quarantined perception calls supply runtime values and branch decisions. Its Observe-Verify-Act workflow supports dynamic UI states, while redundancy-based checks address—but do not eliminate—data-flow attacks.

  • Threat Model: The threat model restricts attackers from modifying the P-LLM while allowing malicious content injection into rendered browser environments.The evaluation focuses on browser-based tasks under an attacker who knows the task and toolset and can approximately predict plans.
  • Dual-LLM Architecture: CaMeL-NOVA separates a privileged planner that emits a complete plan from quarantined perception that interacts with screenshots or DOM content without modifying plan structure.The setup excludes additional use-case-specific data-flow policies, serving as a worst-case Dual-LLM baseline.
  • Plan Execution: The planner compiles user queries into Python-like plans whose conditional tools summarize UI state, return coordinates, and check environmental conditions.The interpreter invokes these tools during execution and binds outputs to variables consumed by later actions and branches.
  • Observe-Verify-Act: Observe gathers visual or DOM state, Verify checks a predicted condition with verify_hypothesis, and Act performs clicks or typing only after verification.This ordering lets plans anticipate failure modes and state transitions without exposing runtime observations to the planner.
  • Planning Methodology: Shared routines such as cookie handling and failed-click recovery are compiled into the system prompt, leaving the planner to reason about task-specific workflow components.This engineering separation improves utility without weakening control flow integrity.
  • Residual Data-Flow Attack Surface: Branch Steering remains possible because false coordinates or state summaries can drive execution into attacker-chosen branches within the fixed plan.The proposed defenses cross-check DOM and screenshot information or seek independent-model consensus, but remain probabilistic.

4 Evaluation

The evaluation measures plan structure, utility, planner effects, cross-substrate cost, and defenses against Branch Steering on OSWorld. NOVA substantially improves utility over the unoptimized baseline, while redundancy defenses remain vulnerable to attacks.

  • Evaluation setup: OSWorld evaluates plan structure, utility retention, planner quality, and redundancy defenses against Branch Steering across realistic computer-use tasks.The benchmark covers applications including Chrome, LibreOffice, and GIMP, using pass@k to measure whether at least one of k attempts succeeds.
  • Plan structure: CUA plans require substantially more calls, code, and branches than AgentDojo plans because runtime UI states need explicit fallback paths.AgentDojo averages 4.9 tool calls and 51.8 code lines, versus 19.8 calls and 71.6 lines for unoptimized CUA plans and 41.1 calls and 213.3 lines for NOVA.
  • Utility: 40-point gap: UITars reaches 58.3% versus 18.3% pass@3 with the unoptimized baseline, showing NOVA’s Observe-Verify-Act planning is essential.NOVA’s branching provides recovery paths for unexpected UI states and closes a 40% gap at pass@3.
  • Planner effects: 65.0%, 66.7%, and 68.3% pass@5: UITars, OpenCUA, and Claude perform within 3.3 points under CaMeL-NOVA, making planner quality the dominant utility factor.Across nine frontier planners, GPT-5 succeeds on 12/17 tasks and Grok-4 on 10/17.
  • Cost: 33.3% pass@1 and 66.7% pass@5: Fides-NOVA reaches comparable utility, but adds 29.6× tokens versus CaMeL-NOVA’s 1.88× overhead.Adding Multi-Modal Consensus to CaMeL-NOVA raises total overhead to 6.57×.
  • Security evaluation: Both cookie-popup and pixel attacks succeed against redundancy defenses, including the strongest Multi-Modal Consensus configuration.DOM Consistency blocks static Google banners but fails on HTML5 banners, while pixel attacks evade screenshot-based and DOM-enhanced verification.

5 Discussion

The discussion argues that isolated single-shot planning is viable for CUAs and provides meaningful control-flow security, oversight, and privacy benefits. It also identifies data dependency, benchmark quality, attack residuals, and inference cost as important boundaries.

  • Discussion: Strict system-centric security is compatible with GUI automation, establishing a baseline that exposes trade-offs among plan rigidity, data dependency, and residual attacks.The paper argues that CUA tasks are often less data-dependent than they appear.
  • Security implications: Control Flow Integrity eliminates arbitrary-instruction injection by making actions absent from the planner’s written plan structurally impossible.Branch Steering remains possible because Q-VLM outputs can act as control signals for selecting among actions already present in the plan.
  • Security implications: Explicit plans make policy protections, pre-execution inspection, user approval, and composition with browser-layer sandboxing tractable.Policy-level protections still require translating coordinate-level actions into semantic ones.
  • Deployment: The Dual-LLM split supports privacy-preserving deployment by keeping sensitive environment content away from proprietary planning models while enabling local perception.The split also reduces cost while preserving user privacy.
  • Feasibility: Single-shot planning is viable because reasoning and memory capabilities of the P-LLM, rather than reactive Q-VLM capabilities, primarily drive performance.The paper suggests improved reasoning models could increase utility without fundamental architectural changes.
  • Limitations: Underspecified tasks force exponentially more branches, degrading performance and increasing costs; prompt tuning can improve this data-dependency trade-off.The proposed remedies include task-distribution fine-tuning, extensive state verification, and explaining general planning reasoning.
  • Limitations: OSWorld includes ill-defined, automatically unmeasurable, or inherently data-independent tasks, motivating better benchmarks.Examples include vague font-editing requests and navigation goals without explicit buttons.
  • Scaling: Pass@5 improves performance, and pass@20 reaches approximately 73% for Claude, indicating favorable scaling as plans are sampled and models become more efficient.Because plans are sampled independently of prior failures, multiple plans can be run in parallel or combined into a super-plan.

6 Conclusion

The work adapts Dual-LLM isolation to CUAs, preserving meaningful utility while exposing Branch Steering as a remaining data-flow vulnerability. It also identifies policy specification and dynamic environment handling as unresolved challenges.

  • Conclusion: Single-shot planning preserves significant CUA utility while shifting execution reasoning to a trusted Privileged Planner.The reported gains are up to 19% utility with smaller open-source models and up to 57% performance with larger closed-source models on CUA tasks.
  • Conclusion: Dual-LLM isolation prevents arbitrary command injection but remains vulnerable when malicious environments steer perception outputs into attacker-chosen plan branches.This Branch Steering vulnerability can redirect execution within a legitimate plan, including toward attacker-controlled websites.
  • Conclusion: The paper frames Dual-LLM as a system-centric defense that separates trusted planning from untrusted perception to constrain control flow.CaMeL uses upfront planning, while Fides generates actions iteratively; both separate control flow from data flow in different ways.
  • Conclusion: Policy-based protection is difficult to scale because policies require manual expertise and may be incorrect in dynamic environments.The authors therefore avoid adding security policies for CUAs and instead restrict Q-LLM calls while adding redundancy defenses.

B Extended methodology section

The methodology adapts Dual-LLM separation to CUAs through one-shot branching plans, quarantined perception, and the Observe-Verify-Act workflow. Practical prompting and multimodal UI information help address the difficulty of predicting complex navigation paths.

  • Architecture: CaMeL-NOVA compiles a complete Python-like CUA plan upfront and executes it using a Privileged Planner, Quarantined Perception model, and verifier.The plan includes screenshot summarization, element finding, hypothesis verification, and completion checks, with security checks during execution.
  • Threat model: The attacker is assumed to control parts of the execution environment while lacking access to the Privileged Planner and underlying agent infrastructure.The threat model includes malicious websites and injected content in trusted sites such as advertisements, forums, and product reviews.
  • Architecture: All planning, reasoning, and action selection remain with the Privileged Planner, while the Q-VLM retrieves only values for conditional logic or tool-call arguments.This separation requires a hierarchical framework rather than an end-to-end CUA model alone.
  • Perception: DOM access complements screenshot perception by providing precise, less biased element information while remaining more verbose and less dynamic.The methodology uses both DOM and visual information because each source exposes different classes of UI elements and failure modes.
  • Observe-Verify-Act: Observe-Verify-Act structures execution by first gathering UI state, then checking a state hypothesis, and only afterward acting.The workflow uses screenshot summaries, page elements, or page text to establish and verify the current computer state.
  • Practicality: Single-shot plans use descriptive finding, branching, looping, cookie-handling, and site-navigation guidance to manage complex UI workflows.The navigation guidance emphasizes exploring the current site, iterating after failures, and progressively narrowing through categories and subcategories.

B.1.4 Ablation: How to make Fides work

Fides-NOVA relaxes Fides’ redaction rule so the planner can inspect boolean environment results, while bounded execution limits information leakage and ensures termination.

  • Fides adaptation: Fides-NOVA allows the planner to check boolean variables returned by environment functions.This relaxation enables conditional behavior that is unavailable when all function outputs remain redacted.
  • Security trade-off: The relaxation can let repeated calls reveal environment information and undermine termination guarantees without additional limits.The risk arises because Fides executes actions iteratively rather than through a fixed-length plan.
  • Security trade-off: max_steps, max_turn, and max_variable_reuse bound GUI actions, total function calls, and reuse of individual outputs.The practical settings are max_steps=15, max_turn=70, and max_variable_reuse=5.

C.1 Verifier Module Architecture

The verifier architecture cross-checks Q-VLM outputs against DOM or independent visual models, but adaptive pixel attacks can still evade these redundancy defenses. Cookie-popup attacks exploit predictable early routines to redirect agents through benign and attacker-controlled pages.

  • Verifier architecture: Verifier modules cross-check environment-derived Q-VLM outputs before they populate plan variables.They use either a second information source, such as the DOM, or independently instantiated models.
  • Verifier architecture: The defense offers best-effort protection because probabilistic verifiers provide no formal guarantees beyond Dual-LLM control-flow protection.Its rationale relies on low transferability of adversarial examples between diverse models.
  • DOM consistency: DOM consistency checking compares proposed coordinates and instructions with webpage structure, helping flag clicks on advertisements or disguised elements.Static Google ad banners expose an Advertisement frame and coordinates that can identify clicks within the ad region.
  • Multimodal consensus: Independent screenshot verification checks visual and semantic consistency, but optimized perturbations can produce outputs that appear legitimate to both models.The defense can be circumvented when attacks are optimized against both verifier inputs.
  • Cookie attacks: Cookie attacks embed fake popups in advertisements to redirect agents, exploiting routine cookie handling that defended CUAs must perform early.The attack can use an intermediary benign site and extend across multiple anticipated page visits, including long-range redirection.
  • DOM consistency: HTML5 ads can bypass DOM defenses because fabricated cookie popups replace the Advertisement frame with a cookie frame in the DOM.The DOM exposes only leaf nodes, allowing the embedded popup to appear structurally legitimate.
  • Attack results: Even the strongest redundancy setting can be evaded by attacks whose Q-VLM outputs, coordinates, and thought traces remain consistent with the task.The resulting redirection could be used to increase traffic to attacker-controlled commercial pages.

D.3.2 Implementation

The pixel attack optimizes a masked screenshot region to make the Q-VLM select an attacker-chosen drug while producing a plausible justification. The example targets “Rhapsido” instead of the Natural Product Database, so the action evades redundancy defense Level 2.

  • Pixel attack implementation: The attack optimizes a masked image region to make UI-TARS click “Rhapsido” instead of “Natural Product Database.”The target sequence is shown in Figure 4.
  • Pixel attack implementation: Backpropagation through the vision encoder optimizes pixel values in patch-embedding space while keeping non-masked patches fixed.Straight-through gradient estimation handles PNG quantization during image processing.
  • Pixel attack implementation: EOT averages gradients over 2 augmented samples per iteration, with Gaussian noise and additional top-center noise to improve robustness.The attack uses σ = 0.005 generally and σ = 0.05 in the clock region.
  • Extensions and Future Work: Multi-objective, multi-Q-VLM, and multi-step pixel attacks are left for future work.The authors state that an n-step attack could chain n one-step attacks.
  • Attack example: Figure 4 shows Q-VLM clicking the specific drug and explaining that it is categorized as a natural product, so redundancy defense Level 2 does not flag the action.The figure’s action is the attacker’s intended redirection rather than the database selection.

E.2 Plan Analysis Methodology

The plan-analysis methodology parses planner-generated Python-like plans into DAGs to compare CUA and typed-API plan structure. The comparison examines tool calls, branches, suites, planners, and performance under sampling.

  • Plan Analysis Methodology: The analysis pipeline parses Python-like plans into abstract syntax trees and extracts directed acyclic graphs containing tool-call nodes and branch points.Sequential edges capture execution order, while data-flow edges connect calls through shared variables.
  • Plan statistics: Table 3 compares CaMeL adaptations using per-plan means with 95% confidence intervals and reports branch-condition categories as percentages.The comparison covers AgentDojo, unoptimized CUA, and NOVA plans.
  • Plan statistics: CUA plans require an order of magnitude more tool calls and branches than AgentDojo plans because OS environments have unbounded state spaces.NOVA plans include many fallback paths for runtime UI states.
  • Plan Analysis Methodology: Figure 5 presents a side-by-side comparison of Observe-Verify-Act planning for OSWorld and a selected AgentDojo task.The figure compares the methodology across the two benchmark settings.

E.7 Redundancy Defense Evaluation

The redundancy defenses are evaluated on benign false positives and cookie-mimicry true positives, but both cookie and pixel Branch Steering attacks remain effective in important settings. DOM structure can block some attacks, while visual consensus struggles to distinguish fabricated from legitimate banners.

  • False positives: Redundancy defenses sometimes misclassify legitimate cookie banners as fake, producing false positives even in the narrow advertisement-banner threat model.The non-Chrome false-positive rate is 0/9.
  • Evaluation setup: Table 7 evaluates false-positive rates on benign tasks and true-positive rates against standard Google-ad and advanced HTML5 cookie attacks.The two attacks are evaluated against DOM Consistency and Multi-Modal Consensus, respectively.
  • False positives: Screenshot-based checking can distinguish legitimate cookie banners from fake ad-banner banners better than DOM-based information because screenshots contain more information than the accessibility tree.This comparison concerns checker inputs rather than overall defense success.
  • True positives: The cookie mimicry attack succeeds 100% when its cookie snippet closely matches the sample, but fails when plans query specific element types that exclude fake buttons.The failure suggests tighter DOM control or element-type restrictions could improve defenses.
  • Defense implications: Restricting queries to specific element types, such as buttons, is identified as a potential defense because fake banners may then yield no coordinates.This boundary is illustrated by the Google advertisement attack.

F Token counts comparison between defenses

The token-count comparison examines defense costs on OSWorld, including planner, Q-VLM, and redundancy components. CaMeL and Fides increase output tokens, redundancy defenses drive the largest cost increase, and long OS-navigation plans make the planner the dominant cost without redundancy defenses.

  • Experimental setup: Table 8 reports token counts and prices for different defense setups on a 17-task OSWorld subset evaluated with Pass@5.The comparison covers the costs associated with alternative defense configurations.
  • Cost accounting: The experiments use GPT-5 for planning and one consensus checker, Claude Haiku 4.5 for a DOM-based checker, and locally deployed UITars at zero counted cost.The pricing follows OpenAI and Anthropic platform token pricing.
  • Cost accounting: Table 9 separates token counts and prices across agent elements, including planner output and Q-VLM functions, for Pass@5 CaMeL runs.Q-VLM functions include all plan functions that use the Q-VLM.
  • Token-count findings: Both CaMeL and Fides generate proportionally more output than input tokens because plans require formatting and environment-state descriptions.Functions such as summarize_screenshot_content contribute to state description.
  • Token-count findings: Redundancy defenses produce the largest token-count and cost increase because they require detailed prompts and expensive GPT-5 and Claude Haiku 4.5 models.The prompts improve distinguishability between real and spoofed advertisement banners.
  • Token-count findings: Without redundancy defenses, the planner accounts for most OSWorld+CaMeL token cost because long plans cover many possible OS-navigation states.Planner output is approximately three times the combined Q-VLM-function output.

H.2 Natural Products database example of an insufficient plan (P-LLM: Gemini 3 Pro)

The natural-products database plan is insufficient because it lacks branching and overconfidently redirects toward search and alternative databases. Its workflow instead performs repeated observation and conditional navigation, including cookie handling, search-result checks, database selection, and browsing.

  • Insufficient plan: The plan is criticized as lacking branches, being too confident, and focusing on returning to search rather than website navigation.It may search for other databases instead of browsing the natural-products database.
  • Observation and cookie handling: The workflow first summarizes the current page and checks whether it is a search engine, a specific website, or a cookie-popup state.After navigation or app launch, it re-observes the page to identify the current website and visible consent controls.
  • Observation and cookie handling: When a cookie popup is detected, the workflow searches for common consent labels and clicks a matching button before refreshing the page summary.It tries labels such as “Accept all,” “Accept cookies,” “I agree,” “Consent,” “Allow all,” and “OK,” with visual search as a fallback.
  • Database navigation: The workflow verifies whether the page is already a natural-products database and, if so, looks for navigation options such as Browse, Explore, Data, Compounds, or Search.It clicks the first matching navigation item and scrolls when no relevant link is found.
  • Search and database selection: If the agent is not already on a database, it verifies search results, searches for “natural products database” when necessary, and then selects a candidate database.Candidates include NPASS, COCONUT, Natural Products Atlas, SuperNatural, and Natural Product Activity and Species Source, with a generic database-link fallback.
  • Database navigation: After selecting a database, the workflow handles cookies again, searches for a Browse control, clicks it when available, and otherwise scrolls or marks failure.If no relevant database link is found in search results, it marks the task as failed.
Loading 2601.09923v3…