PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual Confused Deputy: Exploiting and Defending Perception Failures in Computer-Using Agents

Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, Huamin Chen

arXiv:2603.14707v1cs.CVcs.CL

TL;DR

CUAs act on GUIs through visual perception, but prior work has largely evaluated task success rather than whether an agent acts on the intended object. The paper formalizes misperception as a visual confused deputy and proposes an external guardrail that checks both click targets and agent reasoning. The two channels cover complementary threat surfaces, while the guardrail is limited to per-action click-level checks.

  • Problem

    Prior work largely treats CUA perception failures as performance limitations and rarely evaluates whether the agent acts on the correct object.

    Source: §1 · p. 1

  • Method

    Dual-channel contrastive classification independently checks the visual click target and the LLM’s reasoning against deployment-specific knowledge bases, blocking execution if either channel flags risk.

    Source: §1 · p. 2

  • Results

    Visual and textual verification address complementary threat surfaces: image checks detect target-level grounding failures, while text checks capture dangerous intent behind visually innocuous controls.

    Source: §9 · p. 8

  • Takeaways & Limitations

    Safe CUA deployment requires independently verifying what agents believe they are acting on and why, not only improving their on-screen actions.

    Source: §9 · p. 8

    The guardrail evaluates a single click-target crop and reasoning string per action, so it does not detect content-level harm or risks emerging across multi-step plans.

    Source: §8 · p. 8

Abstract

from arXiv · show

Computer-using agents (CUAs) act directly on graphical user interfaces, yet their perception of the screen is often unreliable. Existing work largely treats these failures as performance limitations, asking whether an action succeeds, rather than whether the agent is acting on the correct object at all. We argue that this is fundamentally a security problem. We formalize the visual confused deputy: a failure mode in which an agent authorizes an action based on a misperceived screen state, due to grounding errors, adversarial screenshot manipulation, or time-of-check-to-time-of-use (TOCTOU) races. This gap is practically exploitable: even simple screen-level manipulations can redirect routine clicks into privileged actions while remaining indistinguishable from ordinary agent mistakes. To mitigate this threat, we propose the first guardrail that operates outside the agent's perceptual loop. Our method, dual-channel contrastive classification, independently evaluates (1) the visual click target and (2) the agent's reasoning about the action against deployment-specific knowledge bases, and blocks execution if either channel indicates risk. The key insight is that these two channels capture complementary failure modes: visual evidence detects target-level mismatches, while textual reasoning reveals dangerous intent behind visually innocuous controls. Across controlled attacks, real GUI screenshots, and agent traces, the combined guardrail consistently outperforms either channel alone. Our results suggest that CUA safety requires not only better action generation, but independent verification of what the agent believes it is clicking and why. Materials are provided\footnote{Model, benchmark, and code: https://github.com/vllm-project/semantic-router}.

1 Introduction

CUAs act through low-level GUI commands whose meaning depends on visual perception, but prior evaluations have largely treated misperception as a performance issue rather than asking whether the agent acts on the intended object. The paper frames these failures as a security vulnerability and proposes an independent guardrail that checks both the visual target and the agent’s reasoning.

1 Introduction

  • CUAs use raw screenshots and low-level GUI commands, making perception the trust anchor that determines what action is authorized.A command such as click(450,320) has no meaning without the screenshot.Source: §1 · p. 1
  • Existing evaluations rarely ask whether an agent acts on the correct object, despite evidence of frequent target misses and harmful GUI actions.OSWorld-MCP reports that 56.7% of CUA actions miss their intended target across 369 tasks; CUAHarm finds frontier models comply with dangerous GUI actions 90% of the time without jailbreaking.Source: §1 · pp. 1–2
  • The visual confused deputy arises when grounding errors, screenshot manipulation, or TOCTOU races make an agent authorize an action it would reject under accurate perception.The paper reports that an eight-line pixel-swapping attack can induce privilege escalation while remaining indistinguishable from ordinary CUA misclicks.Source: §1 · p. 2
  • Current approaches neither verify what the agent actually clicks nor operate outside its perceptual loop, while grounding remains weak across models and scales.The cited comparison reports GPT-4o at 0.8% on ScreenSpot-Pro and Qwen2.5-VL-72B at 43.6% [Li et al., 2025b] [Bai et al., 2025].Source: §1 · p. 2
  • Dual-channel contrastive classification checks click-target crops and the LLM’s reasoning against deployment-specific knowledge bases, blocking execution if either channel flags risk.The paper reports that visual and textual verification address complementary threat surfaces and that their fusion substantially outperforms either channel alone.Source: §1 · p. 2

2 The Visual Confused Deputy

A visual confused deputy occurs when an agent’s perceived screen state differs from the real display, so its click reaches an element other than the one it intended. The paper identifies grounding errors, adversarial screenshot changes, and TOCTOU races as sources of this mismatch, while existing defenses are poorly suited to screen-dependent actions.

2 The Visual Confused Deputy

  • 56.7% of CUA actions target the wrong element [Jia et al., 2025], and professional-GUI grounding accuracy is 18.9% [Li et al., 2025b].The paper characterizes visual grounding errors as the dominant, intrinsic source of perception divergence.Source: §2 · p. 2
  • A compromised runtime can swap screenshot button regions, causing the LLM to click the privileged target it misperceives as a benign control.In ScreenSwap, “Acknowledge Alert” is perceived where the actual click reaches “Admin: Reset Credentials.”Source: Figure 1 · p. 3, §2 · p. 3
  • A visual confused deputy event occurs when the LLM intends one element but its action coordinates map to a different element on the real display.The screen may change between screenshot() and click(), creating a TOCTOU race [Wang et al., 2025].Source: §2 · p. 3
  • The confused-deputy framing generalizes Hardy’s concept: authority resides in the agent, intent originates in the LLM, but perception of the target may be wrong.The agent runtime sits at both the screenshot and action-dispatch boundaries.Source: §2 · pp. 2–3
  • Access controls cannot interpret coordinate-only clicks, and in-agent guardrails share the agent’s grounding errors or can be bypassed by a compromised runtime.The paper also describes LLM self-policing as circular because the same perception drives the safety check.Source: §2 · p. 3

3 ScreenSwap: Evidence of Exploitability

ScreenSwap demonstrates that a compromised runtime can turn a routine GUI action into a privileged operation by swapping button pixels before the screenshot reaches the LLM. The attack uses genuine rendered pixels and is described as minimal, undetectable to the LLM, and indistinguishable from routine CUA errors.

3 ScreenSwap: Evidence of Exploitability

  • In the SOC scenario, the compromised runtime swaps “Acknowledge Alert” and “Admin: Reset Credentials” pixel regions before forwarding the screenshot to the LLM.The setting is a shared jump server where a soc-tier1 agent runtime has been compromised via a malicious tool server.Source: §3 · p. 3
  • The LLM sees “Acknowledge Alert” at the admin button’s coordinates and clicks, triggering a credential reset it did not intend.The attack requires no adversarial perturbations or model-specific exploits.Source: §3 · p. 3
  • ScreenSwap uses eight lines of image code and native-rendered pixels, making the attack indistinguishable from routine CUA errors [Jia et al., 2025].The LLM cannot detect the swap, and coordinate-only access control does not reveal the target’s semantic content.Source: §3 · p. 3

4 Defense: Dual-Channel Contrastive Classification

The proposed guardrail intercepts clicks outside the agent’s perceptual loop and evaluates their visual targets and the LLM’s reasoning through separate channels. Its veto-style fusion blocks an action if either channel flags risk, addressing threat surfaces that either channel alone can miss.

4 Defense: Dual-Channel Contrastive Classification

  • The guardrail intercepts each click before execution, using the screenshot, click coordinates, and LLM reasoning text as inputs.It classifies the image crop at the click location and the reasoning text independently.Source: §4 · p. 4
  • Image-channel F1 is 0.889 on ScreenSpot-Pro but 0.0 on neutral buttons; text-channel F1 is 1.0 for dangerous intent behind neutral buttons.The image channel catches grounding errors but is blind to neutral buttons, while the text channel degrades on heterogeneous cross-application data.Source: §4 · p. 4
  • The OR-veto blocks when either channel flags a restricted action, whereas AND-logic would miss threats behind neutral buttons.The image channel evaluates the click-target crop, and the text channel evaluates the LLM’s reasoning.Source: §4 · p. 4, Figure 2 · p. 4

5 Analysis

The guardrail intercepts clicks outside the untrusted agent process and is the sole enforcement point. It covers grounding errors and some screenshot manipulation, but leaves several risks outside its guarantees.

Threat model

  • The guardrail intercepts each click with the screenshot and reasoning text; its classifier is the sole enforcement point.The agent runtime is untrusted and may produce any action.Source: §5 · p. 4

Threat coverage

  • The dual-channel classifier covers visual grounding errors and some screenshot manipulation, with simultaneous misses remaining as deployment-specific false-negative risk εcls.Measured εcls is 0% on controlled SOC scenarios, 12.6% on 260 real ScreenSpot-Pro screenshots, 0% on neutral-button intent scenarios, and 4.0% on OS-Harm leave-one-out contrastive KB.Source: §5 · p. 4

Limitations

  • The defense does not guarantee content-level safety, detection of harmful multi-click chains, resistance to classifier evasion, or coverage beyond curated KB vocabulary.Each click is evaluated independently, and the classifier is described as pragmatic rather than provably robust.Source: §5 · p. 4

ScreenSpot-Pro evaluation

  • Table 1 covers 260 ScreenSpot-Pro screenshots and 1,040 test cases per configuration; its caption identifies image+text veto as the optimal fusion.The caption directs readers to Appendix E for details.Source: Table 1 · p. 5

6 Evaluation

Across controlled attacks, realistic GUI scenarios, screenshots, and reasoning traces, the evaluation tests complementary visual and textual threat surfaces. The image+text veto improves grounding-error detection over image-only classification, while text classification detects dangerous intent behind neutral controls.

6 Evaluation

  • Evaluation uses a multimodal embedding model and nearest-neighbor max-cosine-similarity classification across six benchmarks targeting different threat surfaces.The model has 120M parameters, with a SigLIP image encoder, MiniLM text encoder, and trained 384-dimensional projection.Source: §6 · p. 5

6.1 ScreenSwap PoC

  • On a GTK3 SOC dashboard running in Xvfb, the dual-channel classifier correctly blocks the ScreenSwap admin target across all 8 controlled scenarios, with an image-channel margin of +0.46.The attack swaps “Acknowledge Alert” and “Admin: Reset Credentials” button regions.Source: §6.1 · p. 5

6.2 OS-Harm Scenarios

  • On 13 reproduced OS-Harm scenarios, nearest-neighbor classification blocks 7/7 harmful cases (100% recall) and allows 4/6 safe cases (67% specificity), with F1 = 0.875.Two false positives involve legitimate Send and user-requested Trash controls that look identical across benign and harmful uses.Source: §6.2 · p. 5

6.3 Image Channel on ScreenSpot-Pro

  • F1 = 0.915 for image+text veto on ScreenSpot-Pro grounding-error detection, versus 0.889 for image-only classification.The veto catches 34 additional grounding errors without adding false positives.Source: §6.3 · p. 5

6.4 Neutral-Button Evaluation: Where Text Excels

  • In 20 neutral-button scenarios, the image channel has F1 = 0 while the text channel has F1 = 1.0; OR-logic preserves the text signal, unlike AND-logic.The buttons are visually identical, so reasoning text distinguishes dangerous from safe intent.Source: §6.4 · pp. 5–6, Figure 3 · p. 6

6.5 Contrastive KB on Real CUA Data

  • On OS-Harm leave-one-out classification, max aggregation achieves 96.0% recall, 98.0% specificity, and F1 = 0.969; mean aggregation falls to F1 = 0.627.Cross-domain transfer from an InjecAgent KB to OS-Harm yields 0% specificity; the passage concludes deployment-domain examples are needed.Source: §6.5 · p. 6

6.6 Text Channel on Real LLM Reasoning

  • On real Claude 3.7 Sonnet traces, instruction–reasoning pairwise ranking accuracy is 96.5%, dangerous-intent classification reaches F1 = 1.000, and hijacked-case detection is 50/50.Combined verification allows 46/50 correct reasoning cases; its 8% false-positive rate is attributed to short generic instructions.Source: §6.6 · pp. 6–7

6.7 Overhead

  • Verification adds approximately 14 ms per click on an AMD MI300X GPU, compared with typical CUA inference latency of approximately 200 ms to tens of seconds.The estimate includes crop extraction, embedding computation, and similarity scoring.Source: §6.7 · p. 7

7 Related Work

Prior CUA benchmarks document frequent errors and concrete harms but do not provide independent pre-execution verification of the GUI target. This work relates to agent authorization and confused-deputy defenses while moving verification outside the agent’s perceptual loop.

CUA benchmarks

Agent security

  • Authorization and runtime safeguards for agents do not resolve CUA click ambiguity, because click(x,y) lacks semantic meaning without the screen state.The passage contrasts CUA interactions with API actions represented by meaningful tool names and arguments.Source: §7 · p. 7

External verification

  • Prior authorization approaches rely on the agent’s task, tool-call, or screen representation, whereas this method independently classifies the visual target and textual rationale outside its perceptual loop.The passage describes in-agent self-policing as circular when perception may already be wrong [Park et al., 2025, Gupta and Singh, 2025, Kim et al., 2025].Source: §7 · p. 7

Confused deputy

  • The visual confused deputy extends the classic confused-deputy framing to screen-based agents, where designation is inferred from potentially unstable pixels.Capability systems align designation with authority [ERights.org, 2000].Source: §7 · p. 7

8 Limitations

The guardrail verifies individual GUI actions, leaving content-dependent harms and risks that emerge across longer action sequences outside its scope. Its coverage is also strongest for pointer-driven workflows, with channel-level weaknesses and knowledge-base curation burdens.

8 Limitations

  • The guardrail checks a single click-target crop and reasoning string, so it cannot directly detect document or interface-content harms or unsafe plans that emerge across otherwise benign actions.Examples include malicious email bodies, harmful terminal commands, and dangerous form text.Source: §8 · p. 8
  • It is best suited to pointer-driven GUI workflows and is less effective for terminal-centric interaction, where risk depends more on command semantics than target selection.All 52 CUAHarm computer-use tasks are terminal-based.Source: §8 · p. 8
  • The image channel remains vulnerable on small or ambiguous targets, while the text channel degrades on heterogeneous cross-application data and both depend on curated deployment-specific knowledge bases.The image channel is more robust than the text channel to knowledge-base curation errors.Source: §8 · p. 8

9 Conclusions and Future Work

The paper frames CUA perception failures as a security vulnerability and presents dual-channel verification as a practical guardrail. Its ScreenSwap demonstration shows how swapping genuine button pixels can redirect a click to a privileged action.

9 Conclusions and Future Work

  • The paper formalizes the visual confused deputy and reports ScreenSwap as a minimal attack that turns routine perception errors into privileged mis-execution.Future work includes sequence-level reasoning, broader computer-use coverage, reduced knowledge-base curation burden, and execution-time controls such as human confirmation for irreversible actions.Source: §9 · p. 8
  • Visual and textual verification address complementary threat surfaces: image checks catch target-level grounding failures, while text checks expose dangerous intent behind innocuous controls.The paper presents their combination as a lightweight verification layer that makes the attack surface observable and enforceable.Source: §9 · p. 8

A.1 Attack Procedure

  • ScreenSwap swaps the genuine pixel regions of “Acknowledge Alert” and “Admin: Reset Credentials,” so the LLM sees the safe label at the privileged button’s coordinates and clicks there.The perceived safe action triggers a credential reset on the actual display.Source: §A.1 · p. 10

A.2 Properties of the Attack

  • The attack needs no adversarial perturbation or model-specific exploit, appears pixel-perfect to the LLM, and is indistinguishable from ordinary clicks to access control.It shifts wrong actions toward high-value targets without changing the overall wrong-action rate.Source: §A.2 · p. 10

B.1 Setup

  • The proof of concept uses a GTK3 SOC dashboard in Xvfb, but it does not include an LLM inference call.It demonstrates that the swap is visually convincing and that coordinates derived from the altered screenshot hit the admin button on the real display.Source: §B.1 · p. 11

B.2 Attack Demonstration

  • On the real Xvfb display, clicking the admin button’s center after the swap produces the “Admin Action: Reset Credentials” confirmation dialog.A control click at the acknowledge button’s coordinates shows the safe action.Source: §B.2 · p. 11

B.3 SOC Dashboard: Single-Modality KB Results

On the SOC dashboard, image-only and dual-channel OR classification both achieve perfect accuracy across danger-styled button scenarios. The image-only KB result favors visual classification, while the guardrail’s scope remains limited to click-level evidence.

B.3 SOC Dashboard: Single-Modality KB Results

  • The image-only KB achieves 4/4 accuracy, and trained projection improves the minimum margin over raw SigLIP by 58% (0.1779 vs. 0.1126).Text-only and text-query-against-image-KB strategies fail because the text encoder was trained on natural-image captions, not UI descriptions.Source: §B.3 · p. 11

B.4 SOC Dashboard: Dual-Channel Comparison

  • Both dual-channel OR and image-only classification achieve 8/8 accuracy on danger-styled buttons, with the image channel alone catching the critical cases where stated intent conflicts with admin-target coordinates.The critical grounding-error scenarios are rows 4–5.Source: §B.4 · p. 12, Table 4 · p. 12

C.1 Scope of the Guardrail

  • A single click-target crop can flag grounding mismatches and explicitly restricted-looking clicks, but the guardrail does not assess typed content, selected files, recipient correctness, harmful user requests, or multi-step chains.The image channel classifies the target independently of the agent’s reasoning.Source: §C.1 · p. 12

C.2 Aggregation Method and KB Ablation

The experiments examine contrastive aggregation, knowledge-base construction, and channel behavior across GUI settings. Results show that aggregation and deployment context matter, while image and text channels have different strengths across tasks.

C.2 Aggregation Method and KB Ablation

  • Original + max achieves F1 = 0.875 and accuracy 84.6%, while expanding the text KB from 10+10 to 20+20 entries reduces specificity from 50% to 17%.The passage attributes the specificity drop to semantically overlapping entries pulling benign actions toward the dangerous centroid.Source: §C.2 · pp. 12–13

C.3 Per-Scenario Results

  • Of seven catches, one is a genuine intent/target mismatch detection and six result from inherently danger-styled buttons; two remaining false positives on neutral buttons motivate contextual KBs or human confirmation.Contextual KBs condition on the full screen state rather than only the button crop.Source: §C.3 · p. 13

D Baseline Knowledge Base Vocabulary; D.1 Text Knowledge Base

D.2 Image Knowledge Base

  • The image KB uses synthetic button crops with danger or warning styling versus success, accent, or neutral styling; click-centered crops are compared against these references.The evaluated incoming crop is 100×100 pixels and encoded through the SigLIP branch.Source: §D.2 · p. 14, Figure 4 · p. 15

E.1 Text-Channel Failure Analysis; E.3 Per-Application Breakdown

  • Cross-app text-only max blocks every action (0% specificity), while image-only max reaches ≥85% accuracy on 16/26 applications.Source: §E.1 · p. 15, §E.3 · p. 15

E.2 Dual AND-Logic Results

  • Dual-agree reaches 70.3% accuracy with per-app text knowledge bases, below image-only’s 84.5%; image+text veto is the recommended architecture.Naive cross-app text knowledge bases yield 16.1% accuracy with AND-logic.Source: §E.2 · p. 15

F Neutral-Button Test Design

  • The neutral-button test uses 20 scenarios across five grey button types, pairing each button with reasoning text that distinguishes safe from dangerous actions.Source: F Neutral-Button Test Design · p. 15

G VLM Model Size and CUA Grounding Accuracy

Loading 2603.14707v1…