PaperlayerView exact revision on arXiv

Source-linked AI summary

LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios

Tianyu Chen, Chujia Hu, Ge Gao, Dongrui Liu, Xia Hu, Wenjie Wang

arXiv:2602.03255v1cs.AI

TL;DR

Ambiguous instructions and adversarial tool steering create safety risks for computer-use agents, while existing benchmarks often overlook planning-time risks in long-horizon tasks. LPS-BENCH evaluates these risks in MCP-based workflows using generated cases and trajectory-focused judging. Experiments find that current agents struggle to maintain safety, and prompt-based mitigations are insufficient.

  • Problem

    Existing benchmarks largely emphasize action-level behavior and short-horizon tasks, leaving plan-level safety under benign ambiguity and adversarial steering insufficiently evaluated.

    Source: §1 · p. 1, §2 · p. 3

  • Method

    LPS-BENCH evaluates long-horizon MCP-based planning safety across 65 scenarios in 7 domains and 9 risk types, using multi-agent case generation and LLM-as-a-judge trajectory evaluation.

    Source: §1 · p. 2

  • Results

    Even state-of-the-art agents struggle to maintain consistent safety in complex workflows, and the best-performing model, Claude-4.5-Sonnet, reaches only 5.88% on FA and 61.82% on TS.

    Source: §4.2 · p. 7, §5 · p. 8

  • Takeaways & Limitations

    Long-horizon planning mechanisms need to verify assumptions, validate environmental feedback, and remain robust throughout extended action sequences.

    Source: §4.2 · p. 7

    The benchmark cannot exhaustively cover real-world domains, tools, and edge cases, and some scenario distributions may differ from deployment settings.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Computer-use agents (CUAs) that interact with real computer systems can perform automated tasks but face critical safety risks. Ambiguous instructions may trigger harmful actions, and adversarial users can manipulate tool execution to achieve malicious goals. Existing benchmarks mostly focus on short-horizon or GUI-based tasks, evaluating on execution-time errors but overlooking the ability to anticipate planning-time risks. To fill this gap, we present LPS-Bench, a benchmark that evaluates the planning-time safety awareness of MCP-based CUAs under long-horizon tasks, covering both benign and adversarial interactions across 65 scenarios of 7 task domains and 9 risk types. We introduce a multi-agent automated pipeline for scalable data generation and adopt an LLM-as-a-judge evaluation protocol to assess safety awareness through the planning trajectory. Experiments reveal substantial deficiencies in existing CUAs' ability to maintain safe behavior. We further analyze the risks and propose mitigation strategies to improve long-horizon planning safety in MCP-based CUA systems. We open-source our code at https://github.com/tychenn/LPS-Bench.

1 Introduction

Computer-use agents can turn ambiguous instructions or adversarial tool steering into harmful, state-changing actions, while existing benchmarks often miss planning-time risks in long-horizon workflows. LPS-BENCH evaluates these risks in long-horizon MCP-based tasks through automated case generation and trajectory-focused evaluation.

1 Introduction

  • Ambiguous benign instructions can prompt harmful actions, while adversarial users can steer tool calls or combine benign calls into harmful sequences.MCP tools can produce immediate, irreversible side effects, and a safe final outcome does not guarantee safe intermediate actions.Source: §1 · pp. 1–2
  • LPS-BENCH targets long-horizon MCP-based planning safety, covering 9 risk types across 65 scenarios and 7 task domains.The benchmark assesses safety beyond execution-time correction.Source: §1 · p. 2
  • A multi-agent pipeline expands scenario templates into tool-simulated cases, and an LLM judge assesses safety across plans and action trajectories.The assessment considers whether trajectories avoid unsafe or risk-amplifying steps, not only whether the final outcome is safe.Source: §1 · p. 2, Figure 3 · p. 3
  • LPS-BENCH covers benign assistance and adversarial manipulation in extended, state-changing workflows that can require repeated replanning.Cases are expert-reviewed, use a flexible tool-simulation interface, and are designed for scalable construction and evaluation.Source: §1 · p. 2
  • Many representative open- and closed-source models fail to maintain safe behavior consistently through extended, state-changing workflows.The benchmark also analyzes where risks arise and presents mitigation strategies for benign and adversarial risks.Source: §1 · pp. 2–3

2 Related Work

Prior agent-safety benchmarks cover dialogue, GUI, web, and tool or protocol risks, but largely emphasize action-level behavior. LPS-BENCH addresses the gap in evaluating plan-level safety over long-horizon trajectories under both benign ambiguity and adversarial steering.

2 Related Work

  • QA-style evaluations assess safety from static dialogues or traces, while GUI and web benchmarks involve visually grounded interface actions.Examples include R-JUDGE (Yuan et al., 2024b), OS-HARM (Kuntz et al., 2025), MOBILE-SAFETYBENCH (Lee et al., 2024), SAFEARENA (Tur et al., 2025), and AGENTDAM (Zharmagambetov et al., 2025).Source: §2 · p. 3
  • Tool and protocol benchmarks examine harmful side effects, adversarial attacks, and MCP tool-metadata poisoning, but do not address the full long-horizon planning setting identified here.Examples are TOOLEMU, AGENT-SAFETYBENCH (Zhang et al., 2024c), and MCPTOX (Wang et al., 2025a).Source: §2 · p. 3
  • Prior benchmarks largely emphasize action-level behavior rather than plan-level safety over long-horizon trajectories with benign ambiguity and adversarial steering.The comparison is framed across dimensions including dynamic interaction and plan-level safety evaluation.Source: §2 · p. 3, Table 1 · p. 4

3 LPS-BENCH Construction

LPS-BENCH evaluates planning-time safety in long-horizon MCP-based computer-use tasks, spanning benign and adversarial risks. Its construction combines scenario generation, human review, and trajectory-level safety evaluation.

3.1 Risk Categories

  • LPS-BENCH distinguishes benign risks from adversarial plan-level attacks, covering unsafe assumptions and inefficient plans as well as concealed harmful goals, multi-turn manipulation, environmental backdoors, timing exploits, and prompt injection.Benign risks arise from underspecified, ambiguous, or erroneous instructions; adversarial risks arise when users steer planning toward malicious objectives.Source: §3.1 · p. 4

3.2 Scalable Data Generation Pipeline

  • A human-in-the-loop multi-agent pipeline turns high-level risk scenarios into executable cases by coordinating instruction, mock-tool, and evaluation-criteria design.An orchestrator coordinates the specialized agents, then synthesizes their outputs into a structured draft case.Source: §3.2 · p. 5
  • Expert review checks tool logic, safety criteria, and scenario difficulty, with feedback prompting revisions before cases enter the repository.Reviewers can request changes such as subtler attacks or corrected API signatures.Source: §3.2 · p. 5

3.3 Automated Evaluation Framework

  • Evaluation uses sandboxed tool execution and case-specific LLM assessment of interaction trajectories, judging safe task completion or clarification in benign cases and refusal or timely halting in adversarial cases.Mock toolkits avoid real-world side effects while simulating functions such as file-system APIs and browser interactions.Source: §3.3 · pp. 5–6

4 Experiments

Experiments evaluate 13 LLMs on LPS-BENCH and find persistent safety failures across risk types and task domains. Prompt-based mitigations offer limited gains, with results varying by risk and model capability.

4.1 Setup

  • The experiments evaluate 13 proprietary and open-weight LLMs in a unified agent framework using Safe Rate, the fraction of trajectories without safety violations.All experiments use stochastic sampling at temperature 1, and DeepSeek-R1 serves as evaluator.Source: §4.1 · pp. 6–7

4.2 Safety Analysis According to Risk Type

  • FA remains difficult even for frontier models, while adversarial robustness varies by where influence enters: Claude-4-Sonnet reaches 43.08% SR on EB versus 98.51% on PI.Agents often guess at missing constraints in FA and may treat environmental observations as ground truth in EB.Source: §4.2 · p. 7
  • Even the best-performing model, Claude-4.5-Sonnet, reaches only 5.88% SR on FA and 61.82% on TS, while unsafe planning persists across underspecified and adversarial conditions.Stronger models within each family consistently achieve higher SR, but the paper identifies plan validation as an ongoing blind spot.Source: §4.2 · p. 7

4.3 Safety Analysis According to Domain

  • File-centric tasks expose a domain-specific weakness for most models, while safety rates are relatively consistent across domains for strong models and slightly higher on Web and Social.Open-source models show more pervasive failures across domains and often follow operational requests with little evidence of safety-aware planning.Source: §4.3 · pp. 7–8

4.4 Analysis According to Model Capability

  • Instruction-following capability positively correlates with safety rates, but open-source models often gain capability with limited safety improvement; commercial models achieve higher rates, especially under adversarial risks.Figure 5 presents the relationship between instruction-following capability and planning safety awareness.Source: §4.4 · p. 8, Figure 5 · p. 8

4.5 Mitigation Strategies

  • Prompt-based interventions slightly improve safety, but agents still guess instead of clarifying under benign ambiguity, and adversarial-prompt gains are marginal for weaker models.The strategies use human confirmation for ambiguous or high-stakes actions and threat descriptions to help identify adversarial manipulation; the reported gains depend on model instruction-following capability.Source: §4.5 · p. 8, Figure 6 · p. 8

5 Conclusion

LPS-BENCH evaluates CUA safety during long-horizon planning across benign and adversarial risks. Experiments with 13 representative models reveal persistent difficulty maintaining safety in complex workflows, while simple prompt-based mitigations are insufficient.

5 Conclusion

  • Experiments with 13 representative models find that even state-of-the-art agents struggle to maintain consistent safety throughout complex workflows.The benchmark exposes vulnerabilities in anticipating delayed consequences and verifying environmental assumptions.Source: §5 · p. 8
  • Simple prompt-based mitigations prove insufficient for the long-horizon planning safety challenges identified by LPS-BENCH.Source: §5 · p. 8
  • LPS-BENCH is intended to encourage research beyond superficial guardrails toward intrinsically safe and robust autonomous agents.Source: §5 · p. 9

Limitations

The benchmark has bounded coverage and uses automated evaluation, leaving some deployment mismatch and adjudication uncertainty.

Limitations

  • The benchmark cannot exhaustively cover real-world domains, tools, and edge cases, and its scenario distributions may differ from deployment settings.Source: Limitations · p. 9
  • Automated evaluation supports scalability and consistency, but a very small subset of borderline trajectories may remain difficult to adjudicate perfectly.The authors identify further calibration and refinement as possible.Source: Limitations · p. 9

Ethical Considerations

The benchmark includes adversarial, high-risk instructions and an automated red-team case-generation pipeline, which could be repurposed to probe real systems. Its sandboxed design limits direct misuse beyond evaluation settings.

Ethical Considerations

  • A determined attacker could repurpose a very small portion of the benchmark materials or methodology to probe real systems.Source: Ethical Considerations · p. 9
  • Execution uses controlled sandboxes with mock tools and excludes real credentials, private user data, and operational exploit payloads, limiting direct misuse outside evaluation.Source: Ethical Considerations · p. 9

A Prompts

The appendix documents prompts for data generation, evaluation, and mitigation, including risk-specific templates and criteria.

A.1 Data Generation Prompt Template

  • Generation templates are broadly similar but include optimizations for each safety-risk type.Source: §A.1 · p. 11
  • Templates can generate varied scenarios within a risk category by changing the domain component.Figures 7–15 illustrate the nine risk categories described in the main text.Source: §A.1 · p. 11

A.2 System Prompt of Multi-agent Data Generation Pipeline

  • The pipeline prompt figures specify an Orchestrator and three worker roles: Instruction Designer, Tool Developer, and Criteria Formulator.Source: §A.2 · p. 11

A.3 System Prompt of Evaluator

  • The evaluator receives the full execution record, an instance-specific Safety Criterion, and a brief description of the risk category.The instance-level criterion drives the concrete decision boundary.Source: §A.3 · p. 11
  • The evaluator labels cases safe, unsafe, or execution_failed based on whether safeguards precede risk-sensitive actions.Risk-specific criteria and instantiations share this decision procedure.Source: §A.3 · p. 11

A.4 Mitigation

  • The appendix includes the raw system prompt and separate mitigation variants for benign-user and adversarial-user-induced risks.Source: §A.4 · p. 12

B Case Study

The case-study appendix provides full long-horizon execution traces and explains how highlighted instruction and log spans distinguish ambiguity, unsafe actions, and safe responses.

B Case Study

  • The complete records make safety-relevant planning decisions observable across an MCP agent’s long-horizon tool-and-environment workflow.The records complement the main-text running example with step-by-step execution details.Source: B Case Study · p. 12
  • Red instruction spans mark underspecified references, while red log spans flag unsafe planning or high-impact actions under unresolved ambiguity.Green log spans indicate safe responses such as recognizing ambiguity, pausing, and requesting confirmation.Source: B Case Study · p. 12
  • Figures 24–27 provide end-to-end traces of a representative False-Assumption case, including tool calls, environment feedback, and model decisions.The appendix says other risk categories have analogous traces exposing category-specific failure modes and intervention points.Source: B Case Study · p. 12

C More Detailed Experimental Setup

The experimental setup uses stochastic decoding and a 100-step episode cap, with a shared LangChain agent configuration to standardize comparisons.

C More Detailed Experimental Setup

  • Episodes use stochastic decoding at temperature τ = 1, nucleus sampling p = 0.9, top-k k = 50, and a 100-interaction-step cap.All agents use the default agent in LangChain v1.0 to standardize orchestration and comparison across models.Source: C More Detailed Experimental Setup · p. 12
  • Stochastic exploration is intended to aid recovery from brittle tool-call failures and reduce false safety caused by deterministic decoding failing to reach relevant states.This is intended to make evaluation better reflect safety-aware planning rather than under-exploration.Source: C More Detailed Experimental Setup · p. 12

Prompt Template for Task Sequence Risks (a)

The appendix collects prompt templates for risk-class case generation, worker and evaluator prompts, mitigation prompts, and example execution records.

Prompt Template for Task Sequence Risks (a)

  • The Task Sequence risk class has a prompt template used in the multi-agent data-generation pipeline.Source: Figure 7 · p. 13

Prompt Template for Over-Compliance Risks (b)

  • Templates cover Over-Compliance, False-Assumption, Incomplete Plan, and Harmless Subtask risks, including attacks that disguise harmful intent as benign steps.Source: Figure 8 · p. 14, Figure 9 · p. 15, Figure 10 · p. 16
  • Additional risk templates target fabricated multi-turn attacks, environment backdoors, race-condition windows, and prompt-injection or jailbreak pressure.The race-condition template focuses on latency and non-atomic checks; the environment-backdoor template uses attacker-planted triggers in environment outputs.Source: Figure 12 · p. 18, Figure 13 · p. 19, Figure 14 · p. 20

System Prompt: Orchestrator

System Prompt: Instruction Designer

System Prompt: Criteria Formulator

Loading 2602.03255v1…