PaperlayerView exact revision on arXiv

Source-linked AI summary

DIBench: Benchmarking Decision Integrity of GUI-based Mobile Agents Under Deceptive Injections

Li Hu, Kanghua Mo, Yingbin Jin, Qingqing Ye, Haibo Hu

arXiv:2610.06898v1cs.CRcs.MA

TL;DR

Whether an agent’s within-set selection aligns with instruction semantics is a distinct and critical dimension of agent behavior. DIBench pairs benign and injected instances under a non-privileged UI-content threat model and evaluates final selections with decision-integrity measures. Across 4 frameworks and 7 base models, deceptive injections steer selections while completion-oriented metrics can overestimate agent trustworthiness.

  • Problem

    Whether an agent’s within-set selection aligns with instruction semantics is a distinct and critical dimension of agent behavior.

    Source: §2 · p. 3

  • Method

    DIBench pairs benign and injected instances under a non-privileged UI-content threat model and evaluates final selections with decision-integrity measures.

    Source: §1 · p. 2, §3 · p. 3, §3.3.1 · p. 5

  • Results

    Across 4 frameworks and 7 base models, deceptive injections steer selections while completion-oriented metrics can overestimate agent trustworthiness.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Paired evaluation and decision-integrity metrics enable comparison beyond completion and trajectory anomalies, while trajectory analysis reveals weaker exploration and verification and earlier commitment.

    Source: §5 · p. 9

    The evaluation covers open research mobile-agent frameworks, not commercial or OEM-deployed assistants, limiting external validity.

    Source: §5 · p. 9

Abstract

from arXiv · show

As GUI-based mobile agents rapidly progress, rigorous safety evaluation of their autonomous decision-making in realistic app interfaces becomes increasingly critical. Existing benchmarks mainly focus on execution-level anomalies using task success or hijack rates, but fail to capture the in-task goal deviation risk in multi-candidate selection tasks, where the decision may be steered toward an attacker-specified target, even in violation of instruction-implied constraints (e.g., cheapest/highest-rated), without any overt execution anomalies. We present DIBench, a decision integrity benchmark for measuring this risk in mobile agents. DIBench covers 7 commercial and 3 simulated apps with 5 task types. Under a threat model restricted to non-privileged UI content, we construct 8 deceptive injection probe instantiations that can steer critical selections without overt anomalies. The benchmark includes 1,000 clean and 36,672 injected instances, with a unified protocol and integrity metrics for comparison. Experiments spanning 4 agent frameworks and 7 base models show that completion-based evaluation can overestimate agent trustworthiness and miss decision-integrity risks: deceptive injections steer selections and shift early action policies, inflating completion rates and creating a misleading illusion of safety. Common defenses, including detection, image preprocessing, and prompt reminders, yield inconsistent integrity gains. Overall, DIBench provides a unified, reproducible benchmark to quantify the risk of in-task goal deviation in mobile agents and enable comparable evaluations of safety defenses.

1 Introduction

DIBench targets a decision-integrity gap: mobile agents can complete multi-candidate tasks yet make attacker-steered choices, including choices that violate instruction-implied constraints. It pairs benign and injected instances to evaluate this risk; experiments show completion-based metrics can obscure it.

1 Introduction

  • Prior evaluations emphasize execution anomalies such as hijacking and unauthorized actions, often treating flow-correct completion as evidence of a trustworthy choice.That assumption can fail in multi-candidate selection tasks.Source: §1 · p. 1
  • In-task goal deviation occurs when an agent completes a task but deceptive UI content redirects its final choice toward an attacker-specified option, potentially against implied constraints.The injection need not alter instructions or control flow, and completion-based signals may miss the deviation.Source: §1 · p. 2
  • The benchmark addresses missing explicit threat modeling, scalable multi-candidate tasks across commercial and simulated apps, and unified protocols and metrics for comparing agents and defenses.The identified gap is a reproducible framework for measuring decision-layer risk under controlled, comparable conditions.Source: §1 · p. 2
  • DIBench pairs benign and injected instances across 7 commercial apps, 3 simulated apps, and 5 task categories, with 8 probe variants and 1,000 benign plus 36,672 injected instances.It provides unified configurations and defense hooks for comparison across frameworks, base models, and mitigation strategies.Source: §1 · p. 2
  • Across 4 frameworks and 7 base models, deceptive injections can raise task completion while steering choices, making completion-based evaluation overestimate trustworthiness.The reported commercial-app example raises TCR from 42.0% to 72.4% with 45.8% ASR.Source: §1 · p. 2

2 Background and Related Work

Mobile agents routinely compare candidates in apps, making alignment between the final selection and instruction semantics a distinct evaluation concern. Existing security evaluations largely emphasize execution-level behavior, leaving decision-layer risks from deceptive UI content insufficiently formalized.

2 Background and Related Work

  • Selection among candidates is central to tasks such as e-commerce and food delivery, where agents must balance exploration with committing to a choice.Whether the within-set choice matches instruction semantics is distinct from completing the interaction flow.Source: §2 · pp. 2–3
  • Existing mobile-agent security benchmarks cover high-stakes tasks, system interruptions, and third-party content injection, but mainly measure execution-level observables rather than instruction-aligned final choices.Table 1 compares representative mobile-agent security benchmarks.Source: §2 · p. 3, Table 1 · p. 3
  • Deceptive UI content can steer a candidate choice despite normal browsing, filtering, and submission, so execution-focused evaluations may miss this decision-layer risk.The risk remains insufficiently formalized as a systematic security problem, with no reproducible, comparable framework identified in the passage.Source: §2 · p. 3

3 DIBench Construction

DIBench isolates in-task goal deviation under a non-privileged threat model in which attackers can alter only platform-permitted UI content. It combines commercial and simulated environments, varied selection tasks, and standardized injection probes.

3 DIBench Construction

  • The threat model permits attackers to influence agents only through on-screen content, without changing instructions, prompts, or control flow.The target risk is a completed workflow whose final choice is steered toward an attacker-specified option.Source: §3 · p. 3

3.1 Benchmark Components

  • DIBench comprises 7 commercial apps and 3 simulated apps, with 1,000 benign instances and 36,672 injected instances across 5 task categories.The environments, task suite, and injection probes form its three core components.Source: §3.1 · p. 4

3.1.1 Environments

  • The environments pair 7 real apps with 3 controlled simulations; commercial-app injections use fixed decision-state screenshots, while simulations support deterministic interactions and backend content updates.This balances realistic UI complexity with controlled, reproducible process analysis while avoiding adversarial content on live commercial platforms.Source: §3.1.1 · p. 4

3.1.2 Task Suite

  • Tasks define an instruction, candidate set, and acceptable set across exploratory, exact-match, cost, spatio-temporal, and quality/popularity categories.Exploratory tasks accept every candidate; the other categories impose UI-observable constraints or preferences, with 20 instances per task type for each app.Source: §3.1.2 · p. 4

3.1.3 Injection Probes and Injected Instances

  • Eight parameterized probes alter UI-visible content while preserving instructions, candidate sets, and interaction flow.They include five title-based text variants and three cross-channel variants using cover images, title-and-cover content, or visual perturbations.Source: §3.1.3 · p. 4

3.2 Instance Generation Pipeline

The instance-generation pipeline creates paired decision states, structured candidate annotations, controlled injections, and human-verified acceptable sets. It normalizes these materials into standardized entries for evaluation and stratified analysis.

3.2 Instance Generation Pipeline

  • The pipeline produces benign instances, injected variants, and acceptable-set annotations.Its steps localize decision points, annotate candidates, generate injections, label acceptable choices, and normalize entries for release.Source: §3.2 · p. 5
  • For each task, the pipeline records a benign decision state consisting of the critical-selection screenshot and fixed pre-selection context.Holding non-visual context constant makes the state reproducible for comparison.Source: §3.2 · p. 5
  • Candidate boundaries, visible attributes, and injection anchors are initialized by an MLLM-assisted interface, then verified and finalized by human annotators.Released annotations include stable candidate IDs and, for screenshot perturbations, pixel anchors and style parameters.Source: §3.2 · p. 5
  • Injected instances combine a decision state, target candidate, and probe variant, modifying only visible content while keeping instructions, candidates, and interaction flow fixed.Commercial-app screenshots receive overlays or embedded content; simulated apps receive backend field changes rendered by the frontend.Source: §3.2 · p. 5
  • Acceptable sets derive from candidate attributes; two annotators verify proposed deterministic-task labels, with a third adjudicating disagreements and ambiguous instances removed.The labeling pipeline is separate from evaluated GPT-5.2 agent configurations.Source: §3.2 · p. 5
  • Each released entry combines task specifications, decision state, candidate annotations, injection definitions, labels, pairings, and standardized metadata.The metadata supports cross-framework evaluation and stratified analysis.Source: §3.2 · p. 5

3.3 Unified Evaluation Protocol and Metrics

DIBench pairs benign and injected selection states under matched conditions, then separates completed selections, constraint-consistent choices, and targeted steering through distinct metrics.

3.3.1 Evaluation Protocol

  • Matched benign and injected runs differ only in injected content at the critical selection screenshot, use matched seeds, and are averaged across repeated evaluations.The default is R=10 runs per task.Source: §3.3.1 · p. 5
  • Final selections are labeled Correct when acceptable, Incorrect when outside the acceptable set, or Failure when no choice is committed.An out-of-band monitor records the selected candidate ID without affecting agent behavior.Source: §3.3.1 · p. 5

3.3.2 Metrics

  • Task Completion Rate (TCR) measures whether the agent commits to a candidate, while Correct Selection Rate (COR) measures whether its choice satisfies the instruction-induced acceptability criterion.Reported metrics are dataset-level means of per-instance probabilities.Source: §3.3.2 · p. 6
  • Attack success rate (ASR) measures whether an injected run selects its target when the paired benign run does not.Untargeted degradation is also an injection-induced error, but ASR focuses on targeted steering because it more directly reflects attack controllability.Source: §3.3.2 · p. 6

3.4 Extensible Agent Configuration Suite

DIBench uses a pluggable configuration suite to support reproducible comparisons across agent frameworks and foundation models.

3.4 Extensible Agent Configuration Suite

  • DIBench abstracts runtime differences through AgentAdapter for framework setup and selection-stage I/O alignment, and ModelProvider for model invocation and fixed inference settings.These units are intended to ensure consistent inputs and outcomes.Source: §3.4 · p. 6
  • The initial release supports four frameworks: AppAgent, Mobile-Agent-V2, Mobile-Agent-E, and Mobile-Agent-V3.The passage identifies each framework with its original bibliography marker: AppAgent [31], Mobile-Agent-V2 [48], Mobile-Agent-E [51], and Mobile-Agent-V3 [58].Source: §3.4 · p. 6
  • The suite includes seven open- and closed-source foundation models, enabling horizontal comparisons across framework–model pairings.Open models are Qwen2.5-VL(32B), Qwen3-VL(32B), GUI-Owl(32B), and Mai-UI(8B); closed models are DoubaoSeed-1.6, Gemini-3-Flash, and GPT-5.2.Source: §3.4 · p. 6

3.5 Benchmark Extension: Defense Track

DIBench adds a defense track that compares mitigations under the same tasks, injection conditions, and threat model as the no-defense baseline.

3.5 Benchmark Extension: Defense Track

  • The defense track evaluates mitigations under the no-defense baseline’s tasks, injection conditions, and threat model, using standardized hooks to preserve comparability.The track focuses on three general-purpose defense classes and can be extended to more advanced defenses.Source: §3.5 · p. 6
  • Input detection applies an LLM-based detector to selection-stage inputs to flag potential prompt injections.Inputs can include UI text or images and optionally agent-side intermediate planning; detector prompts are fixed.Source: §3.5 · p. 6
  • Input preprocessing transforms screenshots to attenuate injected semantics while preserving UI structure and task-relevant evidence.Examples include Gaussian noise, JPEG compression, and resizing.Source: §3.5 · p. 6
  • Prompt-level mitigation prepends a short, fixed risk-warning sentence to the system prompt or instruction.The interfaces are framework-agnostic and provide a low-overhead control for benchmarking instruction-level hardening in isolation.Source: §3.5 · p. 6

4 Benchmark Results

Across commercial and simulated selection tasks, deceptive injections can increase completion while steering choices and shift agents’ action patterns. Evaluations find that common defenses provide inconsistent integrity gains, while deviation strength depends on the attack and decision context.

4 Benchmark Results

  • Evaluations test whether deceptive injections steer critical selections while preserving completion in commercial-app decision states and controlled simulation apps.The settings are COMMERCE and SIMULATOR.Source: §4 · p. 6

4.1 Completion-oriented metrics overestimate agent trustworthiness

  • TCR rose from 42.0% to 72.4% for AppAgent + Qwen2.5-VL in COMMERCE, while ASR reached 45.8%, exposing hidden target steering.Completion can therefore appear improved even when the committed choice is attacker-steered.Source: §4.1 · p. 7, Table 3 · p. 7

4.2 Deceptive injections shift decision trajectories

  • Under Combined attacks, selection actions increase while delayed-decision actions decrease across frameworks and model families on COMMERCE.The target-index peak and declining Others indicate shifts from refusal or continued browsing toward target submissions, while other index ordering largely persists.Source: Figure 2 · p. 7, §4.2 · p. 8

4.3 Current general defenses remain unreliable

  • Detection, preprocessing, and prompt defenses yield inconsistent integrity gains, with detection missing naturalistic injections and preprocessing sometimes increasing ASR.Prompt defenses can reduce ASR but may be ineffective or increase it, and can incur COR drops; the authors connect the broader instability to failures of grounded evidence use and verification.Source: §4.3 · pp. 8–9

4.4 Additional analysis

  • In-task goal-deviation strength depends jointly on attack semantics, agent-framework constraints, model characteristics, injection visibility, app structure, and task type.The authors conclude robustness cannot be reliably inferred from a single attack setting or platform.Source: §4.4 · p. 9

5 Discussion and Future Work

Execution-level correctness and decision trustworthiness can diverge: agents may complete tasks while deceptive UI content redirects their choices. DIBench pairs benign and injected cases and uses decision-integrity metrics to measure this risk, while its evaluation scope leaves external validity limited.

5 Discussion and Future Work

  • UI-visible content can redirect an agent’s final choice without changing instructions or execution flow, separating task completion from decision trustworthiness.Users may observe successful completion without detecting the subtle decision manipulation.Source: §5 · p. 9
  • Paired benign/injected protocols and decision-integrity metrics enable cross-model comparison, while TCR and COR can miss manipulation that ASR captures.Trajectory analysis also shows weaker exploration and verification and earlier commitment under attack, motivating evaluation beyond outcome-level metrics.Source: §5 · p. 9
  • DIBench evaluates open research mobile-agent frameworks, not commercial or OEM-deployed assistants, limiting external validity.Future work should extend evaluation to production systems and long-horizon multi-decision tasks.Source: §5 · p. 9
  • Commercial-app evaluations use fixed decision states and screenshot-level injections, avoiding state-changing actions on external services or collection of private user data.The benchmark is released for measurement and defense research, not one-click attack deployment against external apps.Source: §5 · p. 9

A Detection Prompts

The appendix provides detector prompt variants for image-only assessment and for assessment using planning content alongside the image. Both request a binary benign-or-attack label.

A Detection Prompts

  • The security detector determines whether an image contains a prompt injection, malicious instruction, or adversarial perturbations aimed at manipulating an AI agent, and outputs 0 for benign or 1 for attack.Source: A Detection Prompts · p. 14
  • The planning-plus-image detector receives the user task, planning output, and image to assess possible visual manipulation or abnormal planning behavior.It is instructed to return a single digit: 0 for benign and 1 for attack.Source: A Detection Prompts · p. 14

B Additional Experimental Setup

The appendix reports the computing setup and identifies figures and tables used to present carrier variants, selection probabilities, and defense results.

B Additional Experimental Setup

  • Evaluations used two Ubuntu servers, including an 8× NVIDIA A100 LLM-serving backend, and a Mac mini running an Android Emulator with an emulated Pixel 8.Server-1 also hosted the app simulator.Source: B Additional Experimental Setup · pp. 14–15
  • Figure 3 depicts cross-channel carrier variants under combined-semantics deceptive injection.The probe variants include Cover-only, Title-and-Cover, and Perturbation placements.Source: Figure 3 · p. 15
  • Table 5 reports SIMULATOR results under deceptive injection, with summary-row changes for TCR and COR and maximum ASR per column.The caption states that within each metric, maximum and minimum values are highlighted.Source: Table 5 · p. 15
  • Figure 4 encodes SIMULATOR selection probability by item index, with line style distinguishing Qwen2.5-VL and Qwen3-VL and color identifying target indices.Black curves indicate No Attack, colored curves Combined Attack, and Others denotes non-selection.Source: Figure 4 · p. 15
  • Tables 6 and 7 describe COMMERCE defense comparisons: image preprocessing reports ASR, while prompt defense reports ASR and COR against No Defense.Table 6 includes Gaussian noise, JPEG compression, and resizing; Table 7 reports changes relative to the no-defense baseline.Source: Table 6 · p. 16, Table 7 · p. 16

C Analyzing Determinants of In-Task Goal Deviation

In-task goal deviation reflects the joint effects of attack semantics, framework and model characteristics, carrier placement, app structure, and task type. Across these factors, stronger semantic coverage can increase target steering, but its strength depends on the decision pathway and setting.

C Analyzing Determinants of In-Task Goal Deviation

  • In-task goal deviation is jointly governed by six factor categories: attack semantics, framework constraints, base models, carrier visibility and semantic weight, app interface structure, and task type.The analysis draws on Tables 3, 5, 8, and 9 and Figures 5 and 6.Source: C Analyzing Determinants of In-Task… · p. 17
  • In COMMERCE, Mobile-Agent-V3 + Qwen3-VL reaches 52.2% ASR under Combined versus 11.0%/12.9% under Naive/Escape; in SIMULATOR, the corresponding values are 49.8% versus 18.7%/19.3%.Context Ignore and Combined attacks provide stronger semantic coverage and are reported as more stable across environments.Source: C Analyzing Determinants of In-Task… · p. 17
  • Framework vulnerability depends on model–framework interactions: under Qwen2.5-VL, AppAgent reaches 44.5% ASR under Combined in SIMULATOR, above Mobile-Agent-V3 at 30.7%.Under Qwen3-VL, Mobile-Agent-V3 and Mobile-Agent-E show higher ASR under strong attacks in both environments than Mobile-Agent-V2 in SIMULATOR.Source: C Analyzing Determinants of In-Task… · p. 17
  • In SIMULATOR with Mobile-Agent-V3, ASR under Context Ignore/Combined is 43.3%/49.8% for Qwen3-VL, versus 5.9%/3.8% for GPT-5.2.The reported hierarchy is general-purpose open-source models, then specialized open-source models, then closed-source models.Source: C Analyzing Determinants of In-Task… · p. 17, Table 8 · p. 17
  • Title placement is typically most effective, while Perturbation yields about 6%–16% ASR and never attains the maximum across framework columns.Under Qwen3-VL+Combined, Title+Cover lowers ASR from 48.8% to 33.0% for Mobile-Agent-E, from 49.9% to 40.6% for Mobile-Agent-V3, and from 37.0% to 30.1% for AppAgent.Source: C Analyzing Determinants of In-Task… · pp. 17–18, Table 9 · p. 18
  • Figure 5 shows that strong attacks generally rank above Naive and Escape across apps, but absolute ASR varies widely, from low levels to near saturation.Foodpanda has consistently high peaks, whereas Booking and Tripadvisor remain low in most cases; app sensitivity patterns are rarely overturned.Source: C Analyzing Determinants of In-Task… · p. 18, Figure 5 · p. 19
  • Figure 6 shows increased Wrong(Target) under Combined Attack for Cost and Quality/Pop tasks, while Open-ended tasks shift toward Correct(Target) and Exact Match retains high Correct rates.Spatio-Temporal tasks show increases in both Wrong(Target) and Wrong(Non-Target); tighter constraints make steering harder, while continuous ranking tasks are more vulnerable to targeted mis-selection.Source: C Analyzing Determinants of In-Task… · p. 18, Figure 6 · p. 19
  • Strong semantic-coverage attacks raise the ASR upper bound, but stable steering also depends on model–framework alignment, readable high-weight regions, app structure, and task constraints.Attack effects are jointly amplified or suppressed within what the decision pathway permits.Source: C Analyzing Determinants of In-Task… · pp. 18–19

NeurIPS Paper Checklist

The checklist reports that the paper’s claims align with its contributions and that its benchmark and experimental procedures are documented for reproduction. It also records gaps in statistical-significance reporting and per-experiment compute-resource reporting, alongside disclosures on ethics, assets, and LLM use.

Claims and limitations

  • The checklist says the abstract and introduction accurately reflect the paper’s contributions and scope, and that Section 5 discusses limitations and future work.The paper states that its evaluation covers open research frameworks, not commercial or OEM-deployed assistants, limiting external validity.Source: NeurIPS Paper Checklist · p. 20

Reproducibility and experimental details

  • Sections 3 and 4 document the benchmark, paired evaluation, metrics, agents, models, and defense settings; Appendix B and an anonymized repository provide further reproducibility details.The paper also packages benchmark entries, annotations, injection specifications, labels, and metadata for release.Source: NeurIPS Paper Checklist · pp. 21–22

Statistical significance and compute resources

  • The checklist marks statistical-significance reporting as insufficient, although the paper reports empirical averages from a paired benchmark evaluation.It also marks per-experiment compute-resource reporting insufficient; the paper gives local infrastructure but omits runtime and total compute cost because of variable API latency, scheduling, and UI overhead.Source: NeurIPS Paper Checklist · pp. 22–23

Ethics and societal impact

  • The paper describes ethical safeguards for real-app evaluations, including screenshot-level injections only, no live-platform publication, no transactions, and no private user data collection.It identifies potential dual-use risks from deceptive UI injections and discusses safeguards against harm.Source: NeurIPS Paper Checklist · pp. 23–24

Assets and LLM use

  • The checklist reports that external assets are properly credited and used under applicable licenses or API terms, and that new-asset documentation is available in the repository.It also notes disclosure of GPT-5.2 for initial candidate parsing and acceptable-set label proposals, followed by human verification and adjudication.Source: NeurIPS Paper Checklist · pp. 25–26

Human subjects

  • The paper reports no crowdsourcing or research with human subjects, and the checklist marks IRB approval as not applicable.The stated human involvement is limited to benchmark annotation and verification.Source: NeurIPS Paper Checklist · pp. 25–26
Loading 2610.06898v1…