Jev-Mobile: Jev as an Executor for Mobile GUI Agents

• Screenshots and accessibility trees expose different interface information, and neither determines when a local controller should return control.Source: §1 · p. 1
• The study separately measures whether acceptable candidates are available, whether Jev selects them, and whether control returns appropriately.Its AndroidWorld comparators are step-wise VLM and SeeAct-V with UI-TARS grounding.Source: §1 · p. 2
• Jev-Mobile’s AndroidWorld success is close to SeeAct-V’s, while successful trajectories have lower mean latency and model API cost than both comparators.The comparison evaluates complete systems with distinct execution mechanisms, not Jev’s isolated effect.Source: §1 · p. 2
• The method combines a VLM–Jev loop, observation-bound Android candidates, and no generated post-delegation summary.Source: §1 · p. 2

• Each action yields an observation containing a screenshot, accessibility tree, and observable device context; the VLM can inspect the screenshot and tree.Source: §2 · p. 2
• Jev receives a textual tree, its local goal, prior delegation actions, and executable candidate descriptions, while hidden task parameters and evaluator answers are unavailable to both roles.Source: §2 · p. 2
• The first candidate set is deterministic and derived from the raw tree; the method does not infer missing visual coordinates, repair hierarchy, or compile semantic field-completion rules.Those omissions are measured as candidate-coverage or handoff failures.Source: §2 · p. 2

3.1 Delegation state and control flow

• The VLM emits a local goal and optional exact text values, after which Jev can execute several atomic actions before returning control for another goal or reporting blocked.Neither DONE nor BLOCKED establishes task success; an independent evaluator judges the terminal state.Source: §3.1 · p. 2

3.2 Candidates from the current accessibility tree

• Candidates come from visible, enabled tree nodes with valid bounds and explicit action flags; after each action, a fresh observation rebuilds the set and invalidates old IDs and coordinates.The first version cannot synthesize a coordinate action for a target absent from the tree.Source: §3.2 · pp. 2–3

3.3 Typed local decision and event history

• Jev makes one typed choice among current candidate IDs, DONE, and BLOCKED; the adapter validates the output, and handoff history records submitted actions rather than predicted actions or model-written summaries.Older history may be compacted when it exceeds the configured character limit.Source: §3.3 · p. 3, Figure 1 · p. 3

3.4 Budgets, failures, and implementation boundary

• The implementation logs budget events and lacks a coordinate-producing visual fallback, semantic field binding, and a structured completion predicate.A visual target without an executable tree candidate may remain unreachable even if the VLM reinterprets the goal.Source: §3.4 · p. 3

• Mobile benchmarks and datasets cover interactive tasks, demonstrations, cross-app trajectories, resource or state constraints, and longer workflows; this study uses AndroidWorld’s independent terminal score.Trace labels diagnose failures but do not replace that score.Source: §4 · pp. 3–4
• Prior systems use exploration, reusable procedures, app structure, maps, multimodal history, or execution-state tracking; Jev-Mobile instead regenerates candidates from the live tree without a persistent map.A missing tree target is categorized as a coverage failure, distinct from choosing the wrong available candidate.Source: §4 · p. 4
• Prior work studies trained control, reinforcement learning, hierarchy, and device–cloud division; hierarchy or lightweight execution alone is not Jev-Mobile’s novelty claim.Its focus is typed, per-screen choices over program-generated IDs with explicit VLM handoff.Source: §4 · p. 4
• Screenshot-based grounding and visual GUI agents offer an alternative action path, while Jev-Mobile cannot execute targets missing from its accessibility-tree candidates.The SeeAct-V implementation substitutes UI-TARS-1.5-7B for UGround, as disclosed in the experiment design.Source: §4 · p. 4
• Peer-reviewed GUI-agent work had not systematically studied typed decision models such as Jev as GUI execution models, motivating an AndroidWorld evaluation of success, time, and cost.The stated claim concerns executor design, not hierarchical GUI control broadly.Source: §4 · p. 4

5.1 Benchmark and compared systems

• The full AndroidWorld comparison includes Step-wise VLM, SeeAct-V, and Jev-Mobile, with the systems using distinct execution mechanisms.All general VLM roles use Qwen3.8-Max; SeeAct-V uses UI-TARS-1.5-7B as its grounder, while Jev-Mobile delegates candidate selection to Jev.Source: §5.1 · p. 4

5.2 Metrics and accounting

• Success rate averages terminal success over evaluated tasks, while time and cost metrics are conditional on successful trajectories.Total time includes planning, execution-model requests, device interaction, waits, and retries; cost accounting distinguishes executor-model API charges from all-model cost per success.Source: §5.2 · pp. 4–5

Results

• 0.79 of tasks are completed by Jev-Mobile, versus 0.78 for SeeAct-V and 0.84 for Step-wise VLM.Success rate covers all evaluated task instances; timing and dollar figures use successful trajectories.Source: §6 · p. 5, Table 1 · p. 5
• 32.7% lower mean total online time than Step-wise VLM is reported for Jev-Mobile’s successful trajectories.Mean time is 132.67 seconds for Jev-Mobile versus 197.21 seconds for Step-wise VLM; mean model API cost per successful trajectory is $0.072744 versus $0.273694.Source: §6 · p. 5, Table 1 · p. 5

Results

• 0.79 full-task success shows Jev can select actions as an executor in a VLM-guided mobile GUI agent.This is close to SeeAct-V’s 0.78, while Step-wise VLM reaches 0.84.Source: §7 · p. 5

Execution efficiency

• 32.7% lower mean total time, 94.5% lower execution-model time, and 73.4% lower mean all-model API cost are reported versus Step-wise VLM.Jev selects several actions under one VLM-provided local goal, while the candidate builder refreshes executable choices after each action; the authors say this division of labor is consistent with the short execution-model time observed.Source: §7 · p. 5

Limitations

• The evaluation covers AndroidWorld mobile tasks only, leaving web interfaces untested.The authors note that web DOM structure, page dynamics, and interaction patterns may place different demands on a decision-model executor.Source: §8 · p. 5
• The workflow has not been tested with Jev replaced by a small VLM, and visually apparent targets missing from the accessibility tree cannot become executable candidates.The present input path supports printable ASCII text, and long action histories may require compaction.Source: §8 · p. 5

• 0.79 task success on the full AndroidWorld suite is close to SeeAct-V’s 0.78 and below Step-wise VLM’s 0.84.Source: §9 · p. 5
• Jev-Mobile delegates current-tree action selection to a typed decision model while retaining a VLM for task interpretation and local-goal setting.Source: §9 · p. 5
• The results support Jev as a viable mobile GUI execution model; future work can test web interfaces and compare Jev with a small VLM inside the same delegated controller.Source: §9 · p. 5