PaperlayerView exact revision on arXiv

Source-linked AI summary

MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research

Dingbang Wu, Rui Hao, Haiyang Wang, Shuzhe Wu, Han Xiao, Zhenghong Li, Bojiang Zhou, Zheng Ju, Zichen Liu, Lue Fan, Zhaoxiang Zhang

arXiv:2605.26114v2cs.AIcs.CL

TL;DR

Everyday-app GUI-agent research lacks environments that combine verifiable state-based outcomes with scalable online training. MobileGym addresses this gap with a controllable, browser-hosted simulation platform and benchmark, yielding a +12.8 percentage-point GRPO gain on its 256-task test set.

  • Problem

    Existing GUI-agent environments lack a practical combination of deterministic outcome signals for evaluation and rewards and scalable online training across everyday apps.

  • Method

    MobileGym provides a browser-hosted, fully controllable simulation platform with structured JSON state, declarative tasks, deterministic judges, and parallel rollouts for everyday mobile apps.

  • Results

    +12.8 percentage points was gained by GRPO on the 256-task test set, while real-device execution retained 95.1% of the simulation-side training gain on a 59-task subset.

  • Takeaways & Limitations

    MobileGym makes everyday mobile tasks available for reproducible GUI-agent research and scalable online reinforcement learning.

  • Takeaways & Limitations

    Because MobileGym models interaction semantics rather than real service backends, backend-only or stochastic phenomena are not captured unless explicitly modeled as controllable state.

Abstract

from arXiv · show

We present MobileGym, a browser-hosted, lightweight, fully controllable environment for everyday mobile use, targeting interaction fidelity without replicating proprietary backends. It enables two capabilities previously out of reach for everyday apps: verifiable outcome signals through deterministic state-based judging over structured JSON state, and scalable online RL through low-cost parallel rollouts. The full environment state is captured, configured, forked, and compared as structured JSON, and a single server can host hundreds of parallel instances, with about 400 MB memory per instance and about 3 s cold start. A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale, and a single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense RL rewards. The accompanying MobileGym-Bench provides 416 parameterized task templates, including 256 test and 160 train templates, over 28 apps, with deterministic judges and a structured AnswerSheet protocol that avoids free-text matching failures. In a Sim-to-Real case study, GRPO on Qwen3-VL-4B-Instruct gains +12.8 percentage points on the 256-task test set, and on a 59-task real-device signal subset, real-device execution retains 95.1% of the simulation-side training gain. Project page: https://mobilegym.github.io.

1 Introduction

MobileGym addresses the control, verifiability, and scalability gaps dividing emulator-based and real-device environments by providing a lightweight, browser-hosted Android-like simulator with fully programmable structured state. Its benchmarks and empirical studies demonstrate deterministic evaluation, scalable parallel training, and transfer to real-device signals.

  • Motivation: Existing environments trade off repeatability and scalability against everyday-app coverage, making episodes difficult to control, reproduce, and parallelize.Emulators mainly cover system utilities and open-source apps, while real-device benchmarks incur costs from accounts, backend state, app-version drift, devices, and real-world consequences.
  • Motivation: Unreadable, unwritable, and unforkable everyday-app state prevents reliable outcome checking, resettable evaluation, and parallel online training.Internal state is difficult to inspect, task-relevant state is distributed across proprietary storage and services, and identical initial conditions are needed for group-based methods such as GRPO.
  • MobileGym: MobileGym uses structured JSON for app data, OS state, and device context, while agents observe only screenshots and researchers retain programmatic control.The simulator targets interaction fidelity rather than proprietary-backend replication, producing realistic screens in response to discrete agent actions.
  • MobileGym: 400 MB RAM per browser instance and about 3 s cold starts enable a single machine to host hundreds of parallel instances.The platform is browser-hosted, lightweight, Android-like, sandboxed, and includes 12 everyday apps plus 16 system apps.
  • Platform and benchmark: Structured state supports deterministic judging, snapshot-based rollout forking, side-effect detection, and typed AnswerSheets that avoid free-text matching failures.These mechanisms make state readable, writable, forkable, and verifiable for configuration, reset, parallel rollouts, and outcome checking.
  • Platform and benchmark: 416 task templates comprise 256 test and 160 train tasks, while validation spans 9 agents, GRPO, real devices, and VLM-judge auditing.Results include 9.4%–58.8% success rates, +12.8 points on the 256-task test set, 95.1% simulated-gain retention, and 10.2% VLM-judge misjudgment.

2 Related Work

Prior mobile GUI work spans real-device and emulator environments, trajectory-replay and image-generation simulators, and benchmark suites targeting complementary task dimensions. Related research also establishes verifiable environments beyond mobile and explores online RL for GUI agents.

  • Real-device and emulator route: Real-device and emulator environments judge mobile GUI tasks externally through adb, accessibility trees, UI-tree or XPath rules, and VLM-based screenshot judges.On system utilities and open-source apps, deterministic verification is feasible.
  • Other mobile GUI benchmarks: SPA-Bench, Mobile-Bench, ProBench, MVISU-Bench, UI-NEXUS, and ColorBench contribute mobile GUI task suites across axes orthogonal to environment infrastructure.These benchmarks inform MobileGym-Bench’s taxonomy design.
  • Synthesis and trajectory-replay environments: GUI-Genesis reconstructs real apps as lightweight web environments with code-native rewards, whereas UISim and ViMo use image generation with potentially accumulating long-horizon visual prediction errors.GUI-Genesis covers only a single task trajectory per environment, limiting suitability for RL with deterministic state transitions.
  • Verifiable environments in other domains: Verifiable interactive environments extend beyond mobile to web, desktop operating systems, and simulated Python APIs.Examples include WebShop, WebArena, VisualWebArena, WebGym, AutoWebWorld, InfiniteWeb, OSWorld, macOS-World, and AppWorld.
  • RL-based GUI agent training: GUI-agent training research demonstrates online-RL benefits, scales rollouts with thousands of VMs, reports 77.6% SOTA on AndroidWorld, and explores multi-platform, curriculum-style, and R1-style training.DigiRL compares online RL favorably with SFT; UI-TARS-2 enables large-scale rollouts; UI-Venus-1.5 uses full-trajectory online RL and model fusion; GUI-Owl-1.5 introduces MRPO.

3 The MOBILEGYM Platform

MOBILEGYM is a browser-hosted Android-like simulation environment designed around interaction-surface fidelity and fully structured, controllable state. Its state model supports exact reset, snapshotting, forking, comparison, and deterministic judging for reliable task outcomes and side-effect detection.

  • Structured state: MOBILEGYM represents app data, OS settings, and device properties as structured state that can be configured, reset, snapshotted, forked, and compared.This structured state is the foundation for controllable environment manipulation.
  • Interaction fidelity: The platform targets GUI-agent interaction surfaces—including screens, touch, typing, navigation, handoffs, and task-relevant transitions—rather than proprietary backends or pixel-level Android internals.It supplies Android-like mechanisms such as task stacks, keyboard, notification, permission, resource-sharing, and intent-routing flows.
  • Declarative navigation: Each app’s declarative finite-state-machine specification drives runtime navigation, static analysis, trajectory enumeration, and automatic task generation.The specification is built into a per-app file at development time.
  • Verification and judging: Programmatic judges inspect environment state to provide deterministic, finegrained outcome signals, while full-state comparison identifies mutations outside expected outcomes as unexpected side effects.This can distinguish completing a requested goal from also performing an unintended action, such as sending an unintended message.
  • Layered state model: MOBILEGYM composes read-mostly World Data, a per-environment Runtime Overlay, and OS Runtime into structured state supporting snapshot, reset, fork, and deterministic state-diff judging.The layered design separates large world data from mutable runtime state while preserving controllable views.

4 The MOBILEGYM-BENCH

MOBILEGYM-BENCH is a 416-template benchmark spanning 28 apps, with disjoint train/test splits and a structured taxonomy for scope, objective, composition, difficulty, and capabilities. Runtime parameterization, AnswerSheet-based judging, and diagnostic metrics support diverse, reliable evaluation of mobile GUI agents.

  • Task taxonomy: Tasks are categorized along four orthogonal axes: scope, objective, composition, and difficulty, with L1–L4 difficulty calibrated using eight reference models.Scope ranges from single-app to three-or-more-app tasks; objectives distinguish operate, query, and hybrid tasks.
  • Task taxonomy: Each task additionally receives 1–4 capability tags from a 13-tag vocabulary.The taxonomy and tag definitions are provided in Appendix E.
  • AnswerSheet judging: AnswerSheet replaces free-text heuristics with typed GUI fields and type-specific checks, reducing rejection of equivalent answers and acceptance of leaked reasoning.Query-task agents submit answers by filling an AnswerSheet form with declared field types and format hints.
  • Task instantiation: Runtime instantiation varies instructions, sampled parameters, and environment configuration, increasing task diversity while preserving template structure.Variation sources include equivalent goal phrasings, curated or state-derived slot values, and configured app state.
  • Evaluation metrics: Success Rate is primary, while Progress Rate, False Complete, Unexpected Side Effects, and Overdue Termination diagnose partial progress and execution failures.Evaluation uses fixed step budgets, with task-specific budgets of 15, 30, 45, or 60 steps and an additional 15 steps for AnswerSheet tasks.

5 Experiments

Experiments show that MOBILEGYM differentiates agents across calibrated difficulty levels, exposes unintended state changes, and supports scalable evaluation. GRPO training improves simulation and real-device performance, retaining 95.1% of the simulation-side gain.

  • MOBILEGYM-BENCH results: Overall SR spans 9.4%–58.8%, while L4 retains meaningful performance only for Gemini 3.1 Pro at 21.9%.All 9 models decline monotonically from L1 to L4; other proprietary models reach at most 6.2%, and open-source GUI specialists at most 1.9%.
  • MOBILEGYM-BENCH results: USE ranges from 4.7% to 14.5% across models and differs nearly 2× among open-source GUI specialists with similar SRs.MOBILEGYM’s full-environment state comparison detects off-target changes hidden from screenshot or UI-tree judges.
  • Sim-to-real transfer: Training raises test-set SR from 9.4% to 22.2% (+12.8 pt) after GRPO fine-tuning of Qwen3-VL-4B-Instruct.Training used the 160-task train set for 10 steps with 96 parallel environment instances and a PR-shaped dense reward with multiplicative penalties.
  • Sim-to-real transfer: 95.1% retained gain: real-device pass rate increases from 32.2% to 72.9% (+40.7 pt), versus 33.9% to 76.7% (+42.8 pt) in simulation.The evaluation used 59 signal-bucket tasks on a Redmi Note 12 Turbo; stable-fail controls produced 0/15 success for both models.
  • Sim-to-real transfer: Successful operate trajectories are 5.00 vs. 6.03 steps for base and 10.08 vs. 12.20 for trained models in simulation versus on device.Query/hybrid tasks are excluded because AnswerSheet adds simulator-only steps.
  • Scalability: 256 parallel instances used <10% CPU and ∼100 GB RAM, completing the 256-task benchmark in about 6 minutes on one server.MOBILEGYM uses roughly 1/10 the memory and under 1/100 the disk footprint of emulator-based setups.

6 Conclusion

MOBILEGYM makes everyday mobile use a fully controllable simulation platform for GUI agent research by prioritizing interaction fidelity over proprietary-backend replication. Its readable, writable, and forkable state supports deterministic verification, reset and configuration, and parallel online RL, enabling reproducible and scalable research without real accounts, device farms, or proprietary backends.

  • MOBILEGYM turns everyday mobile use into a fully controllable simulation platform for GUI agent research.
  • Interaction-fidelity simulation makes everyday-app state readable, writable, and forkable for deterministic verification, reset and configuration, and parallel online RL.
  • MOBILEGYM enables reproducible and scalable everyday mobile-task research without real accounts, device farms, or proprietary backends.

Limitations

MobileGym prioritizes controllable, deterministic interaction semantics over perfect visual, backend, and feature-surface fidelity. These limits may affect icon-dependent transfer, exclude backend-only phenomena, and leave less common app features out of scope.

  • Visual appearance modeling: Visual differences mainly involve subtle layouts, animations, and app-specific icons, which may affect transfer on exact-icon recognition tasks.The Sim-to-Real experiment provides one quantitative datapoint supporting behavioral policy transfer at this similarity level.
  • Backend and dynamic-content modeling: MobileGym represents server-driven content as controllable JSON state, excluding backend-only or stochastic phenomena such as live recommendations, fraud checks, and latency spikes.This design favors deterministic reset, reproducible evaluation, and stable RL reward signals.
  • Functional coverage of simulated apps: Each simulated app covers main everyday-use scenarios rather than its real counterpart’s full feature surface, leaving less common features out of scope.Expanding within-app coverage is identified as future work.

Ethical Considerations · A System Implementation Details · A.1 TaskManager and the Activity stack

MobileGym is a sandboxed, offline research environment that avoids real services, accounts, funds, and personal data while supporting evaluation of high-risk GUI operations. Its implementation mirrors Android task and activity management, preserving backgrounded app state for faithful task switching.

  • Ethical Considerations: MobileGym is fully sandboxed, disconnecting commercial-app simulations from real services, accounts, funds, and personal data.The environment runs in the browser and is offline.
  • Ethical Considerations: Commercial-app simulations are independently implemented for academic research and model evaluation without reusing or distributing official code or client components.Their trademarks, brand names, and visual elements remain the property of their respective owners.
  • Ethical Considerations: 14 high-risk tasks comprise 7 Payment operations and 7 Test256-Risk tasks, including account deactivation, large transfers, and bulk deletions.The high-risk subset is described as double-edged because it evaluates high-consequence operation capabilities.
  • Ethical Considerations: 64.3% on Payment and 71.4% on Test256-Risk are achieved by Gemini 3.1 Pro, while smaller open-source GUI specialists remain at ≤10.7% on Payment.These results motivate careful consideration of misuse risks in high-risk GUI-agent evaluation.
  • Ethical Considerations: MobileGym is intended for capability evaluation and safety research rather than production deployment, with recommended uses including alignment, prompt-injection robustness, and refusal training.The paper acknowledges that GUI-agent training infrastructure could potentially automate malicious behavior.
  • Ethical Considerations: Its zero-consequence operations, one-click reset, and built-in difficulty levels support digital-literacy education and other positive social applications.Suggested applications include digital inclusion, customer-service training, and AI-safety education.
  • A.1 TaskManager and the Activity stack: Each app runs in its own Task with an Activity stack, while the TaskManager uses Reducer actions for launching, home navigation, recents, task closure, and activity push/pop.CLOSE_TASK closes and destroys React components.
  • A.1 TaskManager and the Activity stack: Backgrounded Activity containers use display:none rather than unmounting, preserving the React state tree during app switching.This allows an unfinished WeChat draft to remain available after switching to Alipay and returning.

A.2 State layer aligned with the data model of Android … D App Coverage and Representative Tasks

MobileGym models mobile apps with controllable, verifiable state and Android-aligned interaction semantics, while supporting declarative navigation, standardized app construction, and broad synthetic-data coverage. Its implementation spans system communication, input dispatch, action adaptation, and efficient app development across 28 simulated apps.

  • A.2 State layer aligned with the data model of Android: Refreshes preserve user data but reset runtime state, while globally registered stores enable single-call snapshots, resets, and programmatic verification.The policy treats browser refresh as device reboot and supports bit-level consistent reset.
  • A.3 Cross-app communication: Android-style cross-app communication is implemented through Intent resolution, content:// providers for Contacts, Sms, and Media, and broadcast events.Intent handling supports unique transitions, Choosers for multiple matches, and callback-based cross-app calls; providers support CRUD and change notifications.
  • A.4 Back-key dispatch: Back-key events follow priority dispatch from permission dialogs (1000) through the system shade (800), keyboard (700), app page (100), and desktop return (0).The first handler returning true consumes the event, while frame-level back locks prevent duplicate edge-gesture and backdrop-click triggers.
  • A.5 Standardized App-layer architecture: Standardized app modules declare manifests, navigation, Zustand state, and replaceable defaults, enabling OS auto-discovery while retaining app-level implementation work.Each app uses a common directory structure and MemoryRouter-based entry component.
  • A.6 Input injection and coordinate transformation: Screenshot coordinates are mapped to CSS viewport coordinates, resolved with document.elementFromPoint, and injected as PointerEvent or TouchEvent sequences.Apps may additionally declare a designViewportWidth, such as WeChat’s 412 px design width.
  • A.7 LLM-assisted app implementation workflow: Typical everyday-app simulations require about 3 to 4 person-days, system apps usually take less than 1 personday each, and total app simulation takes about 60 person-days.Vite HMR makes code edits effective in <1 second within the LLM-assisted implementation workflow.
  • B EFSM Formalization and Declaration Syntax: EFSM navigation combines UI states, actions, guards, data variables, and updates, allowing data-dependent transitions, data-driven state expansion, and compound UI states.The declarative specification executes navigation and supports consistency checking, BFS path enumeration, shortest-path enumeration, and navigation-graph construction.
  • C Full Action Space; D App Coverage and Representative Tasks: The environment supports a 17-action abstraction, adapter-mapped native action spaces, uniformly normalized coordinates in [0, 1000] × [0, 1000], 28 apps, and over 190K synthetic entities.The 28-app coverage comprises 12 everyday and 16 system apps, with over 350K structured records supporting information-dense parameterized tasks.

E Detailed Task Taxonomy … H.6 Same-outcome Trajectory-Length Breakdown

The paper defines a disjoint, parameterized mobile-task benchmark with structured AnswerSheet evaluation and configurable training procedures. Results decompose performance and sim-to-real behavior across taxonomy dimensions, task outcomes, and same-outcome trajectory lengths.

  • E Detailed Task Taxonomy: Test and Train are strictly disjoint, with 36% of Test comprising cross-app tasks for OOD-generalization diagnostics; all tasks support parameter sampling.L3/L4 tasks additionally provide 2–3 instruction variants that combine orthogonally with parameters.
  • F AnswerSheet Protocol Design Details: The AnswerSheet uses typed fields and matchers for choices, numbers, text, and repeatable multi-value lists, enabling structured answer submission.Number fields use floating-point tolerance, while text fields support exact, date, time, and duration matching.
  • F.2 Design motivation: Structured matching reduces natural-language false negatives while distinguishing single values from mixed-in alternatives through physically different GUI states.The judge compares numeric values with tolerance instead of relying on preset string-normalization rules.
  • F.3 Format expectations and hints: AnswerSheet hints specify required formats, but incorrect formats remain wrong; query tasks receive 15 additional steps to cover app switching, form filling, and submission.The resulting budgets are L1: 15+15 = 30 and L4: 60+15 = 75.
  • G Detailed Experimental Configuration: GRPO training uses Qwen3-VL-4B-Instruct with 96 parallel environment instances, while rewards derive from structured rollout progress and discount incorrect AnswerSheet submissions.The base reward is r = p, where p is the fraction of goal checks passed; incorrect submissions lose the bookkeeping credit.
  • H.1 SR by taxonomy dimension: Test-set analysis covers Success Rate across difficulty, objective, and composition, while trajectory diagnostics show successful paths clustering around 8–14 steps across models.Overall episode lengths vary substantially because weaker models more often exhaust the step budget without succeeding.
  • H.3 High-Risk subset: The High-Risk subset contains 14 irreversible or high-consequence tasks and measures completion success rather than refusal behavior.It combines 7 standalone Payment tasks with 7 high-risk tasks from test256.
  • H.5 Sim-to-Real outcome-stratified task sampling: On 59 real-device signal tasks, 8 are excluded for irreproducible operations or preset states; same-outcome operate successes are only modestly longer on real devices, while failures reflect differing stopping protocols.Trained and base real-device operate trajectories are respectively +2.12 and +1.03 steps longer; query successes are shorter because real devices omit simulator-only AnswerSheet submission, and many base failures reach the real-device step budget.

I Reference-Model Sensitivity of the L1–L4 Stratification · J Detailed VLM-Judge Misjudgment Audit

Reference-model recalibration changes L1–L4 bucket sizes but preserves separated difficulty profiles and the concentration of Sim-to-Real gains in L1–L2. The real-device VLM audit identifies 12 misjudgments across 9 tasks, while GPT-5.4 reproduces the aggregate error rate with a different distribution.

  • I Reference-Model Sensitivity of the L1–L4 Stratification: L1–L4 strata are calibrated sequentially using joint mean SR and PR thresholds over reference models, with Qwen3-VL-4B-Instruct held out.The four-model sensitivity calibration uses Gemini 3.1 Pro, Doubao-Seed-2.0-Pro, UI-Venus-1.5-8B, and Step-GUI-4B.
  • I Reference-Model Sensitivity of the L1–L4 Stratification: 20/73/83/80 versus 25/99/58/74 are the L1–L4 bucket counts under eight- versus four-model calibration, while mean SR/PR remains well separated.Eight-model means are (88.3, 90.7), (47.0, 64.0), (22.7, 38.3), (5.0, 15.0); four-model means are (89.0, 91.2), (50.4, 63.3), (24.6, 38.5), (3.5, 18.4).
  • I Reference-Model Sensitivity of the L1–L4 Stratification: +21.3/+25.4/+11.1/+0.9 pt versus +23.0/+22.5/+7.3/+0.7 pt shows that Sim-to-Real lift concentrates in L1–L2 and nearly vanishes on L4.The first sequence is from eight-model calibration and the second from four-model calibration.
  • I Reference-Model Sensitivity of the L1–L4 Stratification: 21.9% is Gemini 3.1 Pro’s L4 SR under eight-model calibration, versus ≤6.2% for every other model; under four-model calibration, Gemini remains the only model above 10% at 12.2%.All other models are ≤8.1% under four-model calibration, and the trained 4B model exceeds AutoGLM-Phone-9B on L1–L2 under both calibrations.
  • I Reference-Model Sensitivity of the L1–L4 Stratification: 9.4% →22.2%, +12.8 pt is the invariant overall SR improvement for trained versus base models because the test set remains fixed at 256 tasks.The invariance is by construction rather than a consequence of the recalibration.
  • J Detailed VLM-Judge Misjudgment Audit: 12 misjudgment instances span 9 unique tasks in the real-device audit, while 30 stable-fail trajectories across 15 tasks and 2 models incur 0 misjudgments.Qwen3.6-Plus supplies the VLM pass/fail labels, which are corrected using the manual audit.
  • J Detailed VLM-Judge Misjudgment Audit: GPT-5.4 re-judges the same saved real-device trajectories against manually audited ground truth without rerunning agents, excluding protocol-level manual exceptions.The robustness check tests whether errors are specific to Qwen3.6-Plus.
  • J Detailed VLM-Judge Misjudgment Audit: GPT-5.4 yields the same aggregate error rate as Qwen3.6-Plus, but distributes errors differently across base and trained trajectories.The distribution is reported in Table 15.

K Case Study · L Broader Uses of MOBILEGYM · M Detailed Footnotes for the Resource-Efficiency Comparison

MOBILEGYM demonstrates Sim-to-Real transfer on an unseen real-device posting constraint, supports controllable benchmark, robustness, online-RL, and data-synthesis research, and offers lightweight resource advantages over emulator-based setups. Its structured state restoration and in-process parallelism underpin these uses.

  • K Case Study: The trained model recovered from a disabled Reddit posting gate by identifying the required flair field, selecting a tag, and successfully publishing.The base model repeatedly clicked the grayed-out Post button until truncation, whereas the trained model used the required-field cue to re-plan.
  • K Case Study: The recovery occurred on a real-device condition absent from training rollouts, illustrating Sim-to-Real OOD generalization for individual interaction primitives.The case transfers learned behavior to a community-specific posting constraint while simulator-based online RL avoids the difficulty of scaling real-device training.
  • K Case Study: Across 19 AnswerSheet tasks, trained-model performance was closely aligned between simulation and real devices: sim 71.1% versus real 73.7%.The AnswerSheet adds a format-and-submission barrier without leaking answers or reducing tasks to multiple choice.
  • L Broader Uses of MOBILEGYM: MOBILEGYM’s modular apps, world data, templates, and judges support domain-specific mobile environments and benchmarks with the same reset, snapshot, and state-based judging interface.Example domains include mobile finance, travel planning, social-media safety, and digital-literacy training.
  • L Broader Uses of MOBILEGYM: Programmable state and event injection enable systematic robustness and safety evaluation across balances, permissions, network states, messages, popups, and phishing-like content.These controls support studies of prompt-injection susceptibility, caution gating, side effects, and recovery behavior.
  • L Broader Uses of MOBILEGYM: Forking identical initial states into many lightweight browser instances makes MOBILEGYM a practical testbed for reproducible online RL without large emulator clusters or real-device farms.Researchers can compare reward designs, state-diff penalties, rollout grouping, and Sim-to-Real behavior under deterministic outcome signals.
  • L Broader Uses of MOBILEGYM: MOBILEGYM generates controllable paired visual and structured state-transition data for training UI world models, state predictors, reward models, and trajectory verifiers.Each interaction step yields a five-tuple, enabling intentional state coverage rather than incidental device logs.
  • M Detailed Footnotes for the Resource-Efficiency Comparison: MOBILEGYM restores task state through direct JSON setState injection and manages multiple browser contexts within one process, avoiding duplicated OS and system-image overhead.By contrast, AndroidWorld’s task restoration uses file-level app snapshots, and concurrent sessions require independent Docker containers with linearly scaling overhead.

N Cost Table if Switching to a VLM Judge

The paper estimates VLM-judge API costs for a full 256-task evaluation run from sampled screenshot trajectories, and shows that GPT-5.4 pricing is substantially more expensive than Qwen. These estimates exclude cloud-device or emulator costs, which can dominate complete real-environment RL training.

  • Evaluation-run cost: 546 screenshots yielded an average trajectory cost of ∼29.8K input tokens and ∼924 output tokens for one 256-task evaluation run.The estimate includes screenshots and is based on a sampled VLM audit.
  • Evaluation-run cost: 8.75×: the GPT-5.4 path is roughly 8.75× as expensive as the Qwen path.The comparison uses Aliyun Bailian pricing of ¥2/M input and ¥12/M output versus OpenAI GPT-5.4 pricing of $2.50/M input and $15/M output, converted at a 7× exchange rate.
  • RL-training cost: VLM-judge API estimates cover only the judging API, while complete real-environment RL training also incurs cloud-device or emulator rental costs.GUI-Genesis reports costs as high as $240 per step for a real-environment plus VLM-reward configuration in WeChat mini-program experiments.
Loading 2605.26114v2…