PaperlayerView exact revision on arXiv

Source-linked AI summary

MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments

Panrong Tong, Quyu Kong, Pengxiang Li, Tianyu Xia, Hanzhang Zhou, Lei Wang, Chenglin Cai, Long Li, Long Chen, Jianan Zhang, Zhengwen Zeng, Xu Zhang, Gongjie Zhang, Yue Wang, Steven Hoi

arXiv:2610.20939v1cs.AI

TL;DR

GUI-agent RL is difficult to scale because interactive environments are costly and realistic training tasks are scarce. MAI-Gym unifies evaluation, data collection, and online RL while synthesizing apps and tasks; MAI-Gym-9B achieves state-of-the-art results among open GUI agents under 10B parameters.

  • Problem

    GUI-agent RL remains difficult to scale because interactive environments are costly and realistic training tasks are scarce.

    Source: §1 · p. 3

  • Method

    MAI-Gym unifies evaluation, data collection, and online RL across mobile, computer, and web environments, and synthesizes resettable apps and cross-app tasks.

    Source: §1 · pp. 3–4

  • Results

    78.4% on MobileWorld and 76.2% on OSWorld-Verified give MAI-Gym-9B state-of-the-art results among open GUI agents under 10B parameters.

    Source: §1 · p. 4, §6 · p. 20

  • Takeaways & Limitations

    RL in a mocked train-ticketing app raised success on 52 real-app tasks evaluated on a physical phone from 28.8% to 48.1%.

    Source: §1 · p. 4

    The paper does not evaluate repeated rounds of failure-driven task generation.

    Source: §3.3 · p. 10

Abstract

from arXiv · show

Online reinforcement learning (RL) with verifiable rewards has driven rapid progress in mathematics, coding, and tool use, but scaling this success to GUI agents remains challenging due to the high cost of interactive environments and the scarcity of realistic training tasks. We introduce MAI-Gym, an open all-in-one platform that unifies GUI agent evaluation, data collection, and online RL training across mobile, computer, and web environments. Through a shared rollout pipeline, MAI-Gym ensures consistent agent behavior across evaluation and training, supports ten established benchmarks, and scales to 20,000 parallel environment instances. Beyond scalable infrastructure, MAI-Gym enables automated synthesis of diverse GUI applications and challenging tasks, including resettable mocked apps and coherent cross-app worlds with execution-validated task verifiers. Using MAI-Gym, we train MAI-Gym-9B, a unified 9B agent for mobile and computer use, achieving state-of-the-art performance among open GUI agents under 10B parameters: 78.4% on MobileWorld and 76.2% on OSWorld-Verified. Importantly, RL in a mocked train-ticketing app transfers to its real-world counterpart, improving task success on a phys- ical phone from 28.8% to 48.1%. We release the framework, synthesis skills, mocked apps, 2,000 verified tasks, and the model checkpoint to support repro- ducible research and further advances in scalable GUI agent training.

1 Introduction

MAI-Gym addresses the cost and consistency challenges of GUI-agent RL with a shared platform for evaluation, data collection, and training, plus synthesized apps and tasks. The paper reports strong results for MAI-Gym-9B and releases resources for reproducible research.

1 Introduction

  • GUI-agent RL is costly to scale and maintain consistently, while realistic workflows need diverse tasks grounded in coherent, resettable states across apps.Existing gyms tend to focus either on reproducible infrastructure or on new apps and tasks, leaving a need to unify scalable execution and synthesis.Source: §1 · p. 3
  • MAI-Gym uses a shared rollout path for evaluation, data collection, and online RL across mobile, computer, and web environments, supporting ten benchmarks and 20,000 parallel instances.The platform reuses observation processing, context construction, and action execution across these activities.Source: §1 · pp. 3–4
  • MAI-Gym synthesizes resettable mocked apps and cross-app tasks grounded in coherent worlds, with task requirements set before execution-validated code verifiers are generated.Failure-driven synthesis uses unsuccessful agent trajectories to target weaknesses.Source: §1 · pp. 3–4
  • RL in a mocked train-ticketing app raised success on 52 real-app tasks evaluated on a physical phone from 28.8% to 48.1%.Training on novel apps also raised held-out task success from 25.8% to 45.2%.Source: §1 · p. 4
  • 78.4% on MobileWorld and 76.2% on OSWorld-Verified establish state-of-the-art results for MAI-Gym-9B among open GUI agents under 10B parameters.RL also improves performance over the SFT warm-up checkpoint across five benchmarks outside the training environments.Source: §1 · p. 4
  • The release includes evaluation and online-RL code, synthesis skills, 49 mocked apps with tasks, and 2,000 execution-tested tasks.The tasks include 1,000 each for MobileWorld and OSWorld.Source: §1 · p. 5

2 Preliminaries

This section defines executable GUI tasks, the agent’s harness and action space, and online RL over a task pool. Tasks pair an environment and initial state with an instruction and verifier; agents interact through a harness, and training requires fresh rollouts.

2.1 Executable Tasks in GUI Agents

  • Each executable task pairs an environment, initial state, instruction, and verifier; changing the initial state can change the correct outcome for the same instruction.Each attempt runs in an isolated instance reset to that initial state.Source: §2.1 · p. 5
  • Task reward gives partial credit across weighted binary criteria, while success requires every required criterion; checks can use code or a VLM judge.The verifier checks the final state and, when needed, the trajectory.Source: §2.1 · p. 5

2.2 Agent Harness and Action Space

  • The harness builds context from the instruction, interaction history, and observation, parses policy output into actions, and executes them; computer agents can also use CLI commands, while mobile and web use GUI only.A step may emit an ordered batch of GUI and CLI actions; failed commands return error feedback rather than ending the episode.Source: §2.2 · p. 5, Table 2 · p. 5

2.3 Online Agentic RL

  • Online RL trains on fresh rollouts from the current policy, and the recipe uses GRPO’s group baseline; groups with identical rewards have zero advantage and contribute no gradient.Each rollout resets an instance to the task’s initial state and ends with a single terminal reward.Source: §2.3 · p. 6

3 MAI-Gym

MAI-Gym organizes GUI evaluation and online RL around a shared rollout path, then loops from evaluation and diagnosis through synthesized practice and retraining.

3.1 System Architecture

  • One harness and rollout path serve evaluation, data collection, and RL; an environment manager exposes common operations across mobile, computer, and web deployments.Data collection stores trajectories without a policy update, while RL training uses them to update the policy.Source: §3.1 · pp. 6–7, Figure 2 · p. 7

3.2 Scalable and Efficient Rollouts

  • MAI-Gym has tested 20,000 parallel instances and combines platform-specific resets with asynchronous warm-pool provisioning and windowed histories that reuse KV caches.Window-wise histories also underpin training samples, with loss applied only to new outputs.Source: §3.2 · pp. 8–9, Figure 3 · p. 8

3.3 Evaluation and Diagnostics

  • Evaluation preserves benchmark-native metrics and reruns infrastructure failures; dashboards trace scores to attempts, while agent diagnosis targets weak capabilities, though repeated loops are not evaluated.The dashboard aligns screenshots, model outputs, actions, and verifier results for each attempt.Source: §3.3 · pp. 9–10

3.4.1 Environment Synthesis

  • Synthesis releases 49 mocked apps—seven real-world clones and 42 novel apps—with resettable workflows; novel apps are deterministic and checked against reference solutions and resets.Training in the 12306 clone improves success in its real-app counterpart on a physical phone.Source: §3.4.1 · pp. 10–11

3.4.2 Cross-App World Construction

  • Cross-app worlds use personas and scenarios to populate consistent files and app records, so tasks require combining evidence across applications.Initialization templates load and check the resulting state, which becomes the task’s reproducible s0.Source: §3.4.2 · p. 11

3.4.3 Task and Verifier Synthesis

  • Tasks target diagnosed weaknesses, fix requirements before independent golden-code and verifier synthesis, then pass reset-state execution checks and rollout review; 2,000 verified tasks are released.Verifier checks require scores of 0 initially, 1 after golden code, and 0 again after reset.Source: §3.4.3 · p. 12

3.5 Online RL Training

  • Training combines policy-adaptive sampling, window-wise sequences that avoid duplicate loss, and token-level train–rollout consistency checks gated by KL divergence.Tasks the policy always or never solves receive smaller but nonzero attempt budgets, while intermittently solved tasks receive the full group.Source: §3.5 · p. 13

4 Experiments

The experiments evaluate MAI-Gym-9B across mobile and computer benchmarks, then analyze training gains, infrastructure efficiency, joint training, synthesized-task scaling, transfer, novel-app learning, and RL algorithms.

4.2 Main Results

  • 78.4% on MobileWorld and 76.2% on OSWorld-Verified place MAI-Gym-9B among the strongest comparable-size GUI agents on unseen tasks in training environments.It leads similarly sized open agents on both benchmarks; only Qwen-CUA scores higher on OSWorld-Verified.Source: §4.2 · p. 14, Table 4 · p. 15
  • 75.9 on AndroidWorld and 46.5 on MobileGym keep MAI-Gym-9B within 4.3 and 6.2 points of UI-Venus-2-9B; it leads similarly sized open agents on three computer Env-OOD benchmarks.Those computer scores are 3.7/11.3 on OSWorld 2.0, 41.3 on MacAgentBench, and 15.2 on MyPCBench.Source: §4.2 · pp. 14–15, Table 4 · p. 15
  • 44.1 to 50.5 Cross Avg follows joint RL, while SFT raises Task-OOD Avg from 29.9 to 66.4 and Env-OOD Avg from 18.8 to 32.3.RL improves both domain averages and every Env-OOD benchmark.Source: Figure 6 · p. 15, §4.2 · p. 15

4.3.1 Infrastructure Runs Efficiently at Scale

  • 328.4 seconds brings up 1,000 mobile environments, at least 6× faster than the estimated serial KVM-style setup; window-wise history cuts update time from 2166.4 to 810.9 seconds.Window-wise history increases prefix-cache reuse and lowers training token cost, while moderately reducing rollout throughput.Source: Table 5 · p. 16, Table 6 · p. 16, §4.3.1 · p. 17

4.3.2 Joint RL Beats Expert Merging

  • 78.4% on MobileWorld and 76.2% on OSWorld-Verified exceed the best expert merges, which reach 76.4% and 74.8%, respectively.No single merge leads in both domains.Source: Table 7 · p. 17, §4.3.2 · p. 17

4.3.4 Clone Training Transfers to the Real App

  • 48.1% real-app success after clone-only RL, up from 28.8%, indicates transfer to the physical 12306 app, though the clone–real gap widens to 25.0 points.The comparison uses 52 paired tasks; clone success rises from 36.5% to 73.1%.Source: Figure 9 · p. 17, §4.3.4 · p. 18

4.3.3 Scaling Synthesized Tasks Improves Generalization

  • 69.2% to 78.4% on MobileWorld and 69.7% to 76.2% on OSWorld-Verified accompany scaling synthesized training tasks from 500 to 5,000.The rollout budget is held constant, and benchmark tasks are not used in training.Source: Figure 8 · p. 17, §4.3.3 · p. 18

4.3.5 Agents Learn Unfamiliar Apps from Reward

  • 45.2% success on held-out novel-app tasks follows RL, versus 25.8% before; SAO shows no clear overall training-time advantage over GRPO in the tested configurations.Novel-app RL uses verifier rewards without demonstrations; it also reduces budget-exhausting attempts from 27 to 5 and mean trajectories from 33.3 to 16.4 steps.Source: Figure 10 · p. 17, §4.3.5 · p. 18, Figure 12 · p. 19

5 Related Work

Related work covers GUI-agent RL recipes, infrastructure and task-supplying gyms, and approaches to synthesizing environments and verifiable tasks.

RL for GUI Agents

  • Prior GUI-agent RL work often contributes a domain-specific recipe and its own environment stack, while some systems train one policy across domains or merge separate domain experts.MAI-Gym is presented as a shared open platform for running recipes across domains and training mobile and computer tasks together.Source: §5 · p. 19

GUI Gyms and Training Infrastructure

  • Open GUI gyms divide between infrastructure for running agents and data-focused systems supplying tasks, while general RL frameworks do not manage GUI environments.Separate evaluation and training harnesses can change a fixed model’s score.Source: §5 · p. 20

Environment and Task Synthesis

  • GUI task synthesis trades reward reliability against scope; MAI-Gym combines cross-app worlds, rollout-failure diagnoses, fixed success criteria, and execution-tested code verifiers.Its mocked apps include real-world clones and novel apps for studying unfamiliar-interface adaptation.Source: §5 · p. 20

6 Conclusion

The conclusion presents MAI-Gym as a shared platform for GUI evaluation and online RL, reports MAI-Gym-9B’s benchmark performance and transfer findings, and lists released resources.

6 Conclusion

  • 78.4% on MobileWorld and 76.2% on OSWorld-Verified are the best reported results among open GUI agents under 10B parameters.RL improves on the SFT warm-up across all five benchmarks outside the training environments.Source: §6 · p. 20
  • Evaluation, data collection, and training share one rollout path and agent harness across mobile, computer, and web, with ten benchmarks and 20,000 parallel instances.The platform supplies mocked apps, cross-app worlds, and tasks with execution-tested code verifiers.Source: §6 · p. 20
  • Clone-only RL raises success on a physical phone from 28.8% to 48.1%; the release includes the framework, synthesis skills, mocked apps, 2,000 verified tasks, and checkpoint.The conclusion also reports that joint training outperforms expert merging and larger synthesized task pools strengthen the policy.Source: §6 · p. 20

A.1 The MAI-Gym Dashboard

The dashboard provides views for fleet capacity, runs, benchmarks, task results, action replay, and model output.

A.1 The MAI-Gym Dashboard

  • Fleet capacity, usage history, and sessions are shown in dashboard panel (a).These views track capacity and usage alongside sessions.Source: Figure 13 · p. 27, §A.1 · p. 27
  • Panel (b) presents runs, evaluation rollouts, and training batches.The dashboard also supports run navigation and step-by-step attempt replay.Source: Figure 13 · p. 27, §A.1 · p. 27
  • Panel (c) lists benchmarks, scores, and runs, while panel (d) shows task and verifier results, action replay, and model output.The dashboard includes step-by-step attempt replay.Source: Figure 13 · p. 27, §A.1 · p. 27

A.2 Gym Interface and Environment Contract

MAI-Gym separates environment-specific operations from shared environment management. Leases and caller-supplied identifiers protect sessions and mutating operations from stale workers and duplicate retries.

A.2 Gym Interface and Environment Contract

  • Each environment adapter implements provisioning, observation, reset, action execution, and task verification operations.The environment manager handles allocation, session management, trajectory recording, and reward delivery above that shared contract.Source: §A.2 · p. 27
  • The environment manager implements allocation, session management, trajectory recording, and reward delivery across adapters.Source: §A.2 · p. 27
  • Leases prevent replaced workers from continuing to act, while caller-supplied identifiers prevent retried mutations from applying an action twice.Source: §A.2 · p. 27

A.3 Action Space

MAI-Gym’s action vocabulary combines GUI actions with command- and API-based operations, user interaction, and task termination.

A.3 Action Space

  • MAI-Gym’s action vocabulary combines coordinate-based GUI operations with CLI and API calls, user interaction, and task termination.Source: §A.3 · p. 27, Table 9 · p. 28

A.4 Hyperparameters

The shared experimental setup initializes a warmed-up Qwen3.5-9B policy and uses GRPO by default. It specifies optimization, task-sampling, and evaluation settings, with individual experiments allowed to differ.

A.4 Hyperparameters

  • The policy starts from a Qwen3.5-9B backbone warmed up by supervised fine-tuning on mixed mobile and computer data.The joint RL run and expert runs start from this warm-up checkpoint.Source: §A.4 · p. 28
  • GRPO is the default algorithm, using group size 8 and a per-token loss that assigns each trajectory token its advantage.Source: §A.4 · p. 28
  • The objective includes a KL loss coefficient of 10^-3 and no entropy bonus; KL constrains the loss, not the reward.Source: §A.4 · p. 28
  • Training uses a learning rate of 10^-6 and weight decay 0.1; history uses windows of K=10 steps with overlap O=2, and the trainer counts windows rather than steps or trajectories.Source: §A.4 · p. 29
  • Task sampling uses a success threshold of 0.8, with budgets of 4 attempts for never-solved tasks and 2 for mastered tasks, against a full group of 8.Source: §A.4 · p. 29
  • Evaluation uses each benchmark’s verifier and step limit, with temperature 0.0 decoding and 3 trials per task.Source: §A.4 · p. 29

A.5 Training Configuration

A single YAML file organizes the run’s task pools, environments, algorithm, rollout budget, history format, and training backend. It leaves experiment-specific values configurable and reads credentials from the environment at load time.

A.5 Training Configuration

  • A single YAML file configures task pools and environments, algorithm, rollout budget, history format, and training backend.Values in angle brackets are set per experiment, and credentials are read from the environment at load time.Source: §A.5 · p. 29
  • The configuration identifies an SFT warm-up checkpoint, policy variant, tool parser, and mobile and computer task pools with their environment backends.Source: §A.5 · p. 29
  • The listing exposes GRPO settings, policy-adaptive sampling budgets, rollout concurrency and limits, and history-window options.The listed defaults include group size 8, learning rate 1.0e-6, KL loss coefficient 1.0e-3, window 10, and overlap 2.Source: §A.5 · p. 29
  • Screenshot transport can be configured for inline transfer or object-storage references.Source: §A.5 · p. 30

A.6 Synthesized Task Examples

The examples illustrate synthesized desktop and mobile tasks, alongside descriptions of real-world app clones and a collection of deterministic novel apps with executable checks.

A.6 Synthesized Task Examples

  • Synthesized desktop tasks combine information across files for spreadsheet and presentation work, while the mobile street-tree task applies counting and unit-conversion rules from a generated protocol.Source: §A.6 · p. 30

A.7 Mocked Apps

  • MAI-Gym releases seven real-world app clones and 42 novel apps.Source: §A.7 · p. 30

A.7.1 Real-World Clones

  • Seven app clones reproduce main user flows over resettable backends; testers compared screens while carrying out the same tasks in clones and real counterparts.Source: §A.7.1 · p. 30

A.7.2 Novel Apps

  • 42 novel apps provide 840 three-level tasks with executable checks and reference sequences, validated for completion, reset restoration, and rejection of untouched or random-action states.Source: §A.7.2 · p. 31

Related papers

Loading 2610.20939v1…