Source-linked AI summary
MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment
Yucheng Shi, Wenhao Yu, Zaitang Li, Yonglin Wang, Hongming Zhang, Ninghao Liu, Haitao Mi, Dong Yu
TL;DR
GUI-agent training is constrained by reliance on offline trajectories and costly annotations, while dynamic interfaces challenge policy robustness. MobileGUI-RL trains agents online using self-explored and filtered task curricula plus trajectory-aware GRPO with rewards for success and efficiency. The framework reports consistent gains across mobile GUI benchmarks, including stronger performance from its larger model, while trajectory-level feedback remains ambiguous for distinguishing which shared actions caused success.
Problem
Offline GUI-agent training relies on static trajectories and costly detailed annotations, limiting scalable adaptation to changing interfaces.
Method
MobileGUI-RL combines an interactive online environment, self-exploration and filtering for curriculum generation, and trajectory-aware GRPO with multi-component rewards.
Results
MobileGUI-RL produces consistent gains across challenging GUI benchmarks, with MobileGUI-32B surpassing its base model and leading closed-source competitors.
Takeaways & Limitations
Online reinforcement learning from trajectory-level feedback is presented as a powerful paradigm for developing more robust and capable GUI agents.
Takeaways & Limitations
Trajectory-level rewards can provide ambiguous learning signals when successful and failed trajectories share common steps, limiting action-level credit assignment.
Abstract
from arXiv · showhide
Recently, there has been a surge of vision-based GUI agents designed to automate everyday mobile and web tasks. These agents interpret raw GUI screenshots and autonomously decide where to click, scroll, or type, which bypasses handcrafted rules and app-specific APIs. However, most existing methods trained GUI agent in the offline environment using pre-collected trajectories. This approach limits scalability, causes overfitting to specific UI templates, and leads to brittle policies when faced with unseen environment. We present MobileGUI-RL, a scalable framework that trains GUI agent in online environment. MobileGUI-RL contains two key components. It (i) synthesizes a curriculum of learnable tasks through self-exploration and filtering, and (ii) adapts GRPO to GUI navigation with trajectory-aware advantages and composite rewards that balance task success and execution efficiency. Experiments on three online mobile-agent benchmarks show consistent gains, validating the effectiveness of our approach.
1 Introduction
MobileGUI-RL addresses the scalability and robustness limits of offline GUI-agent training by learning through online interaction. It combines synthetic curriculum generation with trajectory-level reinforcement learning to improve mobile GUI agents across online and offline evaluations.
- Motivation: Offline GUI-agent training depends on static trajectories and labor-intensive step-level annotations, limiting scalability.These methods also struggle when interface elements or GUI states change unpredictably.
- Motivation: Online training enables agents to interact continuously and update policies, but requires executing and observing every action, creating substantial computational cost.The cost becomes especially challenging for complex applications and mobile devices.
- Framework: MobileGUI-RL combines self-exploration, task filtering, and trajectory-level reinforcement learning in a scalable training pipeline.Its structured rewards balance task success with execution efficiency.
- Framework: The framework synthesizes learnable tasks through self-exploration and filtering, tailoring the curriculum to the agent’s current capabilities.This addresses the need for scalable task generation in online environments.
- Results: Experiments on four mobile-agent benchmarks show improved online and offline performance, with online performance steadily increasing during reinforcement learning.The reported gains indicate that the framework improves both evaluation settings.
2 Related Work
GUI agents have shifted from multi-stage, supervised pipelines toward end-to-end LVLM-based systems and reinforcement learning. Recent work increasingly targets the long-horizon, sparse-reward, and data-cost challenges that make GUI interaction distinct from ordinary policy optimization.
- Vision-based GUI agents: LVLM-based GUI agents increasingly operate directly on screenshots and raw pixels rather than using separate planning and grounding stages (Zheng et al., 2024; Gou et al., 2025; Hong et al., 2024; Xu et al., 2025; Qin et al., 2025).This shift aims to provide a more scalable and human-like interaction framework.
- Reinforcement learning: Reinforcement learning is replacing supervised fine-tuning as a route to improve GUI-agent generalization and decision-making, following the DeepSeek-R1 paradigm (Guo et al., 2025).GUI-R1 is cited as achieving state-of-the-art results across multiple platforms with relatively limited data.
- Specialized RL methods: GUI and web agents require specialized methods because long-horizon interaction, sparse rewards, and high data costs complicate direct reinforcement learning.GiGPO targets credit assignment, ARPO targets sample efficiency, and DigiRL reduces reliance on human annotations (Feng et al., 2025; Lu et al., 2025a; Bai et al., 2024).
3 MobileGUI-RL
MobileGUI-RL trains mobile GUI agents online through a scalable environment, synthetic curriculum generation, and a GRPO-based optimization method tailored to long-horizon navigation. Its design combines self-exploration and filtering with trajectory-level advantages and rewards that jointly reflect task success and execution efficiency.
- 3.1 Overview: MobileGUI-RL provides continuous online learning through batched virtual execution and real-time interaction across mobile GUI environments.Parallel emulator instances collect diverse trajectories asynchronously while policy optimization runs separately on GPU servers.
- 3.2 Scalable and Interactable Environment: The agent observes screenshots, the task instruction, and interaction history, then emits structured mobile actions such as gestures and text input.Trajectories are evaluated at termination by a vision-language oracle using final screenshots and the initial instruction.
- 3.3 Synthetic Task Generation: The task pipeline uses self-exploration and filtering to create a dynamic curriculum of synthetic tasks matched to the agent’s evolving capabilities.Exploration discovers meaningful interface interactions, while filtering removes ambiguous or infeasible tasks before training.
- 3.4 Trajectory-Aware Policy Optimization: MobGRPO assigns a trajectory-level advantage uniformly across action steps, addressing noisy credit assignment in variable-length, long-horizon GUI trajectories.A completed trajectory receives a scalar reward, which is normalized across trajectories for the same task and shared across its steps.
- 3.4 Multi-Component Reward Design: The optimization and reward design jointly encourage successful, efficient completion while preserving learning signals when successful rollouts would otherwise receive identical rewards.The approach combines trajectory-level aggregation with efficiency-sensitive reward shaping and handles degenerate all-failure batches by filtering them during training.
4 Experiments
MobileGUI-RL is evaluated on three online mobile GUI benchmarks using scalable Android environments and 7B/32B agents. It improves base-model performance across benchmarks, while ablations show that filtering, curriculum learning, and decaying rewards each materially contribute to success.
- 4.2 Main Results: MobileGUI-RL substantially improves its Qwen2.5-VL base models across AndroidWorld, AITW-Gen, and AITW-Web.The framework is evaluated with MobileGUI-7B and MobileGUI-32B against closed- and open-source baselines.
- 4.2 Main Results: MobileGUI-7B raises AITW-Gen SR from 49.0% to 65.3% and AndroidWorld SR from 22.0% to 30.0% over Qwen2.5-VL-7B.The AITW-Gen improvement is 16.3 points, although UI-TARS-7B is slightly higher on AndroidWorld.
- 4.2 Main Results: MobileGUI-32B reaches 44.8% SR on AndroidWorld, exceeding GPT-4o at 34.5% and Qwen2.5-VL-72B at 35.0%.It also achieves 58.0% on AITW-Gen and 30.7% on AITW-Web.
- 4.3 Ablation Study: Removing task filtering reduces performance by 1.5 points for 7B and 3.8 points for 32B after pruning 1,251 generated tasks to 436 solvable, unambiguous tasks.The unfiltered variant wastes computation on unproductive trajectories and destabilizes policy optimization.
- 4.3 Ablation Study: Removing complexity-based curriculum ordering lowers success by 5 points for 7B and 10.8 points for 32B.The curriculum starts with simpler tasks and progressively introduces more complex multi-step tasks; training curves show rising early reward followed by lower reward as task difficulty increases, while final benchmark performance improves.
- 4.3 Ablation Study: Replacing the decaying reward with binary success rewards decreases performance by 6.5 points for 7B and 9.3 points for 32B.Decay differentiates successful trajectories by efficiency and preserves normalized advantage signals when all sampled trajectories succeed.
5 Conclusion
MobileGUI-RL combines online-environment training, synthetic curriculum generation, and MobGRPO to optimize GUI task success and interaction efficiency. Experiments show gains across challenging GUI benchmarks, with MobileGUI-32B surpassing its base model and leading closed-source competitors.
- 5 Conclusion: MobileGUI-RL combines a synthetic task pipeline with a text-based world model for curriculum filtering in online mobile environments.The pipeline supports data generation and adapts tasks to the agent’s capabilities.
- 5 Conclusion: MobGRPO uses trajectory-aware advantages and multi-component rewards to optimize both task success and interaction efficiency.Its reward design targets these two objectives jointly.
- 5 Conclusion: MobileGUI-32B surpasses both its base model and leading closed-source competitors across challenging GUI benchmarks.The reported gains come from fine-tuning strong base models with MobileGUI-RL.
6 Future Directions
Future work targets more realistic long-horizon tasks, finer-grained reward supervision, visual lookahead planning, and personalized on-device adaptation.
- 6 Future Directions: Future task generation will move beyond self-exploration toward realistic long-horizon tasks using human-in-the-loop curation and hierarchical decomposition.These approaches are proposed to increase task complexity.
- 6 Future Directions: Step-wise supervision is proposed because sparse trajectory-level rewards can ambiguously credit actions shared by successful and failed trajectories.The ambiguity can make it difficult to identify which actions contributed to success.
- 6 Future Directions: Visual world models are proposed to support multi-step lookahead by predicting future screens for error correction and strategic execution.The proposed models would improve planning over multiple GUI steps.
- 6 Future Directions: On-device continual adaptation is proposed to personalize agents to individual users’ patterns and preferences.This is intended to support more integrated user-specific experiences.
A Detailed Training Configuration
The detailed training configuration uses AndroidWorldAvd with dynamically scaled emulator instances and fixed per-emulator resource allocations.
- A Detailed Training Configuration: Training uses the AndroidWorldAvd base virtual device with emulator instances dynamically scaled to batch size.The configuration specifies the virtual device and scaling policy.
- A Detailed Training Configuration: Each emulator is allocated 3072 MB of memory and 2 CPU cores.These are the stated per-emulator resource allocations.
- A Detailed Training Configuration: GPU acceleration is configured in auto mode.The setting leaves GPU acceleration to automatic selection.
A.2 Model and Training Hyperparameters
The model and training configuration uses Qwen2.5-VL base models with Flash Attention 2, gradient checkpointing, and mixed-precision computation.
- A.2 Model and Training Hyperparameters: Training uses Qwen2.5-VL-7B-Instruct and Qwen2.5-VL-32B-Instruct as base models.Both model sizes are listed in the configuration.
- A.2 Model and Training Hyperparameters: The configuration uses Flash Attention 2 for attention computation and enables gradient checkpointing.Both are listed as training implementation settings.
- A.2 Model and Training Hyperparameters: Mixed precision uses BFloat16 for parameters and FP32 for reduction.The configuration assigns different numeric formats to parameters and reduction.
A.2.2 GRPO Training Parameters
The GRPO training configuration specifies batch sizes, optimizer settings, clipping, and trajectory-based advantage normalization.
- The configuration sets global batch size to 128, with update and experience micro-batches of 4 and 16 per device.
- Optimization uses Adam betas of (0.9, 0.999) and a PPO clip ratio of 0.2.
- GRPO uses trajectory-based normalization for advantage estimation.
A.2.3 Rollout Configuration
The rollout setup limits response length, rollout count, episode length, and GPU parallelism while exposing mobile actions through a structured tool interface with outcome tracking.
- Rollouts use a 2048-token maximum response length, tensor parallel size 2, and GPU memory utilization of 0.5.
- Each prompt receives 8 rollouts, and each episode allows at most 15 steps.
- The agent uses a structured mobile-use function supporting clicks, swipes, typing, system buttons, waiting, coordinates, text, time, and status outputs.
- The interface tracks action history and outcomes, while evaluation returns binary success or failure with detailed task-satisfaction reasoning.
C Evaluation Details
Evaluation uses official or adapted benchmark procedures, with task curation for executable, nonredundant test sets in standardized environments.
- AndroidWorld evaluation follows the original authors’ official code and procedures for direct comparability.
- AITW-Gen is filtered to executable tasks, yielding 300 unique tasks after removing infeasible third-party-application tasks.
- AITW-Web duplicates are merged, producing a curated benchmark of 150 unique tasks.
- The curation steps target fair and consistent evaluation within the standardized environment.
- The case study compares pre-reinforcement-learning and post-reinforcement-learning executions of a calendar-event task; the pre-RL execution omits the description and end time.