Source-linked AI summary
UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning
Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, Hongsheng Li
TL;DR
Rule-based reinforcement learning has been under-explored for multimodal GUI action prediction, despite its success in language models. UI-R1 addresses this gap with a rule-based action reward and a small curated mobile dataset, achieving improved in-domain and out-of-domain performance while also producing an optimized grounding variant.
Problem
Rule-based reinforcement learning has shown reasoning benefits in language models, but its application to multimodal GUI action prediction remains under-explored.
Method
UI-R1 generates multiple reasoning trajectories for low-level GUI instructions, scores action types, arguments, and formats with rule-based rewards, and optimizes the policy using reinforcement learning.
Results
UI-R1-3B achieves significant performance improvements and strong out-of-domain generalization using only 130+ mobile-domain training samples, while UI-R1-E-3B improves grounding efficiency and accuracy.
Takeaways & Limitations
The results support rule-based reinforcement learning as a data-efficient approach for specialized GUI action prediction across mobile, desktop, and web platforms.
Takeaways & Limitations
The effectiveness of SFT diminishes in out-of-domain scenarios, limiting its generalization beyond the specific tasks represented by its training data.
Abstract
from arXiv · showhide
The recent DeepSeek-R1 has showcased the emergence of reasoning capabilities in LLMs through reinforcement learning (RL) with rule-based rewards. Despite its success in language models, its application in multi-modal domains, particularly in graphic user interface (GUI) agent tasks, remains under-explored. To address this issue, we propose UI-R1, the first framework to explore how rule-based RL can enhance the reasoning capabilities of multimodal large language models (MLLMs) for GUI action prediction tasks. Specifically, UI-R1 introduces a novel rule-based action reward, enabling model optimization via policy-based algorithms such as Group Relative Policy Optimization (GRPO). For efficient training, we curate a small yet high-quality dataset of 136 challenging tasks, encompassing five common action types on mobile devices. Experimental results demonstrate that our proposed UI-R1-3B achieves significant improvements over the base model (i.e. Qwen2.5-VL-3B) on both in-domain (ID) and out-of-domain (OOD) tasks, with average accuracy gains of 22.1% on ScreenSpot, 6.0% on ScreenSpot-Pro, and 12.7% on ANDROIDCONTROL. Furthermore, UI-R1-3B delivers competitive performance compared to larger models (e.g., OS-Atlas-7B) trained via supervised fine-tuning (SFT) on 76K samples. We additionally develop an optimized version, UI-R1-E-3B, which significantly improves both grounding efficiency and accuracy. These results underscore the potential of rule-based reinforcement learning to advance GUI understanding and control, paving the way for future research in this domain. Code website: https://github.com/lll6gg/UI-R1.
1 Introduction
UI-R1 extends rule-based reinforcement learning to GUI action prediction, using task-specific rewards and a small, curated mobile dataset to improve action reasoning and out-of-domain performance.
- SFT for GUI agents requires large, high-quality labeled datasets, increasing training costs and often performing poorly in out-of-domain scenarios.
- UI-R1 applies rule-based reinforcement learning to GUI action prediction, where multiple reasoning trajectories are scored and optimized with policy methods such as GRPO.Its reward combines action type, action argument, and format rewards.
- 130+ mobile training samples are selected for difficulty, diversity, and quality, making the approach data-efficient while targeting cross-domain GUI generalization.
- UI-R1 is presented as the first framework using DeepSeek-R1-style reinforcement learning to enhance MLLM reasoning for GUI action prediction.
- The action reward is designed to align with common GUI objectives and support self-refinement through iterative policy optimization.
- UI-R1-E-3B is an optimized version that improves grounding efficiency and accuracy.
2 Related Work
Prior GUI-agent research uses commercial models, prompt engineering, and collaboration, while rule-based reinforcement learning has mainly been applied to general vision or language tasks and reasoning efficiency.
- MLLM-based GUI agents address device control, task completion, and GUI understanding, with some systems relying on commercial models for planning and prediction.
- Prompt engineering and multi-agent collaboration make AppAgent and Mobile-Agent systems adaptable but dependent on careful manual design.
- Rule-based reinforcement learning guides model behavior with predefined reward functions, including final-answer correctness checks in mathematical tasks.
- Recent reasoning-efficiency studies introduce length rewards to encourage concise, accurate reasoning and penalize redundant analysis.
3 Method
UI-R1 trains GUI agents with rule-based rewards and GRPO, combining action-type, coordinate, format, and optional length signals. Its compact mobile dataset and fast-grounding variants target accurate and efficient low-level action prediction.
- Rule-Based Reinforcement Learning: UI-R1 generates multiple reasoning-and-answer trajectories, scores them with task-specific rewards, and updates the policy using Group Relative Policy Optimization (GRPO).GRPO compares candidate responses within a group without requiring a critic model.
- Rule-Based Action Rewards: The rule-based action reward targets GUI grounding by evaluating predicted action types and click coordinates rather than bounding-box Intersection over Union.The predicted action contains an action type and coordinate; coordinate reward is based on whether the click lies inside the ground-truth box.
- Rule-Based Action Rewards: The reward combines action-type, coordinate-accuracy, and response-format components to guide structured reasoning and final action generation.The action space covers Click, Scroll, Back, Open App, and Input Text.
- Fast Grounding: The optimized UI-R1-Efficient-3B variant uses DAST followed by NOTHINK training to adapt reasoning length and enable direct grounding.DAST uses a token-length budget, while NOTHINK removes explicit <think> tags during training and inference.
- Fast Grounding: Reasoning is reported as nonessential for simpler GUI-grounding tasks, motivating direct prediction in the optimized variant.The ablation conclusion specifically concerns simpler tasks such as GUI grounding.
- Training Data: A three-stage Quality, Difficulty, and Diversity selection process produces a high-quality mobile training dataset of 136 samples.The selected data are used to train with rule-based reinforcement learning, emphasizing efficiency with limited supervision.
4 Experiments
Experiments evaluate UI-R1 on GUI grounding, low-level action prediction, data selection, reasoning length, and reward and thinking-training ablations. Results show improved performance with rule-based RL and carefully selected data, while reward components and training order affect outcomes.
- 4.1 GUI Grounding Capability: UI-R1 improves 3B-model GUI grounding by +20% on ScreenSpot and +6% on ScreenSpot-Pro, surpassing most 7B models.The evaluation covers ScreenSpot, ScreenSpot-Pro, and ScreenSpot-V2; correctness is based on predicted click coordinates relative to ground-truth annotations.
- 4.1 GUI Grounding Capability: SFT improves in-domain performance but its effectiveness diminishes out of domain because it depends on task-specific labeled data.This limits adaptation to unseen environments, whereas the RL approach uses task-specific reward optimization with fewer samples.
- 4.2 Action Prediction Capability: UI-R1 improves action-type accuracy by 15% and click-element grounding accuracy by 20% on ANDROIDCONTROL using 136 training data points.ANDROIDCONTROL evaluates action type and grounding separately, with the average score computed from both.
- 4.3 Key Factor Study: Performance improves as training data size increases but gradually saturates, while difficulty-based selection outperforms random selection.Difficulty-based selection uses tasks with long reasoning traces that the base model fails to solve.
- 4.3 Key Factor Study: Longer reasoning answers tend to correspond to lower accuracy, while reinforcement learning produces larger gains on more challenging questions.The analysis interprets longer reasoning as an indicator of greater question difficulty.
- 4.4 Ablation Study: The coordinate reward RC is more effective than the IoU-based reward RIoU for click-element grounding, while the action reward can hinder grounding.The ablation attributes the negative action-reward effect to ambiguity from a larger action space and emphasizes balancing reward components.
- 4.4 Ablation Study: The three-stage data-selection pipeline outperforms both random selection and using the entire dataset across domains.These results indicate that a smaller set of high-quality data can achieve higher performance.
5 Conclusion
UI-R1 extends rule-based reinforcement learning to GUI action prediction and uses a novel reward function with only 130+ mobile-domain samples. UI-R1-3B improves performance and generalizes to out-of-domain desktop and web datasets.
- UI-R1 extends rule-based reinforcement learning to GUI action prediction as a scalable alternative to supervised fine-tuning.
- The novel reward function evaluates both action type and arguments, enabling learning with reduced task complexity.
- Using only 130+ mobile-domain training samples, UI-R1-3B achieves significant performance improvements and strong out-of-domain generalization.
- The results demonstrate adaptability and data efficiency for rule-based reinforcement learning on specialized GUI tasks.
A.1 Setting
The experiments use the hyperparameter settings listed in Table 6 and train the base model on eight NVIDIA 4090 GPUs for approximately eight hours.
- Training uses 8 NVIDIA 4090 GPUs and takes approximately 8 hours.
- The experiment hyperparameters are listed in Table 6.
- Table 6 provides the hyperparameter settings used in the experiments.
A.2 Dataset Distribution
The paper reports dataset distribution statistics in Table 7 and describes Figure 5 as showing variable progression throughout training.
- Table 7 lists statistics for the training and evaluation datasets.
- Evaluation uses Desktop and Web subsets, as indicated by the asterisk in Table 7.
- Figure 5 illustrates the progression of various variables throughout the training process.
- Figure 5 is labeled as the UI-R1 training process.
B Other Evaluation
The paper evaluates the base model’s reasoning capability on ScreenSpot, treating reasoning ability as central to reinforcement fine-tuning and prediction accuracy.
- The base model’s reasoning capability is central to UI-R1’s reinforcement fine-tuning approach.
- Reasoning tags are incorporated into prompts to enhance reasoning ability and prediction accuracy.
- Figure 6 assesses Qwen2.5-VL-3B’s reasoning performance on the ScreenSpot task.
- Figure 6 is titled as Qwen2.5-VL-3B’s reasoning capability on ScreenSpot.
C.1 Training epoches
The training and inference setup examines epoch choice and the maximum-pixel trade-off, while coordinate rescaling compensates for image resizing. A case study illustrates successful task completion by selecting a screenshot checkbox.
- Training epochs: Training was finalized at 8 epochs based on performance across different training epochs.
- Maximum-pixel setting: Maximum-pixel settings trade off memory use and prediction accuracy, with excessively high values risking OOM errors and low values harming accuracy.
- Maximum-pixel setting: The training maximum-pixel value was set to 12,845,056 for improved OOD performance, while evaluation uses a smaller value to conserve memory.
- Coordinate rescaling: Coordinate scaling maps predictions from resized images back to the original image using width and height ratios after smart resizing.
- Case study: The case study shows the trained model completing a task by clicking the checkbox labeled “Remember me” near the bottom of a login form.