PaperlayerView exact revision on arXiv

Source-linked AI summary

GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents

Yuqi Zhou, Sunhao Dai, Shuai Wang, Kaiwen Zhou, Qinglin Jia, Jun Xu

arXiv:2505.15810v2cs.CLcs.AIcs.CV

TL;DR

R1-style GUI grounding applies online RL and explicit reasoning, but general-purpose training components introduce reasoning, reward-hacking, and optimization biases. The paper analyzes these components, proposes faster thinking, box-size-constrained rewards, and difficulty-aware policy updates, and reports state-of-the-art grounding results for GUI-G1-3B.

  • Problem

    R1-style GUI grounding training has component-specific issues: longer reasoning harms grounding, reward functions enable box-size hacking, and GRPO under-optimizes difficult samples.

  • Method

    The paper analyzes input design, output evaluation, and policy updates, then introduces a Fast Thinking Template, a box-size reward constraint, and modified GRPO weighting without length normalization.

  • Results

    GUI-G1-3B achieves state-of-the-art grounding performance, including 90.3% accuracy on ScreenSpot and 37.1% on ScreenSpot-Pro.

  • Takeaways & Limitations

    The targeted training changes yield strong GUI grounding with about 17K public grounding samples and fewer tokens and training stages than InfiGUI-R1.

  • Takeaways & Limitations

    The study focuses on grounding and does not cover action prediction or long-horizon planning.

Abstract

from arXiv · show

Recent Graphical User Interface (GUI) agents replicate the R1-Zero paradigm, coupling online Reinforcement Learning (RL) with explicit chain-of-thought reasoning prior to object grounding and thereby achieving substantial performance gains. In this paper, we first conduct extensive analysis experiments of three key components of that training pipeline: input design, output evaluation, and policy update-each revealing distinct challenges arising from blindly applying general-purpose RL without adapting to GUI grounding tasks. Input design: Current templates encourage the model to generate chain-of-thought reasoning, but longer chains unexpectedly lead to worse grounding performance. Output evaluation: Reward functions based on hit signals or box area allow models to exploit box size, leading to reward hacking and poor localization quality. Policy update: Online RL tends to overfit easy examples due to biases in length and sample difficulty, leading to under-optimization on harder cases. To address these issues, we propose three targeted solutions. First, we adopt a Fast Thinking Template that encourages direct answer generation, reducing excessive reasoning during training. Second, we incorporate a box size constraint into the reward function to mitigate reward hacking. Third, we revise the RL objective by adjusting length normalization and adding a difficulty-aware scaling factor, enabling better optimization on hard samples. Our GUI-G1-3B, trained on 17K public samples with Qwen2.5-VL-3B-Instruct, achieves 90.3% accuracy on ScreenSpot and 37.1% on ScreenSpot-Pro. This surpasses all prior models of similar size and even outperforms the larger UI-TARS-7B, establishing a new state-of-the-art in GUI agent grounding. The project repository is available at https://github.com/Yuqi-Zhou/GUI-G1.

1 Introduction

The paper rethinks R1-Zero-like GUI grounding by isolating challenges in input design, reward evaluation, and policy updates, then proposes targeted modifications to each component. GUI-G1-3B achieves state-of-the-art grounding performance with limited public supervision.

  • Input design: Longer reasoning can degrade grounding performance, motivating a Fast Thinking Template that encourages direct answer generation.The paper attributes this issue to grounding’s reliance on image tokens.
  • Output evaluation: Hit-based and IoU-based rewards induce opposite box-size hacking behaviors during grounding optimization.Hit-based rewards favor smaller boxes, whereas IoU-based rewards favor larger boxes.
  • Policy update: GRPO exhibits length and difficulty biases that encourage longer incorrect responses and hinder learning from challenging samples.The proposed update removes length normalization and adds difficulty-based weighting.
  • Results: 90.3% accuracy on ScreenSpot and 37.1% on ScreenSpot-Pro establish GUI-G1-3B as a state-of-the-art grounding model.The model uses Qwen2.5-VL-3B-Instruct and about 17K grounding samples.

2 R1-Zero-Like Training Paradigm for GUI Grounding

The grounding paradigm treats GUI localization as language generation from a screenshot and instruction, with reinforcement learning evaluating multiple candidate responses. GRPO computes relative advantages from rule-based rewards and optimizes the policy with KL-divergence regularization.

  • Grounding formulation: Given a screenshot and textual description, the MLLM predicts a target location represented as a bounding box or point.The implementation represents the predicted box as Bpred = (x̂1, ŷ1, x̂2, ŷ2) and evaluates it against Bgt.
  • Grounding formulation: Grounding is formulated as language generation, so each response includes the predicted location and may include reasoning or objective descriptions.The output contains a reasoning process and a final answer with the predicted location.
  • RL training: GRPO samples N candidate responses, evaluates them with rule-based rewards, and computes a relative advantage for each response.The advantage is derived from each reward relative to the group mean and standard deviation.
  • RL training: The policy is optimized with the GRPO objective and KL-divergence regularization after reward-based candidate evaluation.Mean and standard deviation are used in interpreting the reward normalization.

3 How R1-Zero-Like Training Affects Grounding for GUI Agents?

The analysis isolates three problems in R1-Zero-like GUI grounding training: excessive reasoning, size-sensitive reward hacking, and GRPO biases that under-optimize difficult samples. GUI-G1 addresses these issues with faster responses, box-size regularization, and bias-aware policy updates.

  • Overview: The study decomposes R1-Zero-like GUI grounding training into input design, output evaluation, and policy update.This structure is used to analyze templates, reward functions, and the GRPO objective separately.
  • 3.1 Analysis on Template: Longer reasoning chains consistently reduce ScreenSpot grounding accuracy, especially for textual targets.The analysis contrasts text and icon targets and finds that additional textual reasoning is less beneficial than appropriately scaled visual input.
  • 3.1 Analysis on Template: The Fast Thinking Template removes explicit deliberation and directly requests bounding-box coordinates in JSON format.It is designed to avoid excessive reasoning during grounding training.
  • 3.2 Analysis on Reward Function: Separately optimizing RHit or RIoU produces opposing reward-hacking behaviors: smaller boxes improve hit accuracy, whereas larger boxes improve overlap.Figure 3 tracks accuracy, IoU, and relative box size across policy iterations, while Figure 4 illustrates the underlying box-size preference.
  • 3.2 Analysis on Reward Function: RBox regularizes predicted box size and is intended to work alongside RHit and RIoU rather than replace them.The paper notes that jointly optimizing RHit and RIoU can still be dominated by one objective.
  • 3.3 Analysis on GRPO Objective: GRPO favors shorter correct responses and longer incorrect ones, while difficulty bias leaves harder samples under-optimized.The revised objective removes length normalization and adds difficulty weighting; experiments report improved performance and fewer extreme cases on hard samples.
  • 3.4 GUI-G1: A Tailored RL Visual Grounding Model: GUI-G1 combines the three targeted changes in a tailored RL grounding model.The paper presents this integrated approach after identifying limitations in existing training paradigms.

4 Experiments

The experiments evaluate GUI-G1-3B’s training setup, data, and benchmark performance. Trained on 17K grounding samples, it achieves state-of-the-art results on ScreenSpot and competitive results on ScreenSpot-Pro.

  • Implementation Details: GUI-G1-3B is built on Qwen2.5-VL-3B-Instruct and trained using the VLM-R1 framework.Training uses a combined hit, IoU, and box-size reward.
  • Training Dataset and Evaluation Benchmarks: The 17K-sample grounding dataset covers Mobile, Web, and Desktop domains.Samples with consistently correct or incorrect responses after eight prompts are discarded.
  • Performance Comparison on ScreenSpot: GUI-G1-3B achieves state-of-the-art performance on ScreenSpot across Mobile, Desktop, and Web.It outperforms proprietary, general-purpose, supervised, and R1-style GUI agents, including larger OS-Atlas-7B.
  • Performance Comparison on ScreenSpot-Pro: 37.1% average accuracy on ScreenSpot-Pro surpasses UI-TARS-7B and InfiGUI-R1-3B, both at 35.7%.GUI-G1-3B scores 16.1% on the OS subset versus 16.8% for OS-Atlas-7B despite using the same training dataset.

5 Related Work

Related work spans GUI grounding methods and R1-Zero-like training for multimodal models. GUI-G1 differs by studying minimal-data R1-Zero-like training to unlock GUI grounding capabilities.

  • Grounding for GUI Agents: Earlier GUI grounding methods aligned screenshots, instructions, and modalities using BERT-based representations.Subsequent work used reinforcement learning or fine-tuned pretrained models across platforms.
  • Grounding for GUI Agents: Recent GUI grounding approaches achieve strong results through large datasets and supervised fine-tuning across platforms.GUI-G1 instead explores minimal-data R1-Zero-like training for MLLM grounding.
  • R1-Zero-Like Training for MLLMs: R1-Zero-like training has been extended from DeepSeek-R1-Zero to vision-language and multimodal reasoning models.Examples include Vision-R1, MM-EUREKA, VisualThinker-R1-Zero, and LMM-R1.

6 Conclusion

The paper revisits R1-Zero-like GUI grounding training through input design, output evaluation, and policy updates. Its targeted changes produce a compact model that outperforms larger and other R1-style models on both reported benchmarks.

  • Input Design: Longer text reasoning can degrade GUI grounding, motivating the Fast Thinking Template for training.The template encourages direct answer generation.
  • Output Evaluation: Existing reward designs exhibit opposing reward-hacking issues, motivating a box-size constraint.The constraint regularizes predicted box size during optimization.
  • Policy Update: GRPO’s length and difficulty biases are addressed by removing length normalization and adding difficulty-based weighting.These changes target under-optimization of harder grounding examples.
  • Overall Results: GUI-G1-3B trained on 17K samples outperforms larger and other R1-style models on ScreenSpot and ScreenSpot-Pro.It also requires fewer tokens and training stages.

B Limitations

The paper’s limitations concern task coverage, the scope of its RL analysis, and the size and diversity of its training data.

  • Focus on grounding: The study focuses on grounding and does not cover action prediction or long-horizon planning.Future work could extend the approach to full GUI interaction decision-making.
  • Scope of RL analysis: The RL analysis mainly examines online reinforcement learning, especially GRPO.Dataset composition, model design, and hyperparameter tuning are not fully explored.
  • Limited training data: Training uses a relatively small set of public datasets, constraining the model’s performance ceiling.The paper proposes scaling training with larger and more diverse datasets to improve generalization and robustness.

C.1 Training Details

Training assigns higher difficulty weights to samples with smaller relative bounding boxes. The weights are derived by inverting and normalizing relative box size, then rescaling the result to a bounded interval.

  • Relative box size: Relative box size measures the predicted bounding box using the input image dimensions.IMAGE_WIDTH and IMAGE_HEIGHT denote the input image’s pixel width and height, respectively.
  • Difficulty-aware weighting: Inverted box-size values are normalized to [0, 1] before difficulty weighting.This normalization precedes the final linear rescaling step.
  • Difficulty-aware weighting: Final difficulty weights are linearly rescaled to the interval (0.5, 1.5].The resulting range keeps weighting stable and bounded.
  • Difficulty-aware weighting: Smaller relative boxes receive higher difficulty weights because they are treated as harder samples.The method first takes the inverse of relative box size to reflect this difficulty intuition.

D.1 Training Data Composition

The grounding dataset contains 17K samples spanning Mobile, Web, and Desktop domains. These domains draw from UI-BERT and OS-Atlas and contain balanced grounding instances pairing natural-language commands with UI elements.

  • Dataset composition: 17K grounding samples are distributed across Mobile, Web, and Desktop domains.The resource is designed to cover diverse platforms.
  • Mobile: Mobile data comes from UI-BERT and consists of user-interface data collected from Android applications.The Mobile domain represents Android application interfaces.
  • Web: Web data comes from OS-Atlas and includes interactive web elements and browser-based environments.This domain covers browser-oriented interaction settings.
  • Desktop: Desktop data also comes from OS-Atlas and focuses on native applications across Windows, Linux, and MacOS.Each domain contains balanced instances pairing natural-language commands with corresponding UI elements.

D.2 Output Token Efficiency Analysis

GUI-G1-3B produces substantially shorter outputs than InfiGUI-R1-3B during ScreenSpot inference across Mobile, Desktop, and Web domains. The compact output reduces computational cost while maintaining or improving task accuracy.

  • Token efficiency: GUI-G1-3B generates approximately one-third as many output tokens as InfiGUI-R1-3B on average across all three domains.The comparison covers Mobile, Desktop, and Web inference on ScreenSpot.
  • Accuracy and efficiency: The shorter outputs maintain or even improve task accuracy relative to InfiGUI-R1-3B.The paper connects compact generation with efficient inference without reporting a separate accuracy value here.
  • Token efficiency: Compact output reduces computational cost and reflects the model’s ability to produce concise responses.The passage presents this as an inference-efficiency benefit.
  • Deployment considerations: Real-world deployment requires care because reward design may cause reward hacking or bias toward easy cases.The paper encourages further study of fairness, robustness, and privacy in GUI-agent training and deployment.
Loading 2505.15810v2…