PaperlayerView exact revision on arXiv

Source-linked AI summary

UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning

Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, Wanjun Zhong, Yining Ye, Yujia Qin, Yuwen Xiong, Yuxin Song, Zhiyong Wu, Aoyan Li, Bo Li, Chen Dun, Chong Liu, Daoguang Zan, Fuxing Leng, Hanbin Wang, Hao Yu, Haobin Chen, Hongyi Guo, Jing Su, Jingjia Huang, Kai Shen, Kaiyu Shi, Lin Yan, Peiyao Zhao, Pengfei Liu, Qinghao Ye, Renjie Zheng, Shulin Xin, Wayne Xin Zhao, Wen Heng, Wenhao Huang, Wenqian Wang, Xiaobo Qin, Yi Lin, Youbin Wu, Zehui Chen, Zihao Wang, Baoquan Zhong, Xinchun Zhang, Xujing Li, Yuanfan Li, Zhongkai Zhao, Chengquan Jiang, Faming Wu, Haotian Zhou, Jinlin Pang, Li Han, Qi Liu, Qianli Ma, Siyao Liu, Songhua Cai, Wenqi Fu, Xin Liu, Yaohui Wang, Zhi Zhang, Bo Zhou, Guoliang Li, Jiajun Shi, Jiale Yang, Jie Tang, Li Li, Qihua Han, Taoran Lu, Woyu Lin, Xiaokang Tong, Xinyao Li, Yichi Zhang, Yu Miao, Zhengxuan Jiang, Zili Li, Ziyuan Zhao, Chenxin Li, Dehua Ma, Feng Lin, Ge Zhang, Haihua Yang, Hangyu Guo, Hongda Zhu, Jiaheng Liu, Junda Du, Kai Cai, Kuanye Li, Lichen Yuan, Meilan Han, Minchao Wang, Shuyue Guo, Tianhao Cheng, Xiaobo Ma, Xiaojun Xiao, Xiaolong Huang, Xinjie Chen, Yidi Du, Yilin Chen, Yiwen Wang, Zhaojian Li, Zhenzhu Yang, Zhiyuan Zeng, Chaolin Jin, Chen Li, Hao Chen, Haoli Chen, Jian Chen, Qinghao Zhao, Guang Shi

arXiv:2509.02544v2cs.AIcs.CLcs.CVcs.HC

TL;DR

GUI agents still face scalable data, stable multi-turn RL, GUI-only limitations, and environment challenges. UI-TARS-2 addresses these issues with a data flywheel, stabilized multi-turn RL, hybrid environments, and unified rollout infrastructure, achieving balanced performance across GUI, browser, mobile, game, information-seeking, and software-engineering tasks. Its analyses also identify remaining headroom in long-horizon reasoning and planning.

  • Problem

    Robust GUI agents lack scalable interactive trajectories and stable long-horizon multi-turn RL, while GUI-only operation and environment stability remain open challenges.

  • Method

    UI-TARS-2 combines continual pre-training, supervised fine-tuning, rejection sampling, multi-turn RL, hybrid GUI-SDK environments, and unified sandbox infrastructure.

  • Results

    UI-TARS-2 achieves balanced, competitive performance across GUI, browser, mobile, game, information-seeking, and software-engineering tasks, improving over UI-TARS-1.5.

  • Takeaways & Limitations

    The results support UI-TARS-2 as a robust GUI-centered agent that generalizes across diverse interactive domains and yields practical insights into multi-turn agent RL.

  • Takeaways & Limitations

    Some games show plateaus or temporary regressions, indicating a reasoning ceiling tied to the starting backbone and remaining need for stronger long-horizon planning.

Abstract

from arXiv · show

The development of autonomous agents for graphical user interfaces (GUIs) presents major challenges in artificial intelligence. While recent advances in native agent models have shown promise by unifying perception, reasoning, action, and memory through end-to-end learning, open problems remain in data scalability, multi-turn reinforcement learning (RL), the limitations of GUI-only operation, and environment stability. In this technical report, we present UI-TARS-2, a native GUI-centered agent model that addresses these challenges through a systematic training methodology: a data flywheel for scalable data generation, a stabilized multi-turn RL framework, a hybrid GUI environment that integrates file systems and terminals, and a unified sandbox platform for large-scale rollouts. Empirical evaluation demonstrates that UI-TARS-2 achieves significant improvements over its predecessor UI-TARS-1.5. On GUI benchmarks, it reaches 88.2 on Online-Mind2Web, 47.5 on OSWorld, 50.6 on WindowsAgentArena, and 73.3 on AndroidWorld, outperforming strong baselines such as Claude and OpenAI agents. In game environments, it attains a mean normalized score of 59.8 across a 15-game suite-roughly 60% of human-level performance-and remains competitive with frontier proprietary models (e.g., OpenAI o3) on LMGame-Bench. Additionally, the model can generalize to long-horizon information-seeking tasks and software engineering benchmarks, highlighting its robustness across diverse agent tasks. Detailed analyses of training dynamics further provide insights into achieving stability and efficiency in large-scale agent RL. These results underscore UI-TARS-2's potential to advance the state of GUI agents and exhibit strong generalization to real-world interactive scenarios.

1 Introduction

UI-TARS-2 addresses data scarcity and unstable long-horizon GUI reinforcement learning through a scalable training methodology. It improves performance across GUI and game environments while extending to broader interactive tasks.

  • Scalable long-horizon GUI learning remains difficult because trajectories containing reasoning, actions, states, and feedback are costly to collect.
  • The Data Flywheel co-evolves the model and corpus through continual pre-training, supervised fine-tuning, rejection sampling, and multi-turn RL.It continually supplies diverse trajectories while iteratively improving both data and model.
  • 88.2 on Online-Mind2Web, 47.5 on OSWorld, 50.6 on WindowsAgentArena, and 73.3 on AndroidWorld demonstrate strong GUI benchmark performance.The report describes these as gains over UI-TARS-1.5 and improvements over strong baselines in multiple cases.
  • 59.8 mean normalized score across a 15-game suite corresponds to roughly 60% of human-level performance.UI-TARS-2 also remains competitive with frontier proprietary models on LMGame-Bench.
  • 29.6 on BrowseComp, 45.3 on Terminal Bench, and 68.7 on SWE-Bench Verified indicate performance beyond GUI and game tasks.

2 UI-TARS-2

UI-TARS-2 is presented as a unified framework for advanced GUI-centered agents. Its methodology combines agent formulation, sandbox environments, data construction, multi-turn RL, and parameter interpolation.

  • The methodology integrates formal agent formulation, all-in-one sandbox environments, a data flywheel, multi-turn reinforcement learning, and parameter interpolation.

2.1 Formulation

UI-TARS-2 models interaction as an evolving ReAct loop in which reasoning, actions, observations, and hierarchical memory jointly determine behavior. Its action space covers GUI operations and system-level tool use.

  • ReAct interaction: At each timestep, the agent interleaves reasoning, action, and observation in a structured ReAct loop.Reasoning includes planning and reflection; actions affect the environment; observations update the agent state.
  • Action space: The action space includes GUI manipulation such as clicking, typing, scrolling, and gameplay interactions.
  • Action space: Pre-defined SDK functions extend interaction to terminal commands, file management, software development, and external-service tool calls.
  • Trajectory formulation: A trajectory of length T consists of complete ReAct cycles containing reasoning, action, and observation.
  • Hierarchical memory: Working Memory preserves recent steps in high fidelity, while Episodic Memory stores compressed summaries of past episodes.Direct context is restricted to recent working-memory steps while episodic memory supports longer-term recall.
  • Policy: The policy predicts the next thought and action conditioned on the instruction, current observation, working memory, and episodic memory.

2.2 Environment: All-in-One GUI Sandbox

The all-in-one sandbox unifies GUI operations, SDK functions, files, terminals, and tool calling across desktop, mobile, and browser environments. It is designed for stable, reproducible, high-throughput multi-turn rollouts.

  • Unified sandbox: A universal sandbox merges GUI operations and SDK functions into one platform, including file-system and tool-calling capabilities.
  • Unified sandbox: A shared file system lets an agent download a browser file and immediately process it with shell commands in the same containerized instance.
  • GUI environment: The virtual-machine platform supports Windows, Ubuntu, and Android through PyAutoGUI and ADB interfaces with a unified interaction SDK.
  • Infrastructure: The infrastructure combines high-concurrency execution with session tracking and resource reclamation for consistent multi-round interactions.
  • System-level workflows: Terminal-launched services can be exposed through proxy URLs so GUI agents can preview front-end and back-end components.
  • Game environment: The browser sandbox executes HTML5/WebGL mini-games while returning synchronous inputs, screenshots, scores, and levels through unified page APIs.
  • Game environment: Multiple browser instances per container, elastic scheduling, crash recovery, checkpointing, and garbage collection support concurrency and long-running stability.

2.3 Data Flywheel Overview

UI-TARS-2 uses a data flywheel in which model-generated trajectories are evaluated and routed across continual pre-training, supervised fine-tuning, and reinforcement learning, allowing model and data quality to co-evolve.

  • Training Stages: The flywheel cycles trajectories through continual pre-training, supervised fine-tuning, and reinforcement learning, with each stage receiving data matched to its purpose.Continual pre-training emphasizes broad knowledge, supervised fine-tuning emphasizes task-specific quality, and reinforcement learning optimizes interactive performance.
  • Cold-start Data Sources: Initial continual-pretraining data combines tutorials, instructional videos, internet demonstrations, and in-house data, while supervised-fine-tuning data comes from synthetic generation and human annotation.Agent-specific data is a small fraction of continual pre-training but a larger fraction of supervised fine-tuning.
  • Iterative Data Flow: Each iteration generates new trajectories through rejection sampling or interactive annotation and evaluates samples with a binary validation function.The validation function maps each sample to V(s) ∈ {0, 1}.
  • Iterative Data Flow: Validated samples with V(s) = 1 enter the next supervised-fine-tuning dataset, whereas lower-quality samples with V(s) = 0 are routed to continual pre-training.This routing keeps recent verified data in supervised fine-tuning while expanding continual pre-training with broader knowledge.
  • Iterative Data Flow: As iterations progress, the improved model generates a higher proportion of validated outputs, and every sample is reused at an appropriate training stage.The reported relationship is P(V(s) = 1 | t) > P(V(s) = 1 | t − 1).

2.4 CT & SFT Data Preparation

UI-TARS-2 addresses scarce, off-policy GUI-agent data through in-situ human annotation and quality-controlled reasoning–action trajectories, while expanding coverage with systematic task construction.

  • Motivation: Agent interaction trajectories are scarce because multi-turn tasks require sustained reasoning and tool manipulation that are difficult to obtain at scale.The paper contrasts this scarcity with abundant mathematical and coding data in human corpora.
  • Data Limitations: Existing GUI data is quickly exhausted, lacks sufficient Chinese-language coverage, and often records procedural actions without complete cognitive processes.These limitations motivate a broader human-centered collection strategy.
  • In-Situ Annotation: The in-situ annotation system runs unobtrusively on annotators’ personal computers, collecting authentic interaction data during realistic everyday workflows.The design is intended to avoid disrupting normal usage.
  • Annotation Protocol: Annotators verbalize thoughts while completing tasks, and the audio is aligned with corresponding UI interactions to capture reasoning–action trajectories.This think-aloud protocol replaces ineffective retroactive reconstruction of annotators’ thoughts.
  • Task Design and Collection: Candidate applications are selected using industry coverage, user engagement, and market penetration, then organized into hierarchical task graphs with task-importance scores.The pipeline targets representative mainstream websites and desktop applications.
  • Curation Pipeline: Collected data undergoes executability checks, deduplication, dual-annotator review, transcription, LLM refinement, synchronization, and linguistic augmentation.The resulting trajectories are prepared for continual pre-training.
  • Interactive Annotation: Interactive annotation keeps supervision on-policy by allowing annotators to accept or override the current model’s proposed reasoning and actions during live rollouts.Immediate feedback avoids post-hoc correction, while refreshed models and task pools keep collection aligned with current behavior.

2.5 Multi-turn Reinforcement Learning

UI-TARS-2 trains on scalable, verifiable multi-turn interactions across GUI, browsing, general web, and game domains, using reward designs and rollout infrastructure intended to improve long-horizon RL stability and efficiency.

  • Framework: The framework uses RLVR to train agents through real-time multi-turn environment interactions with continuously observed state transitions and feedback.Domain-specific pipelines synthesize large-scale verifiable tasks across multiple domains.
  • Task Data: High-quality interactive RL data remains scarce, motivating task construction that combines diverse formats with reliable verification signals.The section frames verifiability as a prerequisite for end-to-end RL data.
  • GUI-Browsing: GUI-Browsing tasks require agents to answer information-seeking questions solely from screenshots without search APIs.The synthesis pipeline uses multi-condition obfuscation and multi-hop chains to create indirect reasoning requirements.
  • GUI-General: GUI-General tasks are synthesized from selected mainstream websites after filtering inaccessible, login-gated, trivial, and casual-game pages.VLMs identify core website functionalities before task synthesis.
  • Gameplay: The game dataset combines browser-runnable HTML5/WebGL games with LLM-synthesized games exposing explicit state interfaces and JavaScript verification scripts.Both real and synthesized games are designed for sandbox execution and evaluation.
  • Reward Design: Reward design distinguishes deterministically verifiable tasks, which use binary or judged correctness, from open-ended tasks, which use a trajectory-conditioned outcome reward model.The outcome reward model consumes text history and the last five screenshots to produce a task-success score.
  • Infrastructure: The rollout infrastructure targets training stability and efficient organization of multi-turn interactions and samples.Traditional batch rollouts are described as vulnerable to long-tail bottlenecks, inefficiency, and off-policy drift.
  • Infrastructure: Asynchronous server inference decouples agent-framework implementation from policy execution while supporting new interaction handlers and more efficient inference.Policy inference runs through an online server architecture.

2.6 Merging Vertical Agents via Parameter Interpolation

Instead of jointly optimizing substantially different environments, UI-TARS-2 trains specialized agents separately and merges their parameters, preserving vertical performance while enabling cross-domain generalization.

  • Motivation: Joint reinforcement learning across environments is unstable and computationally prohibitive because domains differ in action spaces, state spaces, horizons, and rollout complexity.The paper therefore adopts separate environment-specific training runs.
  • Specialized Training: Starting from shared supervised-fine-tuning initialization, UI-TARS-2 trains specialized models for GUI-Browsing, GUI-General, games, and other tool-oriented domains.The specialized models are then merged through parameter interpolation.
  • Results: Parameter interpolation preserves each specialized vertical’s performance while enabling strong cross-domain generalization without additional optimization cost.On composite tasks, the merged model performs almost comparably to the best relevant specialized model.

3 Experiments

UI-TARS-2 improves across GUI and game benchmarks while showing broad gains from multi-turn RL, tool augmentation, and cross-interface training. Analyses identify mechanisms for training stability, exploration, inference-time scaling, and remaining long-horizon reasoning limits.

  • GUI Main Results: 47.5% on OSWorld, 50.6% on WindowsAgentArena, 73.3% on AndroidWorld, and 88.2% on Online-Mind2Web exceed UI-TARS-1.5 across reported GUI benchmarks.The model also outperforms strong baselines, including OpenAI CUA-o3 and Claude 4, across computer, mobile, and browser settings.
  • Game Main Results: 59.8 mean normalized score across 15 games reaches nearly 60% of human-level performance, while LMGame-Bench results remain competitive with frontier models.The model is close to human level on several titles and surpasses human performance on Shapes, but is weaker on Tetris and Sokoban.
  • Training Rewards and Entropy: Multi-turn RL steadily increases rewards across GUI-Browsing, GUI-General, and Game tasks, indicating policy improvement under RL supervision.The consistent upward trend spans structured GUI tasks and dynamic game environments.
  • Training Rewards and Entropy: Rising entropy in GUI and web-game experiments suggests that training expands exploration rather than collapsing prematurely into narrow exploitation.This differs from the monotonic entropy reduction often observed in reasoning-focused RL.
  • PPO v.s. GRPO: PPO maintains higher rewards with lower volatility than GRPO, motivating its selection for the main experiments.The comparison is based on preliminary evaluation of training dynamics.
  • Hybrid Training: Hybrid training transfers GUI-SDK knowledge to pure GUI tasks, outperforming the GUI-only baseline despite halved per-interface training data.A shared value model also yields higher explained variance than interface-specific baselines, improving stability and reward estimation.

4 Related Work

Research on interactive agents has progressed from modular and workflow-based systems toward generalist, end-to-end agents evaluated across GUI, games, tools, and software engineering.

  • Early LLM agents coupled reasoning with acting through prompting recipes or tool-augmented interfaces, including ReAct, MRKL, and Toolformer.
  • Web and desktop GUI research established realistic interaction benchmarks such as Mind2Web, WebArena, and OSWorld.
  • GUI-agent research shifted from grounding individual elements toward end-to-end completion of complete GUI tasks as early datasets approached saturation.
  • Game-agent methods emphasize long-horizon control and strategic exploration, often using explicit memory, planning, or title-specific VLM fine-tuning.
  • Tool protocols and end-to-end reinforcement learning provide complementary directions for flexible tool integration and multi-step interactive optimization.
  • Software-engineering agents increasingly rely on repository-level benchmarks, sandboxed execution feedback, and command-line environments.

5 Conclusion

UI-TARS-2 is a unified native GUI-centered agent for structured computer-use and dynamic game-like environments. Its iterative training supports balanced performance and capability transfer across heterogeneous interactive tasks.

  • UI-TARS-2 handles both structured computer-use tasks and dynamic, game-like interactive environments within one native GUI-centered model.
  • Its training pipeline combines multi-turn reinforcement learning, supervised fine-tuning, rejection sampling, and continual pre-training.
  • UI-TARS-2 achieves balanced and competitive performance across GUI, browser, mobile, and game tasks, although domain-specialized variants can peak on individual benchmarks.
  • Analyses of training dynamics and interaction scaling provide practical insights into multi-turn agent reinforcement learning.
  • Training on diverse environments promotes parameter sharing and capability transfer, producing hybrid graphical-interaction, reasoning, and decision-making skills.

6 Contributions

The contributions section presents the paper’s author list, arranged alphabetically by first name, with some names corresponding to internal company aliases.

  • The author list is alphabetized by first name, and some names are internal aliases used within the company.
  • The listed authors include Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, and Junjie Fang.
  • The listed authors include Baoquan Zhong, Xinchun Zhang, Xujing Li, Yuanfan Li, and Zhongkai Zhao.
Loading 2509.02544v2…