UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning

1 Introduction

• The report identifies costly trajectory collection, sparse or delayed rewards, credit assignment, GUI-only workflows, and fragile large-scale environments as barriers to robust GUI agents.Large-scale environments must support reproducible, fault-tolerant rollouts across browsers, VMs, and simulators.Source: §1 · p. 3
• The four pillars are a co-evolving Data Flywheel, stabilized long-horizon RL, a hybrid GUI environment with external resources, and a unified sandbox for heterogeneous environments.The RL framework uses asynchronous stateful rollouts, streaming updates, reward shaping, adaptive advantage estimation, and value pretraining; the sandbox supports reliable, high-throughput rollouts.Source: §1 · p. 3
• UI-TARS-2 improves over UI-TARS-1.5, scoring 88.2 on Online-Mind2Web, 47.5 on OSWorld, 50.6 on WindowsAgentArena, and 73.3 on AndroidWorld.It also attains a 59.8 mean normalized score across 15 games, surpassing OpenAI CUA and Claude Computer Use by factors of 2.4× and 2.8×, respectively; it remains competitive with frontier proprietary models on LMGame-Bench.Source: §1 · p. 3
• With GUI-SDK, UI-TARS-2 scores 29.6 on BrowseComp, 45.3 on Terminal Bench, and 68.7 on SWE-Bench Verified.The report presents these results as evidence that its training methodology transfers to information-seeking and software-engineering tasks.Source: §1 · pp. 3–4

2 UI-TARS-2

• UI-TARS-2 combines formal agent formulation, all-in-one sandbox environments, a data flywheel, multi-turn reinforcement learning, and parameter interpolation across vertical agents.Figure 1 illustrates a demo trajectory.Source: §2 · p. 4

Formulation

• The agent is modeled as a parameterized policy over historical context, memory, and the current environment, with ReAct [79] interleaving reasoning, action, and observation.Reasoning includes context analysis, memory recall, planning, and self-reflection; observations provide feedback used to update the agent’s state.Source: §2.1 · pp. 4–5
• The action space combines GUI primitives following UI-TARS [49] with predefined SDK functions for terminal operations and MCP tool use.GUI primitives include clicks, typing, and scrolling, which are also reused for gameplay; SDK functions support file management, software development, and external-service orchestration.Source: §2.1 · p. 5
• A step is one complete ReAct cycle, and a trajectory consists of a sequence of thought, action, and observation triples.The trajectory is denoted τ = {(t0, a0, o0), (t1, a1, o1), . . . , (tT, aT, oT)}.Source: §2.1 · p. 5
• Hierarchical memory pairs high-fidelity recent steps in working memory with semantically compressed episodic summaries, using the last N working-memory steps alongside episodic recall.Episodic summaries preserve key intentions and outcomes, supporting longer-term recall under long trajectories.Source: §2.1 · p. 5
• At each timestep, the policy predicts the next thought and action conditioned on the instruction, working memory, current observation, and episodic memory.The policy is specified as P(t_n, a_n | instruction, W_n, o_n, E_n).Source: §2.1 · p. 5

2.2 Environment: All-in-One GUI Sandbox

• The environment addresses the demands of general-purpose agents by supporting diverse task types, heterogeneous tools, and long-lived state across complex interactions.The shared sandbox joins GUI operations and SDK functions within one platform.Source: §2.2 · p. 5
• The universal sandbox combines GUI operations, SDK tools, and a shared file system, enabling browser-to-shell workflows in a stable, reproducible environment.It supports high-throughput training as well as annotation, evaluation, and inference; coding workflows include terminal-launched services that the GUI agent can preview.Source: §2.2 · pp. 5–6

GUI Env: Cloud Virtual Machine

• The cloud VM platform runs Windows, Ubuntu, and Android, integrating PyAutoGUI and ADB through a unified SDK for allocation, interaction, observation, and evaluation.The cluster comprises several thousand instances and supports several thousand QPS; session IDs preserve state across rounds, and lease-based cleanup reclaims resources.Source: §2.2 · pp. 5–6

Game Env: Hardware-Accelerated Browser Sandbox

• The game sandbox uses a hardware-accelerated browser to run HTML5/WebGL mini-games and expose a synchronous action-to-state loop with screenshots, scores, and levels.Multiple browser instances per container, elastic scheduling, crash recovery, and checkpointing support concurrency and reproducibility; DevTools Protocol/Playwright access, GPU acceleration, and adjustable timing improve control and sampling efficiency.Source: §2.2 · pp. 6–7, Figure 2 · p. 6

2.3 Data Flywheel Overview

• Each cycle routes high-quality model-generated trajectories to SFT and lower-quality ones to CT, creating a self-reinforcing loop of improving models and data.The figure depicts this dynamic reallocation across training cycles.Source: §2.3 · p. 7
• Starting from Seed1.6, training proceeds through CT for broad knowledge, SFT for task-specific instruction tuning, and RL for verifiable interactive-task optimization.The model is retrained sequentially on the updated CT, SFT, and RL stages.Source: §2.3 · p. 7
• Cold-start CT data combines tutorials, instructional videos, internet demonstrations, and in-house data, while SFT data is generated synthetically and through human annotation.Both stages mix agent-specific data with general-purpose chat and reasoning data; agent data is a small fraction of CT but a larger share of SFT.Source: §2.3 · p. 7
• After initial RL training, the current model generates trajectories through rejection sampling or interactive annotation, and a binary validation function routes accepted samples to SFT and rejected samples to CT.This keeps recent verified high-quality data in SFT while expanding CT with broader, less polished knowledge without contaminating the supervised signal.Source: §2.3 · p. 7
• SFT and RL run more frequently than CT, and each cycle transfers general-purpose RL gains to agent-specific domains as the proportion of high-quality outputs rises.The report says every generated sample is reused at an appropriate stage, with model and data quality co-evolving to drive continual performance gains.Source: §2.3 · p. 7

2.4.1 In-Situ Annotation for Continual Pre-training

• In-situ annotation captures authentic reasoning and grounded GUI actions during everyday computer use, addressing scarce data and gaps in Chinese-language and cognitively rich coverage.Annotators verbalize their thoughts while performing tasks; experts demonstrate complex tasks, while novices explore unfamiliar ones. The platform runs unobtrusively on annotators’ personal computers.Source: §2.4.1 · p. 8
• A human–LLM workflow builds multilevel GUI tasks from representative applications, while difficulty criteria balance task complexity and coverage.Applications are selected by industry coverage, user engagement, and market penetration; task importance reflects usage frequency, user benefit, and transferability. Tasks span novice-to-expert levels and single- or multi-application settings.Source: §2.4.1 · p. 8
• Quality control verifies executability, removes duplicates, and reviews annotations; ASR transcripts are refined and aligned with on-screen actions to produce high-fidelity training trajectories.The pipeline also augments linguistic diversity and enriches reasoning chains.Source: §2.4.1 · pp. 8–9

2.4.2 Interactive Annotation for Supervised Fine-tuning

• Interactive SFT annotation lets annotators accept or override the current model’s proposed actions and reasoning during rollout, keeping supervision on-policy.Tasks run in controlled virtual environments; task pools and annotation models are periodically refreshed to target weaknesses of the latest agent.Source: §2.4.2 · pp. 9–10

2.5 Multi-turn Reinforcement Learning

• Training covers three domains with different tools: GUI-Browsing, GUI-General, and browser-sandbox gameplay, using multi-turn interaction and verifiable rewards for policy improvement.The framework addresses scarce, challenging, verifiable end-to-end RL tasks. Agents observe state transitions and environmental feedback until task completion.Source: §2.5 · p. 10, §2.5.1 · p. 10

2.5.1 Task Design

• Task design combines screenshot-only browsing tasks, GUI interaction tasks from 690 websites, and real or synthesized games with runtime-verification scripts.Browsing tasks use multi-condition obfuscation and recursively constructed multi-hop chains, filtering out tasks solvable through prior knowledge or a single-turn search. GUI-General tasks are synthesized from website functions; game records encode rewards, termination flags, and verification metadata.Source: §2.5.1 · pp. 10–11

2.5.2 Reward Design

• Reward design uses deterministic verifiers or reference-answer judging when available, and a generative outcome reward model for open-ended GUI-General tasks.For GUI-Browsing, an LLM judge compares predictions with target answers; for games, function-based verifiers provide binary correctness rewards. The outcome reward model scores the trajectory using its text history and last five screenshots, and is trained with targeted annotation and single-turn RL.Source: §2.5.2 · p. 11

2.5.3 Asynchronous Agent Rollout via Stateful Environment

• Asynchronous, stateful rollouts target training stability and efficiency by decoupling inference, training on partially filled rollout pools, and preserving environment state across tool calls.Training starts when completed traces reach the minimum batch threshold while incomplete traces remain available for later iterations. Stateful environments retain context across extended problem-solving sessions.Source: §2.5.3 · pp. 11–12

2.5.4 RL Training Algorithm

• PPO training shapes rewards around final-outcome correctness and uses format rewards or length penalties in some scenarios, with asymmetric clipping to expand exploration while limiting token removal.A larger upper clip bound allows greater increases in low-probability action likelihoods, while a low lower bound helps avoid premature token elimination and output-diversity collapse.Source: §2.5.4 · pp. 12–13
• Long-horizon PPO stability is supported by separate policy and critic GAE coefficients, sequence-length-adaptive policy λ, and value pretraining under a fixed policy.Length-Adaptive GAE uses α = 0.05; value pretraining uses GAE with λ = 1.0 and continues until value loss and explained variance indicate convergence.Source: §2.5.4 · p. 13

• Because heterogeneous domains make joint RL unstable and computationally prohibitive, UI-TARS-2 trains vertical agents from shared SFT initialization and merges their parameters, leveraging approximate linear mode connectivity [48].Specialized runs cover GUI-Browsing, GUI-General, and Game environments, alongside other domains and tools.Source: §2.6 · p. 13
• The merged model performs almost comparably to the best specialized model on composite tasks while enabling strong cross-domain generalization without additional optimization cost.Source: §2.6 · p. 13

3.1 Experimental Setup

• Evaluations cover computer, mobile, and browser use plus in-domain and out-of-domain games, with GUI benchmarks permitting GUI operations or GUI SDK and games tested screenshot-only in a browser sandbox.Compared baselines include Claude 4, OpenAI-o3, and OpenAI CUA-o3 for GUI benchmarks, and Claude, OpenAI, and Gemini models for games.Source: §3.1 · p. 14

3.2 Main Results

• UI-TARS-2 surpasses UI-TARS-1.5 on all reported GUI benchmarks, scoring 47.5% on OSWorld, 50.6% on WindowsAgentArena, 73.3% on AndroidWorld, and 88.2% on Online-Mind2Web.With GUI-SDK, it scores 45.3% on Terminal Bench, 68.7% on SWE-Bench, 50.5% on BrowseComp-zh, and 29.6% on BrowseComp-en; GUI-only scores on the latter two are 32.1% and 7.0%. RL also raises Online-Mind2Web from 83.7% to 88.2%, OSWorld from 43.0% to 47.5%, and AndroidWorld from 64.6% to 73.3%.Source: §3.2 · p. 14, Table 1 · p. 15
• UI-TARS-2 reaches a 59.8 mean normalized score across 15 games, outperforming OpenAI CUA by 35.0 points and Claude Computer Use by 38.2 points.On LMGame-Bench, it remains competitive with frontier models, including 163.2 on Candy Crush versus o3’s 106.0, while weaker Tetris and Sokoban results indicate challenges with very long-horizon planning.Source: Table 2 · p. 15, §3.2 · p. 15, Table 3 · p. 16

3.3 Detailed Analyses

• Across GUI and game tasks, RL training rewards show a clear upward trend; GUI and web-game training frequently shows rising entropy, and manual reward inspection found no substantial reward hacking.On 300 human-annotated GUI agent traces, UI-TARS-2 achieved an F1 score of 83.8 as a generative ORM in binary classification.Source: §3.3 · p. 16
• As RL progresses, GUI step-level think length declines and GUI-General tasks require fewer interaction steps, while game think length rises at new curriculum difficulty levels before declining as the agent adapts.The report notes that interaction-round scaling is not always positively correlated with performance and suggests step budgets in reward design to encourage efficiency-performance balance.Source: Figure 9 · p. 17, §3.3 · pp. 17–18, Figure 10 · p. 18
• Value-model pretraining yields consistently higher PPO rewards, and PPO maintains higher, less volatile rewards than GRPO, so the main experiments use PPO.Source: Figure 10 · p. 18, §3.3 · pp. 18–19, Figure 12 · p. 19
• Increasing inference-time step budgets steadily improves game and OSWorld performance, with UI-TARS-2 continuing to gain where baseline curves flatten.The OSWorld gains persist despite RL training incentivizing task completion in fewer steps, indicating the policy can still use additional interaction steps at inference time.Source: Figure 11 · p. 18, §3.3 · pp. 18–19
• Game RL brings several initially weak titles to nontrivial scores, while some plateau or temporarily regress, and progress often arrives in bursts when task-specific subgoals become reliably attainable.Knowledge acquired through GUI-SDK interaction transfers to GUI-only tasks, outperforming the GUI-only baseline on pure GUI tasks. A shared value model trained across both interfaces yields higher explained variance than interface-specific baselines.Source: §3.3 · pp. 19, 21

Related work

• Early LLM agents combined reasoning with actions or tool interfaces, while specialized benchmarks later targeted web interaction, desktop applications, and repository-level software engineering.Examples include ReAct [79], MRKL [29], Toolformer [54], and Gato [53]; Mind2Web and WebArena cover web tasks, while OSWorld [75] evaluates desktop applications.Source: §4 · p. 22

GUI Agents

• GUI research shifted from saturated element-grounding datasets to end-to-end task agents, as demonstration-trained systems showed limited generalization and robustness and newer work explored reinforcement learning.Examples include CogAgent [23], OS-Atlas [74], Aguvis [76], and RL systems ARPO [35] and Mobile-GUI-R1 [58].Source: §4 · p. 22

Game Agents

• Unlike game agents relying on textual observations and game APIs, this framework uses visual input and native GUI actions, unifying games with computer-use tasks without handcrafted modules.Earlier game agents often targeted single titles; recent LLM/VLM approaches also use explicit memory or planning workflows, or specialize models to particular games.Source: §4 · p. 22

Other Relevant Directions

• Tool-agent research spans end-to-end reinforcement learning and scripted workflows, while harder benchmarks such as BrowseComp expose the limits of training on simple tasks and motivate harder datasets and planning-based systems.Earlier benchmarks included HotpotQA and MathQA; MCP standardized flexible tool integration.Source: §4 · p. 22
• Repository-level coding benchmarks and richer datasets have expanded agent task coverage, while sandboxed execution feedback and command-line benchmarks emphasize interactive coding skills.The landscape includes SWE-bench [28], SWE-Gym [47], SWE-Bench-Extra, SWE-ReBench, Multi-SWE-RL [84], OpenHands [71], and Terminal Bench.Source: §4 · pp. 22–23

5 Conclusion

• UI-TARS-2 delivers balanced, competitive performance across GUI, browser, mobile, and game tasks in one system, although specialized variants can peak on individual benchmarks.The report presents it as a step toward more capable, reliable, and versatile computer-use agents.Source: §5 · p. 23
• Its iterative training pipeline combines multi-turn reinforcement learning, supervised fine-tuning, rejection sampling, and continual pre-training to improve across heterogeneous domains.The report also identifies practical insights into multi-turn agent RL through analyses of training dynamics and interaction scaling.Source: §5 · p. 23
• Training on diverse environments promotes parameter sharing and capability transfer, enabling hybrid skills that combine graphical interaction with complex reasoning and decision-making.The report presents these findings alongside empirical evidence and methodological principles intended to guide future research.Source: §5 · p. 23