GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training

1. Introduction

• Interactive post-training supports long-horizon planning and error recovery, whereas static datasets often fail on out-of-distribution states and complex workflows requiring trial and error.Reinforcement learning in interactive GUIs enables continuous exploration and requires environments that support high-throughput sampling.Source: §1 · p. 1
• Real-world apps make post-training costly and unstable through login and remote-backend dependencies, while hidden task state forces noisy, expensive VLM reward estimates.Visual hallucinations and alignment errors can undermine reinforcement learning with verifiable rewards.Source: §1 · p. 1
• GUI-GENESIS uses VLMs and code LLMs to turn user traces into standalone web apps that preserve logical fidelity, then embeds executable assertions as deterministic code-native reward oracles.These assertions provide precise feedback for stable policy optimization while eliminating network and resource overhead.Source: §1 · pp. 1–2
• GUI-GENESIS yields robust zero-shot sim-to-real transfer on 149 tasks: 14.54% relative improvement over the base model, 3.27% over target-app-trained agents, 10× lower latency, and over $28,000 saved per epoch versus cloud-based VLM-reward training.The savings and latency comparison is against cloud-based training with VLM rewards.Source: §1 · p. 2
• The synthesis-navigation gap describes models that can construct valid environments they cannot yet solve, suggesting a pathway for self-improving agents.The paper reports this gap as an observation from its experiments.Source: §1 · p. 2

GUI agents and the shift to RL

• VLM advances enable GUI agents, but static-dataset SFT often struggles with error recovery and long-horizon planning, motivating interaction-based RL; real-app training remains costly, slow, and network-unstable.Prior SFT work includes Chen et al. (2024b), Lin et al. (2025), Lu et al. (2024), Wu et al. (2024), and Yang et al. (2025); RL approaches include Lu et al. (2025), Wang et al. (2025a), Xu et al. (2025a), and Ye et al. (2025).Source: §2 · p. 2

Reinforcement learning with verifiable reward

• RLVR has improved reasoning in mathematics and code generation, but general-purpose reward engineering remains difficult, and GUI training typically relies on noisy visual proxies rather than ground-truth states.Existing approaches use hand-crafted rule-based rewards or proxy tasks (Chen et al., 2025; Wang et al., 2025b).Source: §2 · p. 2

Environment synthesis in agentic tasks

• Model-generated interactive environments are emerging, but realistic app synthesis still faces GUI reconstruction and sim-to-real transfer challenges not adequately addressed by prior work.Prior tool-use pipelines have used model-simulated user interactions or manually engineered workflow environments (Chen et al., 2024a; Yao et al., 2024); emerging environment-generation work includes Cai et al. (2025) and Xu et al. (2025c).Source: §2 · p. 2

3. Problem Formulation

• The target application is modeled as a POMDP with latent states, screenshot observations, and rewards depending on unobservable true state, while agents learn policies from observations.Source: §3 · p. 2
• Real-app training is bottlenecked by seconds-long interactions that make 10^6-10^7-step RL intractable and by latent rewards that require noisy visual proxies, risking reward hacking and added VLM inference cost.Parallelization can cost ∼$24K/day for 100 concurrent cloud instances.Source: §3 · p. 3
• Given interaction traces D, the objective is to construct a task-conditioned surrogate Msyn on the target task manifold with deterministic code-assertion rewards and millisecond rather than seconds-long steps.The specified objective emphasizes reward verifiability and substantially lower step latency than Mreal.Source: §3 · p. 3
• Unlike learned neural simulators such as world models (Li et al., 2025; Luo et al., 2025), Msyn is executable code, enabling white-box reward verification and near-instantaneous simulation.Source: §3 · p. 3

4.1. Overview

• GUI-GENESIS creates a standalone Flask environment for each RL task, simulating only the task-relevant state and interactions rather than an entire application.Its three-stage workflow acquires trace context, synthesizes the environment and rewards, then verifies executability.Source: §4.1 · p. 3

4.2. Trace-Driven Context Acquisition

• For each training task, a baseline agent’s real-app screenshots and actions provide visual and logical context for synthesis, whether or not the attempt succeeds.Even failed traces capture design patterns and page transitions, helping reproduce the target app’s look and feel without inventing UI elements from scratch.Source: §4.2 · p. 3

4.3. Hierarchical Code Synthesis

• Hierarchical synthesis combines meta-prompted system design with plan-and-execute coding, enforcing visual, functional, viewport, and distractor constraints while implementing files sequentially.The model first creates a file manifest, then implements components in multi-turn dialogue to separate frontend fidelity from backend logic.Source: §4.3 · pp. 3–4

4.4. Code-Native Reward Injection

• Code-native calculate_reward() checks backend state to provide deterministic executable assertions and continuous rewards from 0 to 1 for partial condition satisfaction.This replaces noisy visual estimation with granular feedback for dense RL supervision.Source: §4.4 · p. 4

4.5. Automated Self-Verification

• A two-stage self-verification loop retries generation up to K = 5 times, combining code reflection on reward logic with Playwright tests of the task’s golden path.Only environments whose widgets are interactable and whose backend transitions trigger rewards pass dynamic testing and enter the training pool.Source: §4.5 · p. 4

Datasets and Tasks

• The WeChat Mini-App evaluation uses 969 training instructions and 149 non-overlapping evaluation instructions, all requiring multi-step reasoning and precise GUI interactions.The proprietary platform has over one billion active users and spans dynamic DOM structures and service domains including retail, food delivery, and public services (Tencent, 2024).Source: §5 · p. 4

Baselines and Training Environments

• Four training sources and reward setups are compared while holding agent architecture fixed: snapshot-only, live-app VLM reward, synthesized-environment VLM reward, and synthesized-environment code-native reward.The base model is Qwen-2.5-VL-32B (Bai et al., 2025b), fine-tuned on static snapshots with ground-truth-coordinate actions; it initializes subsequent RL pipelines. All policies are evaluated zero-shot on real WeChat apps, and GUI-GENESIS-trained agents have no real-world interaction before evaluation.Source: §5 · p. 5

Agent and RL Pipeline

• All experiments use a unified VLM-based GUI agent with a ReAct-style strategy (Yao et al., 2022) and Multistep GRPO (Shao et al., 2024).The controlled protocol samples groups of complete trajectories and optimizes policies using outcome-reward advantages relative to the group average, so performance differences reflect environment quality and reward reliability.Source: §5 · p. 5

Evaluation Metrics

• Evaluation covers real-world success, synthesis success, and environment efficiency across latency and simulation cost.Real-world Success Rate is human-annotated completion on live WeChat apps; synthesis success on the 149 evaluation instructions is measured by VLM Evaluation SR and Code-native SR, with code-native reward success defined as 1.0. Efficiency uses average latency and financial cost per simulation step with rollout length 96.Source: §5 · p. 5

6.1. Main Results

• 71.81% VLM Eval SR and 48.99% Native-code SR are reported for GUI-GENESIS training, versus 63.76% and 38.93% for the base model, respectively.The real-world-trained model is reported at 63.76% VLM Eval SR and 44.30% Native-code SE.Source: §6.1 · p. 5, Table 1 · p. 6
• 48.99% Native-code SR in synthetic environments and 42.28% Human Annotation SR in real-world apps show highly consistent trends for the best model.The paper presents this consistency as evidence that generated environments mirror the logical complexity of real tasks.Source: §6.1 · p. 5
• VLM Eval SR significantly overestimates performance against ground-truth metrics, with VLM judges producing false positives on incomplete tasks.Code-native reward checks underlying program state and aligns more closely with human judgment.Source: §6.1 · p. 5
• A ride-hailing task example illustrates that VLM evaluation can miss an incorrect choice of intermediate stops that code-native identification detects.The passage uses this example to contrast visual judging with verification from program state.Source: §6.1 · p. 5
• 42.28% Real-World SR (Human Annotation) is 14.54% higher than the no-RL baseline and 3.27% higher than real-world training at 40.94%.The paper attributes the gain over real-world training to deterministic, low-latency, noise-free learning signals that avoid instability from network latency, application instability, and VLM-judge hallucinations.Source: §6.1 · p. 5, Table 1 · p. 6
• 41.61% Real-World SR from VLM-judged rewards in synthetic environments exceeds the real-world-training baseline of 40.94% but trails code-native rewards at 42.28%.The comparison supports transferable visual grounding from synthesized GUIs while showing that program-state-based feedback provides a finer-grained, more reliable signal than visual approximation.Source: §6.1 · p. 6

6.2.1. ENVIRONMENT SYNTHESIS COST AND SUCCESS RATE

• GUI-GENESIS generated 1,118 task environments for $302.12, with 82.65% passing self-verification.Most successful environments were generated within the first two attempts; the $302.12 generation cost is a one-time expense, with zero marginal execution cost thereafter.Source: §6.2.1 · p. 6, Table 2 · p. 6

6.2.2. COMPUTATIONAL EFFICIENCY AT TRAINING

• A rollout takes 0.1013 hours in synthesized environments versus 0.2272 hours in real-world environments, and real-world training costs an additional $28,000 per epoch.The rollout latency difference is approximately a 2× speedup; local execution and assertion-based rewards eliminate device rental and VLM API costs, making the real-world alternative require more than twice the training time.Source: §6.2.2 · p. 6, Table 3 · p. 7

6.2.3. ANALYSIS OF CODE-NATIVE REWARD VS. VLM-AS-JUDGE

• Code-native rewards align with VLM judgments while distinguishing progress levels that the VLM groups into a single failure class.More than 75% of VLM-positive cases receive rewards above 0.8, while 75% of VLM-failure cases fall in [0, 0.6]; assertion-based rewards also assign distinct values such as 0.2, 0.3, and 0.4 to incomplete states, producing denser, reproducible learning signals.Source: §6.2.3 · p. 7, Figure 2 · p. 7

6.2.4. COMPLEXITY FIDELITY

• After filtering trajectories longer than 20 steps, synthesized environments retain a comparable trajectory-length spread to real-world tasks, with a mean of approximately 5.63 steps.Real-world tasks sometimes require longer navigation because of extraneous pop-ups or loading states; the observed synthesized-task lengths indicate that task logic is not distinctively trivialized.Source: §6.2.4 · p. 7, Figure 3 · p. 8

6.2.5. SCALING PROPERTIES OF SYNTHESIZED ENVIRONMENTS

• Scaling training from 240 to 969 synthesized environments monotonically improves both code-native and VLM success rates on the 149-environment evaluation set.The study also used a half-scale training set of 480 environments; the authors interpret the trend as evidence that performance is limited by scenario diversity rather than synthesized-environment fidelity.Source: §6.2.5 · pp. 7–8, Figure 4 · p. 8

6.3. Analysis: The Synthesis-Navigation Divergence

• Models sometimes synthesize functional applications that they themselves fail to navigate, demonstrating an asymmetric relationship between synthesis and navigation.Figure 5 presents case studies of this divergence.Source: §6.3 · p. 8, Figure 5 · p. 8
• Exploring environments a model generates could help it acquire novel navigation skills, providing a potential pathway for self-evolutionary training.The analysis proposes this possibility based on cases where synthesis succeeds but navigation fails.Source: §6.3 · p. 8

Conclusion and Future Work

• GUI-GENESIS uses coding LLMs to reverse-engineer interaction traces into lightweight, interactive, verifiable environments with code-native rewards.This decouples training from heavy real-world backends and provides precise reward signals.Source: §7 · p. 8
• Post-trained agents show improved training stability and efficiency and successfully transfer to real-world tasks.The conclusion also reports training throughput accelerated by orders of magnitude.Source: §7 · p. 8
• Future work could co-evolve coding and GUI agents by having coding agents synthesize progressively challenging curricula and GUI-agent failures refine coding capabilities.The authors note that models can synthesize task logic they cannot yet solve as agents.Source: §7 · p. 8

A. Prompt

• The system prompt directs a multimodal coding LLM to build a 410x858 native-style mobile app using Flask or FastAPI, vanilla frontend code, in-memory data, free-text inputs, and 3–5 plausible distractors.It asks the coding model to treat screenshots as style references rather than requirements to reproduce faithfully.Source: A. Prompt · p. 12
• The backend must track target-task progress, print action explanations and normalized cumulative RL_REWARD/NEXT values to stdout, and distribute rewards across steps without rewarding app launch.The prompt specifies sparse, multi-step rewards rather than a binary success signal.Source: A. Prompt · p. 12
• Task-specific flow rules require transaction confirmations, query results, date-selection interfaces, or routing views with start point, end point, and direction; the flow then stops without a generic completion popup.The prompt also forbids automatic restart or redirection after reaching the final page.Source: A. Prompt · p. 12
• The pipeline also requests a Python Playwright test script and a yes-or-no review of whether the implementation follows the reward rules and supports normal interaction.The code-generation instructions request raw code output for each file.Source: A. Prompt · pp. 12–13
• For a weather-query example, the generated Flask/vanilla app targets Chinese 410x858 UI, treats screenshots as subordinate to the user task, and allows free-text city search with plausible alternatives.The example task is checking tomorrow’s weather in Lvliang City; the result page displays the weather details.Source: A. Prompt · p. 13

How generated code use specefied resolution

• The generated stylesheet sets the app resolution to 410px by 858px.The root variables specify --app-width: 410px and --app-height: 858px.Source: B. Generation Examples · p. 14

How generated code imitate mobile app GUI

• The example styles a centered app view against a dark desktop background and uses a system-font stack.The body uses flex alignment to center its contents and fills the viewport height.Source: B. Generation Examples · p. 14

How generated code calculate code-native rewards

• For the city-weather example, code checks Lvliang or city_id 1, then emits RL_REWARD=1.0 and a terminal transition.The example describes the user as viewing tomorrow’s weather for Lvliang.Source: B. Generation Examples · p. 14

How generated code mock database interactions

• The mock database represents seven Shanxi cities, including Lvliang, Taiyuan, and Datong.Each listed record includes an id, name, province, and detail field.Source: B. Generation Examples · p. 14

How generated code write testscripts

• A Playwright test opens the app in a 410-by-858 mobile browser context, waits for the search input, and searches for Lvliang.The script uses a mobile user agent and submits the search by pressing Enter.Source: B. Generation Examples · p. 14

• Figure 6 depicts booking a ride with intermediate stops on a ride-hailing app.Source: Figure 6 · p. 15
• Figure 7 depicts a case where the code model actively generates distractors.Source: Figure 7 · p. 15
• Figure 8 depicts a case where the code model imitates visual-trace details and replicates a service agreement.Source: Figure 8 · p. 16