Source-linked AI summary
GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
Yuan Cao, Dezhi Ran, Mengzhou Wu, Yuzhe Guo, Xin Chen, Ang Li, Gang Cao, Gong Zhi, Hao Yu, Linyi Li, Wei Yang, Tao Xie
TL;DR
GUI-agent post-training needs interactive environments with scalable execution and reliable rewards, but real applications are slow and expose only noisy visual proxies. GUI-GENESIS synthesizes lightweight web environments from interaction traces and embeds executable code-native rewards. On held-out real-world tasks, it improves performance over both the base model and real-world RL while reducing latency and cost, and reveals a gap between environment synthesis and task solving.
Problem
Real-world GUI applications impose high latency and unverifiable rewards based on noisy visual proxies, limiting scalable post-training for generalization and long-horizon planning.
Method
GUI-GENESIS uses multimodal code models to reconstruct trace-grounded lightweight web environments with deterministic code-native rewards.
Results
Agents trained in GUI-GENESIS improve relative performance by 14.54% over the base model and 3.27% over real-world RL baselines, while latency falls 10× and savings exceed $28,000 per epoch.
Takeaways & Limitations
The synthesis-navigation gap, where models construct environments they cannot yet solve, suggests a pathway toward self-improving agents.
Abstract
from arXiv · showhide
Post-training GUI agents in interactive environments is critical for developing generalization and long-horizon planning capabilities. However, training on real-world applications is hindered by high latency, poor reproducibility, and unverifiable rewards relying on noisy visual proxies. To address the limitations, we present GUI-GENESIS, the first framework to automatically synthesize efficient GUI training environments with verifiable rewards. GUI-GENESIS reconstructs real-world applications into lightweight web environments using multimodal code models and equips them with code-native rewards, executable assertions that provide deterministic reward signals and eliminate visual estimation noise. Extensive experiments show that GUI-GENESIS reduces environment latency by 10 times and costs by over $28,000 per epoch compared to training on real applications. Notably, agents trained with GUI-GENESIS outperform the base model by 14.54% and even real-world RL baselines by 3.27% on held-out real-world tasks. Finally, we observe that models can synthesize environments they cannot yet solve, highlighting a pathway for self-improving agents.
1. Introduction
GUI-GENESIS addresses the efficiency and reward-verification bottlenecks of real-world GUI-agent post-training by synthesizing lightweight web environments with deterministic code-native rewards. Experiments show improved transfer and lower training cost, while revealing a synthesis-navigation gap relevant to self-improving agents.
- Motivation and contribution: Real-world GUI applications constrain post-training through external dependencies, computational inefficiency, and difficult reward verification.Login sessions, remote synchronization, and noisy visual reward proxies hinder scalable interactive training.
- Experimental findings: 149 real-world mobile tasks show that agents trained in synthesized environments transfer effectively to held-out real-world applications.The evaluation uses zero-shot sim-to-real transfer rather than requiring training directly on target applications.
- Implications: Models can synthesize valid environments they cannot yet solve, exposing a synthesis-navigation gap and a possible route toward self-improving agents.The finding separates environment construction capability from task-solving capability.
- Motivation and contribution: GUI-GENESIS synthesizes lightweight web environments with code-native rewards using multimodal code models.The framework reconstructs applications from user traces while embedding executable assertions.
- Experimental findings: 14.54% relative improvement over the base model accompanies 10× lower environment latency and over $28,000 saved per epoch.These results quantify both the performance and efficiency benefits reported for GUI-GENESIS.
2. Background and Related Work
Prior work has shifted GUI agents from supervised learning toward interactive reinforcement learning, but scalable environments and reliable rewards remain unresolved for general-purpose tasks. Environment synthesis offers a promising direction, although GUI workflows remain harder to reproduce than logical tool use.
- GUI agents and RL: Supervised fine-tuning on static datasets often struggles with error recovery and long-horizon planning.This motivates reinforcement learning from interaction for GUI agents.
- Verifiable rewards: Reinforcement learning with verifiable rewards has improved reasoning in domains with well-defined verifiers, but general-purpose tasks face reward-engineering bottlenecks.GUI tasks lack the straightforward verification available in mathematics and code generation.
- Environment synthesis: Agentic-task training uses simulated interactions, manually engineered environments, and emerging model-generated environments to scale data production.These approaches span tool use and broader multi-step reinforcement learning settings.
- Environment synthesis: GUI environment synthesis remains challenging because visual interfaces and workflows are harder to reproduce than logical tool use.The related work frames GUI synthesis as an open extension of environment-generation research.
3. Problem Formulation
The problem formulation models real applications as partially observable environments whose latent backend states make interaction slow and rewards difficult to verify. GUI-GENESIS therefore targets task-conditioned executable surrogates that preserve relevant behavior while enabling deterministic, fast feedback.
- Real-world environment: Real GUI applications are modeled as POMDPs with latent states, visual observations, actions, transitions, and state-dependent rewards.Agents optimize policies from screenshots even though the true backend state is unobservable.
- Bottlenecks: 106-107 interaction steps can make real-world training intractable because network, rendering, and I/O latency accumulate across steps.Parallelization is also financially constrained, with approximately $24K/day cited for 100 concurrent cloud instances.
- Bottlenecks: Latent ground-truth rewards force reliance on noisy visual proxies, whose estimation errors can enable reward hacking and add VLM inference cost.The verification problem compounds the efficiency problem.
- Synthesis objective: The synthesized environment must preserve real-environment behavior on the target task manifold while providing deterministic code assertions and much shorter step latency.The objective is task-conditioned rather than globally complete simulation.
- Synthesis objective: Executable code provides white-box reward verification and near-instantaneous simulation unlike learned neural simulators such as world models.The design explicitly ties verifiability to source-code access.
4. Methodology
GUI-GENESIS builds task-conditioned web environments from real interaction traces through hierarchical multimodal code synthesis, code-native reward injection, and automated verification. The resulting environments use backend assertions for granular rewards and are tested for both logical and runtime correctness.
- Design overview: Task-conditioned synthesis avoids requiring a globally complete application by preserving behavioral fidelity on task-relevant states.This reduces the scope of generation to what GUI-agent training requires.
- Design overview: The pipeline acquires trace context, synthesizes hierarchical code with rewards, and verifies the executable environment.These are the three automated stages of GUI-GENESIS.
- Trace-Driven Context Acquisition: Execution traces pair screenshots with user actions, grounding synthesis in real visual and interaction dynamics.The traces are collected from baseline-agent attempts on target applications.
- Trace-Driven Context Acquisition: Failed attempts remain useful because they capture interface styles and logical page flows without requiring the synthesizer to invent them.Trace context includes visual patterns and transitions even when the task is not completed.
- Hierarchical Code Synthesis: A multimodal code model uses meta-prompting and plan-and-execute generation to construct the synthesized application.The hierarchical strategy separates system design from sequential component implementation.
- Hierarchical Code Synthesis: The synthesis constraints enforce mobile viewport alignment, visual fidelity, functional isolation, and adversarial distractors.External requests are mocked with local database logic for offline execution.
- Code-Native Reward Injection: Code-native reward functions inspect backend state and support continuous rewards from partial condition satisfaction.For the burger example, α balances sub-goal weights, while the resulting reward remains deterministic and executable.
- Automated Verification: A two-stage verification loop combines self-reflection with Playwright execution tests and retries generation up to K = 5 times after failure.Dynamic tests verify widget interaction and reward-triggering backend transitions before training use.
5. Experiment Setup
The experiments evaluate synthesized environments for sim-to-real transfer, synthetic-task performance, and training efficiency under a controlled agent and RL protocol.
- The study evaluates synthesized environments and their effectiveness for sim-to-real transfer in general-purpose GUI agents.
- Datasets and Tasks: The benchmark uses non-overlapping training and evaluation task sets from the WeChat Mini-App ecosystem.The training set contains 969 instructions, while the evaluation set contains 149 real-world mobile tasks.
- Baselines and Training Environments: The experiments vary training source and reward mechanism while keeping the agent architecture fixed.Baselines include a static-snapshot base model and real-world training with VLM-based rewards.
- Agent and RL Pipeline: All experiments use a unified VLM-based ReAct-style agent trained with Multistep GRPO.GRPO samples complete trajectory groups and optimizes relative advantages against group-average outcome rewards.
- Evaluation Metrics: Performance is assessed using real-world success rate and synthesis success rates based on VLM and code-native rewards.Real-world SR uses human annotation, while synthetic evaluation reports VLM Evaluation SR and Code-native SR.
- Evaluation Metrics: Environment efficiency is measured by average latency and financial cost per simulation step using rollout length= 96.
6. Results
GUI-GENESIS improves real-world transfer over both the base model and real-world RL training, while synthetic evaluation shows stronger alignment with code-native than visual rewards.
- Sim-to-Real Performance Gain: 42.28% Real-World SR (Human Annotation) exceeds the base model by 5.37 percentage points, a relative improvement of 14.54%.
- Sim-to-Real Performance Gain: 3.27% relative gain separates GUI-GENESIS from agents trained directly in real-world environments, which achieve 40.94% Real-World SR.The paper attributes the difference to instability and noisy reward signals in real-world training.
- Fidelity of Synthetic Evaluation: 71.81% VLM Eval SR and 48.99% Native-code SR for GUI-GENESIS exceed the base model and real-world-trained model on synthetic evaluation.The corresponding base-model values are 63.76% and 38.93%; the real-world-trained values are 63.76% and 44.30%.
- Fidelity of Synthetic Evaluation: Native-code SR trends consistently with Human Annotation SR, while VLM Evaluation SR tends to overestimate performance against ground-truth metrics.
- Ablation on Code-Native Rewards: 41.61% Real-World SR from VLM-judged rewards in synthetic environments exceeds real-world training but remains below the 42.28% code-native-reward result.The comparison supports finer-grained and more reliable feedback from program-state-based rewards.
6.2. Detailed Analysis of Synthesized Environments
GUI-GENESIS synthesizes environments at low one-time cost, executes them efficiently, and provides code-native rewards that align with visual judgments while adding finer-grained feedback. Scaling the synthetic training set improves both evaluation metrics.
- 6.2.1. Environment Synthesis Cost and Success Rate: 1118 environments were generated for $302.12 total, with 82.65% passing self-verification and most successful environments requiring no more than two attempts.The reported average generation cost is less than $0.30 per environment, and the cost is one-time.
- 6.2.2. Computational Efficiency at Training: 10× lower latency and over $28,000 saved per RL epoch distinguish local synthesized execution from real-world training.The epoch estimate uses approximately 1,000 environments and 12 rollouts per environment.
- 6.2.2. Computational Efficiency at Training: Local execution removes network overhead, while assert-based rewards eliminate device-rental and VLM API costs.
- 6.2.3. Analysis of Code-Native Reward vs. VLM-as-Judge: Code-native rewards correlate with VLM judgments while distinguishing early-stage failure from near-completion through denser, reproducible values.Among VLM-classified failures, 75% of code-native rewards fall in [0, 0.6]; over 75% of VLM-positive cases exceed 0.8.
- 6.2.5. Scaling Properties of Synthesized Environments: Trajectory-length distributions compare unique interface states needed in real-world and synthesized environments to assess retained interactive complexity.The figure’s stated comparison concerns whether synthesized environments distinctively trivialize task logic.
- 6.2.5. Scaling Properties of Synthesized Environments: Monotonic improvement in both Code-Native SR and VLM Eval SR occurs as the training set scales from 240 to 969 synthesized environments.Agents are evaluated on 149 synthesized environments corresponding to the evaluation instructions.
6.3. Analysis: The Synthesis-Navigation Divergence
GUI-GENESIS reveals an asymmetric capability gap: a code model can synthesize a functional application yet fail to navigate it. Figure 5 presents cases illustrating this synthesis-navigation divergence.
- The analysis examines whether a code model’s ability to synthesize environments aligns with its ability to navigate them.
- The model sometimes generates a functional application but cannot successfully navigate it.This divergence shows that synthesis and navigation are not simply hierarchical in difficulty.
- Figure 5 shows case studies where the code model generates applications that it cannot itself successfully navigate.
7. Conclusion and Future Work
GUI-GENESIS addresses scalability and reward-verification bottlenecks by synthesizing lightweight, interactive, verifiable environments from user traces. Experiments report improved training stability, efficiency, and transfer to real-world tasks, while motivating co-evolution between coding and GUI agents.
- GUI-GENESIS reverse-engineers user interaction traces into lightweight, interactive, verifiable environments using coding LLMs.The framework decouples training from heavy real-world backends and injects code-native rewards.
- The framework provides precise reward signals and accelerates training throughput by orders of magnitude.
- Agents post-trained in synthesized environments achieve superior training stability and efficiency and transfer successfully to real-world tasks.
- Future Work: A potential co-evolution framework could use coding agents to synthesize progressively challenging environments and GUI-agent failures to refine coding agents.
A. Prompt
The prompt specifies how multimodal coding models should generate mobile web applications, including visual alignment, project files, functional behavior, testing, and reward logic. It also requires code review focused on whether the implementation matches the task’s reward rule and interactions.
- Prompt Design: The prompt asks a multimodal coding LLM to build a fully functional mobile web application from user requirements and reference screenshots.
- Visual Alignment: The generated application must target a 410x858 mobile resolution and simulate native-app layouts rather than desktop viewing.
- Code and Testing: The prompt requires a project file list, raw code outputs for requested files, and a Playwright test script.
- Reward Logic: The reward specification assigns cumulative scores of 0.0, 0.3, 0.6, and 1.0 across task-progress states, terminating at the final weather-detail view.
- Self-Verification: The prompt asks the model to judge whether the reward rule matches the requirement and whether the website supports normal interaction.
B. Generation Examples
The generation examples illustrate mobile-app imitation, code-native reward calculation, mocked data interactions, and automated testing. Together, they show how synthesized applications encode both interface behavior and verifiable task progress.
- Mobile Interface: The generated interface uses mobile-oriented styling and a fixed application width and height to imitate a native GUI.
- Code-Native Rewards: The reward implementation emits a terminal RL_REWARD=1.0 when the user views tomorrow’s weather for Lvliang.
- Mocked Data: The examples include mocked city records to support self-contained database-like interactions.
- Mobile Interface: The examples configure generated applications and tests for a 410x858 mobile viewport.
- Automated Testing: The generated code is tested with a Playwright script that launches a browser using the specified mobile dimensions.
C. Supplementary Case Studies
The supplementary case studies illustrate GUI scenarios involving intermediate stops, generated distractors, and imitation of visual trace details with service-agreement replication.
- A ride-hailing case involves booking a ride with intermediate stops.
- Another case examines active distractor generation by the code model.
- A third case covers visual-trace imitation and replication of a service agreement.