Source-linked AI summary
InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang, Zongyu Guo, Jiahao Li, Bin Li, Yan Lu
TL;DR
GUI-agent training is constrained by scarce environments and the difficulty of constructing realistic, interconnected websites. INFINITEWEB synthesizes functional web environments with shared specifications, task-centric test-driven development, diverse website and design inputs, and verifiable evaluators. It surpasses advanced coding agents in realistic website construction and improves GUI-agent performance on OSWorld, Online-Mind2Web, and MobileWorld.
Problem
GUI-agent training is hindered by scarce environments, while existing benchmarks have limited scale and diversity and realistic multi-page generation remains difficult.
Method
INFINITEWEB generates functional web environments using Unified Specification, task-centric test-driven development, website seeds, reference design images, and verifiable task evaluators.
Results
Training on generated environments improves performance by +6.9% on OSWorld, +5.7% on Online-Mind2Web, and +3.9% on MobileWorld, while surpassing advanced coding agents in visual and functional quality.
Takeaways & Limitations
The results support INFINITEWEB as a scalable source of simulated environments for training more capable and generalizable GUI agents.
Takeaways & Limitations
Current tasks operate within individual websites, so coordinated multi-website environments and cross-site tasks remain future work.
Abstract
from arXiv · showhide
GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants. However, training such agents is hindered by the scarcity of suitable environments. We present InfiniteWeb, a system that automatically generates functional web environments at scale for GUI agent training. While LLMs perform well on generating a single webpage, building a realistic and functional website with many interconnected pages faces challenges. We address these challenges through unified specification, task-centric test-driven development, and a combination of website seed with reference design image to ensure diversity. Our system also generates verifiable task evaluators enabling dense reward signals for reinforcement learning. Experiments show that InfiniteWeb surpasses commercial coding agents at realistic website construction, and GUI agents trained on our generated environments achieve significant performance improvements on OSWorld and Online-Mind2Web, demonstrating the effectiveness of proposed system.
1 Introduction
GUI-agent training is limited by scarce, low-diversity environments and the difficulty of generating realistic multi-page websites. INFINITEWEB addresses these constraints with shared specifications, task-driven testing, diverse inputs, and verifiable evaluators, improving both environment quality and agent performance.
- Motivation: Existing benchmarks provide only tens to hundreds of manually constructed applications, limiting training diversity and generalization across real-world websites.They require substantial effort to design applications, tasks, and evaluation criteria.
- Motivation: LLM-generated websites face consistency, correctness, and diversity challenges across interconnected pages and long-horizon tasks.Independent page generation can create incompatible interfaces, conflicting data formats, inconsistent state, functional bugs, repetitive tasks, and homogeneous styles.
- System: INFINITEWEB automatically generates functional web environments at scale for GUI-agent training.The system produces websites, tasks, and evaluators rather than addressing only isolated webpage generation.
- System: Unified Specification derives shared data models and interfaces from user tasks before generating pages, preserving realistic cross-page interactions.This avoids independently generated pages with incompatible implementations.
- System: Task-centric test-driven development refines generated code until task-relevant tests pass, while website seeds and reference images diversify functionality and visual style.Verifiable evaluators track task-related variables and provide dense reinforcement-learning rewards.
- Results: INFINITEWEB surpasses advanced coding agents on WebGen-Bench in both visual and functional quality.The paper presents this as evidence of superior realistic web-environment construction.
- Results: +6.9% on OSWorld, +5.7% on Online-Mind2Web, and +3.9% on MobileWorld are reported after training GUI agents on generated environments.The results support cross-platform transferability of the simulated environments.
2 Related Work
Related work spans interactive GUI benchmarks, website-generation evaluation, and synthetic environment or trajectory generation. INFINITEWEB focuses on synthesizing complete environments, including websites, tasks, and evaluators, to address scalability at the source.
- GUI Agent Benchmarks: GUI benchmarks have progressed from simplified web tasks toward more realistic and complex interactive environments.They support end-to-end evaluation because GUI-agent abilities require interaction with environments.
- LLM-based Code and Website Generation: UI-to-code and website-generation research evaluates converting visual designs or natural-language descriptions into frontend code and complete websites.These efforts establish related evaluation settings for generated web interfaces.
- Synthetic Environment and Data Generation: Synthetic data and procedural generation have been used to improve agent robustness and scalability, including synthetic trajectories for web agents.Prior work includes procedurally generated game levels and automatically evolved computer-use knowledge.
- INFINITEWEB: INFINITEWEB’s environment synthesis includes websites, tasks, and automatic evaluators, addressing scalability at its source.Its unified specification stage derives shared data models and programming interfaces from generated tasks.
3 Method
InfiniteWeb synthesizes functional web environments through a task-derived unified specification, parallel backend and frontend generation, and automatically generated evaluators. Its pipeline targets cross-page consistency, task-relevant correctness, visual diversity, and dense reinforcement-learning rewards.
- System overview: The pipeline takes a website seed and design image, then outputs a functional website, tasks, and automatic evaluators.Website seeds and design images provide diverse functional and visual references.
- Unified Specification Stage: Unified specifications derive tasks, data models, page architectures, and shared interfaces so interconnected pages use consistent data and APIs.System-managed parameters are hidden from page-facing interfaces, preserving identical signatures and formats across pages.
- Task-Centric Backend: The task-centric backend generates data, tests, and implementation code together, iteratively refining code until task-relevant tests pass.Failed tests provide expected-versus-actual outputs and relevant code to guide subsequent fixes.
- Design-Guided Frontend: The design-guided frontend extracts visual characteristics from reference images and uses them to generate consistent, responsive pages in parallel with backend development.The shared framework includes common headers, footers, CSS variables, and page-specific HTML, CSS, and JavaScript layers.
- Evaluator Generation: Task-specific evaluators inspect intermediate state variables to assess partial completion, enabling dense reward signals rather than only binary success.This evaluator design provides partial credit for completed steps in multi-step tasks.
4 Experiments
Experiments evaluate InfiniteWeb for functional correctness, visual quality, and GUI-agent training transfer. It outperforms website-generation baselines and improves agent performance across desktop, web, and mobile benchmarks.
- Experimental setup: The evaluation covers functional correctness, visual quality, and GUI-agent training effectiveness across WebGen-Bench, pairwise visual comparisons, Online-Mind2Web, OSWorld, and MobileWorld.WebGen-Bench evaluates task execution, while the transfer benchmarks test real-world web, desktop, and mobile interactions.
- Functional correctness: 85.6% overall score on WebGen-Bench, significantly outperforming all baselines in functional website correctness.The method also achieves the best reported performance in Functional Testing at 80.9%.
- Visual quality: 69–85% win rate in pairwise visual comparisons against all baselines, with 91% agreement between human and automated evaluations.The comparisons use the same websites generated for the baseline study.
- GUI-agent training: +6.9% on OSWorld, +5.7% on Online-Mind2Web, and +3.9% on MobileWorld after training on InfiniteWeb-generated environments.Improvements are reported across most OSWorld application categories, and the paper attributes gains to data quality rather than quantity.
- Ablation studies: Removing TCTDD reduces the overall score by 5.0 points, while replacing GPT-5 with GPT-4.1 reduces it by 8.2 points.Without TCTDD, the method still achieves 80.6%; with GPT-4.1, it reaches 77.4% versus Claude-Code’s 75.8%.
- Ablation studies: 767 tasks versus 174 with binary reward were discriminative for reinforcement learning, a 4.4× increase from dense rewards.The comparison uses 4,000 generated tasks with four trajectories per task.
5 Conclusion
InfiniteWeb addresses web-environment synthesis challenges through unified interfaces, task-centric testing, website-seed variation, and design-image guidance. The authors report advantages for constructing realistic environments and training more capable, generalizable GUI agents.
- Conclusion: InfiniteWeb addresses consistency through unified interface design, correctness through task-centric test-driven development, and diversity through website seeds and design images.The system is presented as a way to generate functional web environments for GUI-agent training.
- Conclusion: The system surpasses commercial coding agents in the evaluated website-construction scenario and supports GUI-agent training with released systems and datasets.The conclusion frames the release as support for future research on capable and generalizable GUI agents.
Limitations
InfiniteWeb’s limitations concern scope, mobile coverage, and generation cost. The system currently focuses on single-website web environments, does not generate native mobile apps, and requires substantial automated generation effort.
- Single-Website Scope: Cross-website tasks remain outside the current system’s scope, despite improvement on multi-application OSWorld tasks from 3.8% to 9.7%.The authors identify coordinated multi-website environments and cross-site tasks as future work.
- Mobile Evaluation: The system generates web environments but not native mobile applications, despite positive transfer to MobileWorld of +3.9%.Native mobile application environment generation remains future work.
- Generation Cost: Generating one complete website takes approximately 20 minutes and costs $1.93 using GPT-5 batch pricing.Generation involves task generation, architecture design, code generation, and test validation.
A.1 Cross-Domain Transfer Analysis
Cross-domain transfer analysis identifies three GUI capabilities developed through website training: persistence during exploration, complete workflow execution, and avoidance of repetitive loops. These behaviors are illustrated across desktop applications and supported by evaluators and validation mechanisms that structure training feedback.
- Exploration Persistence: Website training develops exploration persistence, with trained agents continuing alternative actions after initial attempts fail.In the VS Code Arabic-language task, the trained model continued for 15 steps instead of stopping after 5.
- Flow Completeness: Website training improves flow completeness: the trained model completed Spotify installation in 13 steps, whereas the baseline stopped after search results.The completed workflow included installation, authentication, waiting, and verification.
- Loop Avoidance: Website training improves loop avoidance: the trained model completed the Thunderbird attachment task in 5 steps, while the baseline repeated actions for 11 steps.The baseline had already attached the file before entering the futile loop.
- Transferability: These capabilities are described as domain-agnostic and applicable to image editors, office suites, and system utilities.The paper connects diverse website interaction patterns and multi-step transactions with transferable behaviors.
- Reward Evaluation: Weighted evaluator checkpoints provide proportional rewards by validating actions, data consistency, and confirmation state.The design supports partial credit and prevents shortcuts such as directly manipulating localStorage.
- Validation: TCTDD iteratively fixes failing implementations until tests pass; most websites require 1–3 iterations, with a maximum of 8.The validation loop detects failures, applies LLM-based fixes, and re-validates the result.
B Data Collection and Implementation
InfiniteWeb collects website seeds and design images from rendered Common Crawl pages, then uses configured generation and training pipelines to produce and evaluate environments. The implementation specifies limits for pages, tasks, validation iterations, and agent-training rollouts.
- Website Seed and Design Image Extraction: Website seeds and design images are derived from Common Crawl pages rendered in a headless browser.An LLM summarizes screenshots into website seeds, while pages violating robots.txt or containing illegal content are filtered.
- Generation Hyperparameters: Each website uses 8–10 tasks, up to 12 pages, and at most 8 TCTDD validation iterations.The generation configuration uses temperature 0.7 and a maximum output length of 32,000 tokens.
- Agent Training: UI-TARS-1.5-7B is post-trained with GRPO using 128 parallel environments and 8 sampled trajectories per task.The training setup uses a 15-step episode limit, learning rate 1e-6, and discount factor γ = 0.95.
- Baseline Implementation: Direct-prompting baselines use GPT-5 with high reasoning effort and receive the same website seed, functionality requirements, technical requirements, and design image.The baseline prompt supports up to 12 pages and localStorage.
C.1 WebGen-Bench Results
InfiniteWeb significantly outperforms the evaluated website-generation baselines on WebGen-Bench, with improvements supported by both ablation results and statistical tests across independent runs.
- Baseline Comparison: InfiniteWeb significantly outperforms Bolt.diy, Claude-Code, and Codex across three independent WebGen-Bench runs.Welch’s t-tests report t=14.81, p<0.001 versus Bolt.diy; t=6.33, p<0.01 versus Claude-Code; and t=6.57, p<0.05 versus Codex.
- Ablation Study: Replacing GPT-5 with GPT-4.1 or removing TCTDD validation significantly degrades performance.The ablations yield t=7.70, p<0.01 and t=2.82, p<0.05, respectively.
C.2 Online-Mind2Web Results
InfiniteWeb improves Online-Mind2Web performance across task difficulties, and controlled comparisons indicate that generated-data quality—not merely quantity—drives the gains. Its advantage is linked to context-managed generation that reduces cross-page inconsistency and supports more complete business logic.
- Results: 200 tasks achieved a significant baseline comparison (t=2.96, p<0.05), 400 tasks achieved a significant comparison (t=4.47, p<0.05), and 600 tasks achieved a stronger significant comparison (t=6.58, p<0.01).The reported tests compare InfiniteWeb results against the baseline across the three training-task counts.
- Data quality versus quantity: Claude-Code training produces marginal or negative improvements, whereas InfiniteWeb achieves significant gains, indicating that data quality rather than quantity drives the improvement.The comparison holds training task count and GRPO setup constant.
- Context management: InfiniteWeb provides each generation step only directly relevant specifications, while Claude-Code accumulates the full codebase in context.The stated context-management difference is associated with fewer cross-page inconsistencies, greater scalability, and more complete business logic for InfiniteWeb.
- Context management: InfiniteWeb scales to 12.6 pages per website compared with 4.9 pages for Claude-Code-generated websites.The passage links the lower Claude-Code scalability to accumulated context and cross-page inconsistencies.
D Human Evaluation for Visual Quality
A human study assessed whether automated visual comparisons correctly identify the implementation closer to the reference design. Human judgments agreed with GPT-5 evaluations in 91% of sampled cases, with disagreements concentrated among closely matched implementations.
- Study design: Human evaluators compared a reference design image with two website screenshots across 100 randomly sampled cases.The sample covered InfiniteWeb-Codex, InfiniteWeb-Claude, and InfiniteWeb-Bolt comparisons with approximately equal representation.
- Findings: 91% agreement between human judgments and automated GPT-5 evaluations supports the reliability of the automated visual quality assessment.The study evaluated which of two implementations more closely matched the reference design.
- Findings: Disagreements primarily occurred when both implementations were reasonably close to the reference design.These cases made the visual distinction less clear-cut for human evaluators.
E Human Verification of Task and Evaluator Quality
Human verification found that the generated tasks and automatic evaluators were usually clear, executable, and reliable, while the iterative validation process left only a small fraction of websites unfixed. The system’s generation pipeline ties task specifications to architecture, interfaces, data, implementation, testing, instrumentation, and evaluation.
- Verification study: Human evaluators assessed whether generated task descriptions were clear and executable and whether automatic evaluators correctly determined task completion.The study randomly sampled 100 tasks from generated websites.
- Verification study: 95 of 100 sampled tasks passed human verification, confirming high-quality tasks and reliable automatic evaluators.The assessment covered both task executability and evaluator correctness.
- Validation loop: Only 1.5% of generated websites remained unfixed after the maximum 8 TCTDD iterations.The reported statistic is used to demonstrate the effectiveness of the iterative test-driven validation approach for functional correctness.
- Task and architecture design: The pipeline generates realistic user tasks for a website type and uses those tasks to design the primary website architecture.The architecture specifies the pages and primary functions needed to support the tasks, with single-user scope and no authentication pages.
- Data and interface design: It derives data models and user-facing interfaces from the tasks, then wraps interfaces to hide system-managed parameters such as user or session state.The interface design is explicitly targeted at single-user agent training.
- Website construction: The system designs complete page architecture, functionality, components, and visual layouts using website specifications and reference design analysis.The generation prompts cover navigation, page components, design characteristics, component layouts, and shared page frameworks.
- Implementation and testing: It generates HTML, CSS, realistic data, business logic, and flow-based integration tests for the resulting website.The implementation prompts connect HTML structure, CSS styling, data models, business logic, and integration testing.
- Evaluation generation: Instrumentation and task-completion evaluators are generated to track relevant actions and verify whether tasks are completed.The pipeline includes prompts for instrumentation requirements, instrumented code, and evaluator generation with instrumentation support.