PaperlayerView exact revision on arXiv

Source-linked AI summary

WebUniverse: Scaling Verified Training Data for Agentic Web Development

Yaorui Ding, Fengqing Jiang, Yichen Feng, Yuetai Li, Zhangchen Xu, Kaiyuan Zheng, Xuanyu Lu, Bhaskar Ramasubramanian, Basel Alomair, Luyao Niu, Radha Poovendran

arXiv:2610.33149v1cs.SEcs.CV

TL;DR

Front-end development agents need demonstrations of editing, execution, inspection, and revision, but public trajectories and reusable task environments remain limited. WebUniverse constructs and verifies web-development tasks and releases paired environments and trajectories; fine-tuning improves performance across reported benchmarks and beats matched public-data training.

  • Problem

    Front-end agent training needs demonstrations of development actions and reusable task environments, while public trajectory resources remain limited.

    Source: §1 · p. 2

  • Method

    WebUniverse builds executable text- and vision-grounded tasks from repositories and real pages, then collects and verifies development trajectories.

    Source: §1 · p. 2

  • Results

    Fine-tuning on WebUniverse-Mix improves visual, functional, and iterative development performance and outperforms matched public-data training across reported metrics.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    The paired trajectories, verification records, and reusable environments support agent training and further trajectory rollout and collection.

    Source: §5 · p. 10

    Vision-grounded tasks use screenshots at selected viewports and do not explicitly capture temporal visual behavior such as animations and transitions.

    Source: Limitations · p. 10

Abstract

from arXiv · show

Training agents for front-end web development requires demonstrations of code editing, execution, browser inspection and revision, yet public trajectories and reusable task environments remain limited. We introduce WebUniverse, a pipeline that constructs verifiable tasks with verified development trajectories from open-source repositories and real web pages. The tasks cover text-grounded application development in standalone and iterative settings and vision-grounded reproduction across pages and viewports. We collect agent trajectories for these tasks and verify the resulting applications against task-specific functional or visual requirements. We release WebUniverse-Env, containing 15K executable task environments, and WebUniverse-Mix, containing the corresponding 15K development trajectories with verification records and scores. Fine-tuning Qwen3.5-35B-A3B on WebUniverse-Mix raises the Vision2Web average score from 51.7 to 68.3, WebGen-Bench accuracy from 34.5% to 63.9%, and Web-Bench pass@2 from 12.2% to 23.1%. The resulting model also outperforms a public-data baseline trained with the same configuration and sample budget across all reported metrics. Ablations show complementary benefits from text and visual trajectories and continued gains as training data scales from 5K to 15K trajectories.

1. Introduction

Front-end development agents must connect code edits to observable functional and visual outcomes, but public development trajectories and reusable task environments remain limited. WebUniverse provides verified trajectories and executable environments for text- and vision-grounded development, with improvements across web-development benchmarks.

Motivation

  • Front-end agents must implement interactive behavior and intended interfaces, then inspect browser outcomes and revise implementations that fall short.Visual resemblance does not guarantee correct behavior, and functional implementations may differ from a visual reference.Source: §1 · p. 2
  • Instruction–code and screenshot–code pairs supervise implementations but do not demonstrate editing, execution, inspection, and correction in a completed application.Capturing these actions requires executable tasks with a development request, starting workspace, reference materials, and assessment criteria.Source: §1 · p. 2

Existing resources

  • Public front-end development trajectories remain limited, with prior released examples focused on text-specified individual tasks rather than visual references or iterative sequences.WebGen-Agent releases a small collection, while FullStack-Agent does not release its trajectories.Source: §1 · p. 2

Empirical validation

  • Qwen3.5-WebUniverse improves performance on Vision2Web, WebGen-Bench, and Web-Bench, and outperforms matched-sample public-data training.Ablations show complementary benefits from text and visual trajectories and continued gains as data scales.Source: §1 · pp. 2–3

Contribution

  • WebUniverse releases 15K executable task environments and 15K corresponding verified development trajectories with verification records and scores.Tasks cover written requirements and screenshot-based reproduction, including standalone, iterative, multi-page, and multi-viewport settings.Source: §1 · p. 2

2. Related Work

Prior resources provide synthetic supervision, executable software tasks, web interactions, or screenshot-to-code examples, but differ in task focus and released materials. WebUniverse focuses on constructing web applications with browser-checked interfaces and verified trajectories.

Synthetic training data

  • Synthetic datasets cover instruction following, reasoning, code generation, and tool use, while software engineering corpora collect test-passing repository trajectories and web-agent corpora collect site interactions.Examples include generated-code tests [Xu et al., 2025d; Wei et al., 2024a], repository issue-resolution trajectories [Yang et al., 2025; Pan et al., 2025], and web interactions [Xu et al., 2025b; Sun et al., 2025; Murty et al., 2024; Pahuja et al., 2025].Source: §2 · p. 3

Task and environment synthesis

Web page and web application generation

  • WebSight and Web2Code pair screenshots with HTML, while WebGen-Bench, WebGen-Agent, and FullStack-Agent provide text-started development examples from fresh workspaces.WebUniverse instead releases 15K verified trajectories covering visual references and iterative development, with per-sample verification records.Source: §2 · p. 3

3. WebUniverse Construction

WebUniverse constructs front-end tasks from repositories and real websites, collects agent trajectories, verifies outcomes, and selects passing samples. Its release pairs reusable environments with 15K verified trajectories spanning text- and vision-grounded development.

3.1. Task Construction

  • Text tasks derive from repository-based application descriptions and vary in specification detail and standalone or iterative development format.Vision tasks use screenshots for single-page, multi-page, or multi-viewport reproduction; visual tasks are standalone.Source: §3.1 · pp. 4–5

3.2. Environments and Trajectory Collection

  • Agents attempt tasks through independent sampling or by refining failed or interrupted attempts, with iterative tasks handing passing workspaces forward.Refinement can use verifier feedback, while resumed samples train only on the resumed portion of the trajectory.Source: §3.2 · p. 5

3.3. Verification and Feedback

  • Task-specific verification checks functionality with browser scripts or a GUI agent and checks visual appearance by comparing rendered pages with references.Visual evaluation uses a six-level rubric and checks required pages at specified viewports.Source: §3.3 · p. 6

3.4. Data Selection

  • Selection requires completed verification and passing conditions, adequate trajectory records, and a consecutive passing prefix of at least three tasks for iterative sequences.Visual comparisons must reach at least good match (0.75); tasks with confirmed benchmark overlap are excluded.Source: §3.4 · pp. 6–7

3.5. Dataset Analysis

  • WebUniverse-Mix contains 15K verified trajectories from 1,559 repositories and 3,500 website sources, paired with 15K reusable environments.The collection has 10,500 text-grounded and 4,500 vision-grounded trajectories; vision-grounded trajectories have higher median token and tool-call counts than text-grounded ones.Source: §3.5 · p. 7
  • A human audit examines 100 accepted and 100 rejected attempts per verifier, assessing task statements, applications, and verification records.The paper reports accuracy, precision, recall, and F1 for all three verifiers.Source: §3.5 · p. 7

4. Experiments

The experiments evaluate visual reproduction, functional development, and iterative development against base and matched public-data models. WebUniverse-Mix improves all three areas, with complementary gains from text and visual trajectories and continued improvement as the dataset grows.

4.1. Setup

  • The setup compares against the original checkpoint and a public-data baseline fine-tuned on a matched 15K sample budget with the same configuration.Evaluation covers page reproduction, functional development, and iterative development; Web-Bench reports pass@1, pass@2, and their arithmetic mean.Source: §4.1 · pp. 7–8

4.2. Main Results

  • Fine-tuning improves visual reproduction, functional development, and iterative development, with gains across viewport sizes and functional check categories (Table 2).Web-Bench pass rates also improve on successive requests within an existing application.Source: §4.2 · p. 8, Table 2 · p. 8
  • With the same checkpoint, configuration, and sample count, WebUniverse-Mix outperforms StepFun-Mix on every reported metric.The advantage is particularly pronounced in functional development, where StepFun-Mix underperforms the original model.Source: §4.2 · pp. 8–9
  • Supplementary traces show more frequent post-edit checks and error–edit–recheck cycles after fine-tuning, although repetition and retries without implementation changes persist.The reported workflow observations are supported by definitions, statistics, and representative cases in Appendix F.1.Source: §4.2 · p. 9
  • Figure 5 illustrates closer matching of reference page layout and assets, while the fine-tuned phone-operator application passes checks that the original model’s output fails.The checks cover plan details, support chat, and account updates.Source: Figure 5 · p. 9, §4.2 · p. 10

4.3. Ablation

  • Text-grounded training primarily benefits functional and iterative development, while vision-grounded training substantially improves visual reproduction; the full mixture scores highest on every reported metric.Vision-grounded training alone leaves functional accuracy and iterative pass rates below the original model.Source: Table 3 · p. 8, §4.3 · p. 10
  • Across nested 5K, 10K, and 15K subsets, Figure 4 shows monotonic improvement on all three benchmarks, with continued value from additional data at 15K despite diminishing gains.Visual reproduction benefits especially from the first subset, while functional accuracy continues to rise substantially as data increases.Source: Figure 4 · p. 9, §4.3 · p. 10

5. Conclusion

WebUniverse scales verified training data for front-end web development through paired executable task environments and development trajectories. Fine-tuning improves visual, functional, and iterative development over matched public-data training, while ablations show complementary benefits from text and visual trajectories and gains with more data.

5. Conclusion

  • WebUniverse pairs 15K verified development trajectories in WebUniverse-Mix with corresponding executable environments in WebUniverse-Env, covering text-grounded development and vision-grounded reproduction.The environments and trajectories include verification records and scores.Source: §5 · p. 10

Limitations

WebUniverse is limited to front-end development, and screenshot-based visual references do not specify temporal behavior. Supplementary details describe how its text and visual tasks, environments, trajectories, and benchmark screening are constructed.

Limitations

  • The collection covers web front-end development; native mobile interfaces, 3D applications, and games are left for future work.Screenshot-based visual references also omit animations and transitions, which future work could represent with videos or interaction recordings.Source: Limitations · p. 10

A.1. Text sources and task generation

  • Text-task seeds come from licensed GitHub repositories created from 2022 onward, and generated tasks change scenarios while preserving interaction mechanisms.Repositories are filtered to exclude third-party libraries, minified files, and binaries; detailed-task generation also receives a scaffold description.Source: §A.1 · p. 13

A.2. Visual reference collection and filtering

  • Visual references are collected from screened sites, captured at three viewports, reviewed in groups, and packaged with navigation paths and assets.Captures freeze animations and remove advertising and consent overlays; provenance records include URLs, timestamps, screenshot hashes, and asset sources.Source: §A.2 · pp. 13–14

A.3. Project scaffolds

  • Detailed tasks use 15 reusable scaffolds, while high-level tasks begin from an empty application workspace.Scaffolds provide entry files, dependencies, and build configuration; the solver chooses and installs a stack for high-level tasks.Source: §A.3 · p. 14

A.4. The agent harness

  • Harbor manages workspaces and OpenCode runs the coding agent; the solving configuration disables delegated tasks and live-page retrieval.The release reconstructs each sample’s actual prompt, tool definitions, user inputs, and any reference images from recorded sessions.Source: §A.4 · p. 14

A.5. Model roles and conversation continuation

  • Trajectory collection uses six models, while browser checks and GUI-based visual grading are handled by designated models from that pool.When attempts are interrupted for persistent tool-call repetition, continuation uses the preserved workspace and trimmed session; only resumed turns are supervised.Source: §A.5 · p. 14

A.6. Decontamination

  • Benchmark-overlap screening uses text similarity thresholds and source-domain matching for visual tasks, followed by manual review of all hits.The release retains 514 text trajectories flagged by semantic similarity and four visual trajectories confirmed as name collisions; flagged samples are grouped separately.Source: §A.6 · pp. 14–15

A.7. Task packaging and validation

Each trajectory is paired with an executable environment that preserves task materials and verification records. The release replays reference solutions against fixed verifiers, and all tested solutions pass in reconstructed environments.

A.7. Task packaging and validation

  • Each environment package preserves the task statement, starting workspace, reference materials, fixed checks, and reference solution.Iterative tasks may start from a preceding solution, a failed implementation handover, or an interrupted workspace and dialogue.Source: §A.7 · p. 15
  • Verification records retain browser-test outcomes, GUI-agent verdicts, or rendered screenshots and visual assessments, depending on the verifier.The release tools convert packages into executable Harbor tasks and replay saved solutions against fixed verifiers; all tested reference solutions pass.Source: §A.7 · pp. 15–16
  • Table 6 distinguishes technologies in initial workspaces from frameworks observed in trajectory code, while Table 7 summarizes tool calls per trajectory.Trajectory framework counts include existing scaffold code; cells report counts and independently rounded column percentages.Source: Table 6 · p. 16, Table 7 · p. 16, §A.7 · p. 16
  • Grading records can trigger a limited number of retries.Source: §A.7 · p. 16

B.1. Coverage and trajectory statistics

The appendix reports trajectory coverage and length statistics, then describes a human audit of the three verifiers. The audit evaluates verifier decisions against human judgments on a sample balanced by original verifier decision.

B.1. Coverage and trajectory statistics

  • The median trajectory length is 26.8K tokens, with P90 66.9K, P99 91.5K, and a maximum of 98.3K; tool calls have a median of 26 and P90 of 79.Table 8 breaks down text-grounded trajectories and environments by specification level and development setting, with identical counts for both.Source: §B.1 · p. 16, Table 8 · p. 17
  • GPT-5.6 Sol contributed 75.3% of retained trajectories, followed by Qwen3.8-27B at 8.4% and GPT-5.5 at 6.9%.MiniMax-M3 contributed 4.3%, GPT-5.4 contributed 4.2%, and Qwen3.5-35B-A3B initial attempts that passed directly contributed the remaining 1.0%.Source: §B.1 · p. 16

B.2. Verifier audit

  • The verifier audit samples 600 attempts: 100 accepted and 100 rejected attempts per verifier, judged against human assessments.The reported measures are accuracy, precision, recall, and F1; the sample is balanced by each verifier’s original decision.Source: §B.2 · p. 17, Table 9 · p. 17

C. Data Example

The PlotPatch example traces an agent building and checking an interactive garden planner from an empty workspace. The appendix materials also report the training configuration and construction of the StepFun-Mix comparison set.

C. Data Example

  • PlotPatch asks for a garden planner where users place crops and paths, track crop areas, and save and reopen named plans.The agent chooses React and Vite, implements grid interactions and local plan storage, then corrects type-checking and lint issues before rerunning checks.Source: C. Data Example · p. 17, Figure 6 · p. 21

Tool call: write

  • Bed Fill traverses adjacent cells matching the original material, while painting updates one cell and saved plans restore the stored grid.The code excerpts show a visited set and queue for traversal, alongside saving and reopening plans.Source: C. Data Example · p. 17, Tool call: write · p. 19

Tool responses (batch excerpt)

User input

  • The user input specifies palette-based cell painting, connected-area Bed Fill, cell clearing, live crop-area summaries, plot clearing, layout suggestions, and named saved plans.The requested interface has earthy colors, crisp grid boundaries, and controls grouped beside the plot.Source: User input · pp. 17–18

Tool response

Step 26: Report completion

D.1. Training hyperparameters

  • Fine-tuning runs share one configuration except for training data; the appendix lists Qwen3.5-35B-A3B, Megatron-LM via ms-swift, eight H100 NVL GPUs, and two epochs.The listed optimizer is Adam with precision-aware updates, and the global and micro batch sizes are 16 and 1.Source: §D.1 · p. 21

D.2. StepFun-Mix construction

  • StepFun-Mix contains 13,500 webpage-generation examples and 1,500 multi-turn software-engineering trajectories from the public Step-3.5-Flash-SFT corpus.The software-engineering trajectories add tool-use and execution-feedback examples; both data components undergo benchmark-overlap screening before mixing.Source: §D.2 · p. 22

D.3. Evaluation setup

The evaluation uses OpenCode integrations to expose benchmark tasks and workspaces, then applies each benchmark’s evaluation procedure. Comparisons with published leaderboards require accounting for setup differences.

D.3. Evaluation setup

  • OpenCode adapters expose each benchmark’s task statement and workspace, while retaining the tools and system prompt selected by each model’s configuration.Models use their recommended inference settings.Source: §D.3 · p. 22
  • Vision2Web uses GPT-5, WebGen-Bench uses GPT-5.4, and Web-Bench uses its test scripts; each benchmark uses the same judge for all compared models.Vision2Web Level 2 supplementary functional tests use Claude Sonnet 4.5.Source: §D.3 · p. 22
  • Vision2Web Level 1 averages 300 desktop, tablet, and mobile viewport scores; WebGen-Bench pools checks across 101 applications, assigning zero credit to missing evaluations.Vision2Web evaluates 100 tasks, and missing viewport evaluations contribute zero.Source: §D.3 · p. 22
  • Web-Bench evaluates 50 projects with 20 sequential tasks each, so earlier failures can prevent later tasks from being reached.The reported metrics are pass@1 and pass@2 under the official scoring procedure.Source: §D.3 · p. 22
  • Published-leaderboard comparisons require accounting for differences from these OpenCode integrations and fixed judges.Source: §D.3 · p. 22

D.4. Vision2Web Level 2 results

On Vision2Web Level 2, fine-tuning modestly raises the composite score, with the increase concentrated in visual fidelity. Supplementary analyses report more frequent inspection and verification, alongside persistent workflow and rendering errors.

D.4. Vision2Web Level 2 results

  • 40.3 to 42.5: the median Vision2Web Level 2 composite score rises modestly, as median visual score increases from 32.7 to 37.6 while median functional score remains similar at 48.3 versus 48.5.The composite is the mean of visual and functional scores within each run; Table 10 reports medians across three runs.Source: §D.4 · pp. 22–23, Table 10 · p. 23

E. Before and After Fine-Tuning

F.1. Agent behavior before and after fine-tuning

  • Inspection rises from 5.0% to 77.0% on Vision2Web and from 73.8% to 85.2% on Web-Bench, while remaining low on WebGen-Bench at 1.0% and 0.0%.The analysis combines retries for Vision2Web and WebGen-Bench but examines individual Web-Bench attempts.Source: §F.1 · pp. 23–24
  • Post-edit verification and error-triggered revision increase on WebGen-Bench from 50.5% to 95.0% and from 45.5% to 73.3%, respectively.Repeated calls persist, and 23/72 (31.9%) of Web-Bench second attempts contain no recorded implementation edits, versus 2/83 (2.4%) for the original model.Source: §F.1 · pp. 24–25

F.2. Vision2Web Level 1 visual analysis

  • Regions scored 0 or 0.25 fall from 28.1% to 8.1%, while regions scored 1 rise from 3.8% to 18.8%.Percentages use each model’s evaluated regions as the denominator.Source: §F.2 · p. 25
  • Rendering and responsive-layout errors persist; 16/100 fine-tuned outputs exceed the reference mobile width by more than 2%, versus 12/100 original outputs.In one fine-tuned output, lower-page content remained hidden in full-page capture, receiving zero on five of seven desktop regions.Source: §F.2 · p. 25

F.3. Web-Bench results by technology and task progression

Web-Bench gains vary by technology domain, while progression improves through early sequential tasks. A separate Vision2Web Level 2 failure analysis finds browser runtime failures, port mismatches, and execution loops among zero-score tasks.

F.3. Web-Bench results by technology and task progression

  • Qwen3.5-WebUniverse exceeds Qwen3.5-StepFun in pass@2 on seven domains, ties on full-stack frameworks, and scores lower on build tools.Versus the original model, pass@2 rises from 5.0% to 25.0% for UI frameworks and from 23.8% to 36.2% for databases, but falls from 35.0% to 23.3% for build tools.Source: §F.3 · p. 26
  • 56% of projects complete at least the first four tasks with Qwen3.5-WebUniverse, versus 40% with Qwen3.5-StepFun and 28% with the original model.Neither fine-tuned model completes all 20 tasks on any project, even allowing up to two attempts per task.Source: §F.3 · p. 26

F.4. Vision2Web Level 2 failure analysis

  • 13 of 18 zero-score Vision2Web Level 2 tasks produce blank pages; none of those traces records inspection of the rendered screenshot.All 13 include HTTP checks, and ten include build checks.Source: §F.4 · p. 26
  • Four zero-score tasks use the wrong port, while another repeats a directory-creation command 2,459 times and reaches the two-hour limit without delivering a launch script.In one wrong-port case, the agent observes a response on 5173 rather than required port 3000 but leaves the launch configuration unchanged.Source: §F.4 · p. 26
Loading 2610.33149v1…