WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning

1 INTRODUCTION

• Current code agents' reliance on code-execution feedback misses website visual and interaction quality, leaving issues such as misaligned components, poor coloring, unresponsive buttons, and broken links.Source: §1 · p. 1
• WebGen-Agent iteratively refines websites using a VLM to assess appearance and a GUI agent to test functionality and provide targeted suggestions.Source: §1 · p. 2
• Step-GRPO uses screenshot and GUI-agent scores at each step as process supervision for the reasoning model.Source: §1 · p. 2
• Step-GRPO raised Qwen2.5-Coder-7B-Instruct accuracy from 38.9% to 45.4% and appearance score from 3.4 to 3.7 on WebGen-Bench.Source: §1 · p. 2
• Claude-3.5-Sonnet accuracy rose from 26.4% to 51.9% and its appearance score from 3.0 to 3.9, outperforming Bolt.diy.Source: §1 · p. 2

2 METHOD

• WebGen-Agent combines iterative website refinement using screenshot and GUI-agent feedback with score-based backtracking and best-step selection.Source: §2 · p. 2, Figure 1 · p. 3

2.1 WEBGEN-AGENT WORKFLOW

• Each workflow step generates code, executes it, and gathers feedback; a VLM assesses screenshot appearance before a GUI agent tests specified functionality.Source: §2.1 · p. 3
• The workflow stores code states and scores, backtracks after five consecutive execution-error steps, and selects the best step by GUI score, then screenshot score, then recency.Source: Figure 1 · p. 3, §2.1 · p. 4

2.2 STEP-GRPO WITH SCREENSHOT AND GUI-AGENT FEEDBACK

• Step-GRPO addresses the gap between strong proprietary coding models and smaller open models by training with website-generation scores as step-level supervision.Source: §2.2 · p. 4
• Training begins with one epoch of supervised fine-tuning on approximately 700 WebGen-Agent trajectories before Step-GRPO.Source: §2.2 · p. 4
• For each step, Step-GRPO sums screenshot and GUI-agent scores into an immediate reward and assigns different advantages to tokens from different steps.Source: §2.2 · pp. 4–5

3.1 MAIN RESULTS

• WebGen-Bench contains 101 natural-language website-generation instructions and 647 GUI-agent test cases; WebGen-Agent inference allows at most 20 iterations and uses Qwen2.5-VL-32B-Instruct for screenshot and GUI-agent feedback.For Step-GRPO, 500 website-generation instructions were randomly sampled from WebGen-Instruct for one epoch, and ambiguous or underspecified instructions were manually filtered out.Source: §3.1 · p. 5
• 58.2% accuracy and an appearance score of 4.3 are achieved by Qwen3-Coder-480B-A35B-Instruct, while WebGen-Agent outperforms other agents on tested proprietary models.Across seven proprietary models, the authors report consistently high performance and generalizability.Source: §3.1 · pp. 5–6, Table 1 · p. 6
• 45.4% accuracy and a 3.7 appearance score for Qwen2.5-Coder-7B-Instruct follow Step-GRPO, up from 38.9% and 3.4; Qwen3-8B also improves on both metrics.For Qwen3-8B, Step-GRPO raises accuracy from 38.6% to 43.4% and appearance score from 3.4 to 3.6.Source: §3.1 · p. 7, Table 3 · p. 7

3.2 ABLATION STUDIES

• 3.3% is the largest accuracy gain from GUI-agent testing; screenshot feedback raises appearance score from 3.0 to 3.6, while backtracking and select-best mitigate appearance loss.Manual inspection found that 98.3% of sampled instructions achieved high requirement coverage.Source: §3.2 · p. 7, Table 2 · p. 7
• Naive outcome GRPO and cumulative-advantage variants underperform final Step-GRPO, and single-feedback rewards underperform combined screenshot and GUI-agent scores.Average file count and line count increase after SFT and Step-GRPO for both Qwen2.5-Coder-7B-Instruct and Qwen3-8B.Source: Table 3 · p. 7, §3.2 · p. 8, Figure 3 · p. 8
• Replacing Qwen2.5-VL-32B-Instruct feedback with GPT-4o does not notably improve results, while replacing the coding LLM with the smaller VLM significantly worsens performance.The authors use the smaller, less expensive VLM for feedback and a strong coding LLM for code generation; score reliability is reported in Appendix E.Source: §3.2 · pp. 8–9, Table 4 · p. 8

Visual Code Generation

• Prior visual-code-generation work includes screenshot-to-HTML reconstruction and interactive-element benchmarks, while WebGen-Bench targets multi-file website codebases from scratch.The paper contrasts its benchmark with tasks often solved in a single HTML file or through rigid pipelines.Source: §4 · p. 9

Code Agents

• WebGen-Agent specializes in end-to-end website generation, adding screenshot feedback and GUI-agent testing to a field that also includes specialist and general-purpose code agents.The passage identifies Bolt.diy as specializing in multi-file website generation and OpenHands and Aider as general-purpose agents.Source: §4 · p. 9

Fine-tuning and Reinforcement Learning for Agents

• Prior agent-training methods use ground-truth comparisons, execution outputs, or task success; these provide sparse outcome supervision or require detailed ground truth for step supervision.WebGen-Agent instead uses screenshot and GUI-agent scores available at each step for GRPO supervision.Source: §4 · p. 9

5 CONCLUSION

• WebGen-Agent iteratively generates websites using screenshot and GUI-agent testing feedback alongside backtracking and select-best mechanisms.The stated goals are appealing appearance and smooth functionality.Source: §5 · p. 9
• 58.2% accuracy and an appearance score of 4.3 are reported for WebGen-Agent with Qwen3-Coder-480B-A35B-Instruct, the best-performing configuration.Training Qwen2.5-Coder-7B-Instruct and Qwen3-8B with supervised fine-tuning followed by Step-GRPO also improves accuracy and appearance scores.Source: §5 · p. 10
• Step-GRPO uses inherent screenshot and GUI-agent testing scores as step-level supervision during GRPO training.The conclusion reports that training with this approach improves accuracy and appearance scores for Qwen2.5-Coder-7B-Instruct and Qwen3-8B after supervised fine-tuning.Source: §5 · p. 10

6 REPRODUCIBILITY STATEMENT

• The paper releases workflow code, Step-GRPO training code and data, and WebGenAgent-LM model weights in the supplementary material.It identifies sections and appendices covering the workflow, prompts, training, score inspection, and GUI-instruction assessment.Source: §6 · p. 10

A LIMITATIONS AND FUTURE WORK

• WebGen-Agent targets website generation from natural-language instructions by non-expert users; response speed and complex network conditions are outside scope.Training is limited to 7B- and 8B-parameter models because Step-GRPO would take more than 24 hours on 16 NVIDIA A800 GPUs and available compute is insufficient for larger models.Source: A LIMITATIONS AND FUTURE WORK · p. 14

B WEBGEN-AGENT ALGORITHM

• The algorithm iteratively edits and executes the codebase, gathers screenshot and GUI feedback, and archives step states; five consecutive execution errors trigger backtracking.The best prior step is selected by highest GUI score, then highest screenshot score among the tied steps.Source: B WEBGEN-AGENT ALGORITHM · pp. 14–15

C WEBGEN-AGENT PROMPTS

• The prompts cover screenshot-based description and suggestions, visual-quality scoring, and generating GUI-agent testing instructions from the website specification.The appendix points to separate prompts for evaluating GUI-agent trajectories and providing improvement suggestions, but that prompt figure is not among the supplied passages for this request.Source: C WEBGEN-AGENT PROMPTS · p. 15, Figure 4 · p. 16, Figure 5 · p. 17

D EXAMPLES OF WEBGEN-AGENT TRAJECTORIES

• Three example trajectories show WebGen-Agent iteratively improving website appearance and functionality based on screenshot and GUI-agent testing feedback.Source: D EXAMPLES OF WEBGEN-AGENT TRAJECTORIES · p. 18

E ACCURACY OF SCREENSHOT AND GUI-AGENT TESTING SCORES

• Human annotators checked score accuracy by judging each step’s score against its screenshot or GUI-agent trajectory and correcting inaccurate scores.Source: E ACCURACY OF SCREENSHOT AND GUI-AGENT… · p. 18
• 93%–96% screenshot-score accuracy and 89%–93% GUI-agent-score accuracy were reported across experiments, with standard errors of 0.20–0.26 and 0.31–0.44, respectively.Table 5 reports score accuracy and standard error against human scores.Source: E ACCURACY OF SCREENSHOT AND GUI-AGENT… · p. 18, Table 5 · p. 20
• 94.8% to 95.5% screenshot-score accuracy and 91.2% to 92.2% GUI-agent-score accuracy are the marginal gains reported when using GPT-4o instead of Qwen2.5-VL-32B-Instruct.The paper describes Qwen2.5-VL-32B-Instruct as sufficient and significantly more cost-effective.Source: E ACCURACY OF SCREENSHOT AND GUI-AGENT… · p. 20

F ANALYSIS OF THE COMPREHENSIVENESS OF GUI-AGENT TESTING INSTRUCTIONS

• Four coding-model runs were manually assessed on a 1–5 scale based on how completely each instruction converts website requirements into concrete GUI-agent checks.The grading guidelines are presented in Fig. 11.Source: F ANALYSIS OF THE COMPREHENSIVENESS OF… · p. 20
• 77.2% of GUI-agent testing instructions score 5, 98.3% score at least 4, 1.7% score 3, and none score below 3.Table 6 reports the distribution of human comprehensiveness scores and the resulting average score; the supplied prose gives the score distribution.Source: F ANALYSIS OF THE COMPREHENSIVENESS OF… · p. 20
• The authors conclude that the generated GUI-agent instructions comprehensively cover most website requirements.Source: F ANALYSIS OF THE COMPREHENSIVENESS OF… · p. 20

• WebGen-Agent achieves superior performance across all instruction and test-case categories compared with other code-agent systems.Table 7 reports categorical results for proprietary and open-source models on WebGen-Bench.Source: G CATEGORICAL RESULTS · p. 20, Table 7 · p. 21
• For both 7B and 8B models, Step-GRPO improves performance in most categories over the original instruct and SFT models.The reported workflow and training process incorporate screenshot and GUI-agent feedback.Source: G CATEGORICAL RESULTS · pp. 20–21
• The categorical comparison covers various proprietary and open-source models on WebGen-Bench.Table 7 compares WebGen-Agent with other code-agent systems.Source: G CATEGORICAL RESULTS · p. 20, Table 7 · p. 21

• The iteration-limit analysis measures accuracy, appearance score, and the percentage of samples exceeding the limit.DeepSeek-V3 is the coding LLM used.Source: H ANALYSIS OF MAXIMUM ITERATION NUMBERS · p. 22
• Accuracy and appearance score rise as the maximum iteration number increases, while exceed rate falls.The analysis evaluates all three metrics at different limits using DeepSeek-V3.Source: H ANALYSIS OF MAXIMUM ITERATION NUMBERS · p. 22
• At maximum iteration limits of 14–20, accuracy, appearance score, and exceed rate begin to converge.Most samples finish before reaching the limit, diminishing its impact on performance.Source: H ANALYSIS OF MAXIMUM ITERATION NUMBERS · p. 22

• Supervised fine-tuning reduces erroneous or malformed websites and improves adherence to instruction-specified appearance requirements.Examples cover Qwen2.5-Coder-7B-Instruct and Qwen3-8B and their SFT and Step-GRPO versions.Source: I QUALITATIVE ANALYSIS OF SUPERVISED… · p. 22
• Step-GRPO further refines the aesthetics and harmony of generated websites beyond supervised fine-tuning.The qualitative comparisons are shown for both 7B and 8B models.Source: I QUALITATIVE ANALYSIS OF SUPERVISED… · p. 22
• The qualitative comparisons show outputs from original, SFT, and Step-GRPO versions of 7B and 8B models.Source: I QUALITATIVE ANALYSIS OF SUPERVISED… · p. 22

J QUALITATIVE ANALYSIS OF THE WEBGEN-AGENT WORKFLOW

• Trajectory examples show the agent improving visual appeal and functionality in response to screenshot and GUI-agent testing feedback.The illustrated steps are simplified due to space constraints.Source: J QUALITATIVE ANALYSIS OF THE… · p. 22, Figure 17 · p. 31, Figure 22 · p. 36

K USAGE OF LARGE LANGUAGE MODELS IN PAPER WRITING

• The paper is primarily human-written; LLMs check grammar and spelling and occasionally polish wording for fluency.The named models include o3 and DeepSeek-V3.Source: K USAGE OF LARGE LANGUAGE MODELS IN… · p. 22