PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning the Loop, Not Just the Page: Execution-Grounded Loop Learning for Web Generation

Yuxin Meng, Ruixu Zhang, Junjie Wang, Yuhan Suo, Yuhan Sun, Ruining Hu, Yiyao Yu, Yubin Wang, Shouwei Ruan, Bin Wang, Yue Liao, Yuxiang Zhang, Yujiu Yang

arXiv:2610.11543v1cs.CL

TL;DR

Existing execution-based Web generation focuses on final-page quality, leaving diagnosis and repair of imperfect implementations less explored. WebLoop jointly learns generation, critique, and refinement, improving results across benchmarks, scales, and input modalities; Qwen3.5-9B reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench.

  • Problem

    Execution-based Web learning mainly optimizes final outputs, while how to diagnose and repair imperfect implementations is less explored, despite the Critic's influence without a directly executable outcome.

    Source: §1 · pp. 1–2

  • Method

    WebLoop jointly trains Generator, execution-free Critic, and Refiner roles in a shared policy, using requirement-level diagnosis and downstream refinement outcomes to credit the Critic.

    Source: §1 · p. 2

  • Results

    WebLoop improves first-pass generation and final refinement across benchmarks, scales, and modalities; Qwen3.5-9B reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench, gains of 11.3 and 15.4 points over the base model.

    Source: §1 · p. 2, §4.2 · p. 7

  • Takeaways & Limitations

    Controlled analyses show that gains cannot be explained by an additional refinement pass alone, and joint loop optimization outperforms staged alternatives.

    Source: §1 · p. 2, §5.1 · p. 8

Abstract

from arXiv · show

Functional Web generation is increasingly optimized with executable rewards, yet existing methods largely focus on the quality of the final page and leave the process of diagnosing and repairing imperfect implementations underexplored. We identify a central challenge in this setting: the Generator and Refiner produce executable artifacts with direct environment rewards, whereas the intermediate Critic influences downstream behavior without a directly executable outcome. We introduce WebLoop, an execution-grounded framework that jointly learns generation, critique, and refinement within a shared policy. WebLoop trains an execution-free Critic with complementary signals for requirement-level discriminability and downstream helpfulness, first establishing reliable diagnosis and then introducing consequence-aware credit, while all three roles are jointly optimized with group-relative policy learning. With Qwen3.5-9B, WebLoop reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench, improving the base model by 11.3 and 15.4 points, respectively. The gains transfer to first-pass generation, persist at 27B scale, and generalize from text-only training to multimodal inputs. Controlled analyses further show that the improvement cannot be explained by an additional refinement pass alone, highlighting the importance of learning the Critic and the loop itself.

1 INTRODUCTION

Existing execution-based Web generation methods emphasize final-page outcomes, leaving diagnosis and repair of imperfect implementations underexplored. WebLoop jointly learns generation, critique, and refinement, and improves benchmark performance with gains that extend beyond a single refinement pass.

1 INTRODUCTION

  • Existing execution-based learning largely optimizes final outputs, while diagnosis and repair of imperfect implementations remain less explored despite their importance in practical Web development.Generator and Refiner outputs receive direct environment rewards, but the intermediate Critic affects later behavior without a directly executable outcome.Source: §1 · pp. 1–2
  • WebLoop jointly optimizes Generator, Critic, and Refiner roles in a shared policy, training the execution-free Critic with requirement-level diagnosis and downstream refinement improvement.It first establishes reliable diagnosis, then introduces consequence-aware credit, and jointly optimizes all three roles through GRPO.Source: §1 · p. 2
  • Qwen3.5-9B WebLoop reaches 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench, improving the base model by 11.3 and 15.4 points, respectively.Gains transfer to first-pass generation, persist at 27B scale, and generalize from text-only training to Markdown, Sketch, Image, and Video inputs.Source: §1 · p. 2
  • The paper frames intermediate Critic credit assignment as a central challenge and evaluates loop learning across benchmarks, scales, and input modalities.Controlled ablations show gains cannot be attributed to an additional refinement pass alone.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work uses executable evaluation to optimize Web artifacts and uses self-generated feedback to improve outputs. WebLoop distinguishes itself by using execution outcomes to train an intermediate, execution-free Critic jointly with generation and refinement.

Functional Web Generation and Execution-Based Learning

  • Execution rewards have been used to optimize Web policies, while WebGen-Agent incorporates iterative interaction or rendering feedback into step-wise optimization (Jiang et al., 2026; Chen et al., 2026; Lu et al., 2025a).These methods establish execution as an effective signal for optimizing Web artifacts; WebLoop instead assigns learning credit to the intermediate critique that guides revision.Source: §2 · p. 3

Critique-Guided Refinement and Self-Improvement

3 WEBLOOP: LEARNING THE GENERATION–CRITIQUE–REFINEMENT LOOP

WebLoop organizes functional Web generation as a shared-policy sequence of generation, critique, and refinement. Browser outcomes supervise executable pages and provide training credit for the execution-free Critic, while role-specific comparisons support joint optimization.

3.1 LOOP FORMULATION AND EXECUTION REWARDS

  • The Generator creates a page, the Critic diagnoses an imperfect implementation, and the Refiner uses that critique to produce a revised page for browser evaluation.Critiques share a selected valid but imperfect base draft, and the Critic does not see browser results or execution traces.Source: §3.1 · pp. 3–4, Figure 2 · p. 4
  • Browser execution evaluates both initial and refined pages with a shared objective, while the Critic receives no direct executable outcome and instead earns credit from diagnosis and downstream refinement effects.The Critic predicts requirement status and provides repair guidance using task requirements and source code as input.Source: §3.1 · pp. 4–5, §3.2 · p. 5

3.2 CREDIT ASSIGNMENT FOR THE CRITIC

  • Critic credit combines requirement-level diagnostic discriminability with retrospective helpfulness, introduced after reliable diagnosis so delayed refinement gains do not substitute for correct diagnosis.Discriminability uses balanced accuracy over satisfied and violated requirements; helpfulness estimates downstream utility from refinements conditioned on each critique.Source: §3.2 · p. 5

3.3 JOINT GROUP-RELATIVE OPTIMIZATION

  • WebLoop forms role-specific comparison groups for Generator outputs per task, Critic outputs per base draft, and Refiner outputs per critique, then jointly updates the shared policy.Local comparisons preserve each role's distinct reward semantics while all three roles contribute to the optimization batch.Source: §3.3 · p. 6

3.4 EXECUTABLE TRAINING DATA CONSTRUCTION

  • The executable training corpus contains 2,880 tasks across 21 domains and 1,442 scenarios, averaging 11.87 requirements per task and 28.15 executable assertions per Interaction Contract Graph.Task construction pairs natural-language specifications with browser-based functional supervision and includes explicit and implicit requirements.Source: §3.4 · p. 6

4 EXPERIMENTS

The experiments evaluate functional correctness on WebRise and WebGen-Bench, with controlled training on text-conditioned tasks. WebLoop improves final and first-pass results, with gains that persist at larger scale and transfer across input modalities.

4.1 EXPERIMENTAL SETUP

  • WebRise has 442 tasks and reports transition validity, explicit and implicit requirement coverage, and their Overall average; WebGen-Bench has 101 tasks and 647 functional test items.WebRise main results use the Text modality; WebGen-Bench evaluations generate websites from an empty workspace with automatic starter-template selection disabled.Source: §4.1 · pp. 6–7
  • Experiments use Qwen3.5-9B and Qwen3.5-27B; controlled ablations use 9B, and Generator, Critic, and Refiner samples are jointly optimized with GRPO unless stated otherwise.Training uses 3K text-conditioned tasks; the Critic starts with discriminability-only training for 15 updates before combined discriminability and helpfulness reward.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • 41.5 Overall on WebRise and 38.9% accuracy on WebGen-Bench: Qwen3.5-9B WebLoop (Refine) improves its base model by 11.3 and 15.4 points, respectively.It also surpasses Qwen3.5-122B-A10B on both benchmarks.Source: §4.2 · p. 7
  • Qwen3.5-9B WebLoop (Gen) rises from 30.2 to 39.1 on WebRise and from 23.5% to 34.2% on WebGen-Bench; gains persist at 27B.At 27B, WebLoop (Gen) rises from 39.3 to 50.2 on WebRise and from 29.8% to 32.5% on WebGen-Bench; the complete loop reaches 52.2 and 42.8%.Source: §4.2 · p. 7
  • Text-only WebLoop training raises average WebRise scores from 31.9 for the base model to 37.5 for WebLoop (Gen) and 40.1 after the complete loop across five modalities.The reported modalities are Text, Markdown, Sketch, Image, and Video, and gains span all three functional metrics.Source: §4.2 · p. 7

5 ANALYSIS AND DISCUSSION

Controlled comparisons find that WebLoop's gains exceed those of refinement without critique or staged training. The Critic's discriminability and downstream-helpfulness signals are complementary, and its training schedule first establishes diagnosis before introducing downstream utility.

5.1 WHERE DOES THE GAIN COME FROM?

  • Table 3's Qwen3.5-9B WebLoop reaches 41.5 on WebRise and 38.9% on WebGen-Bench, versus 40.0 and 26.7% for refinement without critique and 40.1 and 36.1% for staged training.Generation Only reaches 36.9 on WebRise and 23.0% on WebGen-Bench; the larger gap from staged training is already visible in first-pass generation.Source: §5.1 · pp. 7–8, Table 3 · p. 9

5.2 WHAT MAKES A USEFUL CRITIQUE?

  • Discriminability and helpfulness are complementary: discriminability alone performs better on WebRise, while helpfulness alone is stronger on WebGen-Bench.Mixing both signals from the start still falls short of WebLoop's staged objective.Source: §5.2 · p. 8, Table 4 · p. 9
  • Figure 3 tracks discriminability, downstream helpfulness, and Generator and Refiner execution rewards; its dashed line marks the reward switch after 15 updates.Thin and thick curves show per-step values and moving averages, respectively.Source: Figure 3 · p. 9, §5.2 · p. 9
  • WebLoop's Critic curriculum first optimizes discriminability for 15 updates, then switches to helpfulness plus λD while retaining diagnostic supervision.The schedule delays refinement-dependent utility until the Critic has learned a stable notion of correctness; refined outputs remain stronger than first-pass generations.Source: §5.2 · pp. 9–10

6 CONCLUSION

WebLoop jointly learns generation, critique, and refinement, assigning execution-derived credit to the Critic through discriminability and downstream helpfulness signals. The accompanying materials describe its training and objectives, AI use, and reproducibility provisions.

6 CONCLUSION

  • WebLoop jointly learns generation, critique, and refinement, treating critique as an intermediate decision whose credit comes from discriminability and downstream helpfulness.The conclusion reports gains in first-pass generation and final refinement across benchmarks, scales, and input modalities; controlled ablations show an extra refinement pass alone does not explain them.Source: §6 · p. 10

AI USE STATEMENT

  • AI tools assisted with editing, selected code work, and some training-data generation; authors report filtering and browser validation, and say results were not fabricated or post hoc selected.Authors state they made and verified final research and experimental decisions and take responsibility for the manuscript, implementation, data, and findings.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

  • The paper provides training, data-construction, and evaluation details, with code, configurations, and data-construction assets slated for public release upon acceptance.Supplementary sections include rollout configurations, objective definitions, evaluation details, task construction, and role-specific prompts.Source: REPRODUCIBILITY STATEMENT · p. 11

A IMPLEMENTATION AND TRAINING DETAILS

  • The implementation-details appendix supplies additional training details, objective definitions, and controlled-comparison evaluation protocols.The passage describes these as supplements to the main text.Source: A IMPLEMENTATION AND TRAINING DETAILS · p. 14

A.1 TRAINING AND ROLLOUT CONFIGURATION

  • Training uses GRPO with Qwen3.5-9B and Qwen3.5-27B; each task samples three generations, three critiques, and three refinements per critique, jointly optimized in one update.The Critic uses discriminability alone for the first 15 updates, then helpfulness plus weighted discriminability; the roles share model parameters throughout training.Source: §A.1 · p. 14

A.2 ADDITIONAL OBJECTIVE DETAILS

  • The shared base draft is the median-reward valid but imperfect candidate; if none exists, the task gets no Critic or Refiner branch.Retrospective helpfulness measures expected execution gain from refinements under a critique, rather than the critique’s linguistic form; unparsable critiques receive minimum reward and no downstream branches.Source: §A.2 · pp. 14–15

A.3 BROWSER EVALUATION AND BASELINE ADAPTATION

The appendix specifies how benchmarks are evaluated and how controlled baselines are adapted. It also describes executable supervision that supports page-level rewards and requirement-level Critic learning.

A.3 BROWSER EVALUATION AND BASELINE ADAPTATION

  • WebRise uses its Interaction Contract Graph and browser executor, while WebGen-Bench uses its provided harness with automatic starter-template selection disabled.Pages that fail to launch are reported separately from functional failures after initialization.Source: §A.3 · p. 15
  • Same-backbone controls include untrained one-cycle refinement, generation-only training, refinement without learned critique, and staged training before critique and refinement.These variants separate the effects of an extra pass, learned critique, and joint rather than staged optimization.Source: §A.3 · p. 15
  • Critique-GRPO uses a fixed teacher with requirement-level execution information, whereas Critique-RL optimizes only the Critic while keeping the Generator and Refiner fixed.Controlled variants use the same task distribution and browser evaluator wherever applicable.Source: §A.3 · p. 15
  • Same-backbone methods share initialization, benchmark instances, browser infrastructure, and metric computation; WebLoop’s “gen” and “refine” denote first-pass and critique-conditioned outputs.External models use the benchmark-specific execution harness while retaining their released inference interfaces.Source: §A.3 · p. 15

B EXECUTABLE TRAINING DATA CONSTRUCTION DETAILS

  • The executable corpus pairs natural-language tasks with browser supervision for requested functions and behavior across interactions, supporting Generator and Refiner rewards and Critic supervision.The same construction is intended to provide page-level execution signals and requirement-level signals.Source: B EXECUTABLE TRAINING DATA… · p. 16

B.1 TASK AND REQUIREMENT CONSTRUCTION

The appendix builds functional tasks from explicit and implicit requirements, converts them into executable interaction contracts, and describes validation of the resulting corpus and evaluator.

B.1 TASK AND REQUIREMENT CONSTRUCTION

  • Each task pairs a natural-language description with expected-behavior requirements, including explicit functions and interaction entry points and implicit constraints such as state consistency and reset behavior.These requirements allow the initial implementation and its refinement to be tested against the same functional specification.Source: §B.1 · p. 16
  • Tasks average 11.87 requirements, of which 5.47 are explicit and 6.39 implicit; implicit requirements make up 53.9% of the total.Implicit requirements cover behavioral constraints beyond the presence of interface elements.Source: §B.1 · p. 16

B.2 INTERACTION CONTRACT CONSTRUCTION AND EXECUTABILITY FILTERING

  • The Interaction Contract Graph represents functional requirements as browser interaction trajectories and assertions, enabling checks across multiple states and steps.The contract is built from an initial-state specification and turns requirements into executable behaviors rather than textual or visual similarity judgments.Source: §B.2 · p. 16

B.3 HUMAN CONSISTENCY VALIDATION

  • On 300 sampled interaction cases, the automatic evaluator shows strong consistency with two reviewers across accuracy, error, correlation, and agreement measures; human–human agreement is a reference.The table caption identifies the comparison scope but supplies no numerical values in the provided passage.Source: Table 5 · p. 17, §B.3 · p. 18

B.2 INTERACTION CONTRACT CONSTRUCTION AND EXECUTABILITY FILTERING

  • Executing a page against the contract yields transition validity, explicit and implicit requirement coverage, and requirement-level execution labels; the first three signals reward the Generator and Refiner.Requirement-level labels supervise Critic discriminability, so the same contract supports outcome optimization and intermediate Critic credit assignment.Source: §B.2 · p. 17
  • Tasks are retained only when reference implementations from three models each achieve transition validity of at least 0.60, and audited contracts with invalid or misaligned interactions are excluded.The manual audit checks requirement alignment, interaction coherence, assertion validity, structural consistency, and functional discriminability.Source: §B.2 · p. 17
  • The finalized corpus contains 2,880 tasks across 21 domains and 1,442 scenarios, with associated requirements and interaction graphs for generation, diagnosis, and refinement.The task space covers functional Web applications and multi-step workflows rather than isolated visual layouts.Source: §B.1 · p. 16, §B.2 · p. 17

C.1 TRAINING DYNAMICS AND LOOP CO-ADAPTATION

Cross-paired checkpoints show that later Generator and Critic checkpoints each improve refinement quality, with the strongest score when both are latest. Across training, learned-critique refinement develops a larger advantage over refinement without a learned Critic.

C.1 TRAINING DYNAMICS AND LOOP CO-ADAPTATION

  • Critique-guided refinement develops a larger advantage over refinement without a learned Critic, particularly after downstream helpfulness is introduced.Both approaches improve with training; the trend suggests increasing use of learned intermediate feedback rather than benefit from an extra generation pass alone.Source: §C.1 · p. 18
  • 38.4 for the step-20 Generator–Critic pair rises to 41.5 for the step-45 pair, with the best result when both use the latest checkpoint.For every fixed Generator, later Critic checkpoints improve refinement quality; later Generators also improve performance under every fixed Critic.Source: §C.1 · p. 18

C.2 FINE-GRAINED FUNCTIONAL EVALUATION

WebLoop’s refinement gains span independent views of WebGen-Bench’s 647 checks, though the size and consistency of gains vary by model and category.

C.2 FINE-GRAINED FUNCTIONAL EVALUATION

  • Website categories and check categories are complementary views of the same 647 checks, not six mutually exclusive groups.The website taxonomy covers Content Presentation, User Interaction, and Data Management; the check taxonomy covers Functional Testing, Data Display Testing, and Design Validation.Source: §C.2 · p. 18, Table 6 · p. 19
  • 11.8 points on Functional Testing and 22.0 points on Data Display Testing: Qwen3.5-9B final Refine improves over the base model.At 27B, the corresponding gains are 12.4 and 11.8 points; the result covers categories grouped independently across all 647 checks.Source: Table 6 · p. 19, §C.2 · p. 19
  • At 9B, refinement improves four categories over first-pass WebLoop generation but lowers User Interaction by 1.0 point and Design Validation by 0.8 points.At 27B, refinement improves all six categories, suggesting more consistent use of critique-conditioned revision at that scale.Source: §C.2 · p. 19

C.3 CRITIQUE DIAGNOSTICS AND REPAIR ANALYSIS

Critique length is associated with refinement quality differently across Critic objectives, while an example shows diagnosis-guided edits passing browser checks.

C.3 CRITIQUE DIAGNOSTICS AND REPAIR ANALYSIS

  • WebLoop’s refined WebRise Overall rises from 38.5 in the shortest critique-length bucket to 54.0 in the longest, while other objectives show weaker or non-monotonic trends.The analysis is descriptive because critique length was neither controlled nor independently randomized.Source: §C.3 · p. 19, Figure 5 · p. 20
  • WebLoop shows a stronger positive association between critique length and refinement quality than the alternative Critic objectives.The pattern suggests critique utility depends on how feedback is structured and used, not verbosity alone; it does not establish causality.Source: §C.3 · p. 19, Figure 5 · p. 20
  • On Poll Vote, the Critic identifies premature result disclosure and a missing post identifier, and the Refiner’s fixes turn both affected checks from FAIL to PASS.The example illustrates targeted critique-guided repair.Source: Figure 6 · p. 20

C.4 QUALITATIVE ANALYSIS OF THE IMPROVEMENT LOOP

The qualitative cases separate diagnosis, guidance, and implementation as distinct points where the improvement loop can succeed or break down. The role-specific prompts direct generation, execution-free critique, and critique-conditioned repair.

C.4 QUALITATIVE ANALYSIS OF THE IMPROVEMENT LOOP

  • On Poll Vote and File Search, targeted diagnoses and repairs turn failed checks into passes, with the filter repair restoring all 12 files while preserving control reset.The examples align diagnosed defects, implemented changes, and browser-verified improvement.Source: §C.4 · pp. 20–21, Figure 7 · p. 22
  • A false-positive Critic can flag a hidden-checkbox defect even when visible selection and preview behavior already pass.Synchronizing the checkbox preserves those already-correct behaviors, so the passing revision does not validate the negative verdict.Source: §C.4 · p. 21, Figure 8 · p. 22
  • A correct verdict may still yield mislocalized guidance: edits targeting type matching or modal close leave the missing per-tab counts or signing behavior unresolved.In the paired signing critiques, both share discriminability D = 0.75, but only the critique localizing the state conflation leads to the repair that restores signing.Source: §C.4 · p. 21, Figure 9 · p. 23, Figure 10 · p. 23
  • Guidance can be valid but incomplete: adding the missing listener leaves the header checkbox unchecked when a selector error remains unaddressed.Source: §C.4 · p. 21, Figure 11 · p. 24
  • Refinement can fail through either unfaithful implementation of a specified listener or a regression that conflates user like state with aggregate like count.The latter changes the count predicate from PASS to FAIL despite a correct diagnosis.Source: §C.4 · p. 21, Figure 12 · p. 24, Figure 13 · p. 25
  • The cases distinguish correct requirement judgment, sufficient repair guidance, and faithful Refiner implementation; a failed refinement alone does not locate the breakdown.Discriminability evaluates judgment, while helpfulness captures downstream outcomes jointly determined by feedback and implementation.Source: §C.4 · pp. 20–21

D PROMPT TEMPLATES

Loading 2610.11543v1…