PaperlayerView exact revision on arXiv

Source-linked AI summary

AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards

Xin Gao, Yifan Yang, Xin Li, Pan Wang, Yan Li, Muzhao Tian, Mingxi Cheng, Bei Liu, Kai Qiu, Qi Dai, Dong Chen, Yue Dong, Leandros Tassiulas, Rex Ying, Ji Li, Chong Luo

arXiv:2610.18860v1cs.AI

TL;DR

Code models cannot see how layout, hierarchy, and color work together on the canvas, while generated images can be unreliable content targets. AesCode uses a design graph and decoupled rewards to train code generation with image-based aesthetic guidance. Its models improve visual quality over same-scale Qwen3-VL backbones and reach visual quality comparable to frontier models.

  • Problem

    Code is editable and verifiable, but models cannot see canvas-level layout, while simply supplying noisy references to VLMs can entangle visual cues with text semantics.

    Source: §1 · pp. 1–2

  • Method

    AesCode builds a design graph separating required content from visual cues, then trains code generation with deterministic checks and graph-tied VLM aesthetic rewards.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    31.1 and 22.4 Visual points over their Qwen3-VL backbones for AesCode-8B and AesCode-32B, respectively; visual quality is comparable to frontier models, with fewer content errors.

    Source: §1 · p. 2

  • Takeaways & Limitations

    AesCode retains much of this capability even without reference input at inference.

    Source: §5 · p. 10

    Code generation cannot reliably anticipate final pixel geometry; canvas overflow persists, though it is rare after training, leaving visually grounded refinement for future work.

    Source: §4.4 · p. 10

Abstract

from arXiv · show

Generating information-rich visual artifacts such as slides, posters, and dashboards as code is appealing because the outputs are structured, editable, and verifiable. Yet code models cannot see how their choices of layout and color come together on the canvas, often limiting spatial planning and aesthetic quality. Image generators excel at visual design but can misrender requested content, making their outputs unreliable targets for direct reproduction. We therefore introduce AesCode, which generates code grounded in the content prompt while using an image generated from the same prompt as an aesthetic guidance. To do so, AesCode represents the target as a design graph that describes the whole canvas and makes each property attributable. From this graph, we derive complementary cross-modal rewards: deterministic verifiers score properties parsable from code and rendering, while a VLM judge assesses the non-parsable ones with a sample-specific Visual Graph Rubric (VGR) tied to graph elements and relations. We optimize these decoupled rewards through reinforcement learning with per-channel normalization to jointly improve spatial planning, aesthetic quality, and verifiable correctness. To evaluate this setting, we build a benchmark that scores verifiable correctness and visual quality separately, with visual scores highly consistent with human judgments. We find that reference images substantially improve the visual quality of code generated by existing models, and ablations show that AesCode benefits from references both as model input and as a source of visual rewards. Relative to same-scale Qwen3-VL baselines given the same references, AesCode-8B and AesCode-32B improve visual quality by 31.1 and 22.4 percentage points, reaching scores comparable to those of closed-source frontier models such as GPT-5.5 and Claude Opus 4.8.

1 Introduction

Code offers editable, verifiable outputs, but models cannot see how layout, hierarchy, and color work together on the canvas. AesCode uses generated references as aesthetic guidance while grounding content in the prompt, with design-graph rewards and improved visual results.

1 Introduction

  • Code models cannot see the canvas, while image generators can misrender requested content, making generated images unreliable targets for direct reproduction.The paper motivates using a reference for visual guidance while preserving the prompt as the content requirement.Source: §1 · pp. 1–2
  • The design graph separates required content from visual cues and makes element attributes and relations explicit, enabling reward channels tailored to different properties.Nodes represent elements such as text blocks, tables, and charts; their content and visual attributes are decoupled, and edges encode structural relations.Source: §1 · p. 2
  • AesCode-8B and AesCode-32B gain 31.1 and 22.4 Visual points over their Qwen3-VL backbones, respectively, with visual quality comparable to frontier models and fewer content errors.The benchmark evaluates verifiable correctness and visual quality separately, and its automated visual scores closely match human judgments.Source: §1 · pp. 2–3
  • AesCode uses a same-prompt generated reference as an aesthetic prior for the code model, both as input and as a source of visual training signals.The approach challenges the model to satisfy prompt content while flexibly extracting layout and style from a noisy reference.Source: §1 · pp. 2–3

2 Related Work

Prior work generates visual artifacts as images or code, uses images as reconstruction targets, or relies on limited rules and holistic judges for rewards. AesCode instead trains code models to use generated images as aesthetic priors and derives graph-tied reward criteria.

Visual Design and Code Generation

  • Image generators create appealing pages but can misrender dense text and data, while text-only code generation lacks page visibility and image-to-code methods treat images as ground truth.DesignAsCode and Editable Visual Design let agents consult generated images at inference time, whereas AesCode trains the code model to use them as aesthetic priors.Source: §2 · p. 3

Reward Design for Visual Generation

  • Image-similarity rewards can penalize valid alternatives and copied errors, rule-based rewards check only programmable properties, and holistic MLLM scores can vary with prompt phrasing.AesCode builds a design graph from the prompt and reference, using rules for code-level properties and a Visual Graph Rubric for remaining visual properties.Source: §2 · p. 3

Multi-Objective Reinforcement Learning

  • Summing multiple rewards before normalization can let dense, near-saturated rule rewards drown out sparse aesthetic rewards, so AesCode adopts GDPO to normalize each separately.The cited alternatives include MO-GRPO, which reweights by variance, and GDPO, which normalizes each reward separately.Source: §2 · p. 3

3 Method

AesCode uses graph-structured supervision to combine prompt requirements with non-authoritative reference cues, route properties to complementary reward channels, and train code generation. It separately evaluates factual correctness and human-aligned visual quality.

3 Method

  • AesCode constructs target design graphs from prompt-required content and non-authoritative reference cues, then routes properties to deterministic and perceptual reward channels.The method overview also places SFT and GDPO training after reward construction, followed by a separate evaluation protocol.Source: §3 · pp. 3–4, Figure 2 · p. 4

3.1 Design Graph as a Supervision Target

  • The target graph represents the canvas, elements, attributes, and typed relations, so prompt requirements and reference cues can constrain complementary properties independently.Elements include text, charts, and tables; attributes capture content, geometry, and appearance, while relations include containment, alignment, order, adjacency, and semantic connection.Source: §3.1 · p. 4

3.2 From Design Graphs to Multi-Objective Rewards

  • Properties measurable from code or rendering use deterministic verifiers, while appearance and higher-order relations require sample-specific visual judgment.The generated code and render are jointly parsed into a candidate graph and compared property by property with the target graph to derive rewards.Source: §3.2 · p. 4, §3.1 · p. 4

Deterministic verifiers.

AesCode combines deterministic checks, graph-tied visual questions, and per-channel reward normalization, then evaluates artifact correctness separately from visual quality. Its benchmark uses fixed criteria and human review to establish the visual evaluation protocol.

Deterministic verifiers.

  • Six deterministic verifiers assess renderability, required text, canvas boundaries, table or chart structure, reference-aligned text positions, and whitespace distribution.They draw evidence from code, the DOM, OCR, and rendered pixels; table and chart checks prevent static substitutes from receiving credit.Source: Deterministic verifiers · pp. 4–5

3.3 Training Pipeline

  • Cold-start SFT precedes RL on seven rewards, and GDPO normalizes channels separately so applicable advantages contribute without first collapsing raw rewards into one scalar.Applicability masks exclude irrelevant checks from the policy update.Source: §3.3 · p. 5

3.4 Final Evaluation Protocol

  • Final evaluation reports Rule, Visual, and their mean Overall; Visual uses fixed, sample-specific yes/no criteria for content, layout, and style.The protocol was refined on a separate 50-instance set, then frozen for a disjoint 300-instance benchmark after consistency exceeded 90%; its scores show strong agreement with human judgments.Source: §3.4 · pp. 5–6

4 Experiments

Experiments compare AesCode with prompt-only and reference-conditioned models, evaluate transfer and ablations, and assess human agreement and residual failures.

4.1 Experimental Setup

  • The 300-prompt benchmark compares prompt-only and reference-conditioned models, with additional evaluations on Design2Code, SlidesBench, and GDB/lica.The benchmark’s reference-conditioned models receive generated reference images.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 Main Results

  • Overall rises from 51.93 to 82.94 at 8B and from 61.10 to 85.89 at 32B; AesCode leads in verifiable correctness while matching GPT-5.5 and Claude Opus 4.8 visually.The benchmark averages Rule and Visual scores over three generations per prompt.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • AesCode-32B improves on its Qwen3-VL-32B backbone in all fourteen measures across three external benchmarks, with smaller gains on Design2Code.The gains are largest on design-sensitive measures; GPT-5.5 remains stronger in exact screenshot fidelity.Source: §4.2 · pp. 7–8, Table 2 · p. 8

4.3 Ablation Study

  • Reference-based rewards raise prompt-only Visual from 25.17 to 69.71, while reference-conditioned training scores 5.09 points higher under the same inference setting.Removing the reference only at inference reduces Visual by 1.00 point for AesCode-8B, compared with 19.55 for its backbone and 10.04 for GPT-5.5.Source: §4.3 · pp. 8–9, Table 3 · p. 9
  • GDPO adds 5.12 Visual points over scalar GRPO; VGR reaches 75.80 Visual, versus 68.91 for holistic VLM scoring and 72.61 for direct rubric authoring.The direct-rubric variant has unchanged Rule (90.07 vs. 90.05).Source: §4.3 · p. 9, Table 4 · p. 9
  • Under the same RL budget, SFT initialization raises Overall from 77.97 to 82.94; Layout/Whitespace raises Visual from 66.59 to 68.12.The ablations find that RL drives most gains, while SFT provides a stronger initialization.Source: §4.3 · p. 9, Table 4 · p. 9

4.4 Analysis

  • The Visual score agrees with direct human pairwise choices in 80.66% of comparisons; AesCode-32B has higher Content satisfaction than GPT-5.5 (86.50% vs. 83.47%) and equivalent aesthetic quality.Human-majority judgments agree with training VGR and the evaluation checklist on 89.3% and 93.3% of sampled predicates, respectively.Source: §4.4 · p. 9, Figure 4 · p. 10
  • Residual failures include rare canvas overflow, lost relational structure, and semantic elements degraded into generic marks.The authors identify visually grounded refinement as future work for overflow; relation-level rewards remain sparse and do not localize failed edges.Source: §4.4 · p. 10, Figure 5 · p. 10

5 Conclusion

The conclusion presents AesCode as a way to train code models to use generated images as aesthetic guidance while grounding content in prompts. The accompanying discussion distinguishes this approach from image-to-code reconstruction and inference-time image guidance, and describes the data and training setup.

5 Conclusion

  • AesCode matches frontier systems in human-aligned visual quality, surpasses them in verifiable correctness, and retains much capability without reference input at inference.The conclusion also frames image generators as scalable teachers of visual design for structured generative models.Source: §5 · p. 10

A Extended Related Work

  • Unlike inference-time image-guided pipelines, AesCode trains the code model to use a prompt-generated image as an aesthetic prior while ignoring its content errors.Earlier text-to-code methods compose pages without seeing them, while image-to-code methods treat an existing rendering as the target for closer reconstruction.Source: A Extended Related Work · p. 16
  • Generated references are not exact targets: image-similarity rewards can penalize valid designs, rule-based rewards cover only program-decidable properties, and holistic judges can be unstable or prompt-insensitive.The related-work discussion notes that holistic judges may reflect general quality rather than what suits a particular prompt.Source: A Extended Related Work · p. 16
  • The Visual Graph Rubric derives narrow questions from the prompt and reference together, checking reference-proposed layout and style while tying failures to specific elements.Program-decidable properties remain with rule-based verifiers.Source: A Extended Related Work · p. 16
  • GDPO normalizes each reward channel separately because channels differ in firing frequency and variance, avoiding interference from summing differently scaled rewards before group normalization.The discussion contrasts this choice with MO-GRPO's variance-based reweighting.Source: A Extended Related Work · p. 16

B.1 Source Designs and Stratified Sampling

  • The data pipeline infers content-only prompts and requirements from human-designed artifacts, then uses generated references for RL and benchmarking rather than showing the original designs to the policy.The generated image provides a contemporary visual proposal but may disagree with its prompt in strings or ordering.Source: §B.1 · pp. 16–17
  • The source pool contains approximately 18,000 designs and is sampled across CLIP-cluster distance strata; the SFT, RL, and benchmark splits are disjoint.The collection combines public MIT- and Apache-2.0-licensed sources with internal training-only data; produced artifacts are released instead of source images.Source: §B.1 · p. 17

B.2 Prompt Inference and SFT Demonstrations

  • SFT prompts specify artifact content, major layout zones, and reading flow while leaving detailed visual styling to the image; 3,000 demonstrations are filtered from three HTML candidates per prompt.Each demonstration pairs the prompt and original design with selected code and requirements, and the reward system scores the rendered candidate.Source: §B.2 · p. 17

B.3 RL and Benchmark References

The RL set is selected for prompts where the cold-start policy has room to improve, while the held-out benchmark uses independently drawn items with generated references. Evaluation uses shared prompts and references where supported, averaging multiple unselected rollouts.

B.3 RL and Benchmark References

  • The RL set retains prompts where pass@1 and pass@8 differ, excluding near-universal successes with little group-normalized gradient signal and all-failure prompts with none.Difficulty-filtered prompts are rebalanced across CLIP clusters, yielding 7,408 prompts shared by both backbones; each receives a newly generated reference, and the policy never sees its original design.Source: §B.3 · p. 17
  • The held-out benchmark contains 300 items across four artifact types and CLIP clusters, with no source design shared with SFT or RL.Items are stored independently of model outputs with the request, canvas metadata, verifiable requirements, evaluation checklist, and reference.Source: §B.3 · pp. 17, 19
  • All systems receive the same prompt and reference when image conditioning is supported, and scores average three rollouts per prompt without selection.Rendering failures remain in the denominator; Execution is diagnostic only, while non-conditional Table 3 experiments differ only in whether the policy sees the reference.Source: §B.3 · p. 19

B.4 Dataset Statistics

The dataset is stratified across CLIP clusters and benchmark items carry checkable content requirements. Prompt construction keeps those requirements tied to requested content while rewriting references for image generation; the additional-results section identifies the Rule and Visual score columns used.

B.4 Dataset Statistics

  • The 300-item benchmark is balanced across 25 CLIP clusters, with 9 to 20 items per cluster and a median of 12 verifiable requirements per item (range 3 to 23).The requirement count favors fewer high-confidence checks; the 300 benchmark identifiers have no intersection with the RL training file, including 27 RL-candidate items removed when drawn.Source: §B.4 · p. 19
  • Table B1 summarizes the three splits, and both backbones use the same RL file.Benchmark examples are multilingual and vary in canvas shape, inherited from their source designs.Source: Table B1 · p. 19, §B.4 · p. 19

B.5 Prompts Used for Data Construction

  • Prompt inference returns the generation prompt and verifiable requirements together, with exact-string checks limited to strings explicitly requested in the prompt.Prompts specify layout and reading flow but leave detailed appearance to the reference; charts are requested as ECharts data and non-reproducible visuals as sized placeholders.Source: §B.5 · pp. 19, 21
  • The reference-image prompt is rewritten to describe placeholder illustrations and drop code-only constraints while preserving layout, text, and chart and table values.The code prompt retains placeholder requests, and the judge treats a placeholder as the image being present.Source: §B.5 · p. 21

C Implementation and Additional Results

C.1 Implementation Details

AesCode uses Qwen3-VL backbones trained with cold-start SFT followed by GDPO, with shared RL prompts across model scales. During training, text and chart rewards improve faster than design channels, while training and held-out curves show no visible reward trade-off or overfitting.

C.1 Implementation Details

  • Both Qwen3-VL-8B and Qwen3-VL-32B use 3,000 SFT demonstrations and the same 7,408 GDPO prompts, differing only in capacity.Training runs for 400 steps at 8B and 520 steps at 32B.Source: §C.1 · pp. 21–22
  • Benchmark generation averages three rollouts per prompt, and hosted baselines receive the same prompt and rollout count without benchmark tuning.Reasoning-mode endpoints use provider defaults; other hosted arms use temperature 0.8 and top-p 0.95.Source: §C.1 · p. 22
  • The sandbox renders candidate HTML locally and exports DOM, styles, bounding boxes, console status, and a canvas-clipped screenshot.External requests are blocked except locally served ECharts and placeholder assets.Source: §C.1 · p. 22

C.2 Learning Dynamics of the Decoupled Reward Channels

  • After cold-start SFT, Execution saturates, while Text and Chart improve fastest, with pass@1 rising from ≈0.79 to ≈0.94 for Text and ≈0.58 to ≈0.87 for Chart.Source: §C.2 · p. 22
  • Pass@1 converges toward pass@8, while close training and held-out reward curves show no visible trade-off or overfitting.Boundary stabilizes early with low within-group variance, a signal the authors say scalar aggregation would suppress.Source: §C.2 · p. 22

C.3 Loss of Verifiable Capabilities under Reconstruction Objectives

This section examines how reconstruction objectives can trade verifiable chart correctness for visual similarity, and describes AesCode’s reward records, deterministic checks, and graph-bound visual rubric. It also defines a frozen, prompt-derived checklist for judging whether visual properties and spatial relations are verifiable and whether candidate renderings meet a professional standard.

C.3 Loss of Verifiable Capabilities under Reconstruction Objectives

  • Reconstruction objectives improve visual dimensions but sharply reduce chart correctness, with static lookalikes substituting for live charts.CLIP-similarity GRPO raises Layout from 71.19 to 78.45 and Visual from 60.11 to 65.69; Chart falls from 42.35 on the backbone to 9.67, 9.78, and 15.27 across the reconstruction arms. Among 89 prompts requiring chart or table data, those arms chart 15, 8, and 26 prompts, versus 73 for the backbone and 80 for AesCode-8B.Source: §C.3 · pp. 22–23, Table C1 · p. 23

D.2 Verifiable Requirements

  • Verifiable requirements are extracted from the prompt before generation as exact text, chart or table data, and target canvas dimensions.Pairwise and group relations are supervised by graph-bound VGR predicates during training and evaluated by the prompt-derived Visual checklist at test time.Source: §D.2 · p. 23

C.4 Additional Qualitative Results

  • AesCode-32B qualitative examples span held-out dashboards, diagrams, slides, and posters in multiple languages, with no format-specific prompt or decoding changes.Figures C2 and C3 show randomly sampled, unedited rollouts rendered in the evaluation sandbox.Source: §C.4 · p. 23, Figure C2 · p. 24, Figure C3 · p. 25

D.1 Per-Rollout Reward Record

  • Each rollout stores an applicability mask, raw score, and diagnostic payload per channel, distinguishing inapplicable cases, explicit failures, and infrastructure errors.Not-applicable results are excluded from aggregation, while explicit zero-score failures are included; judge timeouts are not treated as negative design feedback.Source: Table D1 · p. 24

D Reward System

  • AesCode derives complementary reward channels from the prompt, its requirements, and the reference, without requiring pixel-perfect reconstruction.The reward system separates properties verifiable from code or rendering from those evaluated visually; reported scores are defined separately.Source: Figure D1 · p. 26

D.3 Deterministic Verifiers

  • Deterministic verifiers check execution, text, boundaries, tables and charts, semantic layout, and whitespace using rendered output and structured data.Table and chart checks parse candidate rows or live ECharts series against required data; semantic layout assesses content placement, while whitespace compares occupancy grids.Source: Figure D1 · p. 26, §D.3 · pp. 26–27

D.4 Visual Graph Rubric

  • VGR converts a reference-derived scene graph into bound yes/no predicates, which GPT-5.5 scores using only the candidate rendering and fixed questions.Rubrics commit to a layout class before question writing, fold repeated elements into group nodes, and assess opaque image slots by geometry rather than depicted content. Across 7,408 training rubrics, the linter flags 20 (0.3%), 99.6% of questions bind to graph elements, and rubrics average 14.1 questions.Source: §D.4 · pp. 27, 28, 29, 31
  • VGR examples bind visual checks to stored graph nodes, relations, and the canvas, with reference-derived criteria kept out of the generation prompt.The examples ask about properties such as element placement, grid structure, containment, overlap, and connector relations.Source: §D.4 · pp. 32–33, Figure D5

E Evaluation Protocol and Human Study

  • The final benchmark scores candidates from prompts and renderings alone, without reusing reference-conditioned SemanticLayout, Whitespace, or VGR rewards.Rule averages prompt-derived Text, Boundary, and Table/Chart scores; Visual uses the checklist pipeline.Source: E Evaluation Protocol and Human Study · p. 34

E.1 Visual Checklist

  • The frozen checklist contains 10,727 questions across Content, Layout, and Style, with 26–50 questions per prompt and a mean of 35.8.It was developed on 50 separate instances and applied unchanged to 300 benchmark items; dimension scores are question pass rates, averaged for Visual.Source: §E.1 · p. 34
  • Judges must cite visible evidence for each answer, and unresolved or ambiguous questions are scored as failures rather than silently passing.Questions are judged from one rendering without a reference; both yes and no require evidence because question polarity varies.Source: §E.1 · p. 36
  • Spatial-relation checks require locating both named elements and verifying their actual relation; touching, crossing, clipping, or partial containment fails the check.If an element or relation cannot be resolved at the available resolution, the answer is “unclear” rather than an assumed pass.Source: §E.1 · p. 37
  • A property passes only when specific visual evidence is identifiable and a complex construct is properly realized; “unclear” counts as failure, not a guessed pass.The checklist rejects superficial approximations such as boxes without a timeline spine and requires repeatable judgments based on specific pixels.Source: §E.1 · p. 37
  • The canvas-level spread check fails pages with an empty region larger than about a quarter of the page, including a wide blank band, empty side, or central hole.Normal separation between sections does not count as an empty region.Source: §E.1 · p. 40
  • Aesthetic checks judge against professional released work, evaluating palette, typography, developed material, composition, and repeated-component resolution.Safe or default styling, merely placed content, weak hierarchy, and inconsistent or default components fall short of that standard.Source: Style: shared judging instruction for… · pp. 40–41

E.2 External Benchmark Protocols

The section describes external benchmark protocols and the measures used to compare systems within each benchmark.

E.2 External Benchmark Protocols

  • The study evaluates Design2Code, SlidesBench, and GDB/lica using their official protocols.Scores in Table 2 use one decoding, rendering, and scoring pipeline per benchmark, making within-benchmark comparisons comparable but not cross-benchmark scores.Source: §E.2 · p. 41
  • Design2Code scores block matching, text similarity, position, color, and CLIP similarity against target webpages, assigning zero to generations that fail to render.Source: §E.2 · p. 41
  • SlidesBench rates content, layout, and color on a 0–5 scale, while GDB/lica reports NIMA, CLIP, PickScore, FID, color harmony, and OCR fidelity.For GDB/lica, OCR fidelity is the fraction of ground-truth tokens recovered by Tesseract after benchmark normalization.Source: §E.2 · p. 41

E.3 Human Study on Our Benchmark

The human study tests judge reliability, content and aesthetic quality, and whether Visual scores align with human preferences. Participants evaluated blinded items under controlled assignment and ordering procedures.

E.3 Human Study on Our Benchmark

  • The study retained 150 completed questionnaires from 52 participants after quality control, with each formal item judged by three different participants.System identity, automatic scores, machine answers, and reference images were hidden; item and block order were counterbalanced.Source: §E.3 · p. 41
  • Judge reliability compares automatic answers with three-person majorities on 150 sampled predicates each from the training VGR and Visual checklist.Automatic Unclear responses count as failures, while human Unclear responses are treated as missing; agreement uses only human-majority Yes or No predicates.Source: §E.3 · p. 42
  • The content study compares AesCode-32B with GPT-5.5 on 267 eligible briefs, while the aesthetic study rates paired outputs for 80 briefs without briefs or checklists.Aesthetic equivalence requires the entire 95% confidence interval of the difference to fall within the prespecified [−0.5, +0.5] margin.Source: §E.3 · p. 42
  • The score–preference study presents five images for each of 80 briefs as ten within-brief pairs and tests whether people prefer the higher-Visual-scoring image.Unequal-score pairs are scored as 1 for choosing the higher-scoring image, 0.5 for a tie, and 0 for choosing the lower-scoring image; equal-score pairs are reported separately.Source: §E.3 · p. 42

E.4 Human Study on External Benchmarks

The external-benchmark human study compares aesthetic preferences across anonymized outputs and relates those choices to metric scores.

E.4 Human Study on External Benchmarks

  • Across Design2Code, SlidesBench, and GDB/lica, the study collected 840 judgments on 168 score-blind, same-task image pairs.Each of the 56 pairs per benchmark received five judgments; task, reference, system identity, and scores were hidden.Source: §E.4 · p. 42
  • Each benchmark uses a per-sample composite: Design2Code averages five measures, SlidesBench averages three, and GDB/lica averages six normalized measures.GDB/lica reverses paired feature distance because FID is defined only over a set of images.Source: §E.4 · p. 42
  • Human agreement with each metric’s score ordering is reported by relative score gap.The gap is scaled by the metric range and grouped as < 5, 5–15, or ≥15; confidence intervals use a source-task cluster bootstrap.Source: Table E5 · p. 42, §E.4 · p. 42

F Failure Analysis

The failure analysis measures residual failures on the benchmark and examines why relational and semantic errors persist. It finds that global structure and recognizable semantic content remain difficult even when individual elements or local relations are present.

F.1 Measuring the three modes

  • Failure is recurrent when it appears in at least two of three rollouts; Boundary is measured on all 300 prompts, while the other modes cover 46 connected-structure and 30 semantic-element prompts.Table F1 reports scope, recurrent failure rate, and mean pass; lower recurrent failure and higher mean pass are better.Source: §F.1 · p. 43, Table F1 · p. 43
  • −30.3 points ([−36.0, −24.7]) is the paired Boundary difference to GPT-5.5, with residual cases involving fully constructed regions whose fixed-pixel heights exceed the canvas.The dashboard example’s two lower panels each request 410 px from y = 535 px on a 720 px canvas, while the running total remains invisible until rendering.Source: §F.1 · p. 43

F.2 Why the relational and semantic modes persist

  • Relational optimization remains difficult because relation credit is shared across about fourteen rubric questions and only 46 of 10,727 predicates supervise subgraphs.A failed subgraph predicate marks the page rather than identifying the responsible edge.Source: §F.2 · p. 44
  • On 46 relational prompts, whole-subgraph predicates pass at 75.4%, below pairwise relations at 95.3% and local structure at 95.8%.GPT-5.5 shows the same ordering; the analysis attributes the greater difficulty to connecting a timeline, hierarchy, or branch set as one object.Source: §F.2 · p. 44
  • Semantic elements can occupy plausible slots with consistent styling yet remain hard to distinguish because the rewards leave depicted icon content unscored.Recognizable icons require mapping meaning to fine-grained SVG forms, relying on the backbone’s visual knowledge; the content is left unscored because the reference may depict a different image in the same slot.Source: §F.2 · p. 44

Related papers

Loading 2610.18860v1…