WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs

1 Introduction

• Existing datasets are either too small for meaningful generalization or synthetic, unlike real-world pages that can have up to 50 times more tokens, six times as many tags, and double the DOM depth.Modern MLLMs also decline on complex real-world designs with more unique HTML tags.Source: §1 · p. 2
• WebCode2M contains 2.56 million real-world design-image and HTML/CSS pairs, averaging 31,216 tokens, 158 tags, and DOM depth 13.The dataset is presented as diverse and comprehensive for real-world webpage designs.Source: §1 · p. 2
• The dataset pipeline collects webpages from Common Crawl, extracts CSS and images, removes noise, and uses a quality scorer to filter incomplete or aesthetically weak instances.The scorer was trained on consensus annotations from six annotators and achieved 90% validation accuracy.Source: §1 · p. 2
• WebCoder, fine-tuned from Pix2Struct-1.3B, outperforms Design2Code-18B and WebSight VLM-8B across all metrics; GPT-4o leads on visual metrics but trails on substructure recall.WebCoder also outperforms general-purpose MLLMs across most metrics.Source: §1 · p. 2
• The paper introduces TreeBLEU for structural hierarchy recall and releases its code, dataset, and benchmark model for research and development.The resources are available at the project homepage.Source: §1 · p. 2

2.1 Dataset Construction

• WebCode2M starts from a random sample of about 0.5 billion Common Crawl pages and generates screenshots by rendering webpage code.The sample comes from the first segment of CC-MAIN-2023-50.Source: §2.1 · pp. 2–3
• The pipeline filters code by length, removes redundant or non-rendering elements, merges CSS, and renders the purified HTML and CSS with Playwright.Rendering accounts for roughly 80% of processing time, which spans about one month.Source: §2.1 · p. 3
• Six experienced annotators score 10,000 screenshots by consensus; the scorer reaches 75% test accuracy and removes below-two samples, 52.5% of the raw dataset.The manually labeled subset is 80% low quality, with scores between 0 and 1.Source: §2.1 · p. 4
• Each instance also pairs its webpage code and design image with a layout of HTML-element bounding boxes, including size, location, and hierarchy.The authors intend this layout information to help models learn DOM-tree structure more accurately.Source: §2.1 · p. 4

2.2 Ethical Compliance

• The dataset applies image and profanity filters, retaining only samples that pass both to mitigate ethical concerns about inappropriate content.The authors cite explicit or violent content as examples of material the filters target.Source: §2.2 · p. 4

2.3 Dataset De-Duplication and Partition

• After screenshot-hash deduplication, about 2,000 high-scoring entries are candidates for testing; the remaining 2.56 million entries form the training set.The candidate test set is further deduplicated with CLIP similarity and manually inspected; test subsets are divided by code length.Source: §2.3 · pp. 4–5

2.4 Dataset Characteristics

• 2.56 million samples make WebCode2M larger than WebSight’s 0.8 million, while its screenshots show varied real-page layouts and rich elements.Its HTML tag count and DOM structure exceed pix2code and WebSight, while its average code length is about one-tenth of Design2Code’s.Source: §2.4 · p. 5

3 Benchmarking

• The section frames its experiments around training-dataset effectiveness and benchmarking general-purpose pre-trained MLLMs.The WebCoder baseline and comparisons are intended to support fair evaluation of webpage code generation models.Source: §3 · p. 5, §3.2 · p. 6

3.1 WebCoder: A Reference Baseline

• WebCoder is fine-tuned from Pix2Struct-1.3B, a ViT-based model, to translate webpage design images into HTML/CSS code.Pix2Struct is fully fine-tuned on WebCode2M; the section establishes WebCoder as a reference baseline for fair comparison.Source: §3 · p. 5, §3.1 · p. 6

3.2 Setup and Baselines

• The benchmark compares WebCoder with task-specific fine-tuned models and general-purpose MLLMs, using one-pass generation for all baselines.The specialized comparisons include Design2Code-18B and WebSight VLM-8B; general-purpose baselines include LLaVA models, CogAgent-Chat-18B, GPT-4V, GPT-4o, Gemini, and Claude.Source: §3.2 · p. 6

3.3 Evaluation Metrics

• TreeBLEU measures matching between generated and reference HTML DOM trees using 1-height subtrees, while CLIP similarity and Visual Score assess visual similarity.TreeBLEU excludes terminal nodes containing tag attributes; CLIP similarity uses image latent-vector cosine similarity, and Visual Score assesses matching low-level block appearance.Source: §3.3 · p. 6

3.5 Effectiveness of Training Dataset (RQ1); 3.6 Benchmarking on the Test Datasets (RQ2)

• On the real-world test set, WebCoder outperforms specialized baselines across all three metrics, while GPT-4o significantly outperforms specialized and other general-purpose models across all metrics.WebCoder also exceeds most general-purpose models on visual metrics and either matches or outperforms GPT-4V; almost all models’ metrics decline as target code length grows.Source: Table 3 · p. 7, §3.5 · p. 7, Table 4 · p. 7

3.6 Benchmarking on the Test Datasets (RQ2)

• As target code length increases, nearly all models’ metrics decline; GPT-4o significantly outperforms all other models across metrics and remains stable as complexity increases.For specialized models, the highest TreeBLEU falls from 0.35 on WebCode2M-Short to 0.15 on WebCode2M-Long; GPT-4o’s Visual Score is 0.85, 0.81, and 0.82 across Short, Mid, and Long.Source: Table 4 · p. 7, §3.6 · p. 7

3.4 Implementation Details

• Training uses two fine-tuning phases, with sequence lengths of 2,048 and then 1,024 tokens, each lasting three epochs.The first phase uses 90,000 iterations and a maximum learning rate of 5e-5; the second uses 10,000 iterations and a learning rate of 1e-5.Source: §3.4 · p. 7

Image Representation Learning

• Image representation research includes contrastive image-encoder training, ViT patch-based processing, and diffusion models for representation, understanding, and generation.Chen et al. [15] investigated contrastive learning on large-scale datasets; ViT processes fixed-size image patches with a Transformer [16], while diffusion work includes [23] [41].Source: §4 · p. 8

Code Generation

• Code-generation research includes specialized language models and production tools, whereas this paper focuses on code generation from webpage designs.The passage lists CodeT5+ [52], InCoder [17], StarCoder, Code Llama [42], WizardCoder [32], Qwen-Coder [26], DeepSeek-Coder [20], and tools including Copilot.Source: §4 · p. 8

Image to Code

• Image-to-code methods range from synthetic-data GUI translation and wireframe parsing to screenshot-based HTML generation and commercial MLLMs.Prior examples include pix2code [12], sketch2code [40], UI hierarchy prediction [53], and Pix2Struct [28]; this work focuses on a comprehensive real-world dataset for training and testing.Source: §4 · p. 8

5.1 Practical Challenges to Study

• Long HTML challenges MLLM training effectiveness and efficiency; overlapping elements and missing borders complicate hierarchy recovery, while all existing models fail to reproduce design images accurately.GPT-4V captures text and color well but struggles with UI-element hierarchy; the authors call for a framework to generate or extract image elements and assemble them into webpage code.Source: §5.1 · p. 8

5.2 Limitations and Future Directions

• The neural scorer is subjective and about 90% accurate, so some low-quality instances may remain in the dataset.The authors consider this an efficiency–quality trade-off because manually screening millions of data points is impractical.Source: §5.2 · p. 8
• Models show significant variance on some metrics and perform poorly on some test cases, motivating more robust frameworks and better webpage-code evaluation metrics.The authors plan to assess generated code and webpages using LLMs.Source: §5.2 · p. 8
• Because the dataset is web-crawled, it may contain a minimal amount of inappropriate content, such as violent material, despite extensive filtering.Source: §5.2 · p. 8

• The experimental results show that WebCode2M significantly enhances MLLMs’ ability to generate code from webpage designs.Source: §6 · p. 8
• WebCode2M contains over 2.56 million samples for training and testing, and the paper analyzes the curated data’s diversity.Source: §6 · p. 8
• The authors fine-tune WebCoder on the training set and evaluate it against other baselines using two visual metrics and a structural measure.Source: §6 · p. 8
• The authors expect the dataset and benchmark to contribute to advancing research in webpage-code generation.Source: §6 · p. 8

• TreeBLEU parses HTML into a simplified DOM tree containing tag names only, omitting attributes deemed unnecessary for comparing substructures.Source: A The Implementation of TreeBLEU · p. 11
• It represents each one-height subtree with a node’s tag name and its child nodes’ tag names, then traverses the DOM tree and removes duplicate subtrees.Source: A The Implementation of TreeBLEU · p. 11
• TreeBLEU matches the generated and reference subtree sets by string comparison and uses the matches to calculate its score; Figure 7 illustrates an example.Source: A The Implementation of TreeBLEU · p. 11, Figure 7 · p. 11

• English constitutes approximately 50% of the corpus, while Russian, German, Spanish, and French each represent nearly 5%.Source: B More Data Statistics · p. 11, Figure 8 · p. 11

C Filtering Inappropriate Content

• Screenshots are retained only when the detector’s NSFW score is below 0.04.The detector is a ViT fine-tuned on safe and explicit images; lower NSFW scores indicate lower probability of harmful content.Source: C Filtering Inappropriate Content · p. 11
• Web text is filtered with two multilingual harmful-word lists, removing samples with more than 20 occurrences.One list covers inappropriate words and the other adds sensitive or stop words.Source: C Filtering Inappropriate Content · p. 11
• Manual review of first-chunk results found that an NSFW threshold of 0.04 produced the best filtering results.Thresholds were chosen to balance retaining samples against minimizing harmful content; Table 5 compares performance across NSFW thresholds.Source: C Filtering Inappropriate Content · p. 11, Table 5 · p. 12