Source-linked AI summary
WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics
Yuhong Dai, Yanlin Lai, Mitt Huang, Hangyu Guo, Dingming Li, Hongbo Peng, Haodong Li, Yingxiu Zhao, Haoran Lyu, Zheng Ge, Xiangyu Zhang, Daxin Jiang
TL;DR
Existing webpage-generation benchmarks largely rely on text or static screenshots, leaving video-conditioned recreation and dynamic fidelity insufficiently evaluated. WebVR introduces a controlled benchmark with demonstration videos, standardized rendering, and human-aligned visual rubrics, finding substantial difficulty in fine-grained style and motion while achieving 96% agreement with human preferences.
Problem
Existing benchmarks primarily evaluate text- or screenshot-conditioned webpage generation, leaving dynamic behaviors and fine-grained visual and interaction fidelity insufficiently assessed.
Method
WebVR evaluates video-conditioned webpage recreation using 175 controlled-synthesis webpages, standardized sandbox rendering, execution videos, and fine-grained human-aligned visual rubrics.
Results
Current state-of-the-art MLLMs show promise in static layout generation but struggle with nuanced temporal behaviors and dynamic interactions; rubric-based multimodal judging achieves 96% agreement with expert preferences.
Takeaways & Limitations
WebVR provides a rigorous benchmark and interpretable evaluation framework for advancing video-to-webpage generation research.
Takeaways & Limitations
Code-only judging can miss large visual differences and may score blank rendered videos highly, while video-only judging lacks sensitivity to distinguish high-performing models.
Abstract
from arXiv · showhide
Existing web-generation benchmarks rely on text prompts or static screenshots as input. However, videos naturally convey richer signals such as interaction flow, transition timing, and motion continuity, which are essential for faithful webpage recreation. Despite this potential, video-conditioned webpage generation remains largely unexplored, with no dedicated benchmark for this task. To fill this gap, we introduce WebVR, a benchmark that evaluates whether MLLMs can faithfully recreate webpages from demonstration videos. WebVR contains 175 webpages across diverse categories, all constructed through a controlled synthesis pipeline rather than web crawling, ensuring varied and realistic demonstrations without overlap with existing online pages. We also design a fine-grained, human-aligned visual rubric that evaluates the generated webpages across multiple dimensions. Experiments on 19 models reveal substantial gaps in recreating fine-grained style and motion quality, while the rubric-based automatic evaluation achieves 96% agreement with human preferences. We release the dataset, evaluation toolkit, and baseline results to support future research on video-to-webpage generation.
1. Introduction
WebVR addresses the lack of video-conditioned webpage-generation benchmarks and the limited evaluation of dynamic, fine-grained fidelity. It introduces a controlled 175-page benchmark, video-based execution protocol, and human-aligned rubric for scalable assessment.
- Motivation: Existing benchmarks mainly use text prompts or static screenshots, leaving transitions, animations, and fine-grained visual or interaction fidelity insufficiently evaluated.Staged screenshots provide only sparse temporal evidence and do not capture fluid motion.
- WebVR: WebVR is introduced as a benchmark for recreating fully functional webpages from demonstration videos.The benchmark targets visual and interactive fidelity in video-to-webpage generation.
- WebVR: 175 webpages span diverse categories and are built through controlled synthesis with paired visual assets, avoiding overlap with existing online pages.The construction pipeline supports varied demonstrations while allowing models to focus on reconstruction rather than image sourcing.
- Evaluation: The evaluation renders generated code in a standardized sandbox, records execution videos, and compares them with references using fine-grained rubrics.Rubric dimensions include layout, color and typography, component completeness, animation quality, and interaction correctness.
- Evaluation: The rubric-based MLLM judge achieves 96% agreement with human preferences while providing dimension-level feedback and model rankings.The protocol is designed for reproducible and interpretable assessment beyond static snapshots.
2. Related Work
Prior webpage-generation benchmarks cover text, screenshot, reasoning, and safety settings but provide limited supervision for continuous temporal dynamics and interaction fidelity. WebVR extends this landscape with video demonstrations and fine-grained multimodal evaluation.
- Webpage Generation: Text-based benchmarks approximate dynamics through staged screenshots, but sparse sampling misses fine-grained motion patterns, animation timing, and continuous transitions.These benchmarks therefore do not fully represent dynamic visual fidelity.
- Webpage Generation: Image-based benchmarks improve screenshot-to-code fidelity but remain grounded in static inputs without supervision for animation pacing or user-triggered state transitions.Their inputs do not encode the full spatio-temporal signals available in videos.
- MLLM as Judge: Existing evaluation approaches using DOM or pixel metrics can overlook high-level semantics and interaction flows, motivating MLLM-based UI assessment.Recent MLLM judges assess web-development quality and human perception, while WebCoderBench combines rule-based and fine-grained metrics.
- MLLM as Judge: Prior MLLM-as-judge studies face hallucination, prompt sensitivity, and limited awareness of engineering quality; WebVR uses human-aligned fine-grained rubrics to improve interpretability and reliability.The rubric guides multimodal evaluation of rendered webpages rather than relying only on coarse or structural criteria.
3. Method
WebVR defines video-conditioned webpage recreation and builds its benchmark through synthesized, rubric-guided reference implementations. Its pipeline combines video execution, fine-grained visual verification, and automated candidate filtering across four design dimensions.
- Task Definition: WebVR requires MLLMs to generate standalone executable HTML from a reference video, visual assets, and demonstrated layout, style, and interactive behavior.
- Dynamic Execution and Rendering: A standardized sandbox executes generated HTML under a fixed interaction script and records a screen-capture video for direct comparison with the reference.
- Evaluation via Visual Rubrics: Fine-grained binary rubrics verify atomic visual properties, replacing brittle pixel-wise comparison with visually grounded, temporally aligned evaluation.
- Data Synthesis: WebVR synthesizes unseen instances by captioning collected videos, semantically re-theming their specifications while preserving layout and motion, and retrieving grounded visual assets.
- Candidate Generation and Filtering: Five MLLM-generated HTML candidates are rendered, rubric-scored, and filtered or refined to construct the final benchmark set, yielding 175 instances after selection.The pipeline retains the top 50 samples per winning generator model before removing samples scoring below 0.5.
- Rubric Dimensions: The rubric taxonomy covers Global Aesthetics, Navigation and Footer, Section-Specific Layouts, and Interaction and Motion.
4. Experiments
WebVR experiments show that current MLLMs reproduce global visual style more reliably than fine-grained layouts, dynamic interactions, and semantically faithful content. Rubric-guided multimodal evaluation improves human alignment and exposes failure modes that code-only or video-only judging can miss.
- 79.14 is the best overall WebVR score, achieved by Kimi-K2.5, followed by Claude-Sonnet-4.6 at 78.49 and GPT-5.2-Thinking at 77.93.
- GA and NF are strongest, while IM remains the bottleneck at 60.10 for Kimi-K2.5 and 59.97 for GPT-5.2-Thinking.The highest SSL score is 79.26 for Kimi-K2.5, still below the strongest GA and NF results.
- Scaling and newer generations improve results, with Qwen3.5-397B-A17B reaching 61.33, Gemini-3.1-Pro-Preview reaching 76.69, and Seed-2.0-Pro reaching 71.88.Within Qwen, Thinking editions outperform corresponding Instruct editions, but IM remains low even with thinking enabled.
- Across 19 models, average scores follow GA > NF > SSL > IM at 72.57, 72.09, 43.27, and 38.44, respectively.The IM–GA gap exceeds 27 points for top models, including 27.34 for Kimi-K2.5 and 29.79 for GPT-5.2-Thinking.
- Rubric-guided judging raises agreement with UI/UX experts from 59.3–66.7% without rubrics to 76.7–86.7% with them, including a 22.7-point Code-and-Video gain.The rubric anchors evaluation to verifiable checks rather than free-form subjective ratings.
- Code-and-Video evaluation achieves the strongest human alignment, reaching 96.00% for Kimi-K2.5, while video-only averages 79.30% versus 76.70% for code-only.Combining structural code with dynamic visual information helps judges resolve visual ambiguities.
- Rubric-based scores are stable, with an overall standard deviation of 0.48 and an IM standard deviation of 0.96.The authors attribute this reproducibility to atomic visual rubrics that convert open-ended evaluation into logical checks.
- Observed failures range from blank or malformed pages to semantically unrelated templates and incorrect asset placement.These cases reflect failures in renderable HTML/CSS generation, video comprehension, content grounding, and precise spatial grounding.
5. Conclusion
WebVR evaluates high-fidelity webpage recreation from videos and finds that current MLLMs remain strong on static layout but struggle with nuanced temporal behavior and dynamic interaction. A multimodal judge using code, video, and fine-grained rubrics achieves high agreement with expert judgment.
- WebVR is designed to evaluate high-fidelity webpage recreation from video demonstrations.
- Current state-of-the-art MLLMs show promise in static layout generation but struggle significantly with nuanced temporal behaviors and dynamic interactions.
- A multimodal judge guided by fine-grained rubrics achieves high agreement with expert judgment.
- Kimi-K2.5 achieves near-perfect reconstruction in a representative case, while other models produce missing content, unrelated layouts, or misplaced images.
- WebVR is positioned as a testing ground for front-end AI agents pursuing more aesthetically aware and interaction-responsive web synthesis.
1. Benchmark Synthetic Details
The benchmark is built through structured visual captioning, semantic re-theming, asset retrieval, candidate webpage generation, and atomic rubric synthesis. These stages preserve structural and motion information while diversifying themes and enforcing visual, content, layout, and style fidelity.
- Asset Retrieval: GPT-5.2 extracts three non-overlapping sets of concise real-world keywords for Unsplash searches.
- Structured Captioning: Structured captions describe website atmosphere, layout composition, motion personality, and key interactive moments without exact pixel values or lengthy asset inventories.
- Semantic Re-theming: Semantic re-theming creates a new website in a different domain while preserving the structural skeleton, motion logic, and functional UI elements.
- Candidate Webpage Generation: The generation prompt requires production-quality front-end implementations that follow the specified website type, theme, visual language, and design rules.
- Candidate Webpage Generation: Generated content must be concrete, realistic, and domain-appropriate rather than generic or placeholder text.
- Automated Rubric Synthesis: Rubric synthesis uses visual-proof criteria with extreme atomicity, separating existence, layout, and style checks into four fixed categories.
2. Evaluation Details
WebVR evaluates generated webpages with atomic visual rubrics using both execution videos and HTML code, while prioritizing visible rendering evidence. The protocol includes standardized judge settings and separate code-only, video-only, and multimodal judging prompts.
- Multimodal Judging: The benchmark judge uses generated execution video and HTML code as complementary evidence for rubric-based scoring.
- Judge Configuration: Kimi-K2.5 serves as the final evaluation judge with temperature = 1.0, top-p = 0.95, top-k = 20, and max_tokens = 128k.
- Judge Configuration: Unless otherwise specified, evaluated models use temperature = 1.0, top-p = 0.95, top-k = 20, and max_tokens = 128k.
- Code-only Judging: The code-only judge requires explicit evidence in HTML, CSS, or JavaScript for each rubric criterion.
- Video-only Judging: The video-only judge treats the rendered video as the primary truth and uses source code only to verify details that cannot be judged visually.
- Multimodal Judging: The code-and-video judge combines visual assessment with code confirmation for exact details such as colors, fonts, and unseen interaction states.