MulFCoder: Framework-conditioned Multi-agent for MLLM-based Multi-framework Front-end Code Generation

1. Introduction

• Prompt-only transfer across frameworks produces compilation gaps and framework-specific errors because frameworks differ in language rules, component conventions, and project scaffolding.The introduction identifies these differences as difficult to localize and fix reliably, contributing to invalid attempts and manual rework.Source: §1 · p. 1
• MulFCoder decomposes generation among Grounder, Planner, Writer, and Judger, using a framework-specific contract and lightweight checks to constrain generation and repair.Grounder establishes UI elements, content, and macro-layout; Planner creates a layout tree, schedule, and contract; Writer makes bounded edits; Judger accepts or rejects updates and triggers bounded repairs.Source: §1 · p. 2
• The paper identifies multi-framework generation as under-studied and reports compilation discontinuities and framework-specific error modes under prompt-only transfer.The introduction says its experiments show substantial compilation-success improvements and reports code release to facilitate future research.Source: §1 · p. 2

Front-end Code Generation Method

• Front-end generation progressed from CNN–RNN screenshot-to-DSL systems to direct HTML generation and methods using layout recognition or computer-vision pipelines.Recent MLLM work includes large-scale synthesis, hierarchical planning, decomposition, and efficiency or post-training techniques.Source: §2 · p. 2

Front-end Code Generation Benchmark

• Benchmarks evolved from DSL and OCR/detection reconstruction toward normalized HTML/CSS datasets with reproducible rendering and visual or element-level metrics.Recent benchmarks add compilation and execution tests, agentic evaluation, and larger-scale synthetic or interactive generation.Source: §2 · p. 2

MLLM

• MLLMs connect a vision encoder to a language backbone, with training that commonly aligns modalities and then applies visual instruction tuning.Recent models also advance perception and reasoning through higher-resolution tokenization, longer context, and post-training recipes.Source: §2 · p. 3

3. Method

• MulFCoder decomposes screenshot-to-code generation into Grounder, Planner, Writer, and Judger agents, with framework-aware constraints and bounded repairs.The pipeline aims to satisfy framework-specific language and engineering constraints without invoking expensive builds.Source: §3 · p. 3

3.1. Grounder

• Grounder detects UI elements, binds OCR text to detected elements, and builds element and content tables linked by stable identifiers.It also infers macro-layout regions to provide a coarse page scaffold for downstream planning and generation.Source: §3.1 · p. 3

3.2. Planner

• Planner builds a DOM-like tree with layout primitives and creates a task schedule alongside a framework-specific file contract.The contract specifies framework policy, permitted edit windows, and task-to-artifact bindings; planning and scheduling proceed from regional subtrees to root assembly.Source: §3.2 · pp. 3–4

3.3. Writer

• Writer generates code incrementally in FILE mode and makes localized repairs in PATCH mode, staying within contract-permitted files and regions.FILE mode builds the structure before backfilling content; PATCH mode aims to preserve structures that have already passed checks.Source: §3.2 · p. 4, §3.3 · p. 5

3.4. Judger

• Judger applies lightweight checks for structured output, parse feasibility, dependency and scope consistency, and contract compliance, then returns repair signals.A bounded repair budget and failure-pattern checks stop repeated unsuccessful repairs and roll back to the most recent state.Source: §3.4 · p. 5

4.1. Setup

• Across five backbones, two public benchmarks, and CSR and CLIP evaluation, the study compares MulFCoder with prompt transfer.Table 1 compares performance across frameworks.Source: §4.1 · pp. 5–6, Table 1 · p. 6

4.2. Main Results

• Angular gains are largest: MulFCoder improves CSR by about 10.2% on Design2Code and 9.0% on DesignBench, while benefits are larger for weaker backbones.The results also show gains across React and Vue, and Table 1 presents the cross-framework performance comparison.Source: Table 1 · p. 6, §4.2 · pp. 6–7
• In strict frameworks, compilation gains can coincide with saturated or slightly regressed CLIP scores, while Vanilla shows consistent visual improvements.The authors attribute this trade-off to occasional simplification of complex UI components when prioritizing compilation.Source: Table 1 · p. 6, §4.2 · p. 7
• React gains average about 3.8% on Design2Code and 6.2% on DesignBench; Vue gains average about 6.6% and 8.2%, respectively.The paper relates Vue’s gains to sensitivity to template syntax and structural well-formedness.Source: §4.2 · p. 6

4.3. Error Analysis

• Compilation errors vary by framework: React parsing and client-boundary issues, Vue template and attribute errors, and Angular wiring, import, and binding constraints dominate.MulFCoder reduces frequent error categories, but Angular’s remaining errors are more persistent and reflect global project-coherence challenges; Table 2 reports repair ratios, which are lower for Angular and on DesignBench.Source: §4.3 · p. 7, Figure 3 · p. 8

4.4. Case Study

• A case study reports that MulFCoder accurately reproduces structural details, which the authors attribute to fine-grained local code writing in the cooperative multi-agent model.Source: §4.4 · p. 8

• MulFCoder decomposes generation among Grounder, Planner, Writer, and Judger, using a file-level contract and lightweight FastGate verification.Source: Conclusion · p. 8
• Experiments show substantially improved compilation success across React, Vue, and Angular, with consistent visual improvements in Vanilla.Source: Conclusion · p. 8
• In strict frameworks, reducing build errors can require simplifying complex UI components, resulting in saturated or slightly regressed visual similarity scores.Source: Conclusion · p. 8

• The approach may reduce repetitive debugging and manual rework for teams maintaining the same UI across frameworks, and support framework-aware evaluation.Source: Impact Statement · p. 9
• UI artifacts may expose sensitive information if logged, retained, or sent externally without strong governance; generated code may also be deceptive or lack security, accessibility, or maintainability.The statement recommends standard review, testing, dependency auditing, and security scanning before downstream deployment.Source: Impact Statement · p. 9

• Design2Code contains 484 curated test webpages built from real webpages, with screenshots paired to normalized implementations designed to reduce environmental variation.Source: §A.1 · p. 12
• DesignBench contains 900 webpage samples across more than 11 themes and covers Vanilla HTML/CSS, React, Vue, and Angular.Source: §A.1 · p. 12
• DesignBench evaluates generation, editing, and repair, and categorizes nine editing types and six issue types.Source: §A.1 · p. 12

A.2. Baseline

• The baseline changes only the textual instruction to request the target framework, then asks the same backbone to generate a runnable project in one shot.Direct-prompting results serve as the backbone reference for comparison with MulFCoder on the same backbone and framework.Source: §A.2 · p. 12

A.3. Metrics

• CSR measures executability as the ratio of generated projects that pass the framework-specific build process without fatal errors.The build toolchain may be Vite or Webpack; CSR reflects practical usability and cross-framework reliability.Source: §A.3 · p. 12
• CLIP Score measures visual similarity by comparing the input UI screenshot with a screenshot rendered from successfully compiled generated code.The dataset-level score is the average image-level similarity; the paper uses CLIP for semantic and structural alignment.Source: §A.3 · p. 12
• The CLIP vision encoder is chosen over pixel-wise metrics such as RMSE because it better correlates with human perception of layout similarity and tolerates minor rendering artifacts.The encoder is ViT-B/32.Source: §A.3 · p. 12

B.1. React

• A React repair completes an invalid JSX ternary with a valid else branch and parentheses, preserving UI logic and layout while restoring compilable structure.The case attributes the original failure to an incomplete conditional expression and describes constrained, minimal edits.Source: B. Error Case Study · p. 13, §B.1 · p. 14

B.2. Vue

• A Vue repair guards a nullable thumbnail before accessing its URL, restoring compilability without changing DOM hierarchy or styling semantics.The case identifies missing field-nullability handling as the failure mode.Source: B. Error Case Study · p. 13, §B.2 · p. 15

B.3. Angular

• An Angular repair replaces an incompatible mixed binding with a valid property-binding expression, preserving DOM structure and layout semantics.The case attributes the failure to Angular template binding rules rather than general HTML syntax.Source: B. Error Case Study · p. 13, §B.3 · p. 16

C. Performance under different difficulty levels

• Table 3 reports CSR for three frameworks across difficulty levels in DesignBench.The supplied caption identifies the table’s scope but does not state its cell values or comparisons.Source: Table 3 · p. 17