PaperlayerView exact revision on arXiv

Source-linked AI summary

Editable Visual Design

Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li, Zhutao Lv, Jiaxin Lin, Jinhua Yu, Jun He, Zilong Huang, Rui Chen, Weijia Li

arXiv:2609.04034v1cs.CVcs.CL

TL;DR

Diffusion models offer visual quality but produce entangled pixels and error-prone text, while code-based design offers structure but lacks global aesthetic intuition. Editable Visual Design pairs a VLM-led Coding Agent with image-model visual simulation, producing designs reported to balance visual quality with clear typography and decoupled, editable layers.

  • Problem

    Visual generation offers strong composition but weak engineering editability, while code-based generation has structural control but lacks global aesthetic intuition.

    Source: §1 · p. 2

  • Method

    Editable Visual Design assigns a VLM requirement understanding, planning, code construction, and evaluation, while an image-generation model serves as an on-demand visual simulator for concrete visual references.

    Source: §1 · p. 2, §2.2 · p. 3

  • Results

    In comparative analysis, Editable Visual Design combines the image simulator’s color and atmosphere with code reconstruction’s clear typography and layer decoupling, balancing visual quality and engineering editability.

    Source: §3.1 · p. 4

  • Takeaways & Limitations

    The resulting native HTML/CSS artifacts keep text, background, and graphic layers relatively independent for later selection, modification, and export.

    Source: §4.2 · p. 6

    The evaluated cases are single-page designs, and scaling to longer pieces depends heavily on the underlying models’ capabilities.

    Source: §5 · p. 8

Abstract

from arXiv · show

While diffusion base models such as GPT-Image-2 and Nano-Banana exhibit remarkable visual expressiveness, their end-to-end generation inherently yields flattened bitmaps with error-prone text, precluding layer-wise post-editing. Conversely, code-based visual generation via Coding Agents provides precise layout control and decoupled layers, yet remains constrained by a lack of global aesthetic intuition and the difficulty of coding complex visual assets. To address this, we propose Editable Visual Design, a new paradigm driven by a Coding Agent. We designate the VLM as the ``creative brain'' for requirement comprehension, task planning, and aesthetic judgment, while utilizing the image generation model as an on-demand ``visual world simulator'' to synthesize standalone visual assets. Operating under an ``imagine first, then act'' closed-loop workflow, the agent generates isolated assets, writes native HTML/CSS, and iteratively refines the design against visual rendering feedback. Furthermore, Agent Design Replay faithfully reproduces the creative and reasoning trajectory akin to that of professional human designers. Ultimately, the system delivers editable artifacts with decoupled layers and real text, enabling users to perform intuitive mouse dragging and layout adjustments on a graphical user interface. Validations on posters, infographics, and other scenarios show that this paradigm successfully achieves both refined aesthetics and production-grade editability.

01 / CAMPAIGNS · 1 Introduction

Editable Visual Design combines a VLM-driven Coding Agent with an image generation model to pair visual judgment and asset creation with editable, code-based layouts. Its imagine-first workflow and Agent Design Replay produce inspectable design trajectories and artifacts with independently editable text, assets, and layout layers.

1 Introduction

  • Recent coding models, including GPT-5.6 Sol, Claude Fable 5, and Kimi K3 [22], can generate structurally complete, organized layouts directly in HTML, SVG, and CSS.The passage frames this capability as opening opportunities for automated poster and infographic design.Source: §1 · p. 2
  • Code-generated layouts offer clean layers, interactive text, and support for later modification, but current code LLMs lack visual intuition and often produce templated designs.The paper describes their difficulty with two-dimensional spatial sense [13] [34] [41] and visual intuition [42].Source: §1 · p. 2
  • Hand-coding complex assets such as cinematic backgrounds, 3D hero visuals, textures, and illustrations is costly and often yields stiff results or placeholder-like designs.HTML, SVG, and CSS support precise typography and geometric alignment, but complex visual assets can make the final product look unfinished.Source: §1 · p. 2
  • Pure code generation offers structure but lacks spatial and aesthetic intuition, while diffusion models [32] create compelling images that lack editable layers and can distort text [3] [4] [5] [17] [18].The paper identifies these as complementary weaknesses in existing visual-generation approaches.Source: §1 · p. 2
  • Editable Visual Design pairs a VLM “creative brain” for requirements, planning, code construction, and judgment with an on-demand image-generation “visual world simulator” for turning ideas into visual effects.The paradigm is inspired by the World Action Model (WAM) [36] [56].Source: §1 · p. 2
  • The agent imagines a visual first to establish composition, lighting, and color priors, then generates assets, writes native HTML/CSS, and iteratively repairs the rendered design.The workflow uses multiple rounds of render-and-reflect refinement.Source: §1 · p. 2
  • Agent Design Replay exposes the agent’s trajectory from intent planning and asset generation through code reflection and repair, making the process visible, traceable, and reproducible.The replay presents the trajectory in a manner likened to that of a human artist.Source: §1 · p. 2
  • The system delivers artifacts whose text, assets, and layout layers are decoupled for independent selection, dragging, editing, and export, and validates the approach on marketing materials, infographics, long-text layouts, and event posters.The artifacts pass deterministic checks and visual review, combining high-quality aesthetics with full editability.Source: §1 · p. 2

2 Coding-Agent-Driven Design Workflow

The workflow pairs a VLM that plans and judges the design with an on-demand image generation model, moving from visual planning through code, verification, and delivery of editable artifacts with a recorded design trajectory.

2.1 Understanding and Design Planning

  • Before coding, the agent interprets required content, deliverable format and size, and visual register, then generates an imagined finished piece that lets users revise the direction early.The imagined visual helps settle the design plan and offers a low-cost opportunity for user intervention before code exists.Source: §2.1 · p. 3

2.2 Visual Simulation

  • The VLM parses the imagined visual’s color, composition, and style as a global reference, using its rendered pixels to guide aesthetic judgment rather than reproduce it exactly.The model-generated image gives the agent visual feedback before code exists, analogous to inspecting a browser rendering of code already written.Source: §2.2 · p. 3

2.3 Structural Coding and Generation

  • The agent uses the visual prior to plan canvas topology, generates clean standalone assets, and combines them with real text in layered native HTML/CSS [ ].Assets are generated separately rather than cut from the imagined visual; they use alpha channels when supported, or green-screen matting when the subject contains no green. Fixed-size canvases, separately tagged layers, and viewport-independent layouts support meaningful dragging.Source: §2.3 · pp. 3–4

2.4 Verification and Visual Self-Healing

  • Headless-browser rule checks and VLM review of rendered screenshots detect structural and visual flaws, prompting targeted code patches that typically take one or two repair rounds [20] [50] [51].Checks cover overflow, failed external resources, and malformed DOM; visual review assesses balance, alignment, and text readability.Source: §2.4 · p. 4

2.5 Editable Design and Agent Design Replay

  • The delivered artifact separates text, assets, and backgrounds into native DOM layers users can edit, drag, scale, and export, while Agent Design Replay records the design process for tracing and intervention [55].The replay captures requirement decomposition, visual imagination, asset prompts, code evolution, and reflective repair.Source: §2.5 · p. 4

3 Case Studies and Showcase

The case studies examine the paradigm through a baseline comparison, examples across design scenarios, and a real-world creative trajectory.

3.2 Diverse Scenario Showcase

  • Examples span posters, infographics, marketing materials, and long-text layouts, adapting to different information densities while keeping text, backgrounds, and illustrations independently editable as native DOM.Information-dense cases maintain type scale and grid alignment; visually driven cases organize multilayer space sensibly, and elements can be selected, edited, and exported independently.Source: §3.2 · p. 4

3.3 Real-World Case Study of the Agent Design Replay Trajectory

  • In a red panda infographic, Agent Design Replay traces requirement parsing and visual ideation through layout planning, asset generation, HTML/CSS construction, render-based repair, and a layered final design.The case makes the agent’s decisions transparent and traceable as it develops visual expressiveness and structured code.Source: §3.3 · p. 4

3.1 Comparative Analysis

  • Under the same brief, Editable Visual Design balances visual quality and editability, unlike diffusion’s distorted text and entangled layers [3] [18] or pure code’s flat, monotonous output.Figure 3 depicts the diffusion result as a locked bitmap with disordered Chinese characters and the code result as a chaotic layout with large blank areas; the proposed design has clean typography and separable layers.Source: §3.1 · p. 4, Figure 3 · p. 5

4 Conclusion and Discussion

Editable Visual Design combines a reasoning-capable Coding Agent with visual generation as an on-demand simulator, then builds editable artifacts with decoupled layers and real text. Its discussion emphasizes visual simulation as a source of design priors, structured delivery, and visible agent workflows.

4.1 Conclusion

  • Editable Visual Design pairs VLM planning with visual simulation, then constructs native-code artifacts with decoupled layers and editable text.Agent Design Replay records the process from intent planning and asset generation through code adjustment.Source: §4.1 · p. 5

4.2 Discussion

  • Using generated images as visual simulators may provide aesthetic and compositional priors that inform later multimodal decisions and code layout.The discussion contrasts this role with using generative models for rigorous symbolic reasoning, which it describes as a poor fit for their implicit feedback mechanism.Source: §4.2 · pp. 5–6
  • The division of labor assigns planning, code organization, and quality checks to the VLM, while the generation model renders concrete visuals on demand.The report situates this collaboration alongside GenClaw [55] and Mind-Brush [15], combining visual expressiveness with more precise structural control through code.Source: §4.2 · p. 6
  • Native HTML/CSS and decoupled assets keep text, backgrounds, and graphics independently selectable and modifiable, addressing the editing difficulty of entangled bitmap output.The discussion attributes that difficulty to deeply entangled pixels [18] and error-prone text [3].Source: §4.2 · p. 6, Figure 4 · p. 6
  • Agent Design Replay exposes the design trajectory from requirement understanding to code adjustment, supporting transparency and traceability in human-AI collaboration.The examples include a field guide with 120 editable layers in 13 groups, where review repairs layout defects, and a travel poster with 6 layers in 1 group, where review makes no changes; layer structure follows brief density.Source: §4.2 · pp. 6–7, Figure 5 · p. 7

5 Limitations

Editable Visual Design remains bounded by the capabilities of its underlying models, and its aesthetic quality and practical editability resist reduction to objective scores. Its reported cases are single-page designs, leaving longer-form consistency and scalability uncertain.

5 Limitations

  • The system cannot exceed its underlying models: weak layout code limits the page, ordinary image-model composition offers little to build on, and asset generation requires clean background separation.The binding constraint is more often the image model’s design sense, though coding-agent capability can also limit results.Source: §5 · p. 7
  • The evaluated cases are single-page designs; multi-page decks and websites require consistent type, palette, and visual thread, while scalability depends heavily on model capability.Source: §5 · p. 8
  • Aesthetic quality lacks ground truth and involves subjective judgments, so the work reports cases rather than scores and uses VLM judgment as a proxy for a designer’s eye.Source: §5 · p. 8
  • Editability is difficult to measure: layer counts and DOM structure are reportable, but whether a designer can make the file do what they want has no simple numerical measure.Source: §5 · p. 8

6 Related Work

Related work establishes visual code as editable and verifiable, image models as strong in composition and aesthetics, and agentic systems as increasingly plan- and tool-driven. Editable Visual Design combines these lines by using image generation for intermediate assets and code for the final editable artifact.

6.1 Visual Code Generation and LLMs

  • Visual-code methods improve layout fidelity and support editable deliverables, but pure code struggles to create complex textures, backgrounds, and illustrations.The literature includes benchmarks for web and interactive design, layout-guided generation, and iterative visual refinement; Editable Visual Design targets the gap in global aesthetic control and asset generation.Source: §6.1 · p. 8

6.2 Image Generation and Computational Design

  • Image-generation systems and design methods advance composition, aesthetic quality, and text rendering, but their raster outputs limit precise long-text layout, local text edits, and engineering maintenance.Editable Visual Design uses image models’ aesthetic priors and asset-rendering ability as upstream input to structured code rather than as the final deliverable.Source: §6.2 · p. 8

6.3 Agentic and Reasoning-driven Design Generation

  • Agentic design work adds planning, tool scheduling, reflection, and feedback, while Editable Visual Design shifts the route from code-to-image to image-to-code.Unlike GenClaw’s code-driven intermediate canvas, it uses generated images as intermediate assets and an up-front simulator, with code as the final engineering deliverable.Source: §6.3 · pp. 8–9

Related papers

Loading 2609.04034v1…