Source-linked AI summary
Editable Visual Design
Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li, Zhutao Lv, Jiaxin Lin, Jinhua Yu, Jun He, Zilong Huang, Rui Chen, Weijia Li
TL;DR
Existing methods either produce expressive but hard-to-edit raster images or structured code with limited aesthetic intuition and complex-asset capability. Editable Visual Design coordinates a VLM creative brain with an on-demand image simulator in an imagine-first, iterative coding workflow, and delivers editable layered artifacts. Across posters, infographics, marketing materials, and long-text layouts, the paper reports a balance of refined aesthetics, production-grade editability, and traceable design decisions.
Problem
Diffusion models produce flattened images with error-prone text, while code-based generation lacks global aesthetic intuition and struggles to create complex visual assets.
Method
A VLM plans and judges the design while an image model simulates visuals and generates standalone assets, followed by native HTML/CSS construction and iterative render-based repair.
Results
The system delivers artifacts with decoupled layers and editable text across posters, infographics, marketing materials, and long-text layouts.
Takeaways & Limitations
The paradigm offers a practical direction for combining visual expressiveness with deterministic editability and transparent human-AI design interaction.
Takeaways & Limitations
The reported cases are single-page designs, and longer multi-page pieces depend heavily on the capabilities of the underlying models.
Abstract
from arXiv · showhide
While diffusion base models such as GPT-Image-2 and Nano-Banana exhibit remarkable visual expressiveness, their end-to-end generation inherently yields flattened bitmaps with error-prone text, precluding layer-wise post-editing. Conversely, code-based visual generation via Coding Agents provides precise layout control and decoupled layers, yet remains constrained by a lack of global aesthetic intuition and the difficulty of coding complex visual assets. To address this, we propose Editable Visual Design, a new paradigm driven by a Coding Agent. We designate the VLM as the ``creative brain'' for requirement comprehension, task planning, and aesthetic judgment, while utilizing the image generation model as an on-demand ``visual world simulator'' to synthesize standalone visual assets. Operating under an ``imagine first, then act'' closed-loop workflow, the agent generates isolated assets, writes native HTML/CSS, and iteratively refines the design against visual rendering feedback. Furthermore, Agent Design Replay faithfully reproduces the creative and reasoning trajectory akin to that of professional human designers. Ultimately, the system delivers editable artifacts with decoupled layers and real text, enabling users to perform intuitive mouse dragging and layout adjustments on a graphical user interface. Validations on posters, infographics, and other scenarios show that this paradigm successfully achieves both refined aesthetics and production-grade editability.
02 / LONGTEXT DESIGN
The paper presents editable visual design artifacts as the target output of its proposed paradigm.
- Editable design artifacts are generated by the proposed Editable Visual Design approach.
1 Introduction
Existing visual-generation paths trade off aesthetic quality against engineering editability: code generation lacks visual intuition and asset-building ability, while diffusion produces hard-to-edit raster images. Editable Visual Design addresses this gap by coordinating a VLM creative brain with an on-demand visual simulator, then delivering editable artifacts.
- Code-generation models provide structured layouts but struggle with global visual control and refined aesthetics.They are fluent in syntax and layout primitives but lack two-dimensional spatial sense and visual intuition.
- Pure code generation also struggles to create complex visual assets such as cinematic backgrounds, 3D visuals, textures, and elaborate illustrations.These assets are often replaced by simple geometric blocks, gradients, or emoji placeholders.
- Diffusion models offer strong composition, lighting, and texture but produce raster images with distorted text and weak engineering structure.
- Editable Visual Design assigns planning, coding, and result judgment to a VLM while using an image model as an on-demand visual simulator.
- The agent follows an imagine-first loop, generating visual priors and standalone assets before writing native HTML/CSS and repeatedly repairing rendered layouts.
- The system delivers artifacts with decoupled text, asset, and layout layers that users can select, drag, edit, and export independently.The paper validates this combination of aesthetics and editability across marketing materials, infographics, long-text layouts, and event posters.
2 Coding-Agent-Driven Design Workflow
The workflow combines VLM planning, image-based visual simulation, structured code generation, automated checks, visual repair, and editable delivery. It also records the agent’s decisions and revisions as a traceable Design Replay.
- The workflow has five stages: planning, visual simulation, structural coding, verification and self-healing, and editable delivery.
- The agent first interprets the brief and generates an imagined visual as an aesthetic reference for the intended design.
- Visual Simulation: The VLM extracts color tone, compositional distribution, and style from the imagined visual to guide subsequent code construction.The reference is used as a global design prior rather than reproduced one-to-one.
- The Coding Agent uses the visual prior to plan canvas topology, requests standalone assets, and writes structured code for the composition.
- Assets are generated separately by layer, while native HTML/CSS defines fixed-size canvases and independently movable elements.
- Headless-browser checks detect overflow, failed resources, and malformed DOM structures before iterative visual repair addresses remaining layout flaws.
- The final DOM artifact supports direct text editing, asset manipulation, and separate export, while Agent Design Replay serializes the full decision process.The replay makes design intent, manual intervention, and later adjustment traceable.
3 Case Studies and Showcase
Evaluations compare the paradigm with diffusion and pure-code generation, test varied design scenarios, and trace a complete design trajectory. The results emphasize a balance between visual quality, structural editability, and process transparency.
- Comparative Analysis: Under the same brief, Editable Visual Design balances diffusion-based visual quality with code-based typography and layer decoupling.Pure diffusion distorts text and entangles layers, while pure code often lacks global aesthetic control.
- Diverse Scenario Showcase: Across event posters, infographics, marketing materials, and long-text layouts, the workflow adapts to different information densities and style requirements.
- Diverse Scenario Showcase: All showcased artifacts use native DOM, allowing text, backgrounds, and illustrations to be selected, edited, and exported independently.
- Real-World Case Study: The red panda infographic replay shows requirement parsing, visual imagination, asset generation, layout coding, render observation, and reflective repair.
- Real-World Case Study: Agent Design Replay exposes how an initial idea becomes a visually expressive artifact with structured code and traceable decisions.
4 Conclusion and Discussion
Editable Visual Design combines visual generation with coding to produce aesthetically refined, structurally editable artifacts. Agent Design Replay makes the design trajectory visible and traceable across planning, asset generation, coding, and repair.
- Collaborative paradigm: Editable Visual Design combines a VLM’s planning and judgment with an image generation model’s visual simulation to balance expressiveness and maintainability.The VLM leads requirement decomposition, planning, code organization, and quality checking, while the generation model renders concrete visuals on demand.
- Collaborative paradigm: Visual generation supplies aesthetic and compositional priors that assist subsequent code layout and arrangement decisions.The imagined visual provides reference information for composition, color, and style before structured code construction.
- Structured delivery: Native HTML/CSS and decoupled assets keep text, backgrounds, and illustrations independently selectable, editable, and exportable.This structured delivery addresses the entangled pixels and error-prone text of conventional image generation.
- Process visibility: Replay cases show timestamped phases for planning, building, and review, with layer structure varying from 120 layers in 13 groups to 6 layers in 1 group.The information-dense field guide receives layout repairs, while the travel poster passes review without changes.
- Process visibility: Agent Design Replay records the trajectory from requirement understanding and concept simulation through asset generation and code adjustment.The replay improves transparency and traceability of design decisions and supports reproducible human-AI collaboration.
5 Limitations
The approach is bounded by the capabilities of its underlying models and is evaluated mainly through qualitative cases rather than fully objective measures. Its current evidence focuses on single-page designs, while both aesthetic quality and editability remain difficult to quantify.
- The delivered design remains bounded by the capabilities of the coding agent and image generation model.Weak layout code can weaken the page, while compositionally ordinary references provide little for the agent to build on.
- The reported cases are all single-page designs, so scalability to longer decks or websites remains dependent on the underlying models.Longer pieces must preserve type scale, palette, and visual continuity across pages.
- Aesthetic quality lacks ground truth and is assessed through cases and VLM judgment rather than objective scores.The evaluation targets subjective properties such as considered composition, atmospheric color, and deliberate type scale.
- Editability is difficult to reduce to a numerical measure because layer counts and DOM structure do not establish whether designers can achieve their intended changes.
6 Related Work
Related work follows code-based and diffusion-based routes, alongside growing efforts to make visual generation more agentic through planning, tool use, and iterative reasoning. Code methods emphasize structured layouts, while image methods improve visual quality, text rendering, and asset generation.
- 6.1 Code Generation and Visual Design: Frontier coding models can generate interactive interfaces and structured layouts from natural-language instructions, building on benchmarks such as Design2Code and WebSight.
- 6.2 Image Generation and Computational Design: Diffusion-based methods improve text-to-image quality, photorealism, text rendering, and structured visual design for posters and infographics.Representative systems include GPT Image 1/2, Nano Banana 2, Seedream 5.0 Pro, and SenseNova-U1.
- 6.3 Agentic Visual Generation: Agentic visual generation research increasingly combines multimodal tool scheduling, planning, verification, sketch refinement, and reflective iteration.Examples include Visual ChatGPT, GenArtist, Idea2Img, RPG, GoT, T2I-R1, Uni-CoT, Mind-Brush, SCOPE, and GEMS.