Source-linked AI summary
GenClaw: Code-Driven Agentic Image Generation
Junyan Ye, Jun He, Zilong Huang, Dongzhi Jiang, Xuan Yang, Rui Chen, Weijia Li
TL;DR
Existing image-generation agents largely refine prompts while relying on black-box synthesis, limiting direct canvas control. GenClaw uses a Conceptualize–Sketch–Color workflow that combines reasoning, executable visual code, and image generation, offering more controllable composition, text layout, and physical-scene simulation.
Problem
Existing image-generation agents rely almost entirely on black-box text-to-image synthesis, leaving agents without direct, staged control over canvas construction.
Method
GenClaw uses a Conceptualize–Sketch–Color pipeline that combines search and reasoning, executable visual code, and image generation for final textures and photorealism.
Results
GenClaw demonstrates stronger controllability for complex composition, text rendering, physical simulation, and layered editing.
Takeaways & Limitations
Code functions as a controllable intermediate canvas that makes image generation more transparent and traceable across creative stages.
Takeaways & Limitations
GenClaw depends heavily on powerful underlying image-generation models because current open-source models often fail to render abstract sketches photorealistically.
Abstract
from arXiv · showhide
Image generation models have evolved from text-conditioned pixel synthesis toward multimodal agents endowed with visual comprehension and tool invocation capabilities. Yet, existing agents remain at the mercy of underlying black-box image models. Their workflow is trapped in a repetitive cycle of prompt rewriting for generation refinement, leaving them with no mechanism to directly manipulate the canvas. In essence, the potential of LLMs to serve as a genuine "brush" for precise visual construction remains largely untapped. In this paper, we propose GenClaw, a code-driven agentic image generation paradigm that empowers the agent to create like a human artist: first conceptualizing, then sketching, and finally coloring. Specifically, the agent first constructs the conceptual knowledge and context through search and reasoning. It then utilizes code (e.g., SVG, HTML, ThreeJS) to render executable visual sketches. Finally, it employs an image generation model to supplement textures, materials, and photorealism. In this workflow, code serves as a controllable intermediate canvas bridging linguistic reasoning and pixel synthesis, seamlessly integrating programmatic logic with the visual expressiveness of generative models. By transforming image generation from a black-box paradigm into a staged process akin to authentic human creation, GenClaw offers a step toward for highly controllable and interpretable visual generation systems.
1 Introduction
GenClaw proposes a code-driven agentic image-generation paradigm that replaces black-box prompt rewriting with a staged workflow: conceptualize through search and reasoning, sketch with code, and color with an image-generation model. Code acts as a controllable intermediate canvas, enabling more structured visual expression and generation transparency.
- Motivation: Existing image-generation agents can plan, invoke tools, and refine outputs, but final visual synthesis still relies largely on end-to-end text-to-image generation.The agent is therefore constrained to repeated prompt rewriting while the underlying image model remains a stochastic black box.
- Motivation: GenClaw uses code as a controllable digital brush and intermediate representation for expressing visual intent more precisely than natural language alone.Candidate representations include SVG, HTML, and other visual code or layered structures suited to structured visual generation.
- GenClaw Workflow: The workflow follows three stages: search and reasoning conceptualize entities and context, code sketches composition and structure, and the image model colors the sketch with textures, materials, and realism.The sketch can specify object positions, sizes, text layout, layer occlusion, and 3D physical rules before pixel synthesis.
- Advantages: Code-driven generation improves compositional controllability and text-layout reliability by representing coordinates, quantities, and rendered text explicitly rather than treating them as pixel textures.This is intended to alleviate hallucinations in object counts and spatial relations and reduce spelling confusion.
- Significance: GenClaw shifts image generation from end-to-end black-box synthesis toward a human-artist-like process with greater transparency and traceable failure sources.The paper presents the system as an exploratory initial harness for using general-purpose code agents in visual generation.
2 Related Work
Image generation has progressed from text-conditioned synthesis to unified multimodal and increasingly agentic systems that incorporate reasoning, search, review, and tool use. Related work also explores executable visual code as an editable intermediate representation, while identifying limitations in unified architectures, complex-scene composition, and photorealistic detail.
- Unified Multimodal Image Generation: Image generation has evolved from text-conditioned pixel synthesis to unified multimodal models supporting visual understanding and generation, including systems capable of complex typography, bilingual text, and high-fidelity editing.Qwen-Image and successors perform well on complex Chinese/English typography, while Nano-Banana achieves solid performance in complex generation and high-fidelity editing.
- Agentic Image Generation: Recent closed-source systems integrate search and review modules into image-generation loops, marking a shift from passive pixel synthesizers toward tool-using visual agents.Nano-Banana-Pro and FLUX 2 Pro exemplify this emerging agentic direction.
- Agentic Approaches: Code agents and image-generation agents increasingly separate high-level understanding from concrete generation, enabling models to read state, invoke tools, and revise actions using feedback.Think-Then-Generate and GenAgent explicitly decouple understanding from generation, while code-agent systems such as Codex and Claude Code motivate this broader executable-agent paradigm.
- Visual Code Representations: CoCo validates executable programs as intermediate sketches but relies heavily on one unified model for code generation and pixel refinement, limiting its use of decoupled architectures on complex tasks.CoCo uses Matplotlib code to create a structured sketch that is subsequently refined into a final image.
- Visual Code Representations: Executable visual code represents images through editable, losslessly scalable, structurally explicit vector programs containing paths, shapes, text, and hierarchies.This research direction includes representations such as SVG and HTML, motivated by the reasoning and code-authoring strengths of large language models.
- Limitations of Existing Methods: Pure-code image generation remains limited for complex-scene composition and open-domain semantic organization, and struggles with high-frequency photorealistic details such as lighting, hair, and natural texture.Existing research is largely concentrated on regular tasks including icons, UI layouts, and isolated components.
3 Method
GenClaw structures image generation as a three-stage, code-driven workflow: cognitive reasoning and knowledge gathering, executable sketch construction, and image-model-based visual realization. Code serves as an interpretable intermediate canvas that makes structure, text, relations, and constraints directly controllable before visual enhancement.
- Cognitive Structuring Layer: The Cognitive Structuring Layer uses VLM/LLM reasoning, search, knowledge bases, and tools to understand intent, references, world knowledge, and symbolic relations before generation.The agent can retrieve facts for long-tail entities, real-time events, geographic locations, cultural symbols, and professional objects when internal knowledge is insufficient.
- Visual Realization and Review: Generated results are reviewed by a VLM or interactively by the user, enabling precise alignment checks and dynamic, fine-grained adjustments through the transparent workflow.Compared with prompt-only refinement, the framework materializes agent reasoning as an executable canvas state across conception, sketching, and completion.
- Executable Sketch Construction: The agent converts its conception into executable code that renders an intermediate sketch containing the image’s objects, text, layout, and structural relations.Structured records emphasize executability and traceability by specifying visual objects, rendered text, preserved relations, and supporting knowledge facts.
- Executable Sketch Construction: The sketch backend is task-dependent: SVG controls object nodes and spatial constraints, HTML/CSS deterministically renders text, and Python, Canvas, or Three.js models geometric and physical relations.These backends support fixed object counts, strict spatial or occlusion relations, text-heavy layouts, and physical references such as mirror reflections.
- Visual Realization and Review: The final layer feeds the executable code and rendered sketch to image generation or editing models, which supplement texture, lighting, materials, details, and realism instead of planning structure from scratch.Qwen-Image and Nano Banana are examples of models used for final visual realization.
4 Experiments
GenClaw is evaluated across image generation, editing, text rendering, physical-simulation, and knowledge-driven reasoning tasks against open-source and proprietary models. Results attribute its advantages to code-driven intermediate canvases that provide explicit layout, deterministic text rendering, executable physical constraints, and agentic knowledge integration.
- Evaluation setup: GenClaw is evaluated on GenEval++, LongText-Bench, ImgEdit, and Mind-Bench against models including GPT-Image, Qwen-Image, Nano-Banana, and others.The benchmarks cover complex-scene instruction following, long-text rendering, image editing, and world knowledge and reasoning.
- Implementation: Claude-Opus-4.6 translates user intents into structured records and executable canvases, while Gemini-3.1-Flash-Image performs final rendering conditioned on sketches, text layers, masks, or editing constraints.The rendering backend dynamically adapts its code to the task.
- GenEval++: On GenEval++, explicit SVG pattern guidance gives GenClaw advantages on Counting and Spatial tasks over GPT-Image-1.5 and Gemini-3.0 Pro-Image.The passage attributes this advantage to better control than text-only generation for quantities and spatial relationships, which can be compressed or mismatched in natural-language descriptions.
- LongText-Bench: GenClaw achieves clear advantages on Chinese and English LongText-Bench tasks by deterministically rendering text through HTML/SVG layers while leaving backgrounds, styles, and visual details to the image model.This task division avoids requiring one black-box pixel model to organize knowledge, lay out text, and draw characters simultaneously.
- Physical simulation: For physical-simulation scenes, GenClaw builds simplified physical or geometric models in code before generation, converting formalizable world rules into inspectable visual constraints.Three.js can deterministically place mirror materials, light sources, and objects so generated images can follow the sketched reflection positions.
- Editing and reasoning: On ImgEdit, PSNR and SSIM are computed only on unedited regions, while on Mind-Bench GenClaw remains highly competitive as agentic external knowledge and explicit reasoning improve image-model performance.Most baseline editing models substantially alter unedited regions, and GenClaw’s layered representation remains a stated direction for future exploration.
5 Limitations and Future Work
GenClaw improves spatial controllability and compositional accuracy but introduces limitations in model dependence, efficiency, and code-generation stability. These limitations motivate future work on more capable generation models, more efficient pipelines, and more reliable code generation.
- Overall Limitations and Future Work: Although GenClaw enhances spatial controllability and compositional accuracy, its limitations highlight directions for future research.The stated limitations concern dependence on the generation model, efficiency overhead, diminishing returns, and code-generation stability.
- High Dependency on the Underlying Generation Model: GenClaw depends heavily on the underlying image generation model to translate abstract SVG or HTML sketches into high-fidelity, photorealistic images.Current open-source conditional generation models often produce severe artifacts or degraded textures during this translation.
- Efficiency Overhead and Diminishing Returns: The multi-step agentic pipeline introduces significant inference latency and computational overhead, making it redundant and inefficient for simple tasks.Its time cost is justified for complex generation requiring precise control but compares unfavorably with one-shot end-to-end generation for straightforward tasks.
- Stability Risks in Code Generation: Natural-language-to-code translation is unstable, with LLMs sometimes generating coordinate, layer-occlusion, or element-scaling errors.These code-level flaws can produce suboptimal spatial layouts or misaligned details in the final generation.
6 Conclusion
The conclusion frames GenClaw as a shift from one-shot image guessing and prompt engineering toward step-by-step, code-driven construction through a Conceptualize →Sketch →Color workflow. It presents GenClaw as an exploratory step toward proactive visual generation systems with greater controllability, interpretability, and reasoning capabilities.
- GenClaw advocates shifting from increasingly sophisticated prompt engineering to agentic image construction through code, step by step like a human painter.
- The proposed Code-Driven Agentic Image Generation paradigm is instantiated in GenClaw through the Conceptualize →Sketch →Color workflow.
- Future visual generation systems are expected to move from passive, chatbox-based responses toward a more proactive and agentic paradigm.
- GenClaw is presented as an initial exploratory step intended to inspire the community and inform visual creation systems with high controllability, interpretability, and profound reasoning capabilities.