Source-linked AI summary
UniWorld-Design: From Pixel Generation to Layer-Native Design
Zongjian Li, Zhiyuan Yan, Chenxu Bai, Chen Chen, Haoxiang Sun, Shaodong Wang, Feize Wu, Shenghai Yuan, Bin Lin, Zheyuan Liu, Yuwei Niu, Li Yuan
TL;DR
Flat pixel synthesis lacks the editable structure used to create and manipulate visual content. UniWorld-Design makes semantic RGBA layers native to generation and decomposition, with I2L improving fidelity and usability over Qwen-Image-Layered while T2RGBA achieves the highest mean CLIP Score among compared models.
Problem
Flat RGB outputs lack the semantic, independently manipulable layer structure used to compose and edit visual content.
Method
UniWorld-Design represents images as ordered semantic RGBA layers through text-to-asset generation and instruction-controlled image decomposition.
Results
I2L improves per-layer fidelity and usability over Qwen-Image-Layered, while T2RGBA achieves the highest mean CLIP Score; I2L reduces RGB L1 by 37% and improves Alpha Soft IoU by 34%.
Takeaways & Limitations
Semantic RGBA layers provide persistent, language-addressable object states that external design agents can select, manipulate and recompose.
Takeaways & Limitations
Alpha-boundary quality and dense typography, especially Chinese text, remain primary limitations requiring further improvement.
Abstract
from arXiv · showhide
We introduce UniWorld-Design, a framework that redefines image generation from flat pixel synthesis to structured visual composition, with semantic RGBA layers as the atomic units of generation, understanding, and editing. Our key insight is that pixels define how an image is rendered, whereas layers define how an image is created, understood, and edited. Just as human designers create and manipulate visual content through layers rather than raw pixels, UniWorld-Design equips multimodal generative models with a layer-native design space. UniWorld-Design comprises two models. The Text-to-RGBA (T2RGBA) model generates standalone RGBA assets directly from text. The Image-to-Layer (I2L) model conditions on a finished image, a global instruction and per-layer prompts, and jointly produces ordered, complete semantic RGBA layers. Its instruction interface supports top-level decomposition, recursive decomposition and targeted extraction, making layering an instruction-addressable operation for agentic editing. Because I2L learns complete semantic objects rather than visible-pixel partitions, its layers stay usable when moved or removed. On the Crello benchmark, I2L reduces per-layer RGB L1 error by 37% and achieves a 34% relative improvement in Alpha Soft IoU over Qwen-Image-Layered. Separately, T2RGBA achieves the highest CLIP Score, outperforming LayerDiffuse and OmniAlpha.
1 Introduction
UniWorld-Design reframes image generation and editing around ordered semantic RGBA layers rather than flat RGB pixels. It introduces T2RGBA for reusable asset generation and I2L for instruction-addressable decomposition into complete, controllable layers, achieving stronger benchmark results than Qwen-Image-Layered and the highest compared CLIP Score.
- Motivation: Flat RGB synthesis loses explicit object structure, layer order, hierarchy, and occluded content, making later edits require incomplete reconstruction from pixels.Whole-image editing can affect unchanged content, while masks localize edits without restoring the underlying layered structure.
- Layer-native formulation: UniWorld-Design treats semantic RGBA layers as atomic units that are independently addressable, placeable, editable, movable, deletable, and recomposable.The framework exposes persistent object states for structured generation, decomposition, refinement, editing, and recomposition.
- Framework and models: T2RGBA generates reusable RGBA assets from text, while I2L jointly produces ordered, complete semantic RGBA layers from a finished image, global instruction, and per-layer prompts.Both models use RGBA as a native compositional representation rather than merely an output or post-processing format.
- Instruction interface: I2L makes decomposition instruction-addressable through top-level decomposition, recursive refinement, and targeted extraction at controllable granularity.An external agent can select independent layers, refine an existing decomposition, or extract requested content before editing and recomposition.
- Evaluation: 37% lower per-layer RGB L1 and 34% higher Alpha Soft IoU are reported for I2L versus Qwen-Image-Layered, while T2RGBA reaches the highest CLIP Score at 33.03.I2L also reduces blank-layer generation by 63% and scores 20.43 versus 17.60 on the VLM metric; alpha boundaries and dense text remain challenging.
2 Related Work
Related work spans native RGBA generation, semantic layer decomposition, and agentic image editing. UniWorld-Design follows native RGBA modeling while distinguishing standalone asset generation from layered image decomposition and text-addressable editing.
- RGBA generation: RGBA generation follows pretrained RGB models augmented with alpha or native RGBA representations for reconstruction, layered generation, and multitask generation.UniWorld-Design follows the native RGBA direction, using T2RGBA for standalone RGBA assets and I2L for layered image decomposition.
- Layer decomposition: Qwen-Image-Layered, OmniPSD, and LayerD decompose or generate layered visual content, while related work develops layered training data and editability evaluations.Qwen-Image-Layered supports recursive re-decomposition, OmniPSD targets text-to-PSD generation and image-to-PSD recovery, and LayerD sequentially extracts raster-design elements.
- Agentic image editing: Agentic image-editing systems increasingly use planning, verification, and layer-aware execution to decompose requests, edit localized regions, or construct editable hierarchies.GenArtist uses a multimodal planner; Agent Banana treats a mask-localized crop as a local execution patch; ReDesign grows an editable JSON hierarchy using heterogeneous extraction tools.
3 Method
UniWorld-Design uses a layer-native generation pipeline built on RGBA autoencoding, instruction-bound layer decomposition, and flow-based training with progressive adversarial distillation and DiffusionNFT post-training. T2RGBA and I2L use task-specific reward designs, while I2L generates ordered, complete semantic layers that preserve occluded content and support compositing-based reconstruction.
- RGBA representation: RGBA autoencoding extends only the first encoder and last decoder convolutions from three to four channels, preserving RGB initialization while starting with an opaque alpha channel.Training mixes transparent RGBA and opaque RGB images and combines RGB/alpha reconstruction with perceptual, KL, adversarial, and RGB-latent alignment terms.
- Layer-native decomposition: I2L maps a composite image and instruction to N ordered, complete semantic RGBA layers, retaining occluded content and reconstructing the input through back-to-front alpha compositing.Its interface supports semantic-role decomposition, recursive decomposition, and targeted extraction, where the unselected remainder can be represented as a target.
- Layer-native decomposition: Layer–instruction binding restricts each image query to global conditioning and its matching layer prompt, while unrestricted image attention resolves occlusion and stacking across targets.The attention mask broadcasts global instructions to all image queries and binds each layer prompt to its corresponding layer.
- Training and post-training: Progressive adversarial distillation is adopted because distribution-matching distillation produced overly opaque alpha, RGB–alpha misalignment, weaker condition following, and weaker cross-layer consistency.The student is anchored to the teacher’s next flow location with an endpoint L1 term, while a discriminator compares teacher and student endpoints at the same timestep and conditioning.
- Training and post-training: T2RGBA and I2L receive distinct DiffusionNFT reward pipelines: T2RGBA evaluates alpha following and prompt–image similarity, whereas I2L combines per-layer RGBA reconstruction and RGB-only LPIPS rewards.For I2L, reconstruction distances are averaged equally across target layers with equal L1 and LPIPS weights; no recomposed-image reward is used.
4 Evaluation
Evaluation shows that I2L improves per-layer fidelity and editability on Crello while supporting instruction-controlled recursive decomposition. Separately, T2RGBA achieves the highest mean CLIP Score, although OmniAlpha leads on several reconstruction metrics.
- Image-to-Layer evaluation: I2L is evaluated on 512 fixed-seed Crello designs with four requested layers, matched against Qwen-Image-Layered at the same layer count.Crello data are not used for training; I2L runs at 512-equivalent image area with eight steps and no classifier-free guidance.
- Image-to-Layer evaluation: 37% relative reduction in per-layer RGB L1 (0.2014 →0.1264) and 34% relative increase in Alpha Soft IoU (0.5454 →0.7325) show closer RGB and alpha agreement.The comparison is made after stack alignment.
- Image-to-Layer evaluation: 63% fewer blank outputs (0.35 →0.13), Feature Distribution rising from 0.658 to 0.690, and Bad Layers falling from 1.69 to 1.32 indicate improved editability.Glazed outputs also fall from 1.34 to 1.19.
- Image-to-Layer evaluation: VLM evaluation improves content distribution (+0.95), content validity (+0.93), semantic separation (+0.83), and background inpainting (+0.56), while alpha cleanliness declines from 3.33 to 2.90.The four gains account for most of the 2.83-point total improvement.
- Image-to-Layer evaluation: I2L qualitatively preserves the same structural pattern across designs, and a returned layer can be decomposed again by a second call.The examples demonstrate instruction-controlled recursive decomposition.
- Text-to-RGBA evaluation: T2RGBA has the highest mean CLIP Score (33.03 versus 31.00, +6.5%), while OmniAlpha leads on FID (87.86 versus 117.14) and white-composite LPIPS (0.462 versus 0.537).Relative to LayerDiffuse, FID falls by 45%, CLIP Score rises by 13%, and the three paired measures improve by 15–26%.
5 Limitations
UniWorld-I2L remains limited by imperfect alpha edges and typography, especially for dense or Chinese text. These weaknesses leave fine boundaries, glyph fidelity, and layout stability as areas for future improvement.
- 5 Limitations: Alpha cleanliness is lower for UniWorld-I2L than for Qwen-Image-Layered, at 2.90 versus 3.33.Improving fine alpha boundaries remains future work.
- 5 Limitations: Dense typography, particularly Chinese text, can cause missing strokes, wrong characters, and unstable layouts.Typography-intensive design will require stronger text generation in the underlying model.
6 Conclusion
UniWorld-Design makes semantic RGBA layers the native units of generation and interaction, with T2RGBA producing reusable assets and I2L decomposing images into ordered, complete, language-addressable layers. Evaluations show stronger layer fidelity and usability than Qwen-Image-Layered, while alpha-boundary quality and dense typography remain limitations.
- Core framework: UniWorld-Design treats semantic RGBA layers as native units for generation and interaction rather than post-processing outputs.Its layers can be selected, manipulated with external tools, and recomposed as persistent, language-addressable object states.
- Core framework: T2RGBA generates reusable standalone RGBA assets, while I2L produces ordered, complete semantic layers through top-level decomposition, recursive refinement, and targeted extraction.I2L supports instruction-addressable layering for external design agents.
- Evaluation: I2L improves per-layer fidelity and usability over Qwen-Image-Layered, while T2RGBA achieves the highest mean CLIP Score among compared models.These results support the framework’s advantages in both image decomposition and standalone RGBA asset generation.
- Limitations and future work: Alpha-boundary quality and dense typography remain the primary limitations observed in the evaluations.Future work will scale data, model capacity, and compute toward unified native RGBA generation and understanding, including insertion, replacement, constrained composition, and iterative editing.
A Contributions
The paper credits a contributor team of ten researchers and identifies Zhiyuan Yan and Li Yuan as project leads.
- The contributor roster includes Zongjian Li, Chenxu Bai, Chen Chen, Haoxiang Sun, and Shaodong Wang.
- The contributor roster also includes Feize Wu, Shenghai Yuan, Bin Lin, Zheyuan Liu, and Yuwei Niu.
- Zhiyuan Yan and Li Yuan are identified as the project leads.
B Evaluation Details
This section defines ordered layer-quality, editability, VLM, and T2RGBA evaluation protocols, including alignment, alpha-aware scoring, artifact criteria, and generation metrics. The measures assess layer correspondence, independent editability, semantic and compositing quality, and text-to-RGBA fidelity.
- Per-layer metrics (LayerD): LayerD aligns predicted and reference stacks as ordered sequences with dynamic-time-warping edits, potentially merging adjacent layers before scoring.Matched pairs are compared at the smaller resolution, using downsampling only.
- Per-layer metrics (LayerD): RGB L1 is weighted by binary reference support, while Alpha Soft IoU uses continuous alpha and remains sensitive to semi-transparent regions.Both RGB and alpha values lie in [0, 1].
- Editability metrics: Bad Layers counts predicted layers that are blank or glazed, using mean-alpha and RGB-variation thresholds to identify residual-color artifacts limiting independent editability.Blank layers have mean alpha below 0.01; glazed layers have mean alpha in (0.1, 0.9) and RGB standard deviation above 0.05.
- VLM judging: GPT-5.6-terra judges the input design, predicted alpha composite, and labeled layer contact sheet with five integer scores ranging from 0 to 5.The judging dimensions include semantic separation, alpha cleanliness, and background inpainting, among others.
- T2RGBA metrics: T2RGBA evaluation computes FID and CLIP Score over 512 generated samples, while Alpha MSE, SAD, and white-composite LPIPS use 479 successfully matched prompts.FID uses Inception-v3 features at 299×299 against 1000 real layers; CLIP Score uses CLIP ViT-B/32, and white-composite LPIPS uses AlexNet.
C Sequence Construction
The sequence is packed layer-major, with shared crop-resize alignment preserving canvas coordinates across layers. Text begins with a global instruction and then assigns each target layer indexed short and detailed prompts to explicit layer slots.
- Token packing: Tokens are packed layer-major, placing every token of layer 0 before layer 1 and subsequent layers rather than interleaving spatial positions.A shared crop and resize across sample layers makes each (h, w) coordinate refer to the same canvas location in every layer.
- Token packing: Text is packed as a global instruction followed by indexed short and detailed prompts for each target layer.The explicit layer index binds each description to its corresponding layer slot.
D Prompts for the Qualitative T2RGBA Comparison
Figure 7 compares T2RGBA qualitatively using seven single-resource prompts and seven multi-resource prompts. The prompts span isolated illustrated assets, cohesive scrapbook clusters, and promotional badge designs with explicit composition, palette, texture, and transparency constraints.
- Figure organization: Figure 7 presents seven single-resource prompts in the top panel and seven multi-resource prompts in the bottom panel, ordered left to right.The figure’s prompts are organized by panel and follow the displayed column order.
- Single-resource prompts: Single-resource prompts request isolated illustrated assets such as a peach cat, wax seal, vegetable-filled tote, whale, bouquet, juicer, and birthday cake.These prompts consistently specify centered placement, transparent backgrounds, crisp silhouettes, controlled highlights, and a 70-percent canvas occupancy target.
- Multi-resource prompts: Multi-resource prompts create cohesive scrapbook assets, including breakfast stickers, gardening kits, rainy-day clusters, seaside ephemera, and city-weekend travel designs.They combine multiple named objects with coordinated palettes, hand-drawn gouache and colored-pencil texture, sticker borders, paper shadows, balanced spacing, varied scale, and transparent isolation.
- Promotional prompts: The comparison also includes promotional assets requiring exact lettering: “SUMMER SAVINGS” on a summer badge and “DOWNLOAD APP” on an app-download badge.Both prompts specify complete isolated button or badge assets, dimensional highlights, clean shadows, controlled ornament, and readability at thumbnail size.