PaperlayerView exact revision on arXiv

Source-linked AI summary

MRT: Masked Region Transformer for Layered Image Generation and Editing at Scale

Zhicong Tang, Zhao Zhang, Jingye Chen, Mohan Zhou, Yifan Pu, Yuchi Liu, Yalong Bai, Ethan Smith, Yuhui Yuan

arXiv:2605.27235v1cs.CV

TL;DR

Layered image generation remains underdeveloped relative to text-to-image generation. MRT unifies three layered-generation tasks with masked diffusion and reports stronger scaling, out-of-domain generalization, and multi-image fusion while supporting efficient synthesis of editable layers.

  • Problem

    Generative models for layered image generation remain significantly underdeveloped compared with text-to-image models.

  • Method

    MRT unifies text-to-layers, image-to-layers, and layers-to-layers generation and editing through masked diffusion, overflow-aware canvas layers, and distillation.

  • Results

    The study reports improved performance from scaling, strong image-to-layers generalization to out-of-domain images, and support for multi-image fusion in layers-to-layers generation.

  • Takeaways & Limitations

    MRT advances scalable multi-layer generation by enabling efficient synthesis of high-fidelity, semi-transparent, fully editable visual layers.

  • Takeaways & Limitations

    On real-world photographs, MRT often mishandles shadows because its design-only training data lacks physical effects common in natural scenes.

Abstract

from arXiv · show

Layered image generation and editing is a fundamental capability that enables layer-wise reuse, editing, and composition of generated visual content, analogous to word-level editing in natural language. Despite its importance, this remains an underexplored area at scale. To address this gap, we present MRT, a 20B-parameter masked region diffusion model tailored for multi-layer transparent image generation and editing, trained on over 10M multilingual design samples spanning diverse aspect ratios and textual prompts. To fully leverage this scale, we make two key technical contributions. First, we unify three complementary tasks including text-to-layers, image-to-layers, and layers-to-layers within a shared masked region diffusion framework, where selective token masking enables flexible layer-wise generation and editing. Second, to enable overflow layer generation, we introduce an overflow-aware canvas layer that handles boundary inconsistencies and supports semi-transparent background synthesis, enabling complete editable layers extending beyond visible canvas boundaries. Additionally, we apply diffusion distillation to achieve 8-step, real-time multi-layer generation with minimal quality degradation. Extensive experiments demonstrate that our framework substantially outperforms prior state-of-the-art approaches, including various commercial systems, across all three tasks, establishing a new benchmark for multi-layer transparent image generation. Notably, our model significantly outperforms the concurrent Qwen-Image-Layered model in image-to-layers quality according to user-study results, while achieving 10-100\times faster inference and reducing activation GPU memory consumption by 50-90\% during image-to-layer inference.

1. Introduction

The paper addresses large-scale layered image generation and editing with a dataset of ≥10 million samples and a unified masked region transformer for text-to-layers, image-to-layers, and layers-to-layers tasks. It further introduces adaptive masking and distribution matching distillation to improve flexibility and enable few-step generation.

  • Dataset: The dataset contains ≥10 million samples, over 43 million unique layers, and over 7 million unique oversized visual elements across diverse resolutions and aspect ratios.GPT-5 mini generates global captions for all graphic designs, while oversized elements support overflow layer generation.
  • Technical Contributions: The unified masked region transformer supports text-to-layers, image-to-layers, and layers-to-layers generation and editing through adaptive masking.Adaptive masking determines whether each layer is initialized from clean latents or noise according to the task.
  • Technical Contributions: Distribution matching distillation produces a few-step multi-layer generator with minimal quality degradation.
  • Experiments: Scaling the model and dataset elevates performance, while joint multi-task training further improves performance and user experience.The experiments also show that image-to-layers generalizes to out-of-domain design and natural images, and layers-to-layers supports multi-image fusion.

2. Related Work

Prior layered image generation and editing methods follow simultaneous or sequential generation paradigms, while related layout methods either generate layouts from visual elements or control generation through spatial conditioning.

  • Layered Generation and Layout Control: Layered image generation and editing methods divide into simultaneous approaches, including Text2Layer [63], LayerDiff [17], ART [38], PrismLayer [5], and Qwen-Image-Layered [59], and sequential approaches, including LayerDiffuse [62], COLE [22], OpenCOLE [21], and LayerD.Related layout generation and control methods separately focus on generating layouts from visual elements or controlling generation via spatial conditioning.

3. Approach

MRT combines a curated 10M+ multilayer design dataset with a shared masked regional diffusion transformer for text-to-layers, image-to-layers, and layers-to-layers generation. Its overflow-aware canvas layer preserves complete editable content beyond visible boundaries, while layer grouping augmentation and diffusion distillation improve robustness and efficiency.

  • Scaled Layered Dataset: Over 10M professionally created and fully licensed multilayer graphic designs provide the curated training dataset for large-scale modeling.The dataset spans multilingual designs and diverse layer structures, aspect ratios, and visual token counts.
  • Overflow Layer Support: Over 60% of training samples contain overflow layers, motivating a full-size transparent canvas layer that preserves complete elements beyond visible background boundaries.Without overflow support, boundary-crossing content is cropped and becomes non-editable; the proposed design uses complete ground-truth layers available in the dataset.
  • Unified Masked Region Diffusion: MRT unifies text-to-layers, image-to-layers, and layers-to-layers through conditional masking of global image, reference, and existing-layer tokens in one regional diffusion transformer.Masked clean tokens encode pre-existing conditions, while diffusion generates the unmasked layer representations.
  • Image-to-Layers: Image-to-layers decomposes raster inputs into transparent layers through regional diffusion conditioned on a target layout, addressing segmentation and occluded-content inpainting.The method uses human annotations or a layout detector and conditions masked tokens on the composed image representation.
  • Training and Efficiency: Layer grouping augmentation randomly groups overlapping or adjacent layers to improve structural diversity, robustness to ambiguous boundaries, and generalization to noisy out-of-domain layouts.The framework uses a 20B-parameter regional diffusion transformer with WAN-2.1-VAE representations, and DMD distillation compresses the teacher into a few-step student.

4. Experiment … 4.3. Main Results

MRT is evaluated with full-parameter Qwen-Image fine-tuning across layered generation, decomposition, and editing benchmarks. It outperforms prior methods in quality and editability while providing efficient multi-layer inference, though several decomposition challenges remain.

  • 4.1. Implementation Details: Full-parameter fine-tuning of the 60-layer Qwen-Image architecture is used to explore MRT’s performance upper bound rather than restricting training to LoRA weights [5] [38].The model has hidden dimension 3584 and 24 attention heads per layer, with weights initialized from an open-source HuggingFace checkpoint.
  • 4.1. Implementation Details: System-level training uses approximately 70,000 iterations at 512 × 512 on the full 10M dataset followed by approximately 20,000 iterations at 1024 × 1024 on 64×H200 GPUs.This progressive strategy first establishes multi-layer decomposition and then scales to higher resolution.
  • 4.2. Evaluation Protocol: Evaluation combines DESIGN-MULTI-LAYER-BENCH with the overflow-focused OVERFLOWERFLOW-DESIGN-BENCH and measures layer, merged-image, and human-preference quality.The standard benchmark lacks overflow layers, motivating the additional benchmark for complete layers extending beyond visible canvas boundaries.
  • 4.3.1. Text-to-Layers: Comparison with SoTAs: Participants preferred MRT over ART in instruction following, overall aesthetics, and layer quality, while MRT uniquely generates overflow RGBA layers beyond the background boundary.Prior systems such as ART restrict pixels to the background region, causing cropped or missing content.
  • 4.3.2. Image-to-Layers: Comparison with SoTAs: MRT is preferred over LayerD, RoboNeo, and Lovart for image-to-layer quality, content integrity, and decomposition granularity, producing sharper and more reusable RGBA layers.The evaluation uses layouts extracted by a z-order-aware detector, and qualitative results also show generalization to natural scenes.
  • 4.3.3. Image-to-Layers: Comparison with con-current Qwen-Image-Layered: MRT achieves significantly higher SNRmerged and SSIMmerged than Qwen-Image-Layered on a 100-image out-of-domain test set spanning three source groups.The comparison is conducted on creative designs generated by Nano-Banana-Pro and Ideogram 3.0, plus images from the official Qwen-Image-Layered repository.
  • 4.3.4. Layers-to-Layers: Layered Editing: For layered editing, MRT generates or restylizes multiple layers in one pass, preserving cross-layer consistency, global composition, geometry, and target style better than GPT-Image-1.GPT-Image-1 instead performs iterative layer-by-layer generation, which increases latency and can propagate inconsistencies across edits.
  • 4.3.3. Image-to-Layers: Comparison with con-current Qwen-Image-Layered: Up to 108.5× speedup at approximately 20 layers is achieved over Qwen-Image-Layered, while MRT maintains near-constant latency of approximately 5 seconds.The regional diffusion transformer avoids processing full-resolution tokens for every layer, and Figure 19 also reports memory comparisons across layer configurations.

4.4. Ablation Study and Analaysis

Ablations show that scaling model capacity and data improves quality, while overflow-aware generation, unified training, caption ablation, layer grouping, and distillation each support the framework’s effectiveness. Distillation reduces sampling cost substantially with minimal quality degradation, whereas adding layers-to-layers training introduces a slight performance drop.

  • Overflow support: Overflow-aware training preserves overflow elements while achieving comparable FID, PSNR, and SSIM scores.More than 60% of designs contain overflow layers, which prior methods truncate.
  • Multi-task training: Unified random-sampling training integrates text-to-layers, image-to-layers, and layers-to-layers with comparable performance and minimal degradation.Adding layers-to-layers slightly reduces overall performance, attributed to layer-to-layer dataset quality issues.
  • Textual conditioning: Removing global captions from image-to-layers yields modest but consistent metric improvements, showing textual conditioning is not essential.Captions can still aid boundary disambiguation and provide semantic cues for complex overlapping compositions.
  • Layer grouping augmentation: Layer grouping augmentation consistently improves image-to-layers robustness, including on DESIGN-MULTI-LAYER-BENCH with high-quality layout annotations.Randomly merging training layers increases layout diversity and addresses the gap between precise training layouts and noisy test-time layouts.
  • Multi-layer generator distillation: DMD2 distillation [60] [61] reduces generation from 50 to 8 denoising steps, delivering a 6× speedup with minimal performance degradation.FID remains comparable in Table 6 and visual quality is largely preserved in Fig. 23.
  • Scaling: FID drops from 17.79 to 16.15 with model scaling and to 15.63 after expanding the dataset from 0.5M to 10M samples.The comparison uses FLUX.1 [dev] (13B) and Qwen-Image (20B) text-to-layers models under the same initial dataset and limited training budget.

5. Conclusion

The paper presents MRT as the first systematic study of the performance frontier for large-scale multi-layer transparent image generation. MRT unifies three generation tasks through masked regions and combines overflow-aware canvas handling with distillation for efficient, high-fidelity, semi-transparent, fully editable layers.

  • MRT unifies text-to-layers, image-to-layers, and layers-to-layers generation within a shared masked region paradigm at scale.The 20B-parameter model was trained on over 10M multilingual design samples.
  • In layers-to-layers editing, MRT better follows layer-wise addition instructions and improves layer coherence and style consistency over GPT-Image-1.The task supports iterative, flexible user interaction through layer-wise editing.
  • MRT generalizes to out-of-domain natural images despite training only on poster-style design datasets.Figure 24 reports qualitative image-to-layers results on natural scenes.
  • Overflow-aware canvas handling and distribution matching distillation enable efficient synthesis of high-fidelity, semi-transparent, fully editable visual layers.The canvas layer supports complete boundary handling, while distillation enables real-time generation.

Supplementary Material

The supplementary material studies how caption length, distillation, and the number of generated layers affect model training or inference. It also includes an inference-time analysis for the distilled multi-layer generator.

  • Supplementary Material: Table 1 evaluates short, long, and mixed caption training using FID on the VC5K test set with short and long captions.The study compares caption-length training strategies under matching short- and long-caption evaluation conditions.
  • Supplementary Material: Table 2 examines multi-layer generator distillation together with inference time.The table focuses on the relationship between distillation and inference efficiency.
  • Supplementary Material: Table 3 evaluates image-to-layers generation quality across different ranges of generated layer numbers.This analysis isolates the effect of varying the number of layers in generated results.

1. Additional ablation experiments

Additional ablations show that MRT benefits from mixed caption lengths, scales from 2 to 50 layers without architectural changes, and achieves substantially faster inference through DMD2 distillation with limited quality loss.

  • Caption diversity: Mixed caption-length training generalizes best, achieving FID 16.13 on short captions and 15.93 on long captions.Short-caption-only training reaches 18.56 FID on long captions, while long-caption-only training reaches 16.15 FID and remains suboptimal on short captions.
  • Layer-count scalability: The image-to-layers model maintains stable performance across compositions ranging from 2 to 50 layers without architectural modifications.This supports decomposition of both simple designs and complex multi-element compositions.

2. Attention Analysis of Image-to-Layer Model

The attention analysis examines whether the image-to-layer model learns meaningful semantic representations rather than merely memorizing layout priors. Visualized attention maps show correspondence between predicted transparent layers and localized attention activations.

  • The analysis tests whether decomposition attention reflects meaningful semantic representations rather than layout-prior memorization.
  • Fig. 2 visualizes correspondence between generated transparent layers and their associated attention activations.
  • The attention mechanism exhibits strong spatial localization, with heatmaps highly correlating for each predicted layer.

3. User study details

The user studies use blind pairwise comparisons to evaluate text-to-layer and image-to-layers generation quality. Text-to-layer evaluations randomize result positions and assess four dimensions, while image-to-layers comparisons test MRT against three state-of-the-art baselines using a central reference image.

  • Text-to-layer user study: Text-to-layer evaluation used blind, randomized side-by-side pairwise comparisons between MRT and ART.Participants first saw the text prompt, then judged results whose left-right order was randomized to reduce positional bias.
  • Text-to-layer user study: Participants cast three-way forced-choice votes across layout, aesthetics, typography, and layer harmonization.The options were Method A better, Method B better, or Tie.
  • Evaluation interfaces: The text-to-layer interface showed two generated results side-by-side with the caption in a right-hand panel.The image-to-layers interface instead placed the reference input at the center and decomposition results on both sides.
  • Image-to-layers user study: Image-to-layers evaluation ran three independent blind comparisons: MRT versus LayerD, Lovart, and Roboneo.Each comparison displayed the original image centrally, with the two methods’ decomposition results on either side.

4. Limitations

MRT performs strongly on image-to-layer tasks but struggles with real-world photographs, especially shadows, because it was trained exclusively on planar design datasets; nevertheless, it generalizes reasonably well to real images.

  • 4. Limitations: MRT often mishandles shadows in real-world photographs, excluding shadow regions from segmented object layers and leaving them on the background layer.This produces visual inconsistency in the resulting layer decomposition.
  • 4. Limitations: The limitation is primarily attributed to training exclusively on planar design datasets lacking physical effects common in natural scenes, including shadows, reflections, and refractions.
  • 4. Limitations: Despite this domain gap, MRT still generalizes reasonably well to real images.

5. Visualizations and Qualitative Analysis

The visualizations show that MRT generates coherent, editable multi-layer designs, decomposes images robustly, supports controlled layer editing and restylization, and generalizes zero-shot to natural images. They also expose a transparency-decoding failure caused by the current three-channel VAE input.

  • Text-to-layers generation: Text-to-layers examples show coherent, spatially consistent, stylistically harmonious designs, including compositions with over 25 layers and overflow layers extending beyond visible boundaries.The model generates full-size RGBA layers that preserve editability and reusability for downstream composition.
  • Qualitative comparison: MRT produces cleaner boundaries, finer layer granularity, and more complete RGBA layers than LayerD, Lovart, and RoboNeo in qualitative image-to-layers comparisons.RoboNeo often introduces artifacts or fails to preserve transparency, while LayerD tends to group layers excessively, limiting editing flexibility.
  • Robustness and multilingual generation: Image-to-layers decomposition remains stable from 6 to 16 layers, preserving boundaries and content across varying complexity without observed performance degradation.The model also renders accurate visual text across multiple languages, including Chinese.
  • Layer editing and restylization: Layers-to-layers editing adds prompt- and box-specified elements contextually while matching existing lighting, perspective, and artistic style, and restylization preserves asset geometry while adapting appearance.Single-pass generation maintains cross-layer consistency when transferring color palette, texture, and artistic rendering to external assets.
  • Spatial control: The model follows user-provided bounding-box layouts even when text prompts imply conflicting positions, and layout information guides image decomposition as a structural prior.Layouts may come from automatic detectors or manual annotation.
  • Generalization and failure cases: MRT segments natural photographs into transparent layers zero-shot, but complex physical lighting effects reveal a domain-gap limitation; some transparent backgrounds decode as gray because the VAE uses three-channel inputs.A four-channel encoder is suggested as future work to address this ambiguity.
Loading 2605.27235v1…