Source-linked AI summary
Controllable Layer Decomposition for Reversible Multi-Layer Image Generation
Zihao Liu, Zunnan Xu, Shi Shu, Jun Zhou, Ruicheng Zhang, Zhenchao Tang, Xiu Li
TL;DR
Flattening a layered design image makes its original layers difficult to recover for editing, while existing decomposition approaches have limits in controllability and precision. CLD uses bounding-box-guided multi-layer generation and a tailored benchmark; experiments report better decomposition quality, controllability, and visual fidelity than existing methods. Its generated layers are directly usable for downstream editing.
Problem
Compositing raster layers is irreversible, and existing decomposition methods have limitations in controllability and extraction precision, making layer-level editing difficult when only the raster remains.
Source: §1 · p. 2Method
CLD uses user-provided bounding boxes to guide DiT-based generation of fine-grained layers and introduces a benchmark with metrics for controllable decomposition.
Source: §1 · p. 2Results
CLD outperforms existing methods in decomposition quality, controllability, and visual fidelity.
Source: §5 · p. 9Takeaways & Limitations
Generated layers are directly usable for downstream editing, including removal and recomposition.
Source: §5 · p. 9Very small, thin, or intricate content can exceed available spatial resolution, while heavy occlusion may leave insufficient information to reconstruct hidden content accurately.
Source: §12 · p. 13
Abstract
from arXiv · showhide
This work presents Controllable Layer Decomposition (CLD), a method for achieving fine-grained and controllable multi-layer separation of raster images. In practical workflows, designers typically generate and edit each RGBA layer independently before compositing them into a final raster image. However, this process is irreversible: once composited, layer-level editing is no longer possible. Existing methods commonly rely on image matting and inpainting, but remain limited in controllability and segmentation precision. To address these challenges, we propose two key modules: LayerDecompose-DiT (LD-DiT), which decouples image elements into distinct layers and enables fine-grained control; and Multi-Layer Conditional Adapter (MLCA), which injects target image information into multi-layer tokens to achieve precise conditional generation. To enable a comprehensive evaluation, we build a new benchmark and introduce tailored evaluation metrics. Experimental results show that CLD consistently outperforms existing methods in both decomposition quality and controllability. Furthermore, the separated layers produced by CLD can be directly manipulated in commonly used design tools such as PowerPoint, highlighting its practical value and applicability in real-world creative workflows. Our project is available at https://monkek123king.github.io/CLD_page/.
1. Introduction
CLD targets controllable recovery of editable layers from flattened raster images, combining bounding-box guidance with a multi-layer generation framework. The section motivates the approach, defines the task, and describes its backbone and conditioning strategy.
1. Introduction
- Compositing makes layer-level editing difficult to recover; CLD uses user-provided bounding boxes and a new benchmark to target controllable decomposition.The paper identifies incomplete extraction, artifacts, and error propagation in prior pipelines, while LayerD lacks user control.Source: §1 · p. 2
2. Related work
3.1. Preliminaries
- Flow Matching [7] learns a velocity field between noise and data; CLD adopts it for training and fine-tunes the backbone with LoRA [10].The passage contrasts velocity prediction with noise-prediction diffusion models such as DDPM [8].Source: §3.1 · p. 3
3.2. Problem Formulation
- The task maps a raster image and user bounding boxes to aligned, independent RGBA layers, including a full-image background layer.The objective is fine-grained separation with spatial alignment and visual consistency across layers.Source: §3.2 · p. 3
3.3. LayerDecompose-DiT
- LD-DiT crops and concatenates box-defined visual tokens for MMDiT generation, while Layer-Aware Rotary Position Embedding encodes spatial and inter-layer positions.A composite-image token supplies global context; the adapted transparent-image autoencoder preserves alpha information and inter-layer coherence.Source: §3.3 · pp. 3–4
3.4. Multi-Layer Conditional Adapter
- MLCA crops mapped conditional-image features by bounding boxes, aligns them with layer tokens, and adds them to MMDiT hidden states as denoising guidance.The module is designed to provide fine-grained layer control while preserving inter-layer coherence.Source: §3.4 · p. 5
3.5. Dual-Condition Classifier-Free Guidance
- Dual-condition CFG retains the reference image in its unconditional branch while nullifying text, using the image as a structural anchor for isolating text-driven semantics.Subtracting the unconditional prediction from the conditional one cancels shared structural information, according to the paper.Source: §3.5 · p. 5
4. Experiment
The experiments evaluate CLD on a new multi-dimensional benchmark and compare it with LayerD and other approaches. They report advantages across quantitative, qualitative, and editing evaluations, with ablations examining design choices.
4.1. Experiment Setting
- PrismLayersPro provides approximately 20K layered groups across 21 categories, split 90%/5%/5% for training, validation, and testing; FLUX.1[dev] is fine-tuned with LoRA.Training uses LoRA rank 64 for 25K iterations on 1024 × 1024 images.Source: §4.1 · p. 5
4.3. Quantitative Result
- CLD outperforms LayerD across LayerD metrics and Q-Insight’s three dimensions: semantic consistency, visual fidelity, and editability.The comparison uses the Crello test set; Q-Insight assesses perceptual quality.Source: Table 1 · p. 6, §4.3 · p. 7
4.2. Metrics
- Evaluation covers layer quality, alpha-mask alignment, and reconstruction; RGBA outputs are composited over fixed neutral gray for RGB-only metrics.Layer-level and reconstruction metrics include PSNR, SSIM, and FID; mask alignment uses IoU and F1.Source: §4.2 · p. 6
4.4. Qualitative Result
- Bounding-box guidance yields more detailed, controllable layer structures than LayerD, while generative decomposition produces clearer, more coherent layers than matting and segmentation methods in complex scenes.The qualitative comparison highlights semantic and structural alignment across layers, including under heavy occlusion.Source: Figure 4 · p. 7, §4.4 · pp. 7–8
4.3. Quantitative Result
- Despite training only on PrismLayersPro while LayerD uses Crello training data, CLD performs better on the Crello test set.The authors describe this as evidence of generalization across datasets.Source: §4.3 · p. 7
4.5. Ablation Study
- The ART-adapted decoder improves metrics and foreground boundaries, while retaining the image in CFG’s unconditional branch improves results and prevents background leakage.The ablations also find that predicting the composite image improves layer- and mask-level metrics, with the largest gains in reconstruction.Source: Table 2 · p. 6, §4.5 · p. 8
4.6. Application
- The separated layers support removing, copying, and adding layers, and changing layouts directly in PowerPoint.The paper presents these operations as practical layer-based editing in common office software.Source: Figure 7 · p. 9, §4.6 · p. 9
5. Conclusion
CLD separates raster images into fine-grained layers using bounding boxes for user-guided separation. Experiments report gains over existing methods, and the layers can be used in downstream editing.
5. Conclusion
- CLD separates raster images into fine-grained layers using bounding boxes for precise, user-guided separation.The paper presents this as addressing a key limitation in post-production.Source: §5 · p. 9
- Experiments show CLD outperforms existing methods in decomposition quality, controllability, and visual fidelity.Source: §5 · p. 9
- CLD's generated layers are directly usable for downstream editing, including removal and recomposition.The paper presents this as a practical solution for creative design and future multi-layer extensions.Source: §5 · p. 9
6. Further Explanation of the Baseline
The paper uses LayerD as its primary baseline because existing approaches do not match its user-specified bounding-box setting. CLD is reported to outperform LayerD in quantitative metrics and visual quality.
6. Further Explanation of the Baseline
- Existing matting- and VLM-based approaches cannot handle user-specified bounding-box constraints, while DiT-related works focus on text-to-image generation.The paper therefore says no existing work fully matches its task setting.Source: §6 · p. 12
- LayerD's matting-base variant performs strongest among its matting-, YOLO-, and VLM-base designs.LayerD also compares combinations of matting and inpainting models; the paper treats it as the current state of the art among traditional matting/segmentation/VLM-based approaches.Source: §6 · p. 12
- CLD significantly outperforms LayerD in quantitative metrics and visual quality.The paper identifies LayerD as its primary baseline and attributes CLD's advantage to stronger representational capacity and better disentanglement under complex scenarios.Source: §6 · p. 12
7. Q-Insight Additional Details
The paper uses Q-Insight for automated assessment of generated images, complementing traditional metrics with semantic alignment, visual fidelity, and editability. Its evaluation prompt favors finer-grained decompositions when layers remain semantically complete.
7. Q-Insight Additional Details
- Q-Insight assesses generated images across semantic consistency, visual fidelity, and editability.The assessment is intended to complement traditional metrics.Source: §7 · p. 12
- The editability criterion prefers finer-grained decompositions when the individual layers remain complete semantic parts.The prompt says not to penalize over-decomposition when the resulting layers are logical and useful for an editor.Source: §7 · p. 12
- The visual-fidelity rating targets decomposition artifacts rather than the image's inherent artistic style.Examples include halos, incorrect background inpainting, color bleeding, and missing pixels.Source: §7 · p. 12
8. Additional Dataset Specifications
The training data and benchmark use PrismLayersPro, a 20K-sample dataset with 21 visual style categories and complete layer information. The paper splits each style category independently into training, test, and validation partitions.
8. Additional Dataset Specifications
- PrismLayersPro contains 20K samples across 21 visual style categories, with complete layer information and textual annotations.Source: §8 · p. 13
- Each sample includes a global textual description, composite image, RGBA layers, layer descriptions, and foreground-layer bounding boxes.The background layer has no provided bounding box and defaults to the entire image.Source: §8 · p. 13
- Each style category is split independently: 0–90% for training, 90–95% for testing, and 95–100% for validation.The paper describes the partitions as balanced and consistent.Source: §8 · p. 13
9. RGBA-to-RGB Conversion Method
The conversion fills transparent regions with neutral gray before computing RGB-based metrics, using a fixed solid background to avoid color bias and reduce evaluation error.
9. RGBA-to-RGB Conversion Method
- The RGBA-to-RGB process separates the RGB and alpha channels, then composites them over a fixed solid background.Neutral gray at (0.5, 0.5, 0.5) in the normalized [0, 1] range is the default.Source: §9 · p. 13
- Neutral gray is chosen to avoid adding color bias while minimizing evaluation error from filling transparent regions.The passage describes it as a balanced fill that reduces systematic color shifts and worst-case error.Source: §9 · p. 13
- In opaque regions, the placeholder has negligible influence, supporting visually comparable and consistent RGB-based metric evaluation.The text says gray avoids artificial color or brightness shifts in transparent areas and reduces metric bias.Source: §9 · p. 13
10. Additional Results
Additional examples from the curated PrismLayersPro test split illustrate the decomposition pipeline across varied scenes and appearances.
10. Additional Results
- The additional layer-decomposition examples are drawn from the test split of the curated PrismLayersPro dataset.The passage identifies Figures 10 and 11 as the example set.Source: §10 · p. 13
- The examples span diverse scenes and appearance variations.The passage presents them as evidence of pipeline consistency across that variation.Source: §10 · p. 13
- The examples further illustrate the effectiveness and consistency of the decomposition pipeline.This claim is scoped to the additional test-split samples shown in Figures 10 and 11.Source: §10 · p. 13
11. In-The-Wild Case
In-the-wild examples from Crello show the method separating complex design elements beyond the controlled training distribution.
11. In-The-Wild Case
- The in-the-wild examples come from the Crello test split, which contains professionally designed layouts with varied styles.The passage identifies Figures 12 and 13 as these examples.Source: §11 · p. 13
- The model separates icons, text segments, and composite shapes in these examples, demonstrating generalization beyond the controlled training distribution.The passage characterizes the results as handling complex visual elements.Source: §11 · p. 13
- The examples support the framework's robustness on real-world design assets encountered in practical editing scenarios.The scope is the in-the-wild Crello examples described in the passage.Source: §11 · p. 13
12. Failure Case
The reported failures arise with fine, intricate content and complex hierarchical occlusions, where resolution or insufficient visual evidence limits reconstruction.
12. Failure Case
- Very small, thin, or intricate content, including fine text, can exceed the model's ability to capture local cues at limited spatial resolution.Figure 14 shows representative failure cases.Source: §12 · p. 13, Figure 14 · p. 19
- Heavy hierarchical occlusion can leave insufficient image evidence for accurate reconstruction, and auxiliary text descriptions may not recover hidden content.The passage identifies more advanced text cues or explicit structural priors as possible directions for addressing this case.Source: §12 · p. 13
- The authors suggest higher input resolution or multi-scale features for fine details, and stronger text cues or structural priors for occlusions.These are proposed mitigations, not reported fixes.Source: §12 · p. 13