PaperlayerView exact revision on arXiv

Source-linked AI summary

Editing Images into Reusable Layers via Compositional Edit-Prior Distillation

Wen Xue, Yong Liu, Xueheng Li, Si Wu, Yong Xu, Yipeng Sun, Yan Li, Simiu Gu

arXiv:2610.24738v1cs.CV

TL;DR

Flattened editing outputs limit fine-grained manipulation and layer reuse, while separate editing and decomposition stages may yield poorly organized layers. LayerEditor jointly generates an edited canvas and reusable RGBA layers, using distillation to preserve editing capability. Experiments report strong recomposition fidelity, layer separation, and control over layer counts and content.

  • Problem

    Flattened editing outputs limit fine-grained manipulation and layer reuse, while separately optimized editing and decomposition can produce incomplete or poorly organized layers.

    Source: §1 · p. 1

  • Method

    LayerEditor jointly represents an edited canvas and ordered RGBA layers, with C-EPD transferring a pretrained editor’s prior to the canvas and composed layer predictions.

    Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 5

  • Results

    LayerEditor reports strong recomposition fidelity, accurate layer separation, and flexible control over layer counts and content.

    Source: §4.2 · p. 7, §4.3 · p. 8, §5 · p. 10

  • Takeaways & Limitations

    Independently editable predicted layers support fine-grained revisions and content reuse beyond the initial edit.

    Source: §5 · p. 10

    LayerEditor is trained and evaluated with at most ten RGBA layers, and thin structures, complex transparency, and semitransparent boundaries can still artifact during movement or recomposition.

    Source: E LIMITATIONS · p. 22

Abstract

from arXiv · show

High-quality image editing requires precise modification of target regions while preserving the content of unrelated regions. Existing editing models produce flattened outputs that limit precise local manipulation and layer reuse, while decomposition methods typically optimize layer recovery separately from instruction-driven editing. To this end, we introduce LayerEditor, a unified framework that couples instruction-guided image editing with controllable generation of reusable RGBA layers. At its core, In-Context RGB-Alpha Token Modeling jointly represents the edited canvas and an ordered sequence of RGB-Alpha layers within a single token sequence, enabling semantic edits and explicit control over individual layer content. To retain the pretrained model's editing capability during adaptation to layered outputs, we propose Compositional Edit-Prior Distillation, which transfers its editing prior to both the edited canvas and the image composed from the predicted layers. To support training and evaluation, we construct LayerEditor-100K, a large-scale paired image-editing dataset with ordered RGBA layer annotations, and introduce dedicated benchmarks for layered editing. Extensive experiments demonstrate that LayerEditor maintains strong editing fidelity while generating accurate RGBA layers, supporting both flexible layer-level manipulation and multi-layer decomposition.

1 INTRODUCTION

Flattened editing outputs constrain fine-grained manipulation and layer reuse, while separately optimized editing and decomposition can leave layers incomplete or poorly organized. LayerEditor jointly edits images and generates reusable RGBA layers, alongside paired training data and dedicated evaluation benchmarks.

1 INTRODUCTION

  • Flattened edited canvases make fine-grained manipulation and user-provided elements difficult, while repeated revisions risk changing unrelated content and creating boundary artifacts.The introduction motivates reusable layers for downstream uses such as e-commerce content production and advertising design.Source: §1 · p. 1
  • Separately optimized editing and decomposition can produce incomplete or poorly organized layers that require repeated model calls and manual refinement.Editing models prioritize visual fidelity without targeting layer structure, whereas decomposition recovers layers after editing without jointly optimizing the requested edit and structure.Source: §1 · p. 1
  • LayerEditor couples instruction-driven editing with controllable RGBA-layer generation using In-Context RGB–Alpha Token Modeling and on-policy C-EPD.The model represents the edited canvas and ordered layers in one token sequence; C-EPD supervises both the direct canvas and the composition of predicted layers to preserve editing capability during adaptation.Source: §1 · p. 2
  • LayerEditor-100K pairs source images, instructions, edited targets, and ordered RGBA layers across text, object, and appearance edits with variable layer counts.The dataset provides paired supervision for editing outcomes and their underlying layer structures.Source: §1 · p. 2
  • Crello-Testset and ScaleEdit-Testset evaluate editing fidelity, layer quality, and layered controllability through layer-count and user-specified-content settings.The benchmarks address the stated lack of adequate evaluation for instruction-driven layered editing.Source: §1 · p. 2

2 RELATED WORK

Prior work advances instruction-driven editing, layered generation, decomposition, and on-policy distillation, but the paper identifies a gap in jointly editing a flattened source and producing reusable layers. LayerEditor positions its approach against these separate lines of work.

2.1 CONDITIONAL IMAGE EDITING

2.2 LAYERED IMAGE GENERATION

  • Layered generation and design methods create coordinated or editable structures, while decomposition methods such as Qwen-Image-Layered (Yin et al., 2025) recover components from rendered images; these tasks are typically separate from instruction-driven editing.LayerEditor addresses this gap by jointly editing a flattened source image and generating its corresponding reusable RGBA layers.Source: §2.2 · p. 3

2.3 ON-POLICY DISTILLATION

  • On-policy distillation supervises a student on states from its own trajectories, and prior flow-based work explores this strategy for generative models; LayerEditor instead targets preserving editing capability when extending single-image outputs to layered outputs.The passage contrasts this goal with progressive distillation and consistency models, which mainly reduce sampling steps.Source: §2.3 · p. 3

3 METHOD

LayerEditor jointly generates an edited canvas and an ordered, controllable stack of reusable RGBA layers. Its method combines joint RGB–Alpha modeling, teacher distillation, flow-matching training, and paired layered-editing data with dedicated benchmarks.

3 METHOD

  • LayerEditor takes a source image, editing instruction, and requested layer count to jointly generate an edited canvas and an ordered stack of reusable RGBA layers.The stack is ordered back to front, and instructions may specify content for individual layers.Source: §3 · p. 3

3.1 IN-CONTEXT RGB–ALPHA TOKEN MODELING

  • In-Context RGB–Alpha Token Modeling jointly denoises the edited canvas and layer fields, using layer-aware coordinates and a dedicated Alpha decoder for continuous opacity.The slot coordinate distinguishes fields and stacking order; the Alpha decoder specializes in fine structures and semitransparent boundaries.Source: §3.1 · pp. 3–4

3.2 COMPOSITIONAL EDIT-PRIOR DISTILLATION

  • C-EPD uses the frozen pretrained editor as an on-policy teacher to guide both the Direct output and the decoded composition of predicted layers.The composed supervision propagates through student-side decoding and alpha compositing to the layer prediction branch.Source: §3.2 · pp. 4–5

3.3 TRAINING OBJECTIVES

  • Training first applies flow matching to the complete joint velocity sequence, then adds C-EPD guidance while retaining flow matching, with λC controlling guidance strength.The joint noisy sequence is conditioned on the source image, instruction, requested layer count, and noise level.Source: §3.3 · p. 5

3.4 LAYEREDITOR-100K AND EVALUATION BENCHMARKS

  • LayerEditor-100K pairs source images, instructions, edited targets, and ordered RGBA layers; disjoint Crello-Testset and ScaleEdit-Testset evaluate editing and layer controllability.Targets contain 2–10 layers; the evaluation settings include layer-count control and held-out image-to-layer decomposition.Source: §3.4 · pp. 5–6

4 EXPERIMENTS

Experiments evaluate recomposed editing fidelity, layer usability and controllability, image-to-layer decomposition, and the effects of the proposed training components.

4.1 EXPERIMENT SETUPS

  • Evaluation compares editing and decomposition baselines under default or official settings and measures recomposed-image similarity, layer agreement, and five rubric dimensions.LayerEditor is initialized from FireRed-Image-Edit-1.1, adapted with rank-256 LoRA, and jointly optimized with C-EPD after two epochs of flow-matching training.Source: §4.1 · pp. 6–7

4.2 EDITING FIDELITY AND LAYER REUSABILITY

  • LayerEditor achieves the best Composite SSIM/FID and Alpha IoU on both benchmarks; its Crello-Testset rubric mean is 6.00 versus 4.50 for the highest baseline mean.Figure 4 compares outputs rendered from predicted RGBA stacks; competing methods show missing or duplicated content, mixed components, and transparency errors, while LayerEditor produces coherent recompositions and reusable layers.Source: §4.2 · pp. 7–8, Figure 4 · p. 7

4.3 LAYER-STRUCTURE AND CONTENT CONTROLLABILITY

  • LayerEditor exceeds Seedream5-Pro (Joint) by 0.0493 in Alpha IoU and 0.97 points in semantic grouping in the layer-count control setting.Prompts also specify layer content; examples assign separate text rows and birds to the requested reusable layers.Source: §4.3 · p. 8, Table 2 · p. 8

4.4 LAYER FIDELITY AND DECOMPOSITION CAPABILITY

  • The image-to-layer decomposition evaluation uses 1,171 held-out Crello images with 2–10 reference layers, without I2L-specific fine-tuning or reference layer descriptions.Figure 6 covers image-to-layer decomposition on the Crello test set.Source: §4.4 · p. 9, Figure 6 · p. 9

4.5 ABLATION STUDY

  • Removing C-EPD lowers edit consistency from 6.14 to 4.67 and layer segmentation from 6.83 to 5.01; combining both guidance terms improves all four reported metrics.The dedicated Alpha decoder reduces RGBA L1 from 0.0578 to 0.0350 and raises layer segmentation from 5.67 to 6.83 versus averaging the pretrained RGB decoder outputs.Source: §4.5 · p. 9, Table 3 · p. 9, Figure 7 · p. 9

5 CONCLUSIONS

LayerEditor combines instruction-driven editing with controllable reusable RGBA layers, using joint token modeling, an Alpha decoder, and C-EPD. Experiments show strong recomposition fidelity, layer separation, and control over layer counts and content.

5 CONCLUSIONS

  • In-Context RGB–Alpha Token Modeling captures cross-layer dependencies and appearance–transparency interactions, while a dedicated Alpha decoder refines opacity for fine structures.C-EPD transfers the pretrained editing prior to both the edited canvas and composed layer predictions to help preserve editing fidelity during layered adaptation.Source: §5 · pp. 9–10
  • LayerEditor generates independently editable layers alongside the edited canvas, supporting fine-grained revisions and content reuse beyond the initial edit.The work also constructs LayerEditor-100K and establishes dedicated benchmarks for layered image editing.Source: §5 · p. 10
  • The reported experiments demonstrate strong recomposition fidelity, accurate layer separation, and flexible control over layer counts and content.LayerEditor-100K provides paired training supervision for editing outcomes and underlying layer structures.Source: §5 · p. 10

SUPPLEMENTARY MATERIAL

The supplementary material covers implementation, dataset construction, evaluation protocols, and additional experimental results. LayerEditor takes a source image, requested layer count, and editing instruction as inputs.

SUPPLEMENTARY MATERIAL

  • LayerEditor’s inputs are a source image, requested layer count, and textual editing instruction, optionally including desired content for individual layers.Textual conditions are model inputs; RGBA layers, Alpha masks, and related annotations are used only as training supervision or evaluation references.Source: SUPPLEMENTARY MATERIAL · p. 13
  • Layer-content specifications are optional additions to the textual editing instruction.These conditions are inputs to the model.Source: SUPPLEMENTARY MATERIAL · p. 13
  • RGBA layers, Alpha masks, and related annotations serve as supervision or evaluation references rather than model inputs.Source: SUPPLEMENTARY MATERIAL · p. 13

A.1 TRAINING AND INFERENCE CONFIGURATION

LayerEditor adapts a pretrained editor with LoRA and flow matching, then trains a dedicated Alpha decoder to recover opacity for layered outputs. At inference, one denoising trajectory produces the edited canvas and ordered RGB–Alpha fields.

A.1 TRAINING AND INFERENCE CONFIGURATION

  • One inference trajectory conditioned on the source, instruction, and requested layer count produces the edited canvas and ordered RGB–Alpha fields, without teacher calls.The model uses rank-256 LoRA on the DiT backbone; image and text encoders and the pretrained RGB decoder remain frozen.Source: §A.1 · p. 13
  • Backbone adaptation begins with two epochs of flow matching, followed by joint flow-matching and C-EPD optimization; the Alpha decoder is trained separately and then frozen.Both decoders remain frozen during backbone adaptation while retaining input gradients for compositional guidance.Source: §A.1 · p. 13, §A.2 · p. 14

A.2 ALPHA DECODER

  • The dedicated Alpha decoder uses a single-channel sigmoid head to recover continuous opacity, while the Base Model and w/o Alpha Decoder variants average the pretrained RGB decoder’s three channels.The Alpha decoder retains the VAE topology and initializes its head with zero weights and bias −3.5.Source: §A.2 · pp. 13–14
  • Alpha-decoder training combines regional Alpha errors with support, boundary, area, multiscale, and black/white-rendering terms, plus stage-specific composition, perceptual, and adversarial supervision.The auxiliary terms are enabled only in specified training stages.Source: §A.2 · p. 14

A.3 C-EPD TRAINING

C-EPD distills the frozen editor’s prior at states visited by the student, guiding both its Direct canvas and the decoded composition of its predicted layers. Local clean estimates are decoded and composited differentiably at selected trajectory states.

A.3 C-EPD TRAINING

  • The rollout samples 10 nodes from the 30-step inference sigma grid, then samples two query states from high-, medium-, and low-noise bands with probabilities 0.2, 0.5, and 0.3.Rollout states are detached; student velocities are recomputed with gradients, while the teacher prediction is frozen and gradient-free.Source: §A.3 · p. 14
  • At selected student rollout states, C-EPD combines Direct velocity guidance from a frozen teacher with pixel-space guidance on the decoded layer composition.Flow matching remains active on every update; C-EPD is enabled every five post-warm-up updates with a ramped weight.Source: §A.3 · p. 14

A.4 LOCAL DECODING AND PIXEL-SPACE COMPOSITION

  • At each selected query, predicted velocities yield approximate clean layer latents that frozen decoders turn into RGB and continuous Alpha fields for white-background source-over composition.The Direct canvas is not part of the layer stack, and composition uses decoded RGB rather than latent velocities.Source: §A.4 · p. 15
  • The teacher target is a decoded local clean prediction, and differentiable composition lets pixel loss guide RGB and Alpha through their visible effects while flow matching supervises fully occluded content.Composition is not re-encoded, and decoding is needed only at selected queries rather than every rollout step.Source: §A.4 · p. 15

B.1 DATA CONSTRUCTION PIPELINE

LayerEditor-100K pairs edited images with ordered RGBA stacks through native-design and flattened-pair construction routes. Shared annotation checks and disjoint splits support training and layered-editing evaluation.

B.1 DATA CONSTRUCTION PIPELINE

  • LayerEditor-100K combines native layered designs and flattened editing pairs; each accepted record contains a source, instruction, edited target, and ordered full-canvas RGBA stack.The two construction routes use different sources and annotation procedures, with the flattened-pair route screening, repairing, and verifying decomposition candidates.Source: §B.1 · p. 16

B.2 LAYER ANNOTATION AND QUALITY CONTROL

  • Both routes undergo structure, semantic, layer-quality, reconstruction, and deduplication checks, including inspection of isolated RGBA components and Alpha boundaries.Annotations include each layer’s index, role, content, edit status, and bounding box.Source: §B.2 · p. 17, Figure 9 · p. 17

B.3 DATASET STATISTICS AND SPLIT

  • LayerEditor-100K contains 100,000 accepted pairs with 2–10 target layers, and documents and their derived variants remain within a single split.The data includes 48,329 text edits, 36,858 object edits, and 14,813 poster-level appearance edits.Source: §B.3 · p. 17

B.3 DATASET STATISTICS AND SPLIT / C.1 BASELINES

  • Crello-Testset has 2,994 documents and ScaleEdit-Testset has 1,000; controllability settings pool 3,994 source documents, averaging variants within each source before aggregation.The baseline suite includes cascaded editing-plus-decomposition systems, decomposition models, and joint or adapted layered-generation systems.Source: §B.3 · p. 17, §C.1 · p. 18

C.2 METRICS AND LAYER MATCHING

On the 3,994-document instruction-control setting, LayerEditor leads the reported image, layer, and rubric comparisons. The paper also specifies how layer matching, rendered composites, and rubric scores are evaluated.

C.2 METRICS AND LAYER MATCHING

  • Hungarian assignment matches nonempty predicted and reference layers by Alpha-support IoU, then ordered residual and padded pairs contribute to RGBA L1.RGBA L1 uses a fixed reference-count denominator and stored RGB and Alpha values, so visually equivalent transparent pixels can differ in score.Source: §C.2 · pp. 18–19

C.3 RUBRIC-BASED EVALUATION

  • A single Gemini-3.1-Pro evaluator scores five dimensions using recomputed Composite outputs; selected examples illustrate score anchors but do not validate agreement with human judgments.The mean across the five dimensions is an unweighted descriptive summary, not an additional evaluator output.Source: §C.3 · p. 20, Figure 10 · p. 21

D.1 INSTRUCTION CONTROLLABILITY

  • 0.9018/0.8765 Direct/Composite SSIM, 0.5823 Alpha IoU, and 0.0344 RGBA L1 are LayerEditor’s reported instruction-control results; its rubric scores also lead across dimensions.Its edit-consistency score is 5.54 versus 3.32 for Seedream5-Pro (Joint), and its composition score is 5.68 versus 4.34 for FR + QIL-Decomp-FT.Source: §D.1 · pp. 20, 22, Table 6 · p. 22

D.2 ADDITIONAL QUALITATIVE RESULTS

Qualitative examples show LayerEditor following requested edits while separating content into reusable RGBA layers, with recomposed outputs retaining the edits.

D.2 ADDITIONAL QUALITATIVE RESULTS

  • Examples show requested appearance and text changes alongside separately accessible text, objects, and backgrounds in reusable RGBA layers.The examples include pop-art restyling, text replacement, and background editing; prompts specify layer count and, where applicable, desired content separation.Source: §D.2 · p. 22

E LIMITATIONS

LayerEditor’s evaluated scope is bounded by layer count, artifact risks, and limits in reconstruction and reuse validation; responsible deployment also requires attention to deceptive editing and copyright.

E LIMITATIONS

  • Thin structures and complex transparency can produce artifacts during layer movement or recomposition; text remains rasterized, limiting typography manipulation.Training and evaluation use at most ten RGBA layers, leaving larger stacks unexplored; results do not establish lossless reconstruction or artifact-free reuse.Source: E LIMITATIONS · p. 22
  • Evaluation measures layer isolation, opacity, and recomposition in tested layouts, but does not systematically test post-manipulation reconstruction.Editing-prior retention is tested on constructed benchmarks, not an independent general-editing test; annotations and model-based assessments may contain errors or biases.Source: E LIMITATIONS · p. 22
  • Rubric examples illustrate discrimination among output quality levels, not agreement with human ratings.Source: Figure 11 · p. 22

F ETHICS STATEMENT

  • Layered outputs make subsequent visual manipulation easier, and the paper identifies deceptive alteration and unauthorized modification of copyrighted assets as misuse risks.The authors recommend considering provenance, copyright, and safeguards against deceptive editing in deployment.Source: F ETHICS STATEMENT · p. 23
  • The dataset uses existing research datasets and design resources, preserves source provenance, and does not intentionally collect additional private user information.Semantic verification and quality assessment use automated vision-language models alongside deterministic checks and structured quality control.Source: F ETHICS STATEMENT · p. 23

Related papers

Loading 2610.24738v1…