ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation

1. Introduction

• ART addresses the limitation of single-layer text-to-image models by targeting independently editable multi-layer transparent compositions.Layer-by-layer control is important for graphic design and digital art.Source: §1 · p. 1
• Anonymous regions specify locations without region-wise prompts, letting the model infer layer semantics from the global prompt and reducing labor for layouts with many regions.The model associates regional visual tokens with relevant global text tokens; adding region prompts can diminish cross-layer coherence.Source: §1 · pp. 1–2, Figure 1 · p. 1
• Anonymous layouts improve coherence because regional and global visual tokens attend to the same global text tokens rather than balancing incompatible regional and global conditioning.The layout also scales better than recaptioning many crops, whose captions can be noisy.Source: §1 · p. 2

2. Related work

• ART differs from prior simultaneous or sequential methods by generating tens of transparent layers jointly, whereas earlier systems generally support only limited layer counts.Prior examples include Text2Layer [55], LayerDiff [20], LayerDiffuse [54], COLE [25], and OpenCOLE [24].Source: §1 · p. 2, §2 · p. 3

3.1. Multi-Layer Transparent Image Autoencoder

• The framework jointly encodes and decodes a variable number of transparent layers at varying resolutions, including the composed reference image and RGBA foreground layers.The autoencoder concatenates latent tokens for the reference, background, and foreground layers and decodes them in one forward pass.Source: §3 · p. 3, Figure 4 · p. 4, §3.1 · p. 4
• The autoencoder uses gray-background preprocessing and alpha-channel information to support transparency decoding, while tight crops produce variable-length foreground token sequences.The gray background was found sufficient for accurate transparency decoding.Source: §3.1 · pp. 3–4

3.2. Anonymous Region Transformer

• ART applies layout-conditioned multi-layer 3D RoPE to noisy latents and simultaneously generates a reference image, background, and all foreground layers.The design encodes relative width, height, and layer positions while using MMDiT to process text and image tokens.Source: §3.2 · pp. 4–5

3.3. Anonymous Region Layout Planner

• The layout planner predicts anonymous bounding boxes from text using a fine-tuned LLaMA-3.1-8B, and the MLTD dataset supplies approximately 1 million variable-resolution transparent designs.The dataset averages 11 layers per design, with 99.9% containing fewer than 50 layers; the paper reports that 8K high-quality examples can suffice.Source: §3.3 · p. 5, §3.4 · p. 5

4.1. System-level Comparisons

• ART significantly outperforms LayerDiffuse across harmonization, aesthetics, and prompt following, and surpasses COLE on multiple aspects except typography.The comparisons use 30 participants evaluating 50 image pairs per method; COLE retains an advantage in typography because its text is rendered with a typography renderer.Source: §4.1 · pp. 6–7
• Anonymous region layout produces comparable FIDmerged but significantly higher PSNR than semantic layout, indicating stronger layer coherence and global harmonization.The paper attributes lower semantic-layout coherence to conflicts between local region prompts and global visual tokens; over 80% of inferred anonymous-region labels align with human annotations.Source: §4.2 · p. 7, Table 2 · p. 7

4.2. Ablation Study and Analysis

• Regional Full Attention achieves superior FIDmerged while keeping computational costs nearly constant from 10 to 50 layers, unlike Full Attention’s quadratic growth.Regional cropping removes most transparent pixels and constrains predictions to the anonymous regions; comparisons also include Spatial + Temporal Attention.Source: §4.2 · pp. 7–8, Table 4 · p. 7
• 3D-RoPE outperforms absolute layer-position encoding for latent features and achieves the best FIDmerged among tested decoder position-embedding schemes.The decoder analysis finds that simply adding layer-wise absolute embeddings provides minimal improvement, whereas replacing absolute encoding with RoPE substantially improves decoding quality.Source: Table 5 · p. 7, §4.2 · p. 8
• Larger training datasets improve performance, although ART already achieves strong results with 8K samples, demonstrating data efficiency.Performance improves with dataset scale, while only 20 curated high-quality natural images suffice for strong multi-layer natural-image results after fine-tuning.Source: Table 6 · p. 7, §4.2 · p. 8
• ART performs better as the number of transparent layers increases but slightly worse with longer captions, which the paper attributes to training-set distributions.The model also applies directly to multi-layer natural image generation without architectural modifications, and reference composed-image prediction improves FIDmerged at a slight inference-time cost.Source: §4.2 · pp. 7–8, Table 7 · p. 7, Table 8 · p. 7

• ART supports generating a variable number of transparent layers at variable resolution and enables flexible image editing through layer compositing.Source: §5 · p. 9
• The method offers better layer coherence, more scalable annotation, and efficient generation across distinct anonymous region layouts.Source: §5 · p. 9
• ART remains limited by repeated layer generation, combined layer generation, and uncertain generalizability across all potential layouts.Source: §5 · p. 9
• Future work targets autonomous semantic-label identification and improved visual quality and flexibility for generated images.Source: §5 · p. 9

• Tables 2 to 4 document the global prompts and anonymous layouts used in Figures 5 and 6.Source: §1 · p. 12
• The first two rows of Table 4 use global prompts selected from DESIGNINTENTION for fair comparisons.Source: §1 · p. 12
• Tables 5 and 6 provide the instructions for user studies on PHOTO-MULTI-LAYER-BENCH and DESIGN-MULTI-LAYER-BENCH.Source: §1 · p. 12

• Figure 1 marks inconsistent regions and visualizes attention between regional visual tokens, region-caption tokens, and global reference-image tokens.Source: §2 · p. 12
• Regional visual tokens primarily attend to region-wise prompts and rely less on the predicted reference image, resulting in less coherent outputs.Source: §2 · p. 12
• The conflict arises from a disparity between region-wise prompts and the global prompt associated with the same regional crop.Source: §2 · p. 12

3. Analyzing the Inferred Label Assignments within Anonymous Region Layouts

• CLIP scores measure whether generated transparent layers in anonymous regions match human-annotated region-wise prompts.Source: §3 · p. 12
• Figure 2 plots aligned-layer percentages across CLIP thresholds using a test set of 5,000 multi-layer transparent images.Source: Figure 2 · p. 12, §3 · p. 13
• The observed alignment between inferred labels and human annotations is attributed to Schema Theory.Source: §3 · p. 13

4. Qualitative Multi-Layer Transparent Image Generation Results with around 50 Layers

• ART generates high-quality transparent images with 40, 45, and 51 layers from ultra-dense anonymous region layouts.Source: §4 · p. 13

• PyTorch implementations define the layout-conditional multi-layer 3D RoPE and its use within the Attention Module.These implementations are presented in Algorithm 1 and Algorithm 2.Source: §5 · p. 13

• ART adapts semantic concept assignments across various anonymous region layouts given a fixed global prompt.Qualitative results are illustrated in Tables 7 to 15.Source: §6 · p. 13

• ART regenerates editable layers to match prompt changes while keeping other layers unchanged and visually harmonious.The editing pipeline modifies the prompt, regenerates targeted layers, and freezes the remaining layers.Source: §7 · p. 13

• The encoding transforms 4-channel RGBA images into 3-channel RGB images while embedding alpha information for pretrained RGB generation models.Its overall goal is to facilitate reuse of three-channel image generation models.Source: §8 · p. 13
• RGB and alpha channels are linearly normalized from [0, 255] to [−1, 1], with alpha −1 representing full transparency and 1 representing full opacity.The RGB normalization follows standard practice in Flux.1 models.Source: §8 · p. 13
• The alpha-dependent transformation maps transparency from [−1, 1] to [0, 1] before encoding it into the RGB channels.This transformation is applied to embed transparency information into the RGB representation.Source: §8 · p. 13
• Fully opaque pixels retain their RGB values, fully transparent pixels become pure gray (RGB = (0, 0, 0) in [−1, 1]), and semi-transparent pixels change proportionally with alpha.The transformation preserves opaque colors while encoding intermediate transparency levels proportionally.Source: §8 · p. 13

9. Evaluation in text generation

• Replacing CNNs with ViTs and using a multi-layer autoencoder both improve reconstruction performance on MLTD data.The autoencoder evaluation reconstructs images by encoding and decoding the same ground-truth multi-layer image; Table 1 reports the ablation scope.Source: §9 · p. 13, Table 1 · p. 13
• The ViT-based autoencoder improves visual text modeling by predicting more accurate edges than CNN-based autoencoders.LayerDiffuse and Flux-RGBA are identified as CNN-based comparisons.Source: §9 · p. 13
• Jointly encoding and decoding multiple layers enables explicit cross-layer interactions and outperforms single-layer methods.The comparison concerns the multi-layer autoencoder architecture versus single-layer methods.Source: §9 · p. 13
• The generated examples include transparent images with 40, 45, and 51 layers, each conditioned on an anonymous region layout and a global prompt.The figures show merged images, transparent layers, anonymous region layouts, and global prompts.Source: §9 · p. 14, Figure 3 · p. 18, Figure 4 · p. 19
• The method supports layer-wise editing of generated images.The evaluation includes a figure specifically showing layer-wise editing.Source: Figure 6 · p. 33