LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes

Data-efficient supervision

• The recipe decouples visual-prior learning from early language alignment, reducing dependence on expensive paired captions while preserving condition–target compatibility through same-region conditioning.Informative high-resolution crops are resized only mildly for 256^2 training, and the overall data requirement remains moderate.Source: §1 · p. 7

Training pipeline

• The 6B Diffusion Transformer learns its visual prior from image-only pre-training and mid-training before progressively adding language alignment, refinement, and joint generation–editing training.The pipeline processes approximately 220M generation-training samples, with more than 90% image-only training and real images comprising more than 70% of supervised fine-tuning.Source: §1 · p. 7

Unified architecture

• A frozen dLLM-based vision-language module and single-stream DiT unify multimodal understanding, text-to-image generation, and reference-preserving editing in one checkpoint.The design combines text, visual context, and reasoning traces with SigLIP-VQ features and clean VAE latents for editing.Source: §1 · p. 7

Optimization and deployment

• Parameter-free RMSNorm and the Muon optimizer stabilize the progressive training pipeline, while TwinFlow distills the original multi-step model into variants requiring only 2–4 sampling steps.TwinFlow builds on distribution-matching distillation and self-adversarial flow training.Source: §1 · p. 7

Open release

• The authors release model weights, training and inference code, and detailed recipes covering data construction, progressive training, unified generation and editing, and distillation.The release includes LLaDA-Image and LLaDA-Image Turbo checkpoints and FP8 variants.Source: §1 · p. 8

System scope and results

• LLaDA-Image provides full multi-step and 2–4-step Turbo deployment profiles, with broad visual quality, prompt alignment, text rendering, and editing evaluated across four benchmarks.The family includes Base and Turbo checkpoints plus FP8 variants, alongside released training and inference code and progressive recipes.Source: §1 · p. 8

2 Model Design

• The architecture combines a dLLM-based VLM, an understanding-to-generation connector, and a DiT, decoupling semantic interpretation from pixel-space synthesis while supporting generation and editing.The connector bridges the VLM and DiT, while editing additionally uses a direct reference-image pathway.Source: §2 · pp. 8–9

2.1 dLLM-based Vision-Language Model

• The VLM uses the LLaDA 2.0 Mini dLLM backbone with SigLIP-VQ, jointly reasons over multimodal contexts, and excludes editing references from comprehension by injecting them directly into the DiT.Text-to-image uses text only, image-only pre-training uses prompts with visual tokens, and editing processes only the instruction in the VLM.Source: §2.1 · p. 9

2.2 Understanding-to-Generation Connector

• The connector pairs a Residual Query Adapter that elicits generation-relevant information from the frozen VLM with Transformer blocks that project its features into DiT conditioning space.The RQA preserves the VLM’s pretrained understanding while the Transformer Connector aligns feature dimensions and geometry for DiT consumption.Source: §2.2 · p. 9

2.3 Single-stream Diffusion Transformer

• The pure single-stream DiT jointly processes image and condition tokens, enabling semantic conditions to interact with evolving visual tokens through self-attention at every layer.Replacing DiT normalization with parameter-free RMSNorm improves optimization stability over long training horizons.Source: §2.3 · p. 10
• Reference-image editing combines a SigLIP-VQ semantic branch with a clean FLUX.2 VAE latent, strengthening instruction following and reference identity, structure, texture, and background consistency without a separate editing backbone.The semantic pathway supplies high-level reference guidance, while the VAE latent preserves low-level appearance in regions that remain unchanged.Source: §2.3 · p. 10

3 Data Collection and Preparation

• The corpus combines web and curated images, using image-only data for pre-training and mid-training and paired image–text data for SFT.Synthetic images are a smaller part of SFT data and are used mainly for text rendering.Source: §3 · p. 10

3.1 Data Composition

• 98% of 220M generation-training samples are real images, with image-only data exceeding 90% and real paired data remaining above 70%.Pre-training and mid-training use only real images, while SFT combines real and synthetic content organized into nature, design, people, and synthetic groups.Source: §3.1 · p. 10, Figure 4 · p. 11

3.2 Data Processing

• Images pass metadata, aesthetics, and quality filters before square-crop pre-training or aspect-ratio-bucket processing for mid-training and SFT.Filtering requires more than 1024^2 total pixels, at least 0.15 bytes per pixel, ArtiMuse scores of at least 60, and DeQA-Score values of at least 4.0.Source: §3.2 · p. 11
• Captioning models describe visible content and text, after which captions and editing pairs are retained only when they pass agreement, quality, and consistency checks.The pipeline removes hallucinated or inaccurate captions, malformed or privacy-sensitive outputs, watermarks, and edit instructions containing unsupported details.Source: §3.2 · p. 11

• CoT supervised fine-tuning is performed first to enhance the backbone’s performance, following evidence that comprehension can facilitate generation.The overall training and release path is summarized in Fig. 2.Source: §4 · p. 11
• Understanding and generation are organized as independent training pipelines, summarized in Tables 1 and 2.Table 1 provides the detailed configuration for understanding training and defines CoT SFT as CoT Supervised Fine-tuning.Source: §4 · p. 12, Table 1 · p. 12
• Muon is used throughout all generation-training stages.The optimizer is attributed to Liu et al. (2025a).Source: §4 · p. 12

Data Process and Construction

• Fixed-length packing with block-diagonal attention improves utilization without cross-sample leakage, while generation, understanding, and text-only data are mixed at 9:9:2.Corpora are greedily packed into sequences of exactly L = 16,384 tokens, and images are VQ-encoded once at 512^2 target resolution under variable-aspect-ratio cropping.Source: §4.1 · p. 12

Training Objective

• Complementary masking supervises every response token across two optimizer steps while each pass reads a partially clean context.The system prompt, question, and visual tokens remain unmasked and unsupervised; the loss covers masked <think> traces and answers, with per-block normalization.Source: §4.1 · p. 12

Optimization

• AdamW training uses FSDP2 full sharding, mixed precision, gradient checkpointing, linear warmup, and cosine decay, with the schedule doubled for complementary passes.The complementary pass takes its own optimizer step, so the learning-rate schedule spans twice the number of data iterations.Source: §4.1 · p. 12

Image-only Pre-training

• Unlabeled images provide sufficient supervision to learn a strong visual generative prior without large-scale image–text pairs.A frozen VLM extracts image semantics as a self-conditioning signal while remaining intact during this stage.Source: §4.2 · pp. 12–13
• Resolution-aware sampling randomly crops 256^2 or moderately larger native regions before resizing, preserving local detail and preventing caption-alignment distortions from aggressive downsampling.The VLM derives the condition from the processed crop, keeping it faithful to the training target.Source: §4.2 · p. 13
• Masked self-conditioning converts dense reconstruction into sparse-to-dense prediction, requiring the generator to recover missing content from visible image patches.The method masks image tokens while pairing image features with a fixed auxiliary prompt, encouraging contextual and compositional visual structure.Source: §4.2 · p. 13
• Image-only training reuses the downstream RQA–VLM–connector pathway to condition the DiT, changing only multimodal input construction.This preserves the common architecture while applying masked image self-conditioning during pre-training.Source: §4.2 · p. 13
• Flow matching trains the visual generator, residual query adapter, and connector while keeping the vision encoder and VLM frozen, providing initialization for later alignment.The interpolation uses Gaussian noise and uniformly sampled t; masked reconstruction and parameter-efficient self-conditioning absorb visual knowledge from inexpensive image-only data.Source: §4.2 · p. 13

Image-only Mid-training

• The pixel budget increases from 256^2 to 512^2 while retaining image-derived self-conditioning, image-token masking, and the flow-matching objective.This exposes the DiT to finer spatial structure without introducing caption mismatch.Source: §4.3 · p. 14
• Image-only mid-training separates the resolution increase from the later shift to textual supervision, providing a smoother transition while preserving the visual prior.Directly switching to high-resolution image–text training would simultaneously change spatial distribution and supervision source, while captions may lose fine details after downsampling.Source: §4.3 · p. 14
• Aspect-ratio buckets assign each image to a closely matching shape at approximately the same pixel budget, preserving aspect ratio with only minimal cropping.The strategy reduces geometric distortion and content loss while exposing the model to diverse spatial shapes for generation.Source: §4.3 · pp. 14–15
• Nearly constant per-image pixel budgets keep memory and computation closely matched across data-parallel ranks, avoiding rank-specific peaks and stragglers.Different ranks may process different spatial shapes, but the controlled pixel budget limits their resource disparity.Source: §4.3 · pp. 14–15

4.4 Supervised Fine-tuning

• Logit-normal timestep sampling with Pmean = 0.8 and Pstd = 0.8 shifts supervision toward high-noise states, improving early-denoising stability.The strategy retains support across the full denoising trajectory while avoiding repeated oversampling of comparatively well-resolved low-noise states.Source: §4.4 · p. 15

4.4.1 Text-to-Image Training

• A real-image-dominant SFT recipe slows early convergence but substantially improves final realism and raises the model’s overall capability ceiling.Real images consistently comprise more than 70% of the training mixture, with refinement emphasizing text-intensive and portrait-centric examples.Source: §4.4.1 · pp. 15–16

4.4.2 Image Editing Training

• A 1:1 mixture of text-to-image and image-to-image examples gives one checkpoint editing ability while preserving open-ended generation quality and the earlier visual prior.Reference conditioning combines semantic reference tokens with a VAE latent of the clean reference image, supporting instruction following and preservation of unchanged content.Source: §4.4.2 · p. 16

4.4.3 Checkpoint Merging

• Merging nearby converged checkpoints suppresses step-dependent fluctuations and produces a more stable, robust final model across evaluation benchmarks.The method averages parameters from multiple training states to reduce sensitivity to local sampling noise near performance plateaus.Source: §4.4.3 · p. 16

4.5.1 TwinFlow for Few-step Distillation

• TwinFlow distills the multi-step LLaDA-Image into LLaDA-Image Turbo by assigning generator updates to +t and fake-score updates to −t within one shared DiT.The signed time indicates learning roles rather than separate models, with +t ∈[0, 1] and −t ∈[−1, 0].Source: §4.5.1 · pp. 16–17
• DMD alternates positive-time generator optimization against the frozen real score with negative-time fake-score fitting to the student’s evolving distribution.The fake-score estimator uses stop-gradient samples, and its faster optimization time scale tracks the non-stationary student distribution.Source: §4.5.1 · pp. 16–17
• The shared model converts velocity predictions into real and fake scores, eliminating the need for a separate auxiliary score network.The frozen Fbase supplies sreal at +t, while Fθ supplies sfake at −t on the same noised sample.Source: §4.5.1 · p. 17
• Implementation attaches separate fake-score and DMD heads, retains only the DMD head at inference, uses four-step backward simulation, and applies a 1:2 update schedule.The 1:2 schedule reduces fake-score updates relative to DMD2’s 1:5 schedule; reinforcement tuning with TBSM follows distillation.Source: §4.5.1 · p. 17

5.1.1 General Image Generation Performance

• All LLaDA-Image Turbo benchmark results use four sampling steps, raw reported baseline scores, and standard inference without prompt enhancement, reasoning, or test-time rewriting.GenEval and DPG-Bench are supplementary because their rankings can diverge from human preferences and their scores are increasingly saturated.Source: §5.1.1 · pp. 17, 20
• 53.53 and 53.38 on Qwen-Image-Bench English and Chinese tracks exceed Z-Image Turbo by 1.87 and 0.67 points, respectively.LLaDA-Image ranks first among compared open-source models in Quality, Aesthetics, and Alignment in both languages, plus Creative Generation on English, without prompt enhancement or test-time thinking.Source: §5.1.1 · p. 18, Table 3 · p. 18, Table 4 · p. 19
• 0.923 and 0.913 on LongText-Bench English and Chinese indicate balanced bilingual long-text rendering, but scores trail dedicated text-rendering models.The benchmark is used with CVTG-2K because it emphasizes transcription correctness rather than overall visual quality.Source: §5.1.1 · pp. 18–19, Table 5 · p. 20
• 0.875 average word accuracy on CVTG-2K ranks second-highest, with NED 0.945 and CLIPScore 0.818.Word accuracy decreases from 0.892 for two text regions to 0.857 for five, remaining comparatively stable as complexity increases.Source: §5.1.1 · p. 19, Table 6 · p. 20
• 0.85 on GenEval includes 1.00 single-object generation, 0.98 two-object composition, and tied-highest Attribute Binding at 0.84, but Counting is 0.53.GenEval is reported as a historical fine-grained reference rather than definitive evidence of overall generation quality.Source: §5.1.1 · p. 21, Table 7 · p. 21
• 87.48 on DPG-Bench is supported by Entity 92.31, Attribute 92.37, Relation 92.55, and Other 91.96, placing LLaDA-Image ahead of several strong baselines.These results are included for continuity with prior work rather than treated as a definitive measure of practical generation capability.Source: §5.1.1 · p. 21, Table 8 · p. 22

5.2 Image Editing Generation

• 7.336 and 7.294 on GEdit-Bench-EN and CN show that one checkpoint supports bilingual image editing while retaining text-to-image capability.Semantic consistency exceeds perceptual quality, especially in English, at 8.043 versus 7.182; improving perceptual quality against specialized editors remains open.Source: §5.2 · p. 22, Table 9 · p. 23

Contributions

• LLaDA-Image combines a from-scratch 6B DiT, dLLM-based VLM conditioning, real-data-dominant training, and TwinFlow distillation, while releasing weights, code, and recipes.The recipe uses 220M generation samples, 98% real, with more than 90% supporting image-only training and over 70% real images in paired SFT.Source: §6 · p. 23
• LLaDA-Image achieves open-source state-of-the-art performance on both Qwen-Image-Bench tracks, with strong bilingual text rendering and editing, while Turbo reduces inference to 2–4 steps.Parameter-free RMSNorm and Muon stabilize training, and SigLIP-VQ with clean VAE latents preserves reference information.Source: §6 · p. 23

Future Directions

• Future work will improve grounding and typography, broaden knowledge coverage, extend training to native 2K, and explore agentic generation through closed-loop planning, evaluation, and editing.These directions address specialized and culturally specific concepts, factual-context-dependent prompts, and complex multi-region or long text layouts.Source: §6 · p. 23

Limitations

• Current limitations include insufficient broader world knowledge, less reliable long and multi-region text rendering, and native 2K generation remaining unevaluated beyond the nominal 10242-pixel budget.Rendering stability decreases as region count, requested content length, and layout complexity increase.Source: §6 · p. 23

A Aspect-Ratio Bucket Configurations

• Aspect-ratio buckets are retained across image-only mid-training and supervised fine-tuning at approximately 512^2- and 1024^2-pixel budgets, with configurations specified as (W, H) tuples.The budgets denote approximate image pixel counts rather than fixed square resolutions.Source: A Aspect-Ratio Bucket Configurations · p. 29, Table 10 · p. 30
• Each image is assigned to the bucket whose width-to-height ratio is closest to the image’s aspect ratio.For an image with width W and height H, the aspect ratio is a = W/H, and the loader selects among K resolution buckets.Source: A Aspect-Ratio Bucket Configurations · p. 29
• The selected bucket is reached by aspect-ratio-preserving resize-to-fill followed by cropping only the excess dimension to obtain the exact target shape.Because bucket and source aspect ratios are usually close, the crop typically removes only a few pixels and little content.Source: A Aspect-Ratio Bucket Configurations · p. 29
• Approximately equal-area buckets give distributed ranks similar image-token counts, reducing memory peaks and stragglers while avoiding square-induced distortion and aggressive cropping.The approach also teaches generation across a broad range of output aspect ratios.Source: A Aspect-Ratio Bucket Configurations · p. 29

B CoT Supervised Fine-tuning and Reinforcement Learning

• CoT supervised fine-tuning is the only stage that updates the understanding backbone, whose effect is verified separately before generation-stage training.Table 11 reports multimodal understanding results before and after CoT supervised fine-tuning; the CoT SFT scores were evaluated under the protocol in App. B.Source: B CoT Supervised Fine-tuning and… · p. 30, Table 11 · p. 31
• The evaluation covers 18 multimodal understanding benchmarks spanning general VQA, reasoning, OCR/document understanding, and other multimodal tasks.The benchmark suite includes MMStar, MMBench, HallusionBench, RealWorldQA, SimpleVQA, MMMU, MathVista, We-Math, MathVision, ChartQA, DocVQA, InfoVQA, OCRBench, AI2D, CountBenchQA, VL-RewardBench, and V∗.Source: B CoT Supervised Fine-tuning and… · pp. 30–31

Additional Qualitative Comparison

• The comparison includes historically detailed Chinese artifacts, including a Han-dynasty terracotta attendant figurine and a Ming cloisonné enamel vase.The prompts specify materials, ornamentation, colors, aging, and museum-style presentation.Source: C Additional Qualitative Comparison · p. 32

Additional Qualitative Comparison (Continued)

• It also covers a realistic glacier-view travel portrait that combines a casually posed visitor with extensive clothing, facial, environmental, and photographic details.The scene places the subject on a viewing platform before a cracked glacier, rocky mountains, and a clear sky.Source: C Additional Qualitative Comparison · p. 33
• Landscape examples span a Mediterranean mountain town, a Jiangnan water town, and a coastal cliff, with attention to architecture, terrain, water, atmosphere, and scale.The prompts vary from soft morning light and lived-in streets to misty canals and expansive sea cliffs.Source: C Additional Qualitative Comparison · pp. 33–34

Filtering

• Structural checks remove failed, incomplete, refusing, or repetitive captions, while Qwen3.6-35B-A3B evaluates image quality and caption faithfulness.The evaluator uses the image, caption, and visible-text transcription to identify hallucinated, inaccurate, or text-inaccurate captions.Source: D Image Captioning and Filtering · p. 35
• Filtering excludes samples with watermark signals or private information and rejects captions that misdescribe visual content or visible text.The unified checker evaluates privacy, watermarks, render-text usability, and caption faithfulness using explicit categorical criteria.Source: D Image Captioning and Filtering · pp. 35, 37

Captioning

• Captions describe visible subjects, scenes, composition, style, lighting, materials, and text without inventing unsupported content.Detailed prompts vary in length with image complexity and present visible text verbatim with position and visual properties.Source: D Image Captioning and Filtering · pp. 35–36
• Short prompts retain only the most important image elements in concise, direct, natural user-like wording, while omitting illegible or blurred text.Source: D Image Captioning and Filtering · p. 36
• Each annotation outputs category tags, a detailed prompt, a short prompt, and a render-text field containing exact visible-text transcriptions.Outputs are JSON-only, preserve original wording and language, and require quoted visible text inside detailed prompts.Source: D Image Captioning and Filtering · pp. 35–37

Prompts 1–3

• The first three prompts specify highly detailed realistic portraits, including candid park photography, a softly lit indoor portrait with “LLaDA-Image is Released!” text, and a lightly vintage Xiaohongshu cafe scene.They emphasize natural facial features, authentic skin and hair texture, clothing details, environmental context, lighting, and readable text where required.Source: E Prompts Used in The Report · p. 38

Prompts 4–6

• Prompts 4–6 cover documentary and travel-oriented scenes: a produce-market stallholder, three friends posing on a seaside boardwalk, and a couple celebrating a birthday in a restaurant.These prompts constrain subjects, props, signs, clothing, composition, lighting, and everyday atmosphere to produce authentic photographic scenes.Source: E Prompts Used in The Report · pp. 39–40

Prompts 7–9

• Prompts 7–9 present child-centered lifestyle photography in a library, home, and winter amusement park, with carefully specified expressions, clothing, settings, and photographic styles.The scenes range from quiet natural portraits and warm slice-of-life imagery to an aged family-album look with film cast, grain, dust, edge softness, and a date stamp.Source: E Prompts Used in The Report · pp. 40–41

Prompts 10–12

• Prompts 10–12 depict richly detailed natural landscapes, including snowmelt-fed travertine pools, a karst-canyon underground-river outlet, and a tranquil Jiangnan water town.They define geological formations, water colors and reflections, vegetation, architecture, atmospheric conditions, and spatial depth.Source: E Prompts Used in The Report · p. 42

Prompts 13–14

• Prompts 13–14 extend the landscape set to photorealistic Venice canals and a rain-cleared urban boulevard viewed from a pedestrian overpass.The Venice prompt emphasizes historic architecture, a gondola, reflections, and warm light, while the city prompt specifies wet infrastructure, traffic, pedestrians, buildings, and post-rain illumination.Source: E Prompts Used in The Report · p. 43

Urban and landscape photography

• The passages depict layered old-city rooftops, snowy Huangshan with mist-shrouded granite peaks, and a horizontally framed bridge, using restrained colors, realistic structures, and atmospheric depth.The bridge is shown in direct side view across the full frame, with evenly spaced piers, river reflections, natural restrained colors, and no fantastical appearance.Source: E Prompts Used in The Report · pp. 44–45

Wildlife photography

• The passages describe documentary wildlife photographs of a golden snub-nosed monkey in Qinling forests and a giant panda in a damp Sichuan bamboo forest, emphasizing lifelike fur and soft natural light.The monkey is perched on a branch, while the panda holds bamboo and eats on the forest floor.Source: E Prompts Used in The Report · p. 45

Art and cultural artifacts

• The passages depict Chinese ink painting and calligraphy, a David bust, a Qing Kangxi Songhua-stone inkstone, and a Ming-dynasty Gaoyou water-route album leaf.The works include clear inscriptions or seals, historically identified materials and periods, and detailed presentations of antique or crafted surfaces.Source: E Prompts Used in The Report · pp. 46–47

Food photography

• Prompts 25–27 present close or overhead food photography of tonkatsu, Korean barbecue, and Chaoshan beef hot pot, emphasizing crisp textures, steam, rendered fat, and appetizing detail.The compositions preserve recognizable accompaniments, utensils, tableware, restaurant settings, and natural depth of field.Source: E Prompts Used in The Report · p. 48

Poster-style wildlife photography

• Prompt 28 combines a high-speed wildlife photograph of a white goose surging through deep-blue water with a minimalist English and Chinese poster typographic layout.The prompt requires frozen splashes, an asymmetrical composition with blue negative space, and the exact text “Stop explaining yourself.” with specified subtitles.Source: E Prompts Used in The Report · p. 49

Cultural and tourism posters

• Prompts #29–30 specify a museum exhibit-label photograph and a vertical Forbidden City poster combining traditional Chinese gongbi architecture with contemporary graphic design, black ink-brush composition, and abundant negative space.The exhibit label gives dimensions of 45.2 cm mouth diameter, 28.5 cm foot diameter, and 8.3 cm height, with 20±2◦C temperature and 45%-55% humidity.Source: E Prompts Used in The Report · pp. 49–50

Typography and text constraints

• Across the prompts, typography is treated as a structural visual element, with exact placement, scale, alignment, language, and line-breaking requirements integrated into each composition.Examples include museum-label metadata, three-line poster titles, cutout landscape lettering, and strict instructions that only specified Chinese or English text may appear.Source: E Prompts Used in The Report · pp. 49, 50, 51, 52, 55

Editorial and conceptual posters

• The cited prompts present editorial poster concepts featuring a deep-blue nighttime street with a white vending machine, a cosmic composition centered on SAVE THE SUN, and a realistic horse in an office.Source: E Prompts Used in The Report · pp. 52–53

Chinese literary art poster

• Prompt #36 interprets Li Qingzhao’s “月满西楼” through a golden full moon, indigo pagoda, autumn architecture, aged xuan paper, and antique woodblock-print textures.The design requires the title “满西楼” on one horizontal line, a vertical poem excerpt, restrained vintage colors, and carefully preserved architectural detail.Source: E Prompts Used in The Report · pp. 54–55

Chinese prompt

• The Chinese prompt defines a nearly square cherry-tomato poster with a central vine cluster, four surrounding scientific illustrations, bilingual typography, nutrition text, and abundant negative space.It specifies aged paper, botanical watercolor rather than photography or cartoons, and tomato-red, orange-red, green, yellow, and ivory tones.Source: E Prompts Used in The Report · p. 55

English translation

• The English translation preserves the same poster concept, combining vintage field-guide, natural-history, and modern food-editorial aesthetics.It reiterates the central ripe cherry tomatoes, irregular supporting illustrations, Chinese and English titles, concise labels, and restrained text layout.Source: E Prompts Used in The Report · pp. 55–56