Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

• Diffusion models typically require dozens of network evaluations, while adversarial distillation enables one- and few-step synthesis; ADD [49] uses DINOv2 [36] to distill SDXL [38] into a single-step model.Source: §1 · p. 2
• LADD uses pretrained diffusion-model generative features and targeted noise-level sampling to shift discriminator feedback between global and local features.High noise levels bias features toward global behavior, while low noise levels bias them toward local behavior.Source: §1 · p. 2
• Unlike ADD [49], LADD distills in latent space, avoids RGB decoding, enables megapixel training, and supports large student and teacher networks.ADD’s fixed pretrained DINOv2 discriminator restricts training resolution to 518 × 518 pixels and impedes high-resolution training above 5122 pixels.Source: §1 · p. 2
• LADD offers stable, scalable adversarial distillation amid adversarial models’ scaling and tuning difficulties, and the paper systematically studies its scaling behavior.Source: §1 · p. 2
• In four sampling steps, SD3-Turbo matches Stable Diffusion 3’s teacher image quality as a high-resolution, multi-aspect text-to-image generator.Source: §1 · p. 2
• The paper reports a significantly simplified formulation that outperforms ADD [49] and demonstrates LADD for image editing and inpainting.Source: §1 · p. 2

2.1 Diffusion Models

• The paper focuses on rectified flow, where the denoiser predicts clean images from noisy inputs, and states its method applies to other diffusion formalisms.The denoiser can be trained via score matching, and samples are generated by numerically solving a differential equation backward from t=1 to t=0 [57] [26].Source: §2.1 · p. 4

2.2 Diffusion Distillation

• Distillation seeks straighter trajectories that preserve the clean-image distribution, allowing larger steps and fewer denoiser evaluations.Diffusion denoisers must otherwise be evaluated at sufficiently small step sizes to solve the highly nonlinear differential equation.Source: §2.2 · p. 4
• ADD [49] uses a pretrained feature extractor as discriminator and achieves performance on par with SDXL [38] in four steps.Source: §2.2 · p. 4
• Progressive Distillation halves Euler steps iteratively but accumulates error; Reflow trains iteratively on older models’ synthetic data, also accumulating error.Consistency Distillation avoids iterative application but is unstable and requires distillation schedules and extensive hyperparameter tuning; improved techniques for consistency and progressive distillation are also noted [56] [33] [15] [68] [3].Source: §2.2 · p. 4

Distillation in latent space

• LADD trains purely in latent space, using teacher features for adversarial loss and avoiding the pixel-space decoding required by ADD.This reduces memory demands while supporting high-resolution, multi-aspect image synthesis.Source: §3 · p. 4, Figure 3 · p. 5

Unifying teacher and discriminator

• LADD renoises generated latents at logit-normal timesteps following, then feeds teacher token sequences from each attention block to independent discriminator heads.Each head is conditioned on noise level and pooled CLIP embeddings.Source: §3 · p. 5
• Generative features avoid image decoding, provide noise-level-specific feedback, and naturally support multi-aspect ratios through features from a teacher trained on such data.Discriminative pretraining in latent space is non-trivial and top-performing approaches require significant engineering [8] [36].Source: §3 · p. 5
• Generative models’ shape bias resembles human perception, unlike discriminative models’ texture bias, motivating generative features for adversarial training [22].The cited work reports near-human-level accuracy on out-of-distribution tasks [22].Source: §3 · pp. 5–6
• The discriminator reshapes token sequences to spatial layouts and uses 2D convolutions, avoiding varying token strides across aspect ratios that could weaken a 1D discriminator.Source: §3 · p. 6

Leveraging synthetic data

• Because one-shot CFG oversaturates samples rather than improving text alignment, LADD trains on teacher-generated data at constant CFG, avoiding ADD’s auxiliary distillation loss.Teacher prompts are sampled from SD3’s original training dataset.Source: §3 · p. 6

4.1 Teacher noise distribution

• The teacher noise distribution π(t; m = 1, s = 1) is selected: biasing toward low noise harms global coherence, while excessive high-noise bias harms texture and fine detail.Higher-noise bias improves coherence, but very high noise (m = 2, s = 2) loses fine details.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2 Synthetic data

• Training with synthetic data significantly outperforms real data, and adding distillation loss benefits real-data training but not synthetic-data training.Thus, synthetic-data training can use only an adversarial loss.Source: §4.2 · p. 6, Figure 5 · p. 7

4.3 Latent distillation approaches

• LADD outperforms LCM by a large margin in single-step distillation, despite one LADD run versus extensive LCM hyperparameter search.LCM training was more volatile across hyperparameters, seeds, and iterations. The authors note that LCM may improve with broader tuning or larger models, while LADD distills small and large models without extensive tuning.Source: §4.3 · p. 7, Figure 6 · p. 8

4.4 Scaling Behavior

• Student model size affects performance more than teacher size or data quality, whose benefits plateau; prioritizing larger students can preserve performance with smaller teachers.The study varies student, teacher, and data-generator size one at a time while holding the other two at depth 24.Source: §4.4 · pp. 7–8

4.5 Direct preference optimization.

• 56% was the human-preference win rate for the single-step LADD student with re-applied DPO-LoRA matrices against the initial non-DPO LADD student.The teacher was finetuned with rank-256 LoRA for 3k iterations, and the resulting DPO-finetuned model was used for student, teacher, and data generation; DPO was more impactful in the multi-step setting.Source: §4.5 · p. 8

5.1 Text-to-Image Synthesis

• In single-step user preferences, SD3-Turbo beats all baselines for image quality and prompt alignment, with four steps improving results further.The preference study compares image quality and prompt alignment.Source: Figure 9 · p. 9, §5.1 · p. 9
• At four steps, SD3-Turbo matches SD3 image quality (50 steps), slightly trails its prompt alignment, yet beats Midjourney v6.The four-step model is also on par with or outperforms all evaluated text-to-image systems.Source: §5.1 · p. 9, Figure 10 · p. 10

5.2 Image-to-Image Synthesis

• For instruction-based editing, SD3-edit Turbo matches its teacher in one step and improves style edits and object swaps, but loses control over separate image and text guidance strengths.LADD is applied after task-specific diffusion training; editing uses input-image concatenation and paired edit data, including synthetic InstrucPix2Pix and additional control-task data.Source: §5.2 · pp. 10–11, Figure 12 · p. 11
• For inpainting, SD3-inpainting Turbo performs on par with its teacher in one step, while LaMa leads on LPIPS but lags on FID and large non-homogeneous masks.Training conditions on masked inputs using varied mask strategies and adds a distillation loss to improve text alignment.Source: §5.2 · p. 11, Figure 13 · p. 12, Figure 14 · p. 12

6 Limitations

• Four-step SD3 Turbo maintains teacher image quality at the expense of prompt alignment, with entity merging, spatial-prompting errors, and overlooked negations that vary across random seeds.These challenges are not unique to the model and reflect a trade-off among model capacity, prompt alignment, and inference speed.Source: §6 · p. 12, Figure 15 · p. 13
• Image editing lacks adjustable image and text guidance strengths available in comparative methods [7]; training with adjusted strengths and conditioning on them, as proposed in [33], is a potential solution.The passages identify model conditioning on the guidance parameters as part of the proposed approach.Source: §6 · pp. 12–13
• In some cases, the model adheres too closely to the input, making large changes challenging.Source: §6 · p. 13

Appendix A Details on Human Preference Assessment

• The evaluation uses 128 prompts, selected every fourth prompt outside PartiPrompts’ Basic category, with four samples generated per prompt and model.The Basic category was omitted because it predominantly contains single-word prompts.Source: Appendix A · p. 17
• Each 1v1 comparison presents two models’ samples for the same prompt and requires evaluators, who cannot join multiple studies, to choose one.Each prompt is shown four times on average per task when comparing two models.Source: Appendix A · p. 17
• Evaluators judge either which image better represents and follows the prompt or which is higher quality and more aesthetically pleasing.The prompt appears above both images in each task.Source: Appendix A · p. 17

A.1 List of Prompts used for Human Evaluation

• Prompts include animals and objects in unusual or precisely specified compositions, such as a bat landing on a baseball bat and a bookshelf with ten vertically stacked books.Other examples specify written text, spatial relationships, or absent objects, including a bench without cats and a harp without strings.Source: §A.1 · p. 17
• Food and drink prompts range from cocktails and Beef Pho to imaginative scenes such as a tortilla-chip castle in a salsa river with walking burritos.The list also includes detailed coffee-cup latte-art descriptions and a set of dog emoji icons.Source: §A.1 · p. 18
• Graphic and illustration prompts include emoji dogs in blue turtlenecks, geometric shapes, hearts, lettering on a brick wall, and an owl wearing a graduation cap.Examples also include a dinosaur flag and a children’s book cover showing a white dog driving a red pickup truck.Source: §A.1 · p. 18
• Scene prompts depict buildings, landscapes, and activities, from a goat in a small kitchen and a cruise ship at a beach to a grandmother reading to children.Other scenes include a donkey lecturing in a hall, sauropods by a lakeside, and a tennis court with three yellow cones.Source: §A.1 · p. 18
• People, plants, vehicles, and landmarks appear in prompts ranging from a father and son playing tennis and flowers with cat faces to a blue Porsche and Sydney Opera House beside the Eiffel Tower.The list also specifies detailed views of a blue 1977 Ford F-150 on a mountain road and a subway train with frogs reading or seated inside.Source: §A.1 · p. 19