PaperlayerView exact revision on arXiv

Source-linked AI summary

DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation

Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma

arXiv:2610.02188v1cs.CVcs.AI

TL;DR

Few-step distillation can reduce the many sequential denoising steps required for diffusion generation, while score-based distribution matching carries the cost of auxiliary score fitting. DMAD learns distribution matching through classification with teacher and real-data targets, and reports leading results across image and video benchmarks as well as human preference for its joint audio-video student.

  • Problem

    Diffusion generation requires many sequential denoising steps, and distribution matching methods such as DMD require auxiliary score estimation for the evolving student distribution.

    Source: §1 · pp. 1–2

  • Method

    DMAD trains two discriminator heads to distinguish student samples from teacher samples and real data, using linear logit losses to recover the distribution-matching gradient without auxiliary score fitting.

    Source: §1 · pp. 2–3, §3.2 · p. 5

  • Results

    DMAD reports the best values among compared few-step methods and multi-step teachers across image and video benchmarks, including 1.04 FID with one-step generation on ImageNet-64×64.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The results support discriminative learning as an effective approach to distribution-matching distillation across image, video, and joint audio-video generation.

    Source: §5 · p. 10

    Distilled models may inherit biases and inappropriate generation behaviors from their teachers and training data, so safeguards are encouraged against impersonation, misinformation, and non-consensual content.

    Source: Ethics statement · p. 11

Abstract

from arXiv · show

Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across noise levels from the real-data head's empirical logit gap between real and teacher samples. DMAD reaches a Fr\'echet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64x64, 14.47 with four-step SDXL on COCO-10K, and a VBench total score of 85.15 with four-step Wan2.1-T2V-14B, the best values among the compared few-step methods and the multi-step teachers. On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio-video generation, excluding ties. Our code, models and demos are available at https://yzmblog.github.io/projects/DMAD.

1 Introduction

DMAD replaces DMD’s separate score estimation with classification-based distribution matching, using teacher samples and real data as complementary targets. It also adapts teacher supervision across noise levels and reports strong results across image, video, and audio-video generation.

1 Introduction

  • DMD estimates target and evolving-student scores separately, and critic lag can destabilize training; DMD2 uses five critic updates per student update and still evaluates the teacher online.Online teacher evaluation adds memory and computation cost.Source: §1 · pp. 1–2
  • DMAD uses two discriminator heads and linear logit losses to match teacher and real-data targets, recovering DMD’s distribution-matching gradient at the discriminator optimum.It eliminates auxiliary score fitting and online teacher evaluation, using one discriminator update per student update; teacher samples can be generated once and cached.Source: §1 · pp. 2–3
  • On two nearby Gaussian mixtures with a fixed student, the logit-gradient estimate follows the ground-truth field more closely than DMD’s, with normalized mean squared error 0.144 versus 1.539.Figure 2 compares the two gradient estimates against the known ground-truth field.Source: §1 · p. 3, Figure 2 · p. 3
  • The teacher and real-data targets play complementary roles: removing the real-data head raises four-step COCO-10K FID from 14.47 to 21.37, while removing teacher supervision lowers VBench scores.Without teacher supervision, VBench total falls from 84.70 to 82.20 and semantic score from 79.55 to 75.36.Source: §1 · p. 3
  • Gap-based reweighting uses the real-data head’s logit gap between real and teacher samples in each noise band, assigning larger teacher weights to bands with lower gaps.The gap varies across noise levels and depends on the current student and discriminator; the allocation improves image and video generation.Source: §1 · p. 3
  • DMAD reports the best values among compared few-step methods and multi-step teachers, including 1.04 FID with one-step generation on ImageNet-64×64.It also reports 14.47 FID with four-step SDXL on COCO-10K and human preference rates of 79.1% over DMD2 and 84.6% over rCM for the four-step MiniMax-H3 student.Source: §1 · p. 3

2 Related Work

Prior work distills along teacher trajectories or matches output distributions using score-based objectives, while adversarial methods often supplement those objectives. DMAD instead uses discriminator logits directly for distribution matching.

Trajectory and consistency methods

  • Trajectory and consistency methods constrain students to follow a teacher or common trajectory, whereas distribution matching constrains the distribution of student outputs.Examples include progressive step reduction, consistency and flow-map methods, and video extensions.Source: §2 · p. 4

Score-based distribution matching

  • Score-based distribution-matching methods use score estimation, including DMD’s target-minus-student score difference; FSF-DMD removes the auxiliary fake-score network but derives a surrogate from the generator flow map.Other approaches vary the objective or distribute supervision across generation steps or noise intervals.Source: §2 · p. 4

Adversarial distillation

  • Unlike adversarial methods that supplement score-based objectives, DMAD uses discriminator logits to provide the distribution-matching gradient and matches teacher and real samples with separate heads.The balance between the two targets is adapted across noise levels.Source: §2 · p. 4

3 Method

DMAD replaces DMD’s separate score estimation with classification-based matching: discriminator logits estimate the needed score differences and linear losses train the student. It matches both teacher samples and real data, using the real-data head’s logit gap to adapt teacher supervision across noise levels.

3.1 Reformulating Distribution Matching Distillation

  • At each noise level, balanced binary classification estimates a target-to-student log-density ratio whose optimal logit yields the DMD distribution-matching generator gradient.The result assumes positive differentiable densities on common support and sufficient regularity; the analysis uses the same fixed noise and condition distribution for target and student samples.Source: §3.1 · pp. 4–5

3.2 Distillation with Teacher and Real Targets

  • A shared backbone’s teacher and real-data heads each distinguish their target from student samples, and linear logit losses combine both distribution-matching gradients.Ablations report that real supervision improves image FID, while teacher supervision preserves detail and text alignment.Source: §3.2 · pp. 5–6

3.3 Gap-Based Reweighting

  • The real-data head’s mean-logit gap between real and teacher samples varies by noise level and training state; lower-gap bands receive larger teacher weights.The gap is measured by noise band with an exponential moving average, and the weights are centered at the median gap.Source: §3.3 · p. 6
  • Per-sample teacher weights are normalized and held fixed during generator backpropagation, so they scale each loss term without adding a gradient.The weights are derived from each sample’s noise band.Source: §3.3 · p. 6

4 Experiments

DMAD is evaluated on class-conditional images, text-to-image synthesis, and video generation, with additional ablations of its discriminator heads, generator loss, and gap-based reweighting. Its reported results span one- and four-step generation across these tasks.

4.1 Experimental Setup

  • Evaluation covers class-conditional ImageNet images, zero-shot COCO text-to-image synthesis, and four-step Wan2.1 and MiniMax-H3 video generation.Figure 4 compares Wan2.1-T2V-1.3B outputs across methods using identical initial noise and prompts.Source: §4.1 · p. 6, Figure 4 · p. 7

4.2 Main Results

  • 1.04 FID: DMAD’s one-step ImageNet-64×64 model with a projected discriminator surpasses D2O-F and SiD2.The comparison is against recent baselines and EDM teacher samplers.Source: §4.2 · p. 7, Table 1 · p. 8
  • 14.47 FID and 19.88 patch FID: DMAD’s four-step SDXL model outperforms DMD2, SenseFlow, and the SDXL teacher on both metrics.On one-step COCO-10K generation, it achieves 16.11 FID and a CLIP score of 0.338.Source: §4.2 · p. 7, Table 2 · p. 8
  • 85.15 VBench total: four-step Wan2.1-T2V-14B surpasses rCM and the 50-step teacher; DMAD’s 1.3B model scores 84.70.DMAD has the highest total and quality scores among compared methods, while rCM scores higher on semantics.Source: §4.2 · p. 8, Table 3 · p. 9

4.3 Ablation Studies

  • Removing the real head causes the largest FID increases on ImageNet and SDXL; removing the teacher slows ImageNet FID reduction but lowers VBench semantic score by 4.19 points.The ablation also disables gap-based reweighting when the real head is removed.Source: §4.3 · p. 9, Table 4 · p. 10
  • Disabling gap reweighting worsens ImageNet FID and all SDXL metrics and lowers VBench total by 1.14 points; adaptive teacher weights lower FID at matched iterations.The real-data coefficient stays fixed while teacher weights adapt across noise levels.Source: Table 4 · p. 10, Figure 7 · p. 10, §4.3 · p. 10
  • DMAD first records FID below 1.5 at 209K iterations, versus 354K for the non-saturating variant; the linear loss also improves image FID and VBench total.The comparison examines replacing the linear generator loss with the non-saturating GAN loss.Source: §4.3 · p. 9, Figure 7 · p. 10, Table 4 · p. 10

5 Conclusion

DMAD concludes that its discriminative approach improves on the compared few-step methods and multi-step teachers in image FID and video VBench, with favorable MiniMax-H3 user preferences. The paper also states its responsible-use concerns and reproducibility plans.

5 Conclusion

  • DMAD reports improvements over the compared few-step methods and multi-step teachers in image FID and VBench total score, alongside user preference over DMD2 and rCM on MiniMax-H3.The conclusion says users particularly preferred the student in visual and audio quality.Source: §5 · p. 10

AI use statement

  • Large language models assisted with writing polish, literature retrieval and discovery, and drafting portions of the manuscript; the authors take responsibility for its final content.This responsibility includes claims, citations, and AI-assisted text.Source: AI use statement · p. 11

Ethics statement

  • Faster image and video generation can lower the cost of producing misleading or harmful synthetic media, and distilled models may inherit biases and inappropriate behaviors.The authors encourage safeguards against impersonation, misinformation, and non-consensual content.Source: Ethics statement · p. 11

Reproducibility statement

  • The authors will release code, dataset preparation and preprocessing scripts, and experiment configurations to support independent reproduction and further research.The statement directs readers to Sections 3 and 4 and Appendices A and B for method, derivation, setup, and implementation details.Source: Reproducibility statement · p. 11

A Proof of Proposition 1

The proof connects balanced binary classification to log-density ratios, then uses those ratios to recover the distribution-matching generator gradient under stated assumptions. It also specifies limits on the equivalence and extensions.

A Proof of Proposition 1

  • At the balanced classification optimum, the discriminator logit is the target-to-student log-density ratio, and its unique minimizer yields the input gradient used in the proof.The derivation assumes positive differentiable densities on common support and an unrestricted, unregularized discriminator with equal positive and negative class weights.Source: A Proof of Proposition 1 · p. 14
  • The equivalence is a local equality of generator gradients with an optimal, fixed critic, not equality of loss values or differentiation through the critic’s dependence on student parameters.The distribution-matching gradient derivation also assumes sufficient regularity and support independent of the student parameters.Source: A Proof of Proposition 1 · pp. 14–15
  • The gradient equivalence extends to nonnegative noise-level weights independent of student parameters, but not to DMD’s sample-dependent score normalization or differentiated gap-weight evolution.The gap-based weights are treated as fixed during each generator update.Source: A Proof of Proposition 1 · p. 15
  • A non-increasing generator loss can reweight each sample’s ideal distribution-matching gradient, potentially changing the aggregate update direction relative to the linear loss.The linear loss has a unit multiplier, while the non-saturating loss reduces contributions from samples the discriminator already accepts.Source: A Proof of Proposition 1 · p. 15
  • With two fixed targets and nonnegative head coefficients, the weighted KL objective corresponds at each noise level to a geometric mixture of teacher and real-data marginals.This identity does not make a clean-space geometric mixture the global optimum; forward noising need not commute with geometric mixing.Source: A Proof of Proposition 1 · p. 15

B Implementation Details

Implementation settings differ across image, video, and joint audio-video generation, including training scale, evaluation protocols, discriminator architectures, and noise-band reweighting.

B Implementation Details

  • ImageNet-64×64 uses 8 H100 GPUs, batch size 384, and 570K iterations; SDXL uses 8 H100 GPUs, batch size 64, and 17K iterations.Their generator and discriminator learning rates are respectively 2 × 10^-6 and 5 × 10^-7; evaluation follows DMD2 protocols for ImageNet and SDXL.Source: B Implementation Details · p. 16
  • Wan2.1-T2V-14B training uses 64 H100 GPUs, while MiniMax-H3 uses 8 H200 GPUs and rank-128 LoRA; all reported models use four sampling steps for MiniMax-H3 evaluation.The MiniMax-H3 user study compares outputs on 387 prompts, using 774 pairs evaluated by three annotators across five dimensions.Source: B Implementation Details · p. 16
  • ImageNet and SDXL use U-Net-based discriminator heads, Wan uses two MLP heads, and MiniMax-H3 uses two MLP heads after non-affine RMS normalization.Noise-band reweighting uses six bands for ImageNet, 100-timestep bands for SDXL, and ten bands for each video setting.Source: B Implementation Details · p. 16

B.1 Training Speed and Memory

The training-cost comparison measures online update time and peak GPU memory for DMD2 and DMAD under matched batch sizes and input shapes.

B.1 Training Speed and Memory

  • Table 5 compares DMD2 and DMAD online training cost on NVIDIA H200 GPUs with matched batch sizes and input shapes within each setting.ImageNet and Wan-14B use full-parameter training; MiniMax-H3 uses rank-128 LoRA.Source: §B.1 · p. 16
  • The cost comparison uses DMD2’s official 5:1 critic-to-generator update ratio and DMAD’s 1:1 ratio.Reported time includes one generator update and its associated critic updates.Source: §B.1 · p. 16
  • Update times are CUDA-synchronized after warm-up, and peak allocated GPU memory is recorded; evaluation and checkpointing are excluded.Source: §B.1 · p. 16

B.2 Toy Gradient Estimation

This toy experiment compares estimates of a correction field for two-dimensional Gaussian-mixture distributions. It evaluates them with a fixed student distribution and analytic reference quantities.

B.2 Toy Gradient Estimation

  • The experiment uses two-dimensional Gaussian mixtures with known parameters, treating p as real data and q as a fixed student distribution.The analytic scores and density ratios serve only for evaluation; neither estimator receives them as training supervision.Source: §B.2 · pp. 16–17
  • Both estimates are evaluated on the same 2,048 independent held-out samples drawn from q0.5.The experiment uses seed 0 and σ = 0.5.Source: §B.2 · p. 18
  • The two compared estimates yield NMSE values of 1.539462 and 0.144375 in the fixed-student experiment.The experiment isolates single-target gradient estimation; target samples for teacher pretraining and discriminator training are drawn directly from p.Source: §B.2 · p. 18

C Additional Audio-Video Evaluation

The additional evaluations report MiniMax-H3 audio-video benchmark scores, SDXL qualitative comparisons, and MiniMax-H3 teaser prompts. The comparisons include matched prompts and initial noise for distilled models.

C Additional Audio-Video Evaluation

  • DMAD scores 65.4 on MiniMax-H3 AVGen-Bench, versus 62.6 for DMD2 and 59.8 for rCM; the teacher scores 67.2.Scores are normalized to 0–100, and the reported total is the benchmark’s aggregate score.Source: C Additional Audio-Video Evaluation · p. 18

D.1 SDXL Comparisons

  • In SDXL comparisons, DMAD is described as producing more natural colors and textures and better text alignment than DMD2 and SenseFlow.Examples include matching an instruction to exclude cats and generating exactly four people in a family portrait; distilled models use four steps, versus 50 for the teacher.Source: §D.1 · p. 18, Figure 8 · p. 19

D.2 MiniMax-H3 Teaser Prompts

  • MiniMax-H3 teaser prompts range from scripted dialogue and Ukiyo-e fantasy to fashion scenes and surreal object interactions.Examples include a forest-spirit encounter, a flamingo dress in a lagoon, and a squirrel emerging from a photograph.Source: §D.2 · pp. 18–20

D.3 Wan2.1-T2V-14B Samples

Figure 9 presents generated Wan2.1-T2V-14B video samples, with prompts listed for the upper-grid examples. The samples include both representative frames and chronological frame sequences.

D.3 Wan2.1-T2V-14B Samples

  • Figure 9 shows representative frames from nine generated videos and three chronological frames from each of six other videos.The full prompts are provided in Section D.3.Source: Figure 9 · p. 21
  • Listed prompts include a zen garden inside a glass sphere, a color-changing chameleon, and a drone view of the Amalfi Coast.These are among the prompts for the nine samples in Figure 9’s upper grid.Source: §D.3 · p. 21
  • Other listed prompts include a polar bear playing violin and a watercolor rabbit picking mushrooms.Source: §D.3 · p. 22

D.4 ImageNet-64×64 Samples

The appendix shows one-step ImageNet-64×64 samples and additional MiniMax-H3 video comparisons. The video figures align methods by prompt, initial noise, and sampled frame indices.

D.4 ImageNet-64×64 Samples

  • The ImageNet figures each show a 32 × 32 grid of generated images at native 64 × 64 resolution.Source: §D.4 · p. 23
  • FID is 1.24 for Figure 10 and 1.04 for Figure 11 on one-step ImageNet-64×64 samples.Each figure contains a 32 × 32 grid at native 64 × 64 resolution.Source: Figure 10 · p. 23, §D.4 · p. 23, Figure 11 · p. 24

D.5 More results on MiniMax-H3

  • MiniMax-H3 comparisons place Teacher, DMD2, and Ours in successive rows and use the same four frame indices, initial noise, and text prompts across methods.The displayed frame indices are 1, 42, 83, and 124.Source: Figure 12 · p. 25
Loading 2610.02188v1…