VDOT++: Unified Few-Step Video Generation via Unbalanced Optimal Transport Distillation

1 INTRODUCTION

• Balanced OTD can force unreliable student tokens to match and average incompatible teacher targets in T2V and single-frame I2V.Dense conditions can establish reliable within-frame correspondences, but diverse valid appearances and motions need not align across realizations.Source: §1 · p. 1
• As output ambiguity widens, asymmetric unbalanced transport and an ℓ1 cost better preserve valid motion and detail than balanced OTD.The relaxed student marginal downweights unreliable tokens while the teacher marginal maintains coverage; the ℓ1 cost uses a weighted median to limit distant targets.Source: §1 · pp. 1–2
• VDOT++ extends few-step optimal-transport distillation to T2V and I2V alongside condition-based video creation.Its token-level transport plans complement DMD when student and teacher distributions overlap poorly.Source: §1 · pp. 1–2
• VDOT++ combines distribution matching and adversarial refinement through sequential backward passes within the memory budget of large video backbones.The method also uses larger auxiliary score networks to strengthen supervision for a compact generator.Source: §1 · p. 2

2.1 Unified Video Generation and Editing

• Unified systems such as VACE and UNIC support heterogeneous video conditions, but their large diffusion backbones retain iterative-sampling costs.VDOT++ complements backbone design by distilling pretrained models for controllable creation, T2V, and I2V.Source: §2.1 · p. 2

2.2 Few-Step Diffusion Distillation

• Few-step distillation compresses many-step generation through consistency, distribution-matching, or adversarial objectives, among other approaches.DMD provides distribution-level supervision through score differences, but can become unreliable when student samples lie far from the teacher distribution; VDOT adds token-level OT guidance.Source: §2.2 · p. 2

2.3 Optimal Transport for Distribution Alignment

• Unlike generic unbalanced alignment, distillation motivates relaxing student-token mass more than teacher mass because the evolving student may contain unreliable tokens.The teacher remains the coverage target, and an ℓ1 ground cost limits averaging over incompatible teacher tokens.Source: §2.3 · p. 2

3 PRELIMINARIES

• The condition c ranges from text for T2V to text plus an initial image for I2V or text, context frames, and aligned masks for controllable creation.These settings span broad output ambiguity through dense spatial correspondence between paired frames.Source: §3 · p. 2

3.1 Distribution Matching Distillation

• DMD distills a pretrained diffusion model using a frozen teacher score network and a trainable fake-score network to approximate a reverse-KL gradient.When distributions overlap poorly, reverse KL can exert vanishing pressure on missing teacher modes and produce unstable gradients on unsupported student samples.Source: §3.1 · pp. 2–3

3.2 Optimal Transport Discrepancies

• An OT plan couples empirical samples and specifies which pairs exchange mass and how much, using a ground cost between samples.Balanced plans impose hard marginal constraints; partial and unbalanced OT relax participation or marginal equalities when matching is costly or unreliable.Source: §3.2 · p. 3
• Entropic OT adds a negative-entropy regularizer to the transport cost and can be optimized with Sinkhorn iterations.The regularization strength controls the diffuseness of the resulting plan.Source: §3.2 · p. 3

4.1 Overview

• VDOT++ combines distribution matching, frame-wise optimal-transport matching, and adversarial refinement to train a few-step generator.The losses are accumulated through sequential backward passes and applied in one generator update.Source: §4.1 · p. 3

4.2 Notation and Problem Setup

• For each frame, spatially pooled student and teacher score features form token sets whose pairwise ground costs define an independent transport problem.The generator output is re-noised at a sampled diffusion time before both score networks evaluate it.Source: §4.2 · p. 4

4.3 Optimal Transport Distillation

• OTD uses the optimal transport plan to couple student and teacher spatial tokens, supplying geometric guidance when reverse-KL guidance has limited overlap.The plan is computed independently for paired temporal frames, and the resulting gradients are averaged across batch and frame axes.Source: §4.3 · p. 4

4.4 Why Balanced OTD Weakens under Output Diversity

• Balanced OTD can force unrelated tokens to match and aggregate diverse targets poorly when valid videos differ in spatial content.This is less restrictive with dense conditions that align corresponding frames than with T2V or single-frame I2V.Source: §4.4 · p. 5

4.5 Partial Matching: Addressing Whom to Match

• Asymmetric unbalanced OT relaxes student-token mass to attenuate unreliable matches while keeping the teacher marginal tightly constrained.Partial matching applies only to frame-wise spatial-token OTD; DMD and adversarial loss continue to constrain the complete generated distribution.Source: §4.5 · p. 6

4.6 ℓ1 Ground Cost: Addressing How to Aggregate

• The ℓ1 ground cost changes aggregation from a weighted mean to a coordinate-wise weighted median, limiting the pull of distant targets.Partial matching reduces unreliable-pair influence, while the median limits the effect of diverse targets that remain in the plan.Source: §4.6 · pp. 6–7

4.7 Adversarial Discriminator

• A lightweight relativistic discriminator head embedded in the fake-score network provides adversarial refinement alongside distribution matching.The paper motivates this objective by noting that distribution matching alone leaves score-approximation errors that surface as texture artifacts.Source: §4.7 · p. 7

4.8 Model Learning

• Training updates the critic every step and the generator every R critic steps, using task-specific schedules for adversarial warm-up and updates.The generator objective combines its distillation and adversarial terms.Source: §4.8 · p. 7

• The GAN weight is kept small so adversarial training acts as refinement alongside the ℓ1-cost asymmetric OTD objective.Source: §19 · p. 7
• At 14B scale, sequential backward passes free the distillation graph before the adversarial pass, accumulating both gradients for one optimizer step.This preserves the joint objective with the peak memory of a single backward pass.Source: §19 · p. 7

5.2 Main Results

• Four-step VDOT++ remains competitive with many-step teachers across task families, leads normalized average on T2V, and exceeds the teacher and few-step methods on I2V.The comparisons cover UVCBench, VBench, and VBench-I2V.Source: Table 2 · p. 8, §5.2 · pp. 8–9, Table 3 · p. 9

5.1 Experimental Setup

• The evaluation distills VACE-Wan2.1-14B, Wan2.1-T2V-14B, and Wan2.1-I2V-14B into four-step generators, using VBench, VBench-I2V, UVCBench, and the VACE benchmark.The default generator and both score networks use the 14B architecture; adversarial training follows a warm-up.Source: §5 · p. 7, §5.1 · p. 8

5.2 Main Results

• On UVCBench, VDOT++ stays close to VDOT on dense pose, flow, and greyscale conditions but improves normalized average on outpainting and R2V.On VACE, gains likewise concentrate on R2V, outpainting, and extension; Fig. 4 compares controlled-generation examples.Source: §5.2 · p. 8, Table 2 · p. 8, Figure 4 · p. 10

5.3.1 Asymmetric Partial-OT Objective

• On T2V, partial matching and ℓ1 cost each improve over balanced ℓ2, and their combination is strongest, while variants remain close on strongly conditioned VACE.The surfing example shows partial ℓ1 retaining the panda’s torso and surfboard outline as the waves change.Source: Table 5 · p. 9, §5.3.1 · p. 10, Figure 7 · p. 11
• Source-side relaxation yields the highest Dynamic Degree and normalized average, with Imaging Quality (72.83) near the symmetric-relaxation value of 72.95.Symmetric and target-side relaxation underweight difficult teacher tokens; the comparison indicates that relaxation direction matters.Source: Table 6 · p. 10, §5.3.1 · pp. 10–11

5.3.2 Training Strategy

• Adversarial refinement mainly improves imaging quality, and joint accumulation is more stable than alternating updates because distribution matching preserves structure while the discriminator refines texture.Aesthetic Quality is less consistent across schedules, supporting use of the GAN as a weak appearance-refinement signal.Source: Table 7 · p. 11, §5.3.2 · p. 11

5.4 Cross-Scale Distillation

• With 14B score networks, VDOT++ normalized average rises from 79.98 to 80.51 (DMD2: 77.87 to 78.15), and the 1.3B VDOT++ generator surpasses the 14B teacher.The larger score networks are used only in training, so deployment remains a four-step 1.3B generator with unchanged inference cost.Source: Table 8 · p. 11, §5.4 · pp. 11–12

6 CONCLUSION

• VDOT++ uses asymmetric unbalanced transport to suppress unreliable student correspondences while retaining teacher coverage, and an ℓ1 cost to avoid mean aggregation of incompatible targets.Ablations support the conditioning-dependent design: it behaves similarly to balanced OTD under dense control but better preserves valid motion and detail as output ambiguity widens.Source: §6 · p. 12
• Joint adversarial refinement improves local appearance, while cross-scale distillation uses larger auxiliary score networks to strengthen a compact generator.Source: §6 · p. 12
• Spatial-token OT is solved independently within corresponding frames, so it does not transport tokens across frames or jointly optimize a spatiotemporal plan.The paper proposes spatiotemporal or trajectory-aware transport and adaptive marginal penalties as future directions; one-step, streaming, or autoregressive generation would also require handling stronger rollout errors and support mismatch.Source: §6 · p. 12

APPENDIX A NOTATION

• Table 9 consolidates the symbols defined throughout the main paper.Source: Appendix A · p. 15

• The entropic OT value is concave and differentiable in the cost matrix, with gradient equal to the optimal transport plan.Strictly convex entropy ensures the plan is unique, and Danskin’s theorem yields the gradient.Source: Appendix B · p. 15
• As the marginal penalty parameter grows, the unbalanced OT value is non-decreasing and bounded above by the balanced value, and its plans converge to the unique balanced optimum.Source: Appendix B · p. 15
• The weighted-median condition is the optimality condition for the convex, piecewise-linear objective φ.Source: Appendix B · p. 15

• The discriminator pools cross-attended features from selected fake-score layers through learnable register tokens and a time-conditioned MLP, reusing the fake-score backbone instead of a separate encoder.Real logits use a finite-difference R1 approximation of APT [36].Source: Appendix C · p. 15
• Training uses the 14B architecture for the generator and both score networks, with a frozen teacher and 2,100 optimization steps on four NVIDIA H200 GPUs.The passage also specifies AdamW settings, learning rates, and a critic-update ratio.Source: Appendix C · p. 15
• Algorithm 2 provides the complete log-domain solver for computing the asymmetric transport plan by optimizing Eq. (13) under the negative-entropy convention.Source: Appendix C · p. 15
• Adversarial training begins after 900 warm-up steps, and its settings are shared across T2V, I2V, and VACE; cross-scale distillation uses adjusted discriminator and GAN weights.Source: Appendix C · p. 15

APPENDIX D MULTI-TASK DATA AND UVCBENCH CONSTRUC-

• The appendix follows the automated data and benchmark construction pipeline introduced in the conference version.Source: Appendix D · p. 15
• Table 10 presents additional single-condition results on UVCBench.Source: Table 10 · p. 16
• The appendix includes a log-domain generalized Sinkhorn algorithm for computing the asymmetric unbalanced transport plan in Eq. (13).Source: Appendix D · p. 16, §15 · p. 16

15 end

• The training corpus uses 250,000 4K Artgrid videos rescaled to 832 × 480 and captioned with InternVL, with automatically extracted conditions for I2V and VACE-style tasks.Condition-specific curation includes filtering and ranking pose sequences by weighted inter-frame keypoint displacement.Source: §15 · p. 16
• UVCBench contains 8 single-condition and 10 composite tasks, with 20 videos per task, and generated videos are scored on six VBench dimensions plus a normalized average.Inputs and masks are automatically derived, manually verified, and regenerated when necessary.Source: §15 · p. 16

APPENDIX E FULL RESULTS ON THE UVCBENCH BENCHMARK

• Across ten UVCBench composite tasks, VDOT++ has higher normalized averages on some tasks and lower ones on others, with lower imaging quality on most.Table 11 gives the full per-task comparison.Source: Appendix E · p. 16, Table 11 · p. 17
• On FirstFrame+Flow, VDOT++ combines higher Dynamic Degree with lower Imaging Quality.Source: Appendix E · p. 16

APPENDIX F FULL RESULTS ON THE VACE BENCHMARK

• On extension, outpainting, and R2V, stronger normalized averages are driven mainly by increased dynamics while consistency metrics remain high.These tasks specify only part of the target video; the pattern supports the relaxed student marginal's role in avoiding forced matches in uncertain regions.Source: Appendix F · p. 17
• With four NFEs, VDOT++ remains competitive with many-step VACE variants on depth, pose, flow, scribble, and greyscale tasks.Table 12 reports the full per-task comparison.Source: Appendix F · p. 17, Table 12 · p. 18