DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation

• Late-step teacher imitation can consume scarce discrete capacity fitting the teacher’s continuous-space residual instead of improving target detail prediction.At inference, weakly aligned errors in independently quantized conditional and unconditional branches can be amplified by CFG into high-frequency artifacts.Source: §1 · p. 2
• Conventional QAT preserves prompt semantics but severely degrades detail refinement in the middle and late denoising steps.Early steps establish global layout and motion, while middle and late steps refine textures and sharpness.Source: §1 · p. 2
• Denoising-Stage Oriented Supervision shifts from early-step KD toward late-step target loss, while Denoising-Stage Gated Guidance disables CFG in the final steps.The first component frees capacity for detail reconstruction; the second removes CFG’s amplification of late-step quantization errors while retaining earlier guidance.Source: §1 · p. 2
• DSAQuant consistently improves over SOTA QAT baselines across diverse VDMs under W4A4 and W3A3 settings.The evaluated models include CogVideoX-2B/1.5-5B and Wan2.1-1.3B/14B and Wan2.2-5B.Source: §1 · p. 2

• Video diffusion models use billions of parameters and multi-step denoising over long spatio-temporal sequences, creating substantial latency and memory costs.Prior efficiency approaches include step distillation, sparse attention, step caching, and quantization.Source: §2 · p. 2
• Conventional QAT generally preserves layout and motion but severely degrades details and texture, with late-step quantization damaging visual quality more than early-step quantization.Source: Figure 1 · p. 3
• VDM quantization work includes dynamic-activation and mixed-precision PTQ, salient-data and sparse-token distillation, and QAT frameworks such as QVGen.PTQ may lose quality under activation quantization, while rotation-based transformations can add runtime overhead in DiTs.Source: §2 · p. 3

3.1 Video Diffusion Sampling

• Video diffusion sampling follows a noising path from Gaussian noise toward a clean video latent, and the denoising network predicts a target at each step.The target may be noise, velocity, or a data-related quantity.Source: §3.1 · p. 3

3.2 Stage-Wise Quantization Analysis for Video Denoising

• Early low-SNR steps establish global semantics, scene layout, and coarse motion, whereas late steps refine local appearance, high-frequency texture, and frame-level sharpness.Source: §3.2 · p. 3
• Uniform QAT preserves prompt semantics, layout, and coarse motion but produces blurred textures, corrupted details, reduced sharpness, and unstable local motion.Quantizing only early steps slightly perturbs layout while preserving perceptual quality; quantizing middle and late steps reproduces the full failure mode.Source: §3.2 · pp. 3–4

4.1 Training Stage: Denoising-Stage Oriented Supervision

• Late denoising refines local appearance and sharpness, so uniform teacher imitation can spend low-bit capacity matching teacher residuals rather than reducing target error.The paper links late-stage errors to blur, texture corruption, and lost sharpness.Source: §4.1 · p. 5
• A time-dependent weighted sum preserves distillation early and shifts later steps toward the original diffusion or flow target, freeing discrete capacity for high-frequency detail.The schedule fully weights KD at t = 0 and the target loss at t = 1; early KD also anchors layout and stabilizes the STE gradient.Source: §4.1 · p. 5

4.2 Inference Stage: Denoising-Stage Gated Guidance

• CFG amplifies the mismatch between weakly correlated conditional and unconditional quantization errors, which can accumulate rather than cancel.The two branches can have different activation distributions, producing different clipping and rounding under dynamic activation quantization.Source: §4.2 · p. 5
• As ideal guidance diminishes and quantization error grows in trailing steps, the branch mismatch can dominate and lead to high-frequency artifacts.The unconditional branch can be skipped during late steps when it is no longer used.Source: §4.2 · pp. 5–6

5.1 Implementation details

• Experiments cover CogVideoX-2B/1.5-5B and Wan2.1-1.3B/14B plus Wan2.2-5B, evaluated with VBench, VBench-2.0, VisionReward, instruction following, and Laplacian Variance.The study uses symmetric W4A4 and asymmetric W3A3 settings; timestep-conditioned linear layers are excluded from quantization.Source: §5.1 · pp. 6–7

5.2 Main results

• At W4A4, augmented-prompt VBench averages exceed QVGen by 1.40–2.14 points, and at W3A3 by 2.63–6.60 points; Wan2.1-14B also maintains high VBench-2.0 performance at 3-bit.The augmented-prompt comparisons include W4A4 and W3A3 results; the VBench-2.0 result is reported for Wan2.1-14B.Source: Table 2 · p. 7, Table 3 · p. 7, §5.2 · p. 8

5.3 Ablation study

• The two components together yield the best overall ablation performance, supporting complementary roles for stage-oriented supervision and CFG-drop.The component ablation is on Wan2.1-1.3B with W4A4 symmetric quantization; the table defines Base as the engineering-only quantized baseline.Source: Table 4 · p. 8, §5.3 · p. 8
• The exponential supervision schedule with α = 5 performs best, while the more aggressive α = 10 decay harms performance.The comparison evaluates linear and exponential schedules for balancing distillation and target losses.Source: Table 5 · p. 8, §5.3 · p. 8

5.4 Efficiency analysis

• After H100 normalization, training cost falls roughly 2×–7×; W4A4 cuts storage by over 73%, peak memory by up to 45.1%, and A6000 latency by up to 38.0% versus FP16.The reported memory and latency comparison uses 81 frames at 480p; quantized Wan 14B runs on 4090D where FP16 runs out of memory.Source: Table 6 · p. 9, §5.4 · p. 9, Table 7 · p. 9

5.3 Ablation study

• Applying CFG-drop for only the final three timesteps boosts visual clarity and VBench, but extending it beyond ten steps lowers VBench; nine steps is the selected near-optimal threshold.The threshold ablation weighs visual clarity against VBench performance.Source: §5.3 · pp. 8–9, Figure 4 · p. 9

6 Conclusion

• DSAQuant pairs stage-oriented supervision during training with stage-gated guidance during inference to align quantization with video denoising and improve visual quality, sharpness, and temporal stability while maintaining text-video alignment.The conclusion frames stage-aware design as important for high-fidelity video diffusion model quantization.Source: §6 · p. 9
• H100 and H20’s practical training-throughput gap is narrower than their ∼6.7× peak compute ratio because H20’s higher HBM bandwidth partly offsets its compute deficit.The passage identifies attention, optimizer updates, and gradient synchronization as memory-bound operators.Source: §6 · p. 9

A More Related Work

• Image-diffusion quantization work spans timestep-aware PTQ and distillation-based QAT, while video methods add long-token and temporal-consistency concerns that make direct transfer non-trivial.The related-work discussion distinguishes DSAQuant by matching early-step teacher anchoring with late-step target refinement and reduced CFG-induced noise amplification.Source: A More Related Work · p. 13

B Gradient Variance Analysis of Early-Stage Distillation

• Early denoising leaves substantial uncertainty about the clean latent, injecting posterior target variance into STE gradients that can push low-bit weights across quantization-bin boundaries.The target-gradient covariance is 4JQ(z)⊤Σt(z)JQ(z).Source: B Gradient Variance Analysis of… · p. 14
• Conditioned on the same noisy input, teacher distillation removes label-induced gradient covariance and trades target noise for the teacher’s residual.The teacher output is deterministic given z, so Cov(gKD | z) = 0.Source: B Gradient Variance Analysis of… · p. 14
• Because early high-noise steps have large posterior target variance, distillation stabilizes QAT updates and anchors the global denoising trajectory.The teacher provides a stable estimate of the conditional denoising direction in this regime.Source: B Gradient Variance Analysis of… · p. 14
• In later low-noise refinement, the variance-reduction benefit wanes while teacher residual and quantized-model capacity matter more, motivating a shift toward native target-loss supervision.The stage schedule uses KD early and gradually transfers supervision to target loss later.Source: B Gradient Variance Analysis of… · p. 14

C Two-Stage Analysis of Late-Step CFG Quantization Error

• With dynamic activation quantization, condition-dependent activation distributions yield different branch grids, so accumulated conditional and unconditional errors need not cancel.Shared weights and latent input can still create correlated errors; the relevant quantity is the residual mismatch after correlated components cancel.Source: C Two-Stage Analysis of Late-Step CFG… · pp. 14–15

D General and Quantization-Specific Effects of CFG-Drop

• CFG-drop produces substantially larger VBench-average and visual-sharpness gains for both quantized models than for the full-precision model, though the full-precision model improves slightly too.The identical-schedule controls compare BF16 full precision with W4A4 QVGen and Base on Wan2.1-1.3B.Source: Table 8 · p. 15, D General and Quantization-Specific… · pp. 15–16

C Two-Stage Analysis of Late-Step CFG Quantization Error

• Late in denoising, the useful full-precision guidance direction weakens while branch mismatch can grow, allowing CFG’s scaled mismatch to dominate as high-frequency artifacts.CFG scales the mismatch contribution by guidance strength ω.Source: C Two-Stage Analysis of Late-Step CFG… · p. 15
• Late-step CFG-drop uses only the conditional prediction, removing CFG’s multiplier after the useful guidance direction has diminished.This suppresses the second-stage amplification of quantization error.Source: C Two-Stage Analysis of Late-Step CFG… · p. 15

E Detailed Training Settings

• Training uses H20 GPUs with 140GB memory and local batch size 1 per GPU; synthetic training data varies by model family.Wan2.1-1.3B data is synthesized from filtered, LLM-extended VidProM prompts; Wan2.1-14B uses TurboDiffusion latents, while other models use decoded and re-encoded TurboDiffusion videos.Source: E Detailed Training Settings · p. 16

F Detailed Evaluation Settings

• Evaluation assigns seeds by process rank so generated videos use different random noise, with 128 GPUs for Wan2.1-14B inference and 32 GPUs for other models..Source: F Detailed Evaluation Settings · p. 16

G Timestep-Conditioned Linear Layers and LUT Precomputation

• For discrete scheduler timesteps, timestep features can be precomputed and stored on CPU, then loaded for the current step without added GPU-memory overhead.The procedure covers CogVideoX and Wan and replaces timestep-conditioned linear layers with LUT lookups.Source: G Timestep-Conditioned Linear Layers… · p. 16

G.1 CogVideoX-2B

• CogVideoX-2B’s LUT stores per-timestep AdaLN modulation vectors and output-head modulation for its transformer blocks.The described lookup is defined over the fixed timestep set TK.Source: §G.1 · p. 17

G.2 Wan2.1-T2V-1.3B

• Wan2.1-T2V-1.3B needs only global timestep quantities cached, with block-specific modulation formed by lightweight element-wise addition.The appendix contrasts this with CogVideoX’s separate timestep-conditioned layers inside transformer blocks.Source: §G.2 · p. 18

G.4 Storage Comparison

• Table 10 compares timestep-related linear-parameter storage with LUT size for 50 denoising steps..Source: Table 10 · p. 19, §G.4 · p. 19

G.3 Why LUT Precomputation is Feasible

• For both models, cached quantities depend on discrete timesteps and fixed weights, not latent or text activations, so they can be precomputed once and reused by timestep lookup.This removes online timestep-related linear layers while retaining the original normalization, modulation, attention, feed-forward, and residual-gating paths.Source: §G.3 · p. 19

H Supplementary Results for Main Experiments

• The supplementary table reports per-dimension VBench scores under standard prompts..Source: H Supplementary Results for Main… · p. 19
• The reported supplementary setting is the more aggressive W3A3 quantization regime..Source: H Supplementary Results for Main… · p. 19
• The supplementary results cover Wan2.1-1.3B and Wan2.2-5B and complement the main comparison in Table 1..Source: H Supplementary Results for Main… · p. 19

I Full VBench Results For Ablation Studies

• Comprehensive VBench ablation results are provided in supplementary Tables 12 and 13.The main-text ablation section reports five sensitive metrics because of page constraints.Source: I Full VBench Results For Ablation… · p. 19, Table 12 · p. 20, Table 13 · p. 20

J Further inference acceleration

• Inference latency is reported for W4A4 quantization combined with SageAttention, measured at 81 frames and 480p.The supplementary section states that it integrates SageAttention to accelerate inference.Source: Table 14 · p. 20, J Further inference acceleration · p. 20

K Visual Comparison

• Qualitative comparisons cover three Wan backbones using the quantitative evaluation’s prompts and settings, highlighting alignment, texture, detail preservation, and quantization artifacts.Figures 5–10 show W4A4 and W3A3 results for Wan2.1-1.3B, Wan2.2-5B, and Wan2.1-14B, respectively.Source: K Visual Comparison · p. 20

Limitations

• Experiments cover bidirectional video diffusion models, leaving evaluation on autoregressive video generation as future work.The paper identifies autoregressive models’ potential for real-time generation as motivation for that future evaluation.Source: L Limitations, Broader impacts · p. 20

Broader impacts

• Lower deployment costs may broaden access to video generation while also increasing risks such as misleading synthetic videos and privacy-violating media.The paper recommends safeguards including watermarking, provenance tracking, content filtering, safety classifiers, and synthetic-content disclosure.Source: L Limitations, Broader impacts · p. 21