Source-linked AI summary
TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times
Jintao Zhang, Kaiwen Zheng, Kai Jiang, Haoxu Wang, Ion Stoica, Joseph E. Gonzalez, Jianfei Chen, Jun Zhu
TL;DR
Video diffusion generation is costly, motivating a framework that improves speed without substantially reducing quality. TurboDiffusion combines attention acceleration, step distillation, quantization, and engineering optimizations, achieving 100–200× end-to-end speedup and sub-minute generation on one RTX 5090.
Problem
Video diffusion generation requires acceleration that preserves video quality across multiple video-generation models.
Method
TurboDiffusion combines SageAttention and SLA, rCM step distillation, W8A8 quantization, and additional engineering optimizations.
Results
100–200× end-to-end diffusion speedup with negligible quality degradation enables single-video generation in under one minute on one RTX 5090 GPU.
Takeaways & Limitations
TurboDiffusion makes high-quality video generation substantially more efficient and practical on a single GPU.
Abstract
from arXiv · showhide
We introduce TurboDiffusion, a video generation acceleration framework that can speed up end-to-end diffusion generation by 100-200x while maintaining video quality. TurboDiffusion mainly relies on several components for acceleration: (1) Attention acceleration: TurboDiffusion uses low-bit SageAttention and trainable Sparse-Linear Attention (SLA) to speed up attention computation. (2) Step distillation: TurboDiffusion adopts rCM for efficient step distillation. (3) W8A8 quantization: TurboDiffusion quantizes model parameters and activations to 8 bits to accelerate linear layers and compress the model. In addition, TurboDiffusion incorporates several other engineering optimizations. We conduct experiments on the Wan2.2-I2V-14B-720P, Wan2.1-T2V-1.3B-480P, Wan2.1-T2V-14B-720P, and Wan2.1-T2V-14B-480P models. Experimental results show that TurboDiffusion achieves 100-200x speedup for video generation even on a single RTX 5090 GPU, while maintaining comparable video quality. The GitHub repository, which includes model checkpoints and easy-to-use code, is available at https://github.com/thu-ml/TurboDiffusion.
1 Method
TurboDiffusion combines sparse and low-bit attention, step distillation, quantization, and implementation optimizations to accelerate video diffusion. Training adapts a pretrained model to sparsity and fewer sampling steps before inference deployment.
- 1.1 Main Techniques: TurboDiffusion combines SageAttention, Sparse-Linear Attention, rCM step distillation, W8A8 quantization, and additional engineering optimizations.SageSLA replaces SLA during inference, while rCM reduces sampling steps and W8A8 accelerates linear layers.
- 1.2 Training: The training process finetunes a pretrained model with SLA, distills it with rCM, and merges both parameter updates into one model.Training can use real or synthetic data.
- 1.3 Inference: Inference replaces SLA with CUDA-based SageSLA, reduces sampling from 100 steps to 4 or 3, and uses INT8 Tensor Cores for linear layers.SageSLA is built on SageAttention; linear parameters and activations use block-wise INT8 quantization.
- 1.3 Inference: W8A8 quantization roughly halves model size while accelerating Linear layer computation.Parameters and activations are quantized to INT8 with 128 × 128 block-wise granularity.
2 Evaluations
TurboDiffusion is evaluated across four Wan video diffusion models against Original and FastVideo baselines, measuring end-to-end diffusion latency and video quality. The evaluation reports highest efficiency while maintaining video quality, with visual examples spanning text-to-video and image-to-video scenarios.
- 2.1 Setup: The evaluation covers Wan2.2-I2V-A14B-720P and three Wan2.1 text-to-video models, using Original and FastVideo as main baselines.Efficiency is measured by end-to-end diffusion generation latency, excluding text encoding and VAE decoding.
- 2.1 Setup: Experiments primarily use a single RTX 5090 GPU, with substantial acceleration also observed on RTX 4090 and H100 GPUs.The reported acceleration on those additional GPUs is described as smaller than on the RTX 5090.
- 2.2 Efficiency and Quality: TurboDiffusion achieves the highest efficiency while maintaining video quality across the reported comparisons, outperforming FastVideo where that baseline is available.Wan2.2-A14B-I2V-720P is compared only with Original because FastVideo does not provide an accelerated implementation.
- 2.2 Efficiency and Quality: Visual examples include five-second text-to-video and image-to-video generations across varied scenes, prompts, and motion patterns.The examples include cat, lunar rover, liquid-metal, sailor, watercolor, Tokyo, museum, classroom, Minecraft, and food scenes.
3 Conclusion and Future Work
TurboDiffusion delivers 100–200× end-to-end diffusion speedup with negligible quality degradation across four Wan models. The framework reduces single-video generation time to under one minute on one RTX 5090, while future work targets autoregressive video diffusion.
- 3 Conclusion and Future Work: TurboDiffusion achieves 100–200× end-to-end diffusion speedup with negligible quality degradation across four Wan video models.The evaluated models include Wan2.2-I2V-A14B-720P and three Wan2.1 variants.
- 3 Conclusion and Future Work: TurboDiffusion reduces single-video generation time to under one minute on a single RTX 5090 GPU.The conclusion describes this as making high-quality video generation more efficient and practical.
- 3 Conclusion and Future Work: Future work will extend TurboDiffusion to additional video generation paradigms, including autoregressive video diffusion.