Source-linked AI summary
HunyuanVideo 1.5 Technical Report
Bing Wu, Chang Zou, Changlin Li, Duojun Huang, Fang Yang, Hao Tan, Jack Peng, Jianbing Wu, Jiangfeng Xiong, Jie Jiang, Linus, Patrol, Peizhen Zhang, Peng Chen, Penghao Zhao, Qi Tian, Songtao Liu, Weijie Kong, Weiyan Wang, Xiao He, Xin Li, Xinchi Deng, Xuefei Zhe, Yang Li, Yanxin Long, Yuanbo Peng, Yue Wu, Yuhong Liu, Zhenyu Wang, Zuozhuo Dai, Bo Peng, Coopers Li, Gu Gong, Guojian Xiao, Jiahe Tian, Jiaxin Lin, Jie Liu, Jihong Zhang, Jiesong Lian, Kaihang Pan, Lei Wang, Lin Niu, Mingtao Chen, Mingyang Chen, Mingzhe Zheng, Miles Yang, Qiangqiang Hu, Qi Yang, Qiuyong Xiao, Runzhou Wu, Ryan Xu, Rui Yuan, Shanshan Sang, Shisheng Huang, Siruis Gong, Shuo Huang, Weiting Guo, Xiang Yuan, Xiaojia Chen, Xiawei Hu, Wenzhi Sun, Xiele Wu, Xianshun Ren, Xiaoyan Yuan, Xiaoyue Mi, Yepeng Zhang, Yifu Sun, Yiting Lu, Yitong Li, You Huang, Yu Tang, Yixuan Li, Yuhang Deng, Yuan Zhou, Zhichao Hu, Zhiguang Liu, Zhihe Yang, Zilin Yang, Zhenzhi Lu, Zixiang Zhou, Zhao Zhong
TL;DR
HunyuanVideo 1.5 addresses the accessibility and efficiency gap in high-quality video generation with an open-source compact model. It combines a unified DiT pipeline, sparse attention, bilingual text encoding, progressive training, and super-resolution, and is reported to achieve state-of-the-art quality and motion coherence with 8.3 billion parameters.
Problem
Most state-of-the-art video generation models remain proprietary, limiting accessibility and community-driven innovation, while some open models introduce substantial computational costs.
Method
The paper develops an open-source 8.3B unified video generation framework combining curated data, a DiT with SSTA, bilingual glyph-aware encoding, progressive training, and video super-resolution.
Results
The model is reported to establish a new state-of-the-art among open-source video generation models while delivering high visual quality, motion coherence, and efficient inference.
Takeaways & Limitations
The released model and code provide a compact open-source foundation for high-quality text-to-video and image-to-video generation across multiple durations and resolutions.
Abstract
from arXiv · showhide
We present HunyuanVideo 1.5, a lightweight yet powerful open-source video generation model that achieves state-of-the-art visual quality and motion coherence with only 8.3 billion parameters, enabling efficient inference on consumer-grade GPUs. This achievement is built upon several key components, including meticulous data curation, an advanced DiT architecture featuring selective and sliding tile attention (SSTA), enhanced bilingual understanding through glyph-aware text encoding, progressive pre-training and post-training, and an efficient video super-resolution network. Leveraging these designs, we developed a unified framework capable of high-quality text-to-video and image-to-video generation across multiple durations and resolutions. Extensive experiments demonstrate that this compact and proficient model establishes a new state-of-the-art among open-source video generation models. By releasing the code and model weights, we provide the community with a high-performance foundation that lowers the barrier to video creation and research, making advanced video generation accessible to a broader audience. All open-source assets are publicly available at https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5.
1 Introduction
HunyuanVideo 1.5 is an open-source 8.3B video generation model designed to combine high visual quality with efficient inference. Its unified, progressively trained pipeline supports text-to-video and image-to-video generation while improving efficiency, multimodal understanding, and output quality.
- HunyuanVideo 1.5 uses an open-source 8.3B model to target state-of-the-art visual quality with high inference efficiency.
- Its two-stage pipeline generates 480p–720p videos lasting 5–10 seconds, then upscales them to 1080p with a dedicated super-resolution network.
- SSTA dynamically prunes redundant spatiotemporal tokens and achieves a 1.87× end-to-end speedup for 10-second 720p synthesis versus FlashAttention-3.
- Bilingual understanding combines a large multimodal model with ByT5 glyph encoding, while detailed captions are generated for images and videos.
- Muon and multi-phase progressive training improve convergence, motion coherence, aesthetic quality, and alignment with human preferences.
- The report covers data preparation, architecture and algorithms, task-specific training, and evaluation against state-of-the-art models.
2 Data Preparation
The data pipeline combines large-scale acquisition with multi-stage filtering, modality-specific captioning, and reinforcement-learning-based caption post-training. These steps target diverse, high-quality, accurate training data for video generation.
- The training corpus combines image and video data, including 5 billion curated images and more than 10 million hours of raw video.
- Videos are segmented into 2–10-second clips using scene detection and filtered to remove transition effects.
- Spatial cropping removes subtitles, logos, and watermarks, while clips retaining less than 60% of the original frame are discarded.
- A three-level curation pipeline removes structural artifacts and low-motion scenes, assesses sharpness, detail, noise, artifacts, and dynamic range, and filters low-aesthetic clips.
- Approximately 800 million high-quality video segments remained for pre-training after filtering.
- Specialized models produce image captions, structured video descriptions, and image-to-video instructions describing temporal changes.
- OPA-DPO reinforcement learning balances richer captions against hallucinations and factual inconsistencies.
3 Model Design
The model design uses a unified 8.3B DiT for multiple generation tasks, supported by compressed image-video representations, bilingual text encoding, sparse attention, and latent-space super-resolution.
- A two-stage architecture uses an 8.3B multi-task DiT followed by a video super-resolution network to enhance visual quality.
- Multi-Task Training: The unified architecture jointly trains text-to-image, text-to-video, and image-to-video tasks, integrating reference images through VAE latents and SigLip features.
- VAE: A causal 3D transformer VAE jointly encodes images and videos with 16× spatial and 4× temporal compression and 32 latent channels.
- Sparse Attention: SSTA partitions tokens into 3D blocks, selects important block pairs, adds local sliding-window connections, and applies the combined mask.
- Text encoder: The dual-channel text encoder combines Qwen2.5-VL for multimodal scene understanding with multilingual Glyph-ByT5 for accurate text rendering.
- Sparse Attention: SSTA replaces quadratic full attention with selective and sliding tile attention that exploits spatiotemporal redundancy through block-sparse computation.
- Optimizer: Muon reaches lower training loss than AdamW in half the training steps and provides stronger performance across multiple text-to-image benchmarks.
- Video Super-Resolution: The cascaded VSR model performs latent-space super-resolution by concatenating low-resolution latents and separately training a latent upsample block for spatial alignment.
4 Model Training
HunyuanVideo 1.5 uses progressive mixed-task pre-training and multi-stage post-training to develop video generation capabilities, then refines outputs through task-specific alignment and super-resolution.
- Pre-training: Progressive pre-training spans T2I, T2V, and I2V while scaling resolution, frame rate, temporal length, and duration.Training advances from 256p at 16 fps to 480p and 720p at 24 fps, with durations from 2 to 10 seconds.
- Pre-training: A 1:6:3 task ratio combines T2I, T2V, and I2V data to balance semantic understanding, diversity, and video-specific modeling.The regimen mixes image and video tasks throughout pre-training.
- Post-training: Post-training applies continuing training, supervised fine-tuning, and human-feedback alignment separately to T2V and I2V.The stages progressively refine the model toward task-specific output distributions.
- Post-training: Continuing training uses 1 million high-quality video clips per task, emphasizing dynamic motion for T2V and motion transformations for I2V.Both tasks are trained at 480p and 720p during continuing training.
- Post-training: SFT filters clips for aesthetic appeal, clarity, and motion smoothness, while RLHF targets artifacts and motion quality through task-specific strategies.I2V uses online reinforcement learning, whereas T2V begins with offline DPO before online optimization.
- Video super-resolution: The cascaded video super-resolution model is trained on 1 million clips spanning 1K to 4K, lasting 3 to 10 seconds at 24 fps.Low-resolution latents and noise are concatenated as DiT inputs, and the model is fully trainable under flow matching.
5 Model Performance
The evaluation framework measures text-to-video and image-to-video quality across multiple perceptual dimensions using ratings and pairwise GSB comparisons. Tests use balanced inputs, default competitor configurations, single-run generation, and professional assessors.
- Evaluation methodology: GSB compares two models according to overall video perception quality, complementing the multidimensional rating method.
- Evaluation methodology: Rating evaluates T2V across text-video consistency, frame aesthetics, visual quality, structural stability, and motion effects.
- Evaluation methodology: I2V rating evaluates image-video consistency, instruction responsiveness, visual quality, structural stability, and motion effects.
- Evaluation setup: The study uses 300 diverse text prompts and 300 image samples, generating an equal number of samples per model in one run for comparability.Results are not cherry-picked, and competing models use their default configurations.
- Evaluation setup: More than 100 professional assessors conduct the evaluation, with results reported in Tables 5 and 6.
6 Inference Speed and GPU Memory Requirements
The report evaluates inference speed with and without engineering acceleration and measures memory under offloading configurations. The pipeline can run 720p 121-frame generation on a single consumer-grade GPU within 13.6 GB peak memory.
- Evaluation setup: Inference speed measurements use the classifier-free-guidance-distilled model on 8 NVIDIA H800 GPUs with context parallelism.
- Inference speed: Unoptimized speed experiments isolate the described model methods without engineering-level acceleration or custom optimization.FlashAttention v3 is used for configurations without sparse attention.
- Inference speed: Engineering-accelerated experiments combine SageAttention, torch.compile, and feature caching to improve practical inference speed while maintaining nearly identical output quality.The reported configuration targets notable acceleration rather than the most extreme quality-cost trade-off.
- GPU memory: 13.6 GB peak memory enables end-to-end 720p 121-frame T2V/I2V inference on a single consumer-grade GPU with pipeline offloading, group offloading, and VAE tiling.The report gives the RTX 4090 as an example of supported hardware.
7 Conclusion
HunyuanVideo 1.5 is presented as a compact open-source DiT that combines progressive training, SSTA, super-resolution, and multimodal alignment for high-quality T2V and I2V generation. The report concludes that it establishes a new standard among open-source video generators while providing an accessible foundation for creative and research applications.
- Conclusion: HunyuanVideo 1.5 is an 8.3B-parameter open-source DiT for text-to-video and image-to-video synthesis.
- Conclusion: Its pipeline combines meticulous data curation, progressive training, SSTA-based acceleration, and a dedicated video super-resolution network.
- Conclusion: The report attributes the model's capabilities to high visual quality, motion coherence, bilingual prompt understanding, text rendering, and instruction-following.
- Conclusion: The authors state that comprehensive evaluations establish HunyuanVideo 1.5 as a new standard among open-source video generators.
- Conclusion: Releasing the lightweight model provides an accessible foundation for creative and research applications.
8 Project Contributors
The project credits sponsors, a project leader, core contributors organized by technical responsibilities, and a broader alphabetical contributor list.
- Project sponsors include Jie Jiang, Linus, Yuhong Liu, and Peng Chen, with Zhao Zhong identified as project leader.
- Captioning and data contributions are credited to Xin Li, Bing Wu, Duojun Huang, Hao Tan, Xinchi Deng, and Xuefei.
- VAE and model acceleration contributors include Zhe, Zhenyu Wang, Songtao Liu, Changlin Li, Chang Zou, and Fang Yang.
- Pretraining and post-training contributors include Jianbing Wu, Jack Peng, Patrol, Peizhen Zhang, Penghao Zhao, Weiyan Wang, Xiao He, Yang Li, Yuanbo Peng, Yue Wu, Jiangfeng Xiong, Qi Tian, and Weijie Kong.
- Additional contributors are listed alphabetically across the project credits.The alphabetical list spans the names provided in the contributor-credit passages.