HunyuanVideo 1.5 Technical Report

Introduction

• The two-stage framework generates 480p–720p videos lasting 5–10 seconds with a unified DiT, then upscales them to 1080p.It supports both text-to-video and image-to-video generation.Source: §1 · p. 1
• The 8.3B-parameter DiT and 3D causal VAE achieve 16× spatial and 4× temporal compression while targeting performance comparable to state-of-the-art video generation models.Source: §1 · p. 2
• The few-step video super-resolution network upscales outputs to 1080p, enhancing sharpness and correcting distortions to refine detail and visual texture.Source: §1 · p. 2
• SSTA dynamically prunes redundant spatiotemporal tokens, achieving 1.87× end-to-end speedup over FlashAttention-3 [7] for 10-second 720p video synthesis.Source: §1 · p. 2
• A multimodal model supports Chinese-English understanding, while ByT5 glyph encoding improves text generation accuracy in videos and detailed bilingual captions are generated for images and videos.Source: §1 · p. 2
• Muon [8] accelerates convergence, while multi-phase progressive pre-training and post-training enhance motion coherence, aesthetic quality, and alignment with human preferences.Source: §1 · p. 2

2.1 Data Acquisition and Filtering

• The pipeline curated 5 billion images from over 10 billion for pre-training, reserved 1 billion for later stages, and retained approximately 800 million high-quality video segments.Source: §2.1 · pp. 2–3
• Diverse raw-video sourcing yielded more than 10 million hours, segmented into 2–10-second clips using PySceneDetect and transition-classifier filtering.Source: §2.1 · p. 2
• Filtering removes borders, stitching, collages, static scenes, low-quality or unaesthetic clips, and cropped clips retaining less than 60% of the original frame.Visual-quality scoring covers sharpness, detail retention, noise and artifacts, and dynamic range.Source: §2.1 · pp. 2–3

2.2 Captioning

• Three specialized captioners describe static images, structured cinematic and aesthetic video properties, and temporal changes in image-to-video generation.Source: §2.2 · p. 3
• Caption-model post-training uses OPA-DPO [12] reinforcement learning to balance descriptive detail against hallucinations and factual errors.Source: §2.2 · p. 3
• A specialized camera-movement recognizer identifies multiple movement types at clip and sequential levels, converting high-confidence predictions into caption descriptions.The captions aim to support controllable camera movement in generated videos.Source: §2.2 · pp. 3–4

3 Model Design

• An 8.3-billion-parameter multi-task DiT forms the first stage, followed by a video super-resolution network to enhance visual quality.This is the proposed two-stage framework for high-fidelity video synthesis.Source: §3 · p. 4

3.1 Unified Diffusion Transformer

• The unified architecture jointly trains T2I, T2V, and I2V; I2V combines VAE image latents with noisy latents and SigLip semantic features, distinguished by a learnable type embedding.The VAE-based path supports detail reconstruction, while SigLip features support semantic alignment and instruction adherence.Source: §3.1 · p. 4
• The causal 3D transformer jointly encodes images and videos with 16× spatial compression, 4× temporal compression, and 32 latent channels.Source: §3.1 · p. 4
• The dual-channel text encoder pairs Qwen2.5-VL [13] for scene, action, and requirement understanding with multilingual Glyph-ByT5 for accurate text rendering across languages.It combines high-level semantic representations with fine-grained language-specific features.Source: §3.1 · p. 5
• SSTA combines adaptive block selection with sliding-window attention in parameter-free block-sparse attention, and is applied during distillation for efficiency while preserving output quality.Its mask combines dynamically selected blocks and local-window blocks; an engineered acceleration toolkit implements flex_block_attention using ThunderKittens.Source: §3.1 · p. 5
• Muon [8] reaches lower training loss than AdamW in half as many training steps and performs better across multiple text-to-image benchmarks, with weight decay 0.01 for stability.Source: §3.1 · p. 5

3.2 Video Super-Resolution

• The cascaded VSR model enhances fine details and textures for T2V and I2V outputs by concatenating low-resolution latents and spatially aligning them with a separately trained latent upsample block.It follows the 8.3B Video Diffusion Transformer architecture, operates in latent space, and decodes the resulting high-resolution latents into 1080p video.Source: §3.2 · p. 6

4.1 Pre-training

• Pre-training progressively combines T2I, T2V, and I2V tasks while scaling spatiotemporal conditions to build semantic alignment and coherent video synthesis.The mixed-task ratio is 1:6:3; video training advances from 256p at 16 fps to 480p and 720p at 24 fps, with durations of 2–10 seconds. Shift schedules adapt to changing video token lengths to maintain training stability.Source: §4.1 · p. 6
• T2I pre-training uses 256p and 512p stages with aspect-ratio buckets, strengthening text-image alignment and accelerating subsequent T2V and I2V training.Images with varied aspect ratios are grouped into buckets for efficient sampling.Source: §4.1 · p. 6

4.2 Post-training

• Continuing training and supervised fine-tuning refine T2V and I2V outputs for greater stability, visual quality, aesthetic appeal, and temporal consistency.CT uses 1 million high-quality video clips per task at 480p and 720p; T2V prioritizes dynamic motion, while I2V captions describe motion and transformations relative to the first frame. SFT clips are filtered for aesthetic appeal, clarity, and motion smoothness.Source: §4.2 · pp. 6–7
• I2V online RL targets structural and motion artifacts, using a VLM-based reward model and mixed sampling to improve evaluation metrics, especially motion realism.The reward model scores text alignment, image alignment, visual quality, and motion dynamics; exploration uses varied random seeds and CFG scales with the MixGRPO hybrid ODE–SDE solver.Source: §4.2 · p. 7
• T2V alignment combines offline DPO [20] on manually annotated video pairs with online RL to reduce motion artifacts and improve visual quality and semantic-text alignment.The offline stage uses a balanced O(10K) prompt set covering motion, scene, and subject dimensions, then generates and annotates candidate pairs for semantic fidelity, motion quality, and aesthetics.Source: §4.2 · p. 7

4.3 Video Super-Resolution.

• The video super-resolution model is initialized from a pretrained T2V model and trained with low-resolution latents and noise as input to enhance visual quality and temporal coherence.Its training data includes 1 million high-quality video clips spanning 1K–4K resolutions and 3–10 seconds at 24 fps, plus high-resolution images; all model weights are trainable under flow matching.Source: §4.3 · p. 8

5.2 GSB

• GSB compares models’ overall video perception quality for 720P text-to-video and image-to-video generation.Tables 5 and 6 report the respective T2V and I2V comparisons; the supplied passages do not provide their result values.Source: §5 · p. 8, Table 5 · p. 10, Table 6 · p. 10

5.1 Rating

• Rating evaluates text-to-video on five dimensions and image-to-video on five task-specific dimensions, including consistency, visual quality, structural stability, and motion.Text-to-video also measures frame aesthetics; image-to-video measures instruction responsiveness. Tables 3 and 4 cover the T2V and I2V ratings, respectively.Source: §5.1 · p. 8, Table 3 · p. 9, Table 4 · p. 10

5.2 GSB

• The GSB evaluation uses 300 text prompts and 300 image samples, equal video counts per model, one inference per input, default competitor configurations, and over 100 professional assessors.The protocol is designed to ensure comparability and avoid cherry-picking.Source: §5.2 · pp. 9–10

Inference Speed without Engineering Acceleration

• Speed measurements use the CFG-distilled model on 8 NVIDIA H800 GPUs with context parallelism enabled across all devices.Inference speed is measured as wall-clock time per diffusion step averaged across multiple runs.Source: §6 · p. 10
• Without engineering acceleration, sparse attention reduces inference time for long-context sequences, particularly at 241 frames.The unoptimized measurements use standard distributed inference and Flash Attention v3 [7] for configurations without sparse attention.Source: §6 · p. 10

Inference Speed with Engineering Acceleration

• With engineering acceleration, sparse attention further reduces inference time for long-context sequences while the experiment targets nearly identical output quality.The setup uses SageAttention [21], PyTorch torch.compile, and feature caching, and reports total inference time for 50 diffusion steps.Source: §6 · pp. 10–11

GPU Memory Requirements

• 13.6 GB peak memory enables end-to-end 720p 121-frame T2V/I2V inference on a single consumer-grade GPU with pipeline and group offloading and VAE tiling.Memory measurements use a single-GPU configuration; an RTX 4090 is given as an example.Source: §6 · p. 11

• HunyuanVideo 1.5 is a compact 8.3B-parameter DiT for open-source video generation that competes with leading proprietary systems.It is progressively trained from a text-to-image model.Source: §7 · p. 11
• Its multi-stage pipeline combines meticulous data curation, SSTA for accelerated inference, and video super-resolution for high-resolution output.The system supports text-to-video and image-to-video synthesis.Source: §7 · p. 11
• The model delivers visual quality, motion coherence, and multimodal alignment, with bilingual prompt understanding, accurate text rendering, and reliable instruction-following.Source: §7 · p. 11
• Comprehensive evaluations establish HunyuanVideo 1.5 as a new standard among open-source video generators, and its release aims to lower barriers to creative and research applications.Source: §7 · p. 11