Wan: Open and Advanced Large-Scale Video Generative Models

1 INTRODUCTION

• Wan targets persistent open-source gaps in performance, task coverage, and computational efficiency relative to leading closed-source video models.The introduction attributes these gaps to commercial models’ faster development, foundational models’ focus on text-to-video, and demanding computational requirements.Source: §1 · p. 3
• Open-source releases such as HunyuanVideo (Kong et al., 2024), Mochi, and CogVideoX (Yang et al., 2025b) have narrowed but not eliminated the gap with commercial video models.The introduction identifies these projects as publicly releasing foundation-model code and weights while noting continuing commercial advantages.Source: §1 · p. 3
• 14B-parameter Wan is trained on billions of images and videos totaling O(1) trillions of tokens, supporting capabilities across motion, visual text, camera control, adherence, and style.Its architecture combines Diffusion Transformers, Flow Matching, cross-attention text conditioning, computational optimization, and full spatio-temporal attention.Source: §1 · p. 3
• Wan extends beyond text-to-video to image-to-video, instruction-guided video editing, personalized customization, real-time generation, audio generation, and other downstream applications.The report describes this expansion as building on the foundational model’s capabilities.Source: §1 · p. 3
• 8.19G VRAM enables the 1.3B model to run on many consumer-grade GPUs, while its performance exceeds that of many larger open-source models.Wan provides both 1.3B and 14B models for text-to-video and image-to-video at 480p, with the smaller model reducing inference costs.Source: §1 · p. 3
• Wan publicly presents its complete training process, including data construction, the video VAE, training strategies, acceleration techniques, and automated evaluation algorithms.The report also promises comprehensive design details and experimental results to support community development of specialized video foundation models.Source: §1 · p. 3

Closed-source models

• Closed-source video models from major technology companies have rapidly expanded from Sora to updated systems such as Kling 1.6 and Veo 2 (DeepMind, 2024.12).The reviewed releases emphasize high-quality, professional video generation and increasingly support visual customization, physics understanding, and nuanced human movement.Source: §2 · pp. 4–5
• Wan achieves competitive or superior performance to commercial video models across internal and external benchmarks, leading in multiple attributes.This comparison follows intense global competition among systems including Sora, Kling, Luma, Runway, Vidu, Hailuo, Pika, Movie Gen, and Veo 2.Source: §2 · pp. 4–5

Contributions from open-source community

• Open-source video generation commonly combines an autoencoder, text encoder, and diffusion network, with DiT and MM-DiT increasingly replacing U-Net-based designs.Recent work also advances VAE reconstruction and compression, decoder-based detail generation, and text-video alignment through T5, CLIP, or multimodal language models.Source: §2 · p. 5
• Wan carefully designs or selects each critical component and provides design details and ablations to support future open-source video model development.The broader open-source community has produced multiple diffusion-based video models by integrating these modules with effective diffusion optimization techniques.Source: §2 · p. 5

Downstream tasks

• Video-generation research also covers repainting, editing, controllable generation, and frame-reference generation, often using adapter-based or ControlNet-like conditioning.Wan develops downstream applications for these tasks and reports excellent performance.Source: §2 · p. 5

3 DATA PROCESSING PIPELINE

• The pipeline prioritizes quality, diversity, and scale, curating billions of videos and images through automated data construction.These principles define the overall data-processing pipeline for Wan.Source: §3 · p. 5

3.1 PRE-TRAINING DATA

• Pre-training data is deduplicated and filtered through fundamental, visual-quality, and motion-quality processing before semantic-driven selection.Fundamental filtering removes approximately 50% of the initial dataset, while clustering preserves long-tail distributions and motion tiers control sampling or exclusion.Source: §3.1 · pp. 6–7, Figure 3 · p. 7

3.3 DENSE VIDEO CAPTION

• Wan improves visual-text generation by combining hundreds of millions of synthetic Chinese-text images with real text-containing data processed by OCR and Qwen2-VL.The combined data supports rare-word generation with accurate glyphs and high realism.Source: §3.1 · p. 7

3.2 POST-TRAINING DATA

• Post-training separately refines image visual quality and video motion quality, selecting diverse, category-balanced data across 12 major categories.The image set includes millions of curated samples, while videos include millions with simple and complex movements.Source: §3.2 · pp. 7–8

3.3.1 OPEN SOURCE DATASET

• The captioning corpus combines open-source image, video, visual Q&A, and instruction datasets to support detailed, controllable visual descriptions.The collected sources cover actions, counting, OCR, and user-specified caption styles or content.Source: §3.3.1 · p. 8

3.3.2 IN-HOUSE DATASET

• In-house caption data targets identities, counting, OCR, camera motion, fine-grained categories, relations, editing, grouped images, and human-annotated dense captions.Camera-angle and motion annotations improve controllability, while human-annotated captions provide the highest-quality data for final training.Source: §3.3.2 · pp. 8–9

3.3.4 EVALUATION

• The caption model outperforms Gemini 1.5 Pro on video event, camera angle, camera motion, style, and object color, while Gemini leads on five other dimensions.Evaluation uses F1 scores over ten visual dimensions on 1,000 videos, with Fig. 4 reporting the comparison.Source: Figure 4 · p. 9, §3.3.4 · p. 10

3.3.3 MODEL DESIGN

• The caption model uses a LLaVA-style ViT–Qwen architecture with dynamic-resolution images, slow-fast video encoding, and three-stage training.Sampling 3 frames per second with up to 129 frames, slow-fast encoding improves VideoMME performance from 67.6% to 69.1% without subtitles.Source: §3.3.3 · pp. 9–10

4.1.1 MODEL DESIGN

• Wan-VAE compresses a video’s spatio-temporal dimensions by 4 × 8 × 8 while expanding the latent channels to 16, with the first frame receiving only spatial compression.The architecture maps an input of 1 + T frames and spatial dimensions H × W to latent dimensions [1 + T/4, H/8, W/8].Source: Figure 5 · p. 10, §4.1.1 · p. 10

4.1 SPATIO-TEMPORAL VARIATIONAL AUTOENCODER

• The 3D causal VAE addresses video generation challenges from spatio-temporal dependencies, high-dimensional memory costs, and the need to prevent future frames from influencing past frames.Its design combines spatio-temporal compression, reduced memory usage, and temporal causality for scalable diffusion-based generation.Source: §4.1 · p. 10

4.1.1 MODEL DESIGN

• Replacing GroupNorm with RMSNorm preserves temporal causality and enables feature caching, while halving spatial-upsampling input channels reduces inference memory consumption by 33%.Careful base-channel tuning further yields a 127M-parameter model that reduces encoding time and memory usage.Source: §4.1.1 · p. 11

4.1.2 TRAINING

• Wan-VAE uses staged training that inflates a pretrained 2D image VAE into a 3D causal model, begins with 128×128 five-frame videos, and then fine-tunes on varied high-quality videos with 3D-discriminator GAN loss.Training combines L1 reconstruction, KL, and LPIPS losses weighted by 3, 3e-6, and 3, respectively.Source: §4.1.2 · p. 11

4.1.4 EVALUATION

• 2.5 times faster reconstruction than HunYuan Video, while achieving highly competitive video quality and processing efficiency across 200 videos at 720 × 720 resolution.The evaluation uses 25-frame videos and compares PSNR and frames per second under matched or specified compression settings; circle size in Fig. 7 indicates parameter count.Source: Figure 7 · p. 12, §4.1.4 · p. 12

4.1.3 EFFICIENT INFERENCE

• Feature caching processes arbitrarily long videos chunk-wise, limiting each chunk to at most 4 frames to prevent memory overflow while preserving continuity across chunk boundaries.Cached historical features are reused in causal convolutions, supporting stable inference for infinite-length videos.Source: §4.1.3 · pp. 11–12, Figure 6 · p. 11

4.1.4 EVALUATION

• Wan-VAE preserves texture, facial, textual, and high-motion details with reduced blur, distortion, character loss, and motion degradation.Figure 8 evaluates reconstruction across texture, face, text, and high-motion scenes using videos not used in training.Source: §4.1.4 · pp. 12–13, Figure 8 · p. 13

4.2 MODEL TRAINING

• Wan’s architecture combines Wan-VAE, a diffusion transformer, and a text encoder, with the VAE mapping videos into a spatiotemporal latent space.The encoder maps V ∈ R^(1+T)×H×W×3 into x ∈ R^(1+T/4)×H/8×W/8.Source: §4.2 · pp. 13–14

4.2.1 VIDEO DIFFUSION TRANSFORMER

• The diffusion transformer uses 3D patchification, cross-attention for text conditioning, and shared time-embedding modulation across blocks, reducing parameters by approximately 25% while improving performance at the same scale.Its sequence length is L = (1 + T/4) × H/16 × W/16, and cross-attention supports instruction following under long-context modeling.Source: §4.2.1 · p. 14
• Wan uses umT5 (Chung et al., 2023) as its text encoder because it supports Chinese, English, and visual text, improves composition over unidirectional LLMs, and converges faster at the same parameter scale.The text embedding sequence contains 512 tokens during training.Source: §4.2.1 · p. 14, §4.2.2 · p. 15

4.2.2 PRE-TRAINING

• Pre-training uses flow matching across images and videos, beginning with 256 px text-to-image training and progressing through joint image-video stages at 256, 480, and 720 px.The curriculum uses 5-second videos, first at 192 px and 16 fps, then at 480 px and 720 px; low-resolution initialization addresses throughput, memory, and training-stability constraints.Source: §4.2.2 · pp. 14–15

4.2.3 POST-TRAINING

• Post-training initializes from the pre-trained checkpoint while retaining the architecture and optimizer configuration and jointly trains on 480 px and 720 px video data.The post-training dataset is the video dataset detailed in Sec. 3.2.Source: §4.2.3 · p. 15

4.3.1 WORKLOAD ANALYSIS

• At 1 million tokens, attention can consume up to 95% of end-to-end training time, making it Wan’s primary computational bottleneck as sequence length grows quadratically.The DiT accounts for more than 85% of overall training computation, while GPU activation memory scales linearly and can exceed 8 TB for a 14B model at batch size 1.Source: §4.3.1 · pp. 15–16

4.3.2 PARALLELISM STRATEGY

• 2D Context Parallelism combines Ring Attention and Ulysses, reducing communication overhead from over 10% to below 1% at 256K tokens on 16 GPUs.Wan combines FSDP for parameter sharding with Context Parallelism for activation sharding, then uses Data Parallelism for scaling after memory and latency requirements are satisfied.Source: §4.3.2 · p. 16
• Wan switches between DP for the VAE and text encoder and DP plus CP for DiT, reducing redundant upstream computation to 1/CP of an iteration.CP-group devices initially read different data, then broadcast it before CP so all devices receive identical inputs without duplicating VAE and text-encoder work.Source: §4.3.2 · pp. 16–17

4.3.3 MEMORY OPTIMIZATION

• Activation offloading overlaps PCIe transfers with computation, while gradient checkpointing handles CPU-memory pressure and layers with high GPU-memory-to-computation ratios.For one DiT layer, offloading transfers can overlap computation of 1 to 3 DiT layers, avoiding end-to-end performance loss in long-sequence training.Source: §4.3.3 · p. 17

4.3.4 CLUSTER RELIABILITY

• Alibaba Cloud’s training cluster improves stability through healthy-node allocation, faulty-node isolation and repair, automatic task restart, and seamless training resumption.These scheduling and self-healing mechanisms combine reliability with high performance during distributed training.Source: §4.3.4 · p. 17

4.4 INFERENCE

• Inference optimization targets the roughly 50 sampling steps of video generation by reducing per-step computation through quantization, distributed computing, attention reuse, and classifier-free-guidance reuse.These techniques address the latency and computational burden introduced by iterative sampling.Source: §4.4 · p. 17

4.4.1 PARALLEL STRATEGY

• Wan 14B achieves nearly linear inference speedup across multiple GPUs using 2D Context Parallelism with FSDP model sharding.RingAttention forms the outer context-parallel loop, Ulysses the inner loop, and FSDP reduces communication overhead while overlapping computation.Source: Figure 12 · p. 17, §4.4.1 · p. 17

4.4.2 DIFFUSION CACHE

• Diffusion caching improves Wan 14B text-to-video inference performance by 1.62× while preserving lossless performance.Attention outputs and classifier-free-guidance computations are cached across selected sampling steps, with residual compensation used to preserve fine details.Source: §4.4.2 · p. 18

4.4.3 QUANTIZATION

• FP8 GEMM provides twice the performance of BF16 GEMM and yields a 1.13× speedup in the DiT module with minimal performance loss.Wan applies per-tensor weight quantization and per-token activation quantization during sampling; native FP8 FlashAttention3 caused significant quality degradation, motivating the 8-bit mixed-precision design.Source: §4.4.3 · p. 18
• With 8-bit FlashAttention, inference efficiency improves by more than 1.27×, and after block-size optimization to reduce register spilling, it achieves 95% MFU on NVIDIA H20 GPUs.The implementation uses Int8 for S = QKT and FP8 for O = PV, Float32 cross-block accumulation, and fused Float32 accumulation with the rescale operation.Source: §4.4.3 · pp. 18–19

• Wan uses caption augmentation and user-prompt rewriting to align inference inputs with the format and style of training captions.Caption augmentation increases diversity, while rewriting addresses mismatches between user prompts and training captions.Source: §4.5 · p. 19
• Multiple captions span different lengths and styles, covering varied text-to-video mappings and the range of prompts users may provide.Examples include long, medium, and short captions in formal, informal, and poetic styles.Source: §4.5 · p. 19
• LLM rewriting addresses users’ typically short prompts by adding detail without changing meaning and by incorporating category-appropriate natural motion.The rewritten structure follows post-training captions: video style, content abstract, then detailed description.Source: §4.5 · p. 20
• Benchmark results indicate that instruction-following LLMs can generate prompts that enhance video generation, with Qwen2.5-Plus chosen to balance speed and performance.Source: §4.5 · p. 20

Benchmark design

• Wan-Bench evaluates video generation across dynamic quality, image quality, and instruction following with 14 fine-grained metrics.It combines traditional detectors for simpler tasks with multimodal large language models for more complex assessments.Source: §4.6 · p. 20

Dynamic quality

• Dynamic-quality metrics assess motion, artifacts, physical plausibility, smoothness, pixel stability, and identity consistency.They use optical flow, a human-annotated YOLOv3-based artifact detector, Qwen2-VL, and frame-level DINO features.Source: §4.6 · pp. 20–21

Image quality

• Image-quality metrics combine MANIQA, LAION-based aesthetic prediction, MUSIQ, and CLIP-based scene consistency and text alignment.Stylization is evaluated with Qwen2-VL through frame-level question answering.Source: §4.6 · p. 21

Instruction following

• Instruction-following metrics evaluate object attributes, spatial relationships, camera movements, and action execution against textual prompts.The suite uses Qwen2-VL, RAFT optical flow, and keyframe-based queries to assess alignment, completion, and artifacts.Source: §4.6 · p. 21

Human feedback guided weighting strategy

• Human feedback determines dimension weights for the final score rather than averaging dimensions equally.The weighting is derived from Pearson correlations between model-generated scores and human ratings from over 5,000 pairwise video comparisons.Source: §4.6 · p. 21

4.7.1 METRICS AND RESULTS

• Wan outperforms existing commercial and open-source models on Wan-Bench across dynamic quality, image quality, and instruction-following accuracy.The evaluation used 1,035 samples per candidate and weighted total scores according to human preferences; human evaluation also found Wan 14B strongest across all T2V visual-quality dimensions.Source: §4.7.1 · p. 22
• Wan generates diverse cinematic videos with complex motion, physical interactions, varied artistic styles, and accurate Chinese and English text rendering.Figure 15 documents complex motions, creative transitions, cinematic-quality videos, and bilingual text generation from text prompts.Source: §4.7.1 · p. 22, Figure 15 · p. 23
• 86.22% aggregate VBench score, including 86.67% visual quality and 84.44% semantic consistency, makes Wan 14B benchmark-leading; Wan 1.3B scores 83.96%.Wan 14B significantly outperforms OpenAI’s Sora and MiniMax’s Hailuo, while Wan 1.3B surpasses HunyuanVideo, Kling 1.0, and CogVideoX1.5-5B.Source: §4.7.1 · pp. 22, 24, Table 4 · p. 24

4.7.2 ABLATION STUDY

• Full-shared-AdaLN-1.5B achieves the lowest training loss among equal-sized variants, supporting deeper networks over additional AdaLN parameters and motivating fully shared AdaLN.The ablation used 1.3B-scale models trained for 200,000 text-to-image steps, with latent-space L2 training loss as the convergence measure.Source: §4.7.2 · pp. 24–25
• umT5 yields more favorable training losses than Qwen2.5-7B-Instruct and GLM-4-9B, while Qwen-VL-7B second-last-layer features achieve comparable generation performance at larger model size.A bidirectional token refiner was added to the decoder-only LLM encoders, but umT5 remained most favorable.Source: §4.7.2 · p. 25
• VAE consistently achieves lower FID than VAE-D at both 100,000 and 150,000 training steps.VAE-D replaces reconstruction loss with diffusion loss; both autoencoders were evaluated on text-to-image generation for 150,000 steps.Source: §4.7.2 · pp. 25–26

5.1.3 EVALUATION

• Wan-I2V performs favorably across visual quality, motion quality, and matching, while visualizations show realistic, dynamic animation of diverse input images.Table 7 reports pairwise preferences against state-of-the-art techniques; Figures 19 and 20 provide additional qualitative results.Source: Figure 19 · p. 28, Figure 20 · p. 29, §5.1.3 · p. 30

5.2.1 MODEL DESIGN

• VACE unifies controllable video generation and editing by encoding text, frames, and masks into context tokens, while concept decoupling separates editable and preserved content.The framework supports full Wan fine-tuning or a Context Adapter that integrates context without modifying base-model weights, enabling diverse editing and generation tasks.Source: §5.2 · pp. 30–31, §5.2.1 · pp. 31–32

5.2.3 EVALUATION

• VACE achieves high video quality and temporal consistency across diverse tasks, supports capability combinations such as long-video re-rendering, and also applies to image generation and editing.The Wan single model’s results are shown in Figures 22–24, including compositional applications and image-domain extensions.Source: Figure 22 · p. 32, Figure 23 · p. 33, Figure 24 · p. 34

5.4.1 MODEL DESIGN

• Wan personalization directly conditions video generation on face images in Wan-VAE latent space, using masked prepended frames and self-attention to preserve user identity without modifying Wan-T2V.Training randomly drops reference faces to support generation with zero to K references; the dataset contains about O(10)M personalized videos plus about O(1)M videos with synthesized faces.Source: §5.4.1 · pp. 35, 37, §5.4.2 · p. 37

5.3 TEXT-TO-IMAGE GENERATION

• Wan’s joint image-video training transfers knowledge across modalities, with an image dataset nearly ten times larger than its video dataset supporting high-fidelity synthesis across varied image categories.The reported categories include artistic text visuals, photorealistic portraits, creative designs, and professional product photography.Source: Figure 25 · p. 36

5.4.3 EVALUATION

• The personalization model demonstrates competitive performance against leading commercial and closed-source Chinese competitors on unseen-set ArcFace similarity evaluation.Similarity is averaged over faces sampled at 1 FPS from generated videos, with qualitative results shown in Figure 27 and comparisons in Table 8.Source: §5.4.3 · pp. 37–38, Table 8 · p. 37

5.6 REAL-TIME VIDEO GENERATION

• The real-time video system targets interactive environments where conventional methods are slowed by costly, multi-minute generation of short clips.The motivation includes live-streaming, gaming, interactive entertainment, virtual reality, and video production workflows requiring rapid feedback.Source: §5.6 · p. 38

5.6.1 METHOD

• The real-time pipeline adapts a pretrained Wan model from fixed-length clips to continuous streams using incremental token generation and speed optimization.A denoising queue dequeues the oldest token while appending a new one, removing predefined length constraints and targeting real-time playback.Source: §5.6.1 · p. 40

5.6.2 STREAMING VIDEO GENERATION

• Streamer enables arbitrary-length video generation by restricting attention to a sliding temporal window and continuously denoising queued tokens.Cached tokens are reintroduced across windows to maintain smooth transitions and temporal consistency.Source: §5.6.2 · pp. 41–44

5.6.3 CONSISTENCY MODEL DISTILLATION

• 10 - 20 × inference acceleration enables Streamer to render at 8 - 16 FPS while preserving streaming and temporal-coherence capabilities.The integration uses LCM and VideoLCM to distill diffusion and classifier-free guidance into a four-step consistency model.Source: §5.6.3 · p. 44

5.7 AUDIO GENERATION

• The V2A framework synthesizes ambient sound and background music from video and text while excluding speech and vocal elements.Its model uses a flow-matching DiT, a waveform-based 1D-VAE preserving temporal information, synchronized CLIP video features, and multilingual umT5 text features.Source: §5.7 · pp. 44–45, §5.7.1 · p. 45

5.7.1 MODEL DESIGN

• The V2A model generates up to 12-second stereo audio at 44.1 kHz from multimodal inputs using 48 aligned video frames and a 1536-dimensional latent space.Training masks audio captions randomly to strengthen visual-to-audio associations while retaining optional textual control; data comprises O(1) thousand filtered hours with dense, ambient-sound, and music captions.Source: §5.7.1 · pp. 45, 47

5.7.2 EVALUATION

• Our V2A model produces more temporally consistent, cleaner, rhythmic, and coherent audio than MMAudio across the reported qualitative examples.It improves long-term consistency for pouring coffee, cleanliness for typing, and rhythmic sound synthesis for walking, boxing, and clip-clop.Source: §5.7.2 · p. 47
• The V2A method cannot reliably generate human vocal sounds because speech-related vocal data was excluded during training.The authors identify laughter, crying, and speech as limitations and plan to add speech generation in future work.Source: §5.7.2 · p. 47

Limitation

• Wan still struggles to preserve fine-grained details during large motion, reflecting a broader unresolved challenge in video generation.The report calls for substantial efforts to improve fidelity under large-scale motion dynamics.Source: §6 · p. 47
• Inference for the 14B model requires approximately 30 minutes on a single high-end GPU, while domain-specific performance may remain insufficient in areas such as education and medicine.The authors identify efficiency, scalability, and community-driven specialization as priorities for broader access and deployment.Source: §6 · p. 47

Conclusion

• Wan is publicly released with analyses of its Wan-VAE and DiT architectures, training, data curation, evaluation, preprocessing, and downstream applications.The report presents image-to-video, video editing, and personalized video generation as demonstrations of Wan’s versatility and practical utility.Source: §6 · pp. 47–48
• The 1.3B model achieves competitive performance against larger counterparts and enables inference on consumer-grade GPUs, improving accessibility for content creators.The authors plan to scale both data and model architectures to address continuing video-generation challenges.Source: §6 · p. 48

• Authors are listed alphabetically by their first names.Source: §7 · p. 49
• The contributor roster includes the authors listed from Pandeng Li through Ziyu Liu.Source: §7 · p. 49