Source-linked AI summary
MAGI-1: Autoregressive Video Generation at Scale
Sand. ai, Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, W. Q. Zhang, Weifeng Luo, Xiaoyang Kang, Yuchen Sun, Yue Cao, Yunpeng Huang, Yutong Lin, Yuxin Fang, Zewei Tao, Zheng Zhang, Zhongshu Wang, Zixun Liu, Dai Shi, Guoli Su, Hanwen Sun, Hong Pan, Jie Wang, Jiexin Sheng, Min Cui, Min Hu, Ming Yan, Shucheng Yin, Siran Zhang, Tingting Liu, Xianping Yin, Xiaoyu Yang, Xin Song, Xuan Hu, Yankai Zhang, Yuqiao Li
TL;DR
MAGI-1 targets the limitations of full-sequence video diffusion for causal, streaming-compatible generation. It autoregressively denoises fixed-length chunks with supporting architectural and infrastructure innovations, and reports stronger prompt adherence, physical plausibility, and temporal coherence on VBench-I2V and Physics-IQ. Its main stated limitation is that one decoder-style Transformer couples high-level temporal reasoning with low-level visual denoising.
Problem
Existing large-scale video diffusion models process entire sequences with globally conditioned denoising, limiting causal temporal modeling and streaming-compatible inference.
Method
MAGI-1 autoregressively generates fixed-length video chunks using progressive chunk-level denoising, causal temporal attention, distributed attention, and scalable inference infrastructure.
Results
MAGI-1 outperforms existing large-scale video diffusion models on VBench-I2V and Physics-IQ in prompt adherence, physical plausibility, and temporal coherence.
Takeaways & Limitations
MAGI-1 provides a shared, controllable foundation for text-to-video, image-to-video, and video continuation with fixed computational overhead regardless of video length.
Takeaways & Limitations
MAGI-1 couples high-level temporal context fusion and low-level visual denoising within a single large decoder-style Transformer.
Abstract
from arXiv · showhide
We present MAGI-1, a world model that generates videos by autoregressively predicting a sequence of video chunks, defined as fixed-length segments of consecutive frames. Trained to denoise per-chunk noise that increases monotonically over time, MAGI-1 enables causal temporal modeling and naturally supports streaming generation. It achieves strong performance on image-to-video (I2V) tasks conditioned on text instructions, providing high temporal consistency and scalability, which are made possible by several algorithmic innovations and a dedicated infrastructure stack. MAGI-1 facilitates controllable generation via chunk-wise prompting and supports real-time, memory-efficient deployment by maintaining constant peak inference cost, regardless of video length. The largest variant of MAGI-1 comprises 24 billion parameters and supports context lengths of up to 4 million tokens, demonstrating the scalability and robustness of our approach. The code and models are available at https://github.com/SandAI-org/MAGI-1 and https://github.com/SandAI-org/MagiAttention. The product can be accessed at https://sand.ai.
1 Introduction
MAGI-1 addresses the limitations of full-sequence video diffusion by autoregressively generating fixed-length video chunks with causal temporal modeling. Its architecture and infrastructure target temporally coherent, controllable, scalable, and streaming-compatible synthesis, with strong image-to-video benchmark performance.
- Motivation: Video generation requires temporally coherent, photorealistic sequences conditioned on inputs such as natural language, images, or video clips.The task combines spatial perception and temporal reasoning and is relevant to robotics, embodied AI, interactive media, and scientific simulation.
- Problem: Most large-scale video diffusion models denoise entire temporal sequences simultaneously with uniform noise and full-sequence inference access.This globally conditioned design does not address causal temporal modeling or streaming-oriented generation requirements.
- Approach: MAGI-1 autoregressively generates fixed-length chunks of consecutive frames, using chunk-level progressive noise to preserve left-to-right temporal consistency.The chunk-wise formulation captures mid-range temporal dependencies while retaining causal structure and adaptable conditional generation.
- Architecture and infrastructure: MAGI-1 combines bidirectional spatial and causal temporal denoising with distributed attention, scalable execution, and adaptive data curation.MagiAttention handles ultra-long autoregressive contexts, while the infrastructure supports low-latency, parallelized inference and multi-stage training.
- Scalability: Inference-time peak resource usage is independent of total video length because each chunk uses a fixed computational and memory footprint.The largest model has 24 billion parameters and supports context lengths up to 4 million tokens.
- Evaluation: MAGI-1 achieves substantial improvements on VBench-I2V and Physics-IQ, especially for complex motion, semantic alignment, and physically plausible interactions.The evaluation focuses on prompt fidelity, temporal coherence, and subject integrity in image-to-video generation.
2 MAGI-1
MAGI-1 generates videos autoregressively in latent-space chunks, enabling unified task training, chunk-wise control, and scalable long-video inference. Its infrastructure and guidance strategies target temporal coherence, streaming, and practical deployment.
- Autoregressive generation: MAGI-1 denoises fixed-length latent chunks autoregressively, starting each subsequent chunk before the previous one is completely clean and conditioning on preceding chunks.This pipeline supports concurrent processing while preserving left-to-right temporal consistency.
- Unified multi-task training: Text-to-video, image-to-video, and video continuation share one training objective by varying the proportion of clean chunks, without task-specific fine-tuning.The same framework treats image-to-video as a continuation setting and adjusts clean-chunk proportions in the training data.
- Controllability: Chunk-wise text conditions decompose complex actions into shorter segments, supporting fine-grained control and intricate action sequences in long videos.Different text conditions can be supplied for individual chunks rather than being fixed for the entire video.
- Diffusion guidance: Increasing wprev to 1.5 alleviates adjacent-chunk misalignment and flickering, while stronger values can cause saturation artifacts or static frames.The method uses wtext = 7.5 and reports that longer videos can develop artifacts when guidance remains excessively strong.
- Limitations and mitigation: The non-distilled model develops saturation and checkerboard artifacts beyond roughly 5 seconds, while fine-grained guidance adjustment alleviates longer-video temporal artifacts.Uniformly reducing guidance degrades content quality and increases flickering, motivating stage-dependent adjustment.
- Efficient long-video inference: Caching denoised chunk features and constraining the KV range to the preceding 8 seconds makes long-video computation scale linearly with duration.Cached features are reused by later chunks; the bounded KV range limits each new chunk’s dependency history.
- Prompt enhancement: Prompt enhancement improves generation quality, and distillation into an approximately 7B model retains comparable quality while reducing latency and resource use.The distilled prompt enhancer uses a corpus of approximately 2 million examples.
- Real-time streaming: Pipelined inference makes first-chunk display time independent of total video length and reduces inter-chunk latency, enabling real-time streaming generation.The capability is paired with high-performance inference infrastructure for interactive content and live streaming.
3 DATA
MAGI-1’s data system builds and refines a large-scale video and image dataset through shot segmentation, quality filtering, and evaluation-driven distribution adjustment. Its filters target visual defects, unwanted overlays, shot transitions, and motion characteristics.
- The pipeline constructs MAGI-1’s training dataset from tens of petabytes of raw videos and images collected from diverse sources.
- Long videos are cut into single-shot clips, then filtered for low quality, duplicates, and residual problematic cases using increasingly stronger models.PySceneDetect performs initial shot cutting; subsequent processing adds filters and a multimodal large language model.
- Training-data distribution is dynamically adjusted using evaluation results because modeling difficulty varies across semantic concepts.
- Motion selection uses overall, foreground, and background motion statistics, retaining moderate-motion clips between lower and upper thresholds.RAFT optical flow is combined with saliency detection to separate foreground and background motion.
- Additional filters remove borders, excessive text and subtitles, logos, corner faces, complex transitions, poor technical quality, exposure extremes, unstable camera motion, and slide movements.The system uses edge and text detection, Florence-2, face detection, CLIP, DOVER, HSI brightness, and optical-flow-based analyses.
4 Infrastructure
MAGI-1’s infrastructure addresses ultra-long, heterogeneous attention workloads and deployment constraints through distributed attention, load balancing, communication overlap, and specialized inference optimizations. These designs support efficient training and real-time or cost-efficient serving across different GPU configurations.
- Training infrastructure: Custom online packing under data- and parallelism-specific divisibility constraints achieves 99% capacity utilization while reducing training overhead.
- MagiAttention: MagiAttention supports diverse attention masks while targeting linear scalability with respect to context-parallel size across broad scenarios.
- MagiAttention: Flex-Flash-Attention extends FlashAttention-3 with distributed mask representations and mask-aware partitioning to preserve attention-kernel scalability.It is designed for complex mask patterns arising from block-causal attention and packing-and-padding.
- MagiAttention: MagiAttention balances computation across context-parallel ranks and overlaps communication with computation using zero-redundant primitives and adaptive multistage scheduling.
- Training infrastructure: The proposed framework identifies tight coupling between model implementation and parallelization as a collaboration barrier and extends DTensor and Parallel Plan to decouple them.
- Inference infrastructure: The inference stack enables real-time streaming for the 24B model on 3 nodes with 24 H100 GPUs and supports deployment of the 4.5B model on one RTX 4090 and the 24B model on 8 RTX 4090 GPUs.The design uses multi-chunk parallel inference and KV cache for streaming and cost-efficient deployment.
5 Evaluation
MAGI-1 is evaluated with complementary perceptual and physical-dynamics protocols, including an in-house I2V benchmark and Physics-IQ. Results show strong motion, instruction-following, physical modeling, and competitive overall quality, while complex secondary physical effects remain challenging.
- Evaluation framework: The evaluation combines perceptual-quality assessment with physical-dynamics evaluation to characterize both strengths and limitations.The protocol uses human and automated evaluation for perceptual quality and Physics-IQ for physical-rule understanding.
- Evaluation framework: The in-house I2V benchmark evaluates Overall, Motion Quality, Instruction Following, and Visual Quality across a diverse image-prompt dataset.Its metric design emphasizes comprehensiveness, simplicity, and orthogonality, with 100 image-prompt pairs selected for challenging generation conditions.
- Perceptual evaluation: MAGI-1 outperforms Wan-2.1 and clearly exceeds Hailuo(i2v-01) and HunyuanVideo overall, while remaining slightly behind Kling1.6 (HD).It performs particularly strongly on instruction following and motion quality, but visual quality still trails top models.
- Perceptual evaluation: MAGI-1 (2× decoder) reaches a top overall score of 89.28, ranking first among all models in the reported comparison.The model also shows a significant advantage in dynamic Degree while maintaining strong aesthetic quality and motion smoothness.
- Physical evaluation: Video-conditioned MAGI-1 reaches 56.02 on Physics-IQ, outperforming VideoPoet by approximately 27 points; image-conditioned MAGI-1 remains highest at 30.23.The image-conditioned score is lower than the video-conditioned score but still leads the compared models.
- Physical evaluation: MAGI-1 captures primary dynamics but struggles with precise collision responses, material-specific reactions, and post-deformation behavior.Even when predictions diverge from ground truth, the model can produce physically plausible alternatives.
- Physical evaluation: Increasing accessible historical context improves physical prediction, supporting the role of temporal information in modeling real-world dynamics.The comparison varies MAGI-1’s KV range during inference to measure the effect of history length.
6 Related Works
Prior video-generation systems have advanced quality and control, but many remain closed, globally conditioned, or limited in causal and streaming capabilities. MAGI-1 is presented as a large-scale chunk-wise autoregressive diffusion framework addressing these gaps across multiple video-generation tasks.
- Proprietary systems: Recent video-generation systems improve length, resolution, semantic fidelity, temporal fidelity, motion continuity, and stylistic control.Examples include Sora, Kling, Gen-3, DreamMachine, and Pika 1.5.
- Proprietary systems: Many leading systems remain closed-source and architecturally opaque, limiting reproducibility and extensibility.
- Open-source ecosystem: Open-source work extended latent diffusion and transformer architectures to improve frame coherence and video synthesis.The passage situates Stable Diffusion, VDM, AnimateDiff, and SVD within this development.
- Autoregressive and causal modeling: Emerging causal models support streaming or controlled denoising, but remain limited in scale, often lack chunk-wise abstraction, and do not unify continuation with I2V/T2V.
- MAGI-1: MAGI-1 is described as the first large-scale chunk-wise autoregressive diffusion model trained from scratch to unify text-to-video, image-to-video, and video continuation under strict causal constraints.It combines chunk-wise denoising, shortcut distillation, and KV-cached inference for streaming and long-horizon synthesis.
7 Conclusion
MAGI-1 provides a scalable chunk-wise autoregressive diffusion framework for unified video generation. Its reported benchmark results support strong prompt adherence, physical plausibility, and temporal coherence while retaining streaming-oriented efficiency.
- Scalable architecture: Progressively denoising fixed-length chunks under strict causal constraints enables real-time streaming generation with fixed computational overhead regardless of video length.The framework uses a Transformer with block-causal and parallel attention, distributed attention, and training for ultra-long contexts.
- Unified generation: MAGI-1 unifies text-to-video, image-to-video, and video continuation under one training objective without task-specific modifications.Chunk-wise text conditioning provides fine-grained semantic control, while shortcut distillation reduces diffusion steps while maintaining temporal consistency and sample quality.
- Empirical results: MAGI-1 outperforms existing large-scale video diffusion models on VBench-I2V and Physics-IQ in prompt adherence, physical plausibility, and temporal coherence.
8 Limitation and Future Work
The current MAGI-1 architecture couples high-level temporal-semantic reasoning with low-level visual denoising in one large decoder-style Transformer. The authors identify efficiency, optimization, controllability, and future video-understanding challenges motivating a more modular design.
- Current limitations: A single decoder-style Transformer handles both temporal context fusion and fine-grained visual denoising, leaving heterogeneous objectives tightly coupled.
- Current limitations: Repeatedly invoking the same large model across denoising steps creates an inference-latency bottleneck, especially for low-latency streaming.
- Current limitations: Joint semantic planning and pixel-level restoration can cause objective interference and suboptimal scaling behavior.
- Current limitations: The monolithic architecture limits insertion of auxiliary controls such as confidence-based guidance modulation and dynamic temporal constraints.
- Future work: The authors propose structurally separating high-level semantic reasoning from low-level visual synthesis as video generation advances toward coherent long-form content.They identify video understanding as an increasingly important bottleneck in a future generation-understanding loop.
Contributions and Acknowledgments
This section lists contributors and describes the A8W8 SmoothQuant calibration and FP8 quantization setup used for video generation.
- Contributions and Acknowledgments: Names are presented alphabetically by first name, with separate lists for core contributors and contributors.The core-contributor list includes Hansi Teng, Hongyu Jia, Lei Sun, and others; the contributor list includes Dai Shi, Guoli Su, and others.
- Quantization: A8W8 SmoothQuant pre-computes per-channel scaling factors from a calibration dataset and applies an equivalent transformation to activations and weights.The transformation is Y = (X · diag(s)−1) · (diag(s)W), mitigating channel-wise activation outliers.
- Quantization: The calibration dataset spans T2V and I2V scenarios, samples step sizes from [12, 32], and contains approximately 30% I2V samples.The quantization procedure uses FP8 because INT8 introduced noticeable visual artifacts, with α searched over (0.4, 0.6).
A.2 Multi-Node Parallel Inference
The multi-node inference framework overlaps communication with computation and uses CSO to improve chunk-level scheduling and communication balance on PCIe-connected GPUs.
- Ulysses Parallel Inference: A 3-node, 24-GPU Ulysses-based inference framework identifies inter-GPU communication and attention computation as primary bottlenecks.Ulysses performs four all-to-all communication steps for q, k, v, and o tensors.
- Ulysses Parallel Inference: Communication stages overlap with corresponding computations, including v with k-computation, k with q-computation, q with KV-cache updates, and o with cross-attention.These overlaps target the four Ulysses all-to-all communication steps.
- Ulysses Parallel Inference: Less than 3% of total execution time remains communication overhead after applying the overlapping strategy.The reported reduction concerns the multi-node inference framework’s communication overhead.
- Context Shuffle Overlap: CSO assigns each rank a partial view of every chunk, unlike Ulysses’ sequential chunk distribution, enabling chunk-level overlap of computation and communication.For five chunks, CSO overlaps q, k, v, and o communications with computations and KV-cache updates across neighboring chunks.
- Context Shuffle Overlap: CSO splits communication into multiple balanced all-to-all operations, which perform better than unbalanced communication and support efficient subsequent merging.The cited example includes overlap involving later chunk communications and output computation.
B.1 MagiAttention Materials
MagiAttention combines mask-aware distributed-attention primitives with load balancing and dynamic overlap scheduling for heterogeneous, long-context workloads.
- Distributed Attention Primitives: AttnSlice decomposes irregular masks into multiple slices and rearranges them across context-parallel ranks for distributed attention.The illustration explicitly does not consider computation-load balance across context-parallel ranks.
- Distributed Attention Primitives: FFA requires atomic reductions because slices overlap across both QRange rows and KRange columns in forward and backward kernels.The overlap occurs in both dimensions of the slice-level parallelism layout.
- Distributed Attention Primitives: Ring-Attention uses sequential or zigzag sharding for full, causal, and variable-length masks, with fragmentation and padding degrading variable-length causal performance.The sharding strategy differs by mask type and by whether sequences are packed.
- Distributed Attention Primitives: Ring P2P incurs 25% redundant communication for a simple causal mask and over 33% for a variable-length block-causal mask with a final global block.Group-cast and group-reduce are presented as zero-redundancy alternatives using all-to-all-v communication and range gathering.
- Overlap Scheduling: MagiAttention overlaps partial attention computation with KV prefetching and dKV reduction through four forward stages and three backward stages.The scheduling design exposes computation in the final stage while hiding communication overhead where applicable.
- Load Balancing: The greedy dispatch algorithm sorts chunks by area and assigns them to the least-loaded non-full bucket using a min-heap.It returns bucket assignments and the maximum bucket workload.
- Overlap Scheduling: Dynamic overlap-stage search partitions remote KV and dKV requests into fine-grained packages, evaluates candidate stage counts, and synchronizes selected counts across ranks.The search uses offline-profiled cost models for FFA, group-cast, and group-reduce, with separate forward and backward stage counts.
B.2.1 Benchmarking MagiAttention kernel-level performance and flexibility
The FFA benchmark measures forward and backward throughput across sequence lengths and mask patterns, including variable-length and irregular cases, using standardized FLOP calculations and fixed settings.
- Benchmark Scope: FFA is evaluated against leading attention kernels for forward and backward throughput on Hopper GPUs across standard, variable-length, and irregular mask patterns.The benchmark targets ultra-long, heterogeneous-mask training and reports throughput in TFLOPs/s.
- Benchmark Settings: Sequence lengths vary from 4k through 128k with seqlenq = seqlenk = seqlen, while other configurations remain fixed.Variable-length packed data follows the sequence-length distribution of an open-sourced dataset.
- Throughput Calculation: Forward FLOPs are computed as 2 matmuls × 2 flops per matmul × MaskArea(seqlen, mask_type), while backward FLOPs are 2.5 times forward FLOPs.The equations follow Flash-Attention’s FLOP-counting convention and account for backward recomputation.
- Benchmark Settings: Table 11 provides the fixed settings used for the FFA performance and flexibility benchmark.These fixed settings support comparisons focused on sequence length and mask pattern.
- Benchmark Results: Figures 27–31 benchmark full, causal, variable-length full, variable-length causal, and sliding-window causal mask scenarios.In the variable-length and sliding-window figures, E denotes a distributed-attention implementation that raises a CUDA out-of-memory error.
B.2.2 Benchmarking MagiAttention module-level scalability
MagiAttention is benchmarked for attention-module scalability as sequence length and context-parallel size increase, using full-mask and variable-length full-mask scenarios. It maintains linear scalability across forward and backward passes, while baseline methods face scaling limits or degrade with ultra-long contexts.
- Benchmark setup: The benchmark measures per-GPU attention throughput during forward and backward propagation as sequence length and parallel size increase.Experiments compare MagiAttention with Ring-Attention and Ulysses under full-mask and variable-length full-mask settings.
- Benchmark setup: Experiments scale sequence length from 64k to 3M tokens and context-parallel size from 1 to 48, with tensor-parallel size fixed at 8.The corresponding node size also increases from 1 to 48, while sequence parallelism remains enabled.
- Benchmark setup: Each rank uses 64k attention sequence length with 8 query heads and 1 key head, while remaining activations use 8k sequence length with sequence parallelism.This configuration is intended to simulate a common training setup.
- Results: MagiAttention exhibits linear scalability as context length and context-parallel size increase for both forward and backward passes.This behavior holds in both full-mask and variable-length full-mask configurations.
- Results: Baseline methods either have strict scaling limitations or lose performance with ultra-long contexts, with degradation worsening for variable-length mask patterns.The benchmark is limited to full-mask and variable-length full-mask scenarios because irregular-mask support is complex for baselines.
- Caveat: The productive GPU cluster’s specific GPU count and types are withheld for business and confidentiality reasons.This limits the publicly specified hardware context for reproducing the benchmark.