LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity

1. Introduction

• DiTs’ quadratic cost in video resolution and length makes long-video generation expensive, and most existing models produce only 10–20 seconds.High-resolution video generation also requires predicting hundreds of millions of pixels while maintaining consistency across frames.Source: §1 · p. 1
• Linear-complexity SSMs such as Mamba can disrupt video quality because 3D-to-1D token rearrangement separates spatially and temporally adjacent tokens.Existing rearrangements struggle to ensure consistency when scaled to high-resolution, long-video generation.Source: §1 · p. 1
• At 68s and 512p, LinGen achieves 15× speed-up over standard DiT.The authors also report high-quality videos with linear scaling and up to 15× speed-up for minute-length videos at 16 fps.Source: Figure 1 · p. 2, §1 · p. 2
• LinGen replaces DiT self-attention with MATE, combining bidirectional Mamba2, Rotary-Major Scan, and review tokens with Temporal Swin Attention.The branches target short-to-long-range correlations and spatially or temporally adjacent tokens while maintaining linear complexity.Source: §1 · p. 2
• Human and automatic evaluations compare LinGen with DiT-4B and other video-generation models, with results indicating high-quality video and linear scaling.The introduction reports up to 15× speed-up for minute-length videos at 16 fps.Source: §1 · p. 2

2. Related Work

• Sora, Runway Gen3, LumaLabs, and Kling generate high-resolution videos with high consistency, while MovieGen [41] scales a DiT to 30 billion parameters.The passage notes that MovieGen’s quadratic complexity makes minute-length generation difficult.Source: §2 · p. 2
• MATE replaces self-attention with linear-complexity MA and TE branches; the caption describes the MA branch’s Mamba2, RMS, and review tokens, and TE’s Temporal Swin Attention.The TE branch addresses adjacency preservation and improves video consistency.Source: Figure 2 · p. 3
• Efficiency approaches reduce sampling steps or denoising cost, but factorized attention remains quadratic and Mamba-based designs retain global attention or generate only short, low-resolution videos.The passage also contrasts LinGen’s adjacency handling and minute-length video generation with these approaches.Source: §2 · p. 3
• Early minute-length video-generation works [65, 66] are limited by low frame rates, low resolution, or reduced quality from extension-based generation.Source: §2 · p. 3

3.1. Overview

• MATE replaces vanilla DiT self-attention with a linear-complexity block whose MA and TE branches cover short-, medium-, and long-range correlations.LinGen's quadratic self-attention cost motivates this replacement.Source: §3 · p. 3, §3.1 · p. 4

3.2. MA-Branch: Targets Short-to-Long Range

• Bidirectional Mamba2 supplies a complete correlation map, while Rotary-Major Scan alternates four spatial and temporal row- and column-major orders to preserve adjacency.RMS incurs negligible extra latency on large 3D video token tensors, unlike the significant rearrangement latency of Zigzag scan.Source: §3.2 · p. 4
• For extremely long sequences, pooled review tokens prepend a sequence overview for Mamba2, targeting better video understanding and text-video alignment.The review tokens add no parameters.Source: §3.2 · p. 5

3.3. TE-Branch: TEmporal Swin Attention

• TESA attends within 3D windows for adjacent spatial and medium-range temporal correlations; alternating shifts connect window boundaries while its fixed small spatial windows retain linear cost.The spatial window size is 4×4 in practice.Source: §3.3 · p. 5

3.4. Training Recipe

• LinGen-4B is progressively pretrained from 256p text-to-image to 256p–512p video and lengths up to 68 seconds, then fine-tuned on 3K high-quality videos.Text-image and text-video joint training is used during video pretraining; the authors report it improves consistency in some failure cases.Source: §3.4 · p. 5

4.1. Experimental Settings

• Experiments compare 32-layer LinGen-4B with DiT-4B and other models, pretraining on licensed Shutterstock image and video pairs and fine-tuning on 3K videos.The evaluation includes visual examples, human A/B comparisons, and VBench metrics.Source: §4.1 · p. 5, Figure 6 · p. 6, §4.3 · p. 6

4.2. Efficiency: Linear Computational Complexity

• 15× FLOPs and 11.5× latency speed-ups over DiT-4B for 512p 68s videos accompany LinGen’s linear pixel-cost scaling.Latency is measured on a single H100 GPU.Source: Figure 6 · p. 6, §4.2 · p. 6

4.3. Comparing Quality to State-of-the-Art Models

• Human evaluation favors LinGen over DiT, while VBench-Long finds performance comparable to Gen-3 and Kling and significantly better than OpenSora.LinGen also supports a higher maximum raw-frame count on a single GPU than the compared models.Source: §4.3 · p. 6, Table 1 · p. 7

4.5. Ablation Experiments

• TESA improves video quality at marginal latency, while RMS mitigates adjacency loss with negligible latency and matches Zigzag’s loss curve.Mamba2 is 25% faster than Mamba at almost the same quality; removing the MA-branch speeds generation but severely harms quality. Review tokens, hybrid training, and quality tuning are also validated, with review tokens adding marginal latency.Source: §4.5 · pp. 7–8

4.4. Adaptation to Longer Token Sequences

• Loss curves and early human evaluations indicate that LinGen adapts to longer token sequences faster than DiT during transfer from 256p to 512p generation.The checkpoints are selected after 1K pretraining steps.Source: Figure 10 · p. 8

5. Conclusion

• LinGen is a linear-complexity text-to-video framework that enables high-resolution minute-length video generation on a single GPU.MATE replaces DiT self-attention and comprises MA and TE branches.Source: §5 · p. 8
• MATE addresses Mamba’s adjacency-preservation issue and enhances short-, medium-, and long-range correlations, improving generated-video consistency and fidelity.Source: §5 · p. 8
• LinGen achieves up to 11.5× latency speed-up while maintaining high video quality, and the authors identify hour-length and real-time generation as future directions.Source: §5 · p. 8

A. Adjacency Preservation

• RMS achieves the same average adjacent-token distance as Zigzag while being more efficient and scalable, and RMS with TESA addresses adjacency preservation.The distance is averaged over adjacent tokens in a 2 × 2 × 2 cube for an H × W × T tensor with H = W = T = 32.Source: A. Adjacency Preservation · p. 12, Figure 13 · p. 12

B. Visual Examples

• Supplementary examples include 17-second and 68-second LinGen videos, comparisons with open-source, commercial, and minute-length models, and module ablations.The ablations cover TESA, RMS, review tokens, hybrid training, and quality tuning.Source: B. Visual Examples · p. 12

C. Comparisons with Prior Works

• The extended comparisons add VBench results, visual examples, and human evaluations to assess LinGen against prior works and discuss VBench limitations.Source: C. Comparisons with Prior Works · p. 12

C.1. Automatic Metrics: VBench Results

• The supplementary evaluation compares LinGen on VBench-Long, standard VBench, and VBench-Custom, noting that most standard-leaderboard models generate videos shorter than five seconds.Source: §C.1 · p. 12
• VBench-Custom scores are similar at 256p and 512p, although human evaluation gives 512p videos a much higher quality win rate.The authors also report that VBench results do not perfectly align with human preference.Source: §C.1 · p. 12

• Human evaluation shows LinGen outperforming typical open-source video generative models by a large margin.Source: §C.2 · p. 12

D. More Ablation Experiments

• The TESA block, RMS, review tokens, hybrid training, and quality-tuning each contribute to generated-video consistency and quality.Source: D. More Ablation Experiments · pp. 12–13

E.1. Backbone Details

• The backbone uses a TAE with 8× temporal and 8×8 spatial compression, followed by 2×2×1 patchification and factorized learnable positional embeddings.It also uses RMSNorm, SwiGLU, and time-step-conditioned adaptive layer normalization.Source: §E.1 · p. 13
• The selected backbone has 32 layers, 20 heads per layer, and 2560-dimensional embeddings.Source: §E.1 · p. 13

E.2. Mamba and Mamba2

• Mamba2 uses an attention format and supports larger hidden states with lower latency and better tensor-parallel scaling than Mamba.LinGen sets hidden-state sizes to 16 for Mamba and 128 for Mamba2.Source: §E.2 · p. 14
• LinGen is comparable to Gen-3 and Kling and significantly outperforms typical open-source models on VBench-Long.Source: Table 3 · p. 15

E.3.1 Progressive Training Recipe

• LinGen-4B uses progressive pre-training, beginning with text-to-image at 256p and then moving to text-to-video at progressively higher resolutions and longer video lengths, with latent-space token sequence length gradually increasing.Source: §E.3.1 · p. 16

E.3.2 Hybrid Training

• Text-to-video pre-training mixes text-image and text-video pairs at a 1:100 ratio, which the authors report preserves image generation and improves consistency in some failure cases.Source: §E.3.2 · p. 16

• The training data progress from 300M licensed image-text pairs to licensed video sets of 24M pairs, 2.5M videos over 30 seconds, and 145K videos over 60 seconds.Quality tuning uses 3K high-quality 17s videos and 300 minute-length videos for 68s generation.Source: §E.3.3 · p. 17
• Video selection combines aesthetic and motion filtering, concept balancing, manual identification of cinematic videos, and manual captioning.Source: §E.3.3 · p. 17
• At 256p, the section identifies a failure case with abnormally bad consistency.Source: §E.3.3 · p. 18
• Latency comparisons in Figure 21 use 100 inference steps, rather than the 50-step default used in the main paper.Source: Figure 21 · p. 18
• Higher-resolution videos have a much higher human-evaluation win rate but only a slightly higher VBench quality score, which the authors say does not perfectly align with human preference.Source: Table 6 · p. 19