LongVie 2: Multimodal Controllable Ultra-Long Video World Model

• Current world models often lack global semantic control and show visual degradation and temporal drift beyond one minute.The paper identifies comprehensive controllability, long-term fidelity, and long-context consistency as desired properties.Source: §1 · p. 2
• LongVie 2 uses a control-first, then long strategy and three training stages: multi-modal guidance, degradation-aware training, and history-context guidance.Dense and sparse signals supply complementary constraints; degradation-aware training bridges training and long-horizon inference, while preceding frames support consistency across adjacent clips.Source: §1 · p. 2
• On LongVGenBench, a benchmark of 100 high-quality videos each lasting at least one minute, LongVie 2 achieves state-of-the-art controllable long-video generation, producing temporally coherent and visually realistic videos.Source: §1 · p. 2
• LongVGenBench evaluates long-term visual quality and temporal consistency using 100 videos spanning real-world and game-based scenes.The benchmark includes varied day/night and indoor/outdoor settings.Source: §1 · p. 2

2. Methodology

• The three-stage framework injects multimodal controls, trains with degraded first frames, then adds historical frames to improve controllability, long-horizon visual quality, and cross-clip coherence.The stages progressively extend a controllable short-video generator toward long-horizon generation.Source: §2 · p. 3, Figure 3 · p. 3

2.1. Multi-Modal Control Injection

• Depth maps and point trajectories provide dense structural and sparse semantic cues; feature scaling and multiscale or blur perturbations reduce dense-control dominance and encourage balanced integration.The pretrained backbone remains frozen while control branches and fusion layers are trained.Source: §2.1 · pp. 3–4

2.2. Frame Degradation Training

• First-frame degradation simulates long-form input corruption through repeated VAE encoding and decoding or latent noise followed by denoising, with stronger degradations sampled less often.The strategy addresses the quality gap between clean training frames and predicted frames encountered during long-term generation.Source: §2.2 · p. 4, Figure 5 · p. 4

2.3. History Context Guidance

• Degraded history frames are encoded and supplied as context; sampling 0–16 history frames allows inference with or without historical references while promoting cross-clip continuity.The history-context stage addresses discrepancies between adjacent clips that can arise after improving visual fidelity.Source: §2.3 · p. 5, §2.2 · p. 5
• The first three generated frames receive weights {0.05, 0.325, 0.757}, while history, degraded-input low-frequency, and ground-truth high-frequency terms regularize clip boundaries.These terms target continuity with the history segment, structural coherence with degraded inputs, and fine-grained appearance details.Source: §2.3 · p. 5

2.4. Training-free Inter-clip Consistency Strategies

• Unified noise initialization shares one noise latent across clips, while global percentile normalization keeps depth scales consistent before overlapping clip extraction.Global bounds use the full video's 5th and 95th pixel-value percentiles.Source: §2.4 · p. 6

3. Experiments

• LongVGenBench contains 100 one-shot videos exceeding one minute at 1080p, spanning real-world and game-based scenarios; training uses about 100,000 videos across three stages.The benchmark includes rapid scene transitions and complex object motions; the training stages use different data and parameter-update settings.Source: §3 · p. 6

3.1. Qualitative Comparison

• LongVie 2 substantially improves controllability over controllable baselines and surpasses other baselines in temporal consistency and visual quality on LongVGenBench.The paper reports state-of-the-art performance; Table 1 compares base diffusion, controllable-generation, and world-model baselines.Source: §3.1 · p. 6, Table 1 · p. 7
• LongVie 2 achieved the highest human-evaluation scores across visual quality, prompt–video consistency, condition consistency, color consistency, and temporal consistency.Sixty participants ranked five representative models across the five criteria.Source: §3.1 · p. 7, Table 2 · p. 7
• Across motion transfer, style transfer, and mesh-to-video tasks, LongVie 2 outperformed strong baselines and produced finer visual details under the same settings.Figure 7 compares controllability, structural alignment, and appearance across frames.Source: §3.2 · p. 7, Figure 7 · p. 7

3.2. Quantitative Results

• Figure 8 presents videos exceeding two minutes, with reported sustained visual fidelity and long-term temporal coherence.The text also describes world-level guidance simulating flow changes after a faucet is turned on or off.Source: §3.2 · pp. 7–8, Figure 8 · p. 8

3.3. Ablation Study

• Ablations report that control learning improves SSIM and LPIPS, degradation-aware learning enhances visual quality, and history-context guidance improves global temporal consistency.Removing unified initial noise or global normalization also drops performance across all reported metrics.Source: §3.3 · p. 8, Table 3 · p. 8, Table 4 · p. 8

Controllable Video Generation

• Methods use multiple conditioning signals, sparse structural guidance, optical flow, and 3D point maps [13]; Cosmos-Transfer-1 and LongVie integrate multimodal control signals.Source: §4 · p. 8

World Models

• World models including [8] [16] [19] [29] [53] use actions or camera trajectories to model controllable, physically plausible dynamics.The passage associates these systems with large-scale video or gameplay data and interactive scene synthesis.Source: §4 · p. 9

Long Video Generation

• Long-video approaches address temporal drift and quality loss through autoregressive extension, causal constraints, long-range context [4], or error-recycling training [25].Self-Forcing [22] uses video-level distribution-matching objectives for long-horizon stability.Source: §4 · p. 9

5. Conclusion

• The framework extends pretrained video diffusion models through three progressive stages that improve controllability, visual fidelity, and long-term consistency.Dense and sparse controls provide world-level guidance; degradation-aware training bridges training and long-horizon inference; history context models long-range dependencies.Source: §5 · p. 9
• LongVie 2 achieves state-of-the-art performance in controllable long-video generation and can autoregressively synthesize high-quality videos lasting up to 3–5 minutes.The authors report that strengthening controllability before enhancing long-term visual quality is an effective and practical strategy.Source: §5 · p. 9
• LongVGenBench contains 100 one-minute videos spanning diverse real-world and synthetic environments for evaluation.Source: §5 · p. 9

6. Overview of Supplementary Material

• Section 7 provides implementation details on LongVie, including training data, inference adaptation, and model configuration.Source: §6 · p. 12
• Section 8 gives a comprehensive description of LongVGenBench, the dataset used to evaluate long-term consistency and controllability.Source: §6 · p. 12
• Sections 9 and 10 present ablations of frame-degradation training and additional qualitative results across styles, scenarios, and ultra-long videos.Source: §6 · p. 12

7.1. Training Data

• Stages 1 and 2 use approximately 60,000 videos from ACID and ACID-Large, Vchitect_T2V_DataVerse, and MovieNet.The sources contribute aerial footage, annotated Internet videos, and full-length movies; clips are standardized to 81 frames at 16 fps.Source: §7.1 · p. 12
• Stage 3 uses 40,000 sampled 81-frame targets from OmniWorld and SpatialVID, with all preceding frames supplied as history context.Targets begin at each video's 20th frame, and their controls and prompts are generated using the same pipeline as in Stages 1 and 2.Source: §7.1 · p. 12
• Preprocessing removes scene transitions, samples 81-frame clips at 16 fps, and derives depth maps, point tracks, and captions to form approximately 100,000 video–control pairs.Depth is estimated with Video Depth Anything, tracks with SpatialTracker, and captions with Qwen-2.5-VL-7B.Source: §7.1 · p. 12

7.2. Inference Preparation.

• At inference, depth is globally normalized and point tracks are computed independently within overlapping 81-frame clips; Qwen-2.5-VL-7B revises captions after visual transfers.Figure 9 illustrates caption refinement using a new first frame and its original caption.Source: §7.2 · p. 12, Figure 9 · p. 13

7.3. Model Configuration.

• Input-frame degradation is applied with 20% probability, using VAE-based degradation for 20% of those cases and generation degradation for the remainder.For Stage 3, the same degradation strategies are applied to current-frame and history-frame inputs.Source: §7.3 · p. 13
• Feature-level and data-level degradation probabilities are α = 15% and β = 10%; degradation strategies are disabled during the first 2,000 iterations and gradually introduced during the final 1,000 iterations.Source: §7.3 · p. 13
• Dense and sparse branches inherit interleaved halves of Wan2.1’s pretrained weights, reducing parameter count and providing a stable starting point for joint control learning.Source: §7.3 · p. 13

8. LongVGenBench

• LongVGenBench includes diverse real-world and synthetic scenes lasting at least one minute at a minimum resolution of 1080p.Source: §8 · p. 13
• The dataset spans varied camera motions that pose challenges for existing video-generation models.Source: §8 · p. 13
• The model-agnostic benchmark can evaluate any long-horizon video world model; LongVie evaluation uses 81-frame clips with one-frame overlap and extracted captions and controls.Source: §8 · p. 13

9. Additional Ablation Studies

• The degradation ablation reports complementary contributions from encoding and generation degradation to the frame-degradation training pipeline.Table 5 describes progressive Control Learning, Degradation Adaptation, and History Context stages, each enhancing visual quality, controllability, and long-term consistency.Source: §9 · p. 13, Table 5 · p. 14

10. More Qualitative Results

• In 1-minute subject-driven and subject-free scenarios, LongVie preserves scene layout, motion dynamics, and semantic integrity while adapting seasonal appearance.Examples include a person riding a horse and a drone-perspective sequence.Source: §10 · p. 13, Figure 11 · p. 15, Figure 12 · p. 16
• The additional examples cover real-world and synthetic scenarios across indoor and outdoor environments.They are presented as examples for evaluating controllable long-video generation.Source: Figure 10 · p. 14
• Across three seasonal styles, LongVie maintains coherent scene semantics and smooth transitions in the 3-minute scenario.The mountain-valley sequence is shown in Fig. 13.Source: §10 · p. 14, Figure 13 · p. 17
• In two 5-minute scenarios, LongVie preserves global structure, coherent motion, and stable visual appearance throughout the generation.The examples include both subject-driven and subject-free sequences.Source: §10 · p. 14, Figure 14 · p. 18

11. Limitations and Future Work

• LongVie is presented as a framework for controllable world-model generation of coherent visual dynamics over 3–5 minutes.Source: §11 · p. 19
• All experiments use 352×640 resolution, which limits fine-grained details and high-frequency structures.Source: §11 · p. 19
• Future work aims to scale LongVie to higher resolutions to improve visual fidelity and capture richer scene dynamics.Source: §11 · p. 19