PaperlayerView exact revision on arXiv

Source-linked AI summary

Enhancing Train-Free Infinite-Frame Generation for Consistent Long Videos

X. Feng, J. Zhu, M. Wu, C. Chen, F. Mao, H. Guo, J. Wu, X. Chu, K. Huang

arXiv:2605.18233v1cs.CV

TL;DR

Existing train-free frame-level methods struggle with the training-inference gap and long-term consistency in coherent long video generation. MIGA addresses these issues through noise-span alignment and dual consistency enhancement, achieving state-of-the-art results on VBench and NarrLV.

  • Problem

    Training-free extension of foundation video models must support coherent long videos without costly retraining, while existing methods face training-inference gaps and lack explicit long-range dependencies.

  • Method

    MIGA combines two-stage noise-span alignment with dual consistency enhancement through self-reflection and long-range frame guidance.

  • Results

    MIGA achieves state-of-the-art results on VBench and NarrLV compared with existing train-free methods.

  • Takeaways & Limitations

    MIGA provides an effective train-free approach for extending the generated length of existing foundation models.

  • Takeaways & Limitations

    Longer generation can still produce unintended behaviors, including object-part swaps and other hallucinations over time.

Abstract

from arXiv · show

Without incurring significant computational overhead, train-free long video generation aims to enable foundation video generation models to produce longer videos. Frame-level autoregressive frameworks, e.g., FIFO-diffusion, offer the advantage of generating infinitely long videos with constant memory consumption. However, the mismatch between training and inference, coupled with the challenge of maintaining long-term consistency, limits the effective utilization of foundation models. To mitigate these concerns, we propose \textbf{MIGA}, a novel infinite-frame long video generation method. Firstly, we propose an effective two-stage alignment mechanism that mitigates the training-inference gap by reducing the excessive noise span fed to the model. We then introduce an innovative dual consistency enhancement mechanism, where the self-reflection approach corrects early high-noise frames and the long-range frame guidance approach leverages later low-noise frames with broad coverage to steer generation, jointly improving temporal consistency. Extensive experiments on VBench and NarrLV demonstrate the state-of-the-art performance of MIGA. Our project page is available at https://xiaokunfeng.github.io/miga_homepage/.

1. Introduction

MIGA is a train-free method for infinite-frame video generation that addresses the training-inference gap and weak long-term consistency of frame-level autoregressive frameworks. It combines two-stage alignment with dual consistency enhancement and achieves state-of-the-art performance on VBench and NarrLV.

  • Motivation: MIGA targets the training-inference mismatch and insufficient long-term consistency of existing train-free frame-level autoregressive video generation methods.Inference exposes models to multiple noise levels across frames, unlike training with a single noise level; FIFO-Diffusion also mainly models neighboring-chunk interactions.
  • Method: MIGA introduces a two-stage training-inference alignment mechanism that mitigates the gap by optimizing the excessive noise span of inference latents.The mechanism uses two dedicated optimization stages to reduce the noise-span discrepancy between training and inference.
  • Method: MIGA improves long-term consistency through self-reflection for early high-noise latents and long-range frame guidance for later low-noise latents.Self-reflection uses self-similarity among early latents for correction, while long-range guidance exploits later low-noise latents with broad coverage.
  • Contribution: MIGA preserves the benefits of train-free frame-level autoregression while alleviating its training-inference and long-term consistency limitations.The contribution is framed as a novel infinite-frame generation method designed to address both limitations without training a new model.
  • Results: MIGA achieves new state-of-the-art performance on the VBench and NarrLV benchmarks.The paper reports comprehensive experiments on these mainstream benchmarks.

2. Related Works

Related work has progressed from factorized spatial-temporal video generators to full-attention 3D architectures, while long-video research has explored both resource-intensive training and train-free alternatives. Frame-level autoregressive methods enable infinite-frame generation with fixed memory, motivating MIGA’s focus on training-inference gaps and consistency limitations.

  • Text-to-Video Generation: Video generation has evolved from frameworks combining 2D spatial and 1D temporal modeling to advanced 3D full-attention architectures.Examples include VideoCrafter and Stable Video Diffusion for the former, and Video Diffusion Models and CogVideoX for the latter.
  • Long Video Generation: Long-video studies introduce specialized architectures and large-scale training on curated datasets, but their computational and data requirements limit broad adoption.Recent work therefore explores train-free strategies for more efficient long-video generation.
  • Long Video Generation: Frame-level autoregressive diffusion methods use noise-space design to support infinite-frame generation with fixed memory, while MIGA targets their training-inference gap and consistency limitations.MIGA is presented as retaining the autoregressive advantages while addressing these limitations.

3. Methods

MIGA uses two-stage training–inference alignment to reduce the noise-span mismatch in frame-level autoregressive generation, then adds self-reflection and long-range frame guidance for long-term consistency. These mechanisms exploit the maintained latent queue to align inference with training and coordinate local correction with distant-frame information.

  • Motivation: Frame-level autoregressive generation maintains a queue of T latents with progressively increasing noise levels, enabling long videos but creating an excessive noise span.The queue supports repeated generation beyond the foundation model’s native frame capacity, while FIFO-Diffusion processes it through diagonal denoising.
  • Two-Stage Training-Inference Alignment: Stage 1 slows noise changes by updating the queue every L_zig latent frames, producing a smoother noise span and partially denoising the dequeued latents.Unlike methods that change noise at every frame, the zigzag queue alters it in groups of L_zig latents; Stage 1 stops at τ_e > τ_0.
  • Two-Stage Training-Inference Alignment: Stage 2 processes the Stage 1 outputs at a unified noise level, aligning each denoising operation with the foundation model’s training conditions.After n Stage 1 iterations, nL_zig latents share time step τ_e−1 and are processed sequentially with sliding-window denoising.
  • Dual Consistency Enhancement: Self-reflection evaluates consistency among early high-noise queue latents and triggers expanded correction searches when adjacent chunk scores drop beyond δ_adju.References occupy [f_judg−f_ref, f_judg−1], while evaluated chunks occupy [f_judg, f_judg+f_eval−1].
  • Dual Consistency Enhancement: The self-reflection metric identifies consistency disruptions and remains strongly correlated with clean-latent curves at noise level 40, despite a maximum noise level of 50.The method therefore supports consistency evaluation at relatively early, high-noise stages without additional external evaluation models.
  • Dual Consistency Enhancement: Long-range frame guidance sparsely samples m_guid relatively clean latents from earlier queue positions to enable interactions beyond the local window.The method supplements the model’s access to nearby latents with distant earlier frames during local-window processing.

4. Experiments

MIGA is evaluated on VBench and NarrLV using VideoCrafter2 and Wan2.1-1.3B, achieving state-of-the-art consistency and improved narrative expressiveness. Ablations show that TTA and DCE provide complementary gains, while qualitative results demonstrate stable 1k-frame generation.

  • Evaluation Benchmarks: MIGA is evaluated on VBench and NarrLV, using VBench-Long consistency metrics and NarrLV prompts with TNA counts of 2, 3, and 4.Reported VBench-Long metrics include subject consistency, background consistency, motion smoothness, temporal flicker, and Overall Score.
  • VBench: MIGA achieves state-of-the-art performance across all VBench metrics for both VideoCrafter2- and Wan2.1-based models.VideoCrafter2-based comparisons use a standardized generation length of 128 frames; MIGA further improves subject and background consistency over FreeLong.
  • NarrLV: MIGA improves narrative expressiveness on NarrLV beyond FIFO-Diffusion and existing VideoCrafter2-based methods.The evaluation uses changing-prompt guidance, conditioning latents at different temporal positions on different prompts; Wan2.1-based MIGA shows notable advantages.
  • Qualitative Results: Wan2.1-based MIGA generates three 1k-frame videos that closely follow prompts while maintaining strong subject and background consistency.Additional visualizations are provided in the appendix.
  • Ablation Studies: TTA and DCE individually improve Overall Score by 2.03% and 1.73%, respectively, while their combination provides complementary gains.TTA reduces the baseline’s drastic anomalies and suppresses video noise through its two-stage design; DCE combines self-reflection with long-range frame guidance.

5. Conclusion · Appendix · A. Further Details on Our Method

MIGA builds on frame-level autoregressive generation to address the training-inference gap and long-term consistency in infinite-frame video generation. Its two-stage alignment mechanism proactively mitigates the training-inference gap.

  • 5. Conclusion: FIFO-Diffusion represents train-free frame-level autoregressive frameworks for infinite-length video generation.The passage identifies FIFO-Diffusion as a representative framework.
  • 5. Conclusion: These frameworks generate infinitely long videos with constant memory cost.Constant memory cost is presented as a key merit of the framework family.
  • 5. Conclusion: MIGA is introduced as a novel infinite-frame generation method.MIGA is proposed to build on the merits of existing frame-level autoregressive frameworks.
  • 5. Conclusion: MIGA targets limitations in the training-inference gap and long-term consistency.These are the two limitations explicitly identified in the conclusion passage.
  • 5. Conclusion: MIGA develops an effective two-stage training-inference alignment mechanism.The mechanism is described as part of the method’s contribution.
  • 5. Conclusion: The alignment mechanism proactively mitigates the training-inference gap.Its stated purpose is to reduce the gap between training and inference.

A.1. Pseudocode Implementation · A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.

This section explains FIFO-Diffusion’s train-free frame-level autoregressive generation procedure and identifies the training-inference gap motivating Two-Stage Training-Inference Alignment. It also introduces consistency preservation as a central concern for long-video generation.

  • A.1. Pseudocode Implementation: FIFO-Diffusion illustrates the concrete implementation of train-free frame-level autoregressive generation, followed by analysis of its training-inference gap and motivation for TTA.The section also frames consistency preservation as critical for long-video generation.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: FIFO-Diffusion maintains a noise queue Q containing latents with progressively increasing noise levels.The queue is organized across denoising steps and is used throughout frame generation.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: The queue is initialized with clean foundation-model latents and scheduler-controlled noise injection, using the initial latents for the final f0 queue positions.Because queue length T exceeds initial latent count f0, remaining latents are constructed from noisy versions of the first latent.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: The pseudocode separates initial-latent construction from frame-level generation, with the sampler repeatedly updating Q and saving dequeued latents.This organization corresponds to Algorithms 1 and 2 described in the implementation passages.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: At each inference step, all queue latents decrease their noise level by one, the first becomes clean and is dequeued, and Gaussian noise is appended for continued generation.This produces an autoregressive stream while maintaining the queue’s fixed denoising structure.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: One sampler step over the queue requires multiple foundation-model noise predictions because the model processes only f0 latents while the queue contains T latents, with T > f0.FIFO-Diffusion addresses this using sliding windows of size f0 and stride lstride.
  • A.1.1. PRELIMINARIES: TRAIN-FREE FRAME-LEVEL AUTOREGRESSIVE GENERATION.: The one-step inference pseudocode applies sliding-window updates across the queue, using stride lstride and a final window to cover the remaining latents.The implementation sets lstride to [f0/2] and computes the iteration count from T, f0, and lstride.

A.1.2. TWO-STAGE TRAINING-INFERENCE ALIGNMENT MECHANISM. · A.1.3. DUAL CONSISTENCY ENHANCEMENT MECHANISM.

MIGA uses Two-Stage Training-Inference Alignment to reduce the inference noise span and narrow the training-inference gap, then applies Dual Consistency Enhancement to correct early anomalies and guide generation with distant low-noise frames. Together, these mechanisms structure initialization, iterative denoising, self-reflection, and long-range guidance for more consistent infinite-frame generation.

  • A.1.2. TWO-STAGE TRAINING-INFERENCE ALIGNMENT MECHANISM.: TTA mitigates the training-inference gap by reducing the noise span of latents fed to the model during inference.Its implementation is presented through pseudocode.
  • A.1.2. TWO-STAGE TRAINING-INFERENCE ALIGNMENT MECHANISM.: TTA initializes adjacent Lzig latents at the same noise level and progressively guides noisy latents from clean queue-tail latents.This differs from FIFO-Diffusion initialization, which duplicates semantically identical frames.
  • A.1.2. TWO-STAGE TRAINING-INFERENCE ALIGNMENT MECHANISM.: TTA first applies Zigzag Iterative Denoising, dequeuing Lzig partially denoised latents while enqueuing Lzig Gaussian-noise latents, then completes denoising at identical noise levels.Stage 2 begins after N latents have been dequeued.
  • A.1.2. TWO-STAGE TRAINING-INFERENCE ALIGNMENT MECHANISM.: The TTA algorithm constructs and updates the latent queue through scheduled noise assignment, progressive queue guidance, and iterative denoising with ϵθ and Φ.The implementation specifies inputs including T, f0, Lzig, ϵθ, and Φ.
  • A.1.3. DUAL CONSISTENCY ENHANCEMENT MECHANISM.: DCE combines Long-Range Frame Guidance and Self-Reflection to enhance consistency within the TTA generation process.The framework diagram presents both approaches and their interaction with sparse long-range and dense local latents.
  • A.1.3. DUAL CONSISTENCY ENHANCEMENT MECHANISM.: The DCE implementation uses the two-stage process: Stage 1 performs zigzag iterative denoising, while Stage 2 denoises generated latents at a unified noise level.The pseudocode applies Φ with ϵθ in both stages before returning Qgen and t.
  • A.1.3. DUAL CONSISTENCY ENHANCEMENT MECHANISM.: Self-Reflection evaluates early high-noise latents along the queue and promptly corrects latent anomalies before each queue inference.It is integrated before each application of Q ←Φ(Q, t; ϵθ).
  • A.1.3. DUAL CONSISTENCY ENHANCEMENT MECHANISM.: Long-Range Frame Guidance targets low-noise latents at the queue head to enable interactions among distant latents and improve video consistency.It is integrated into every queue inference step and processes long-range sparse and local dense latents together.

A.2. Analysis of Framework Unification · A.3. Multi-prompt Conditional Generation

MIGA unifies TTA and DCE within the frame-level autoregressive framework, aligning queue-level inference with frame-level denoising to improve long-video consistency. The same framework supports multi-prompt control by assigning different text conditions to successive temporal regions.

  • A.2. Analysis of Framework Unification: TTA and DCE are interdependent mechanisms jointly designed within MIGA’s frame-level autoregressive generation paradigm.TTA addresses queue-level inference, while DCE addresses local frame-level denoising within each queue.
  • A.2. Analysis of Framework Unification: The framework maintains a queue of increasingly noisy frames, simultaneously reduces their noise, and repeatedly pops clean frames from the queue head.A sliding window sequentially refines noisy frames because foundation models process only a limited number of frames at once.
  • A.2. Analysis of Framework Unification: TTA divides queue-level inference into two stages with different noise distributions, reducing the noise span supplied to the model at each step.This transfers foundation models’ short-term generative capabilities to long-video generation.
  • A.2. Analysis of Framework Unification: DCE combines long-range guidance from low-noise frames with self-reflection correction for early high-noise frames to enhance long-video consistency.The self-reflection procedure evaluates cosine similarity and performs extended sampling or correction when the consistency change exceeds δadju.
  • A.2. Analysis of Framework Unification: TTA operates along the queue-level vertical dimension, whereas DCE operates along the frame-level horizontal dimension, forming one coherent framework for consistent, high-quality long videos.Algorithm 7 illustrates how the two mechanisms interact during inference.
  • A.3. Multi-prompt Conditional Generation: The frame-level autoregressive framework enables multi-text control by providing different text conditions to latents at different temporal positions.During sliding-window inference, latents at different queue positions can be guided by different conditioning features.
  • A.3. Multi-prompt Conditional Generation: For nprom prompts, each prompt sequentially controls Nprom frames, producing N = npromNprom clean latents in the final video.The active text condition depends on the model’s queue position and the number of dequeued clean latents.
  • A.3. Multi-prompt Conditional Generation: A Wan2.1-based MIGA example demonstrates multi-prompt controlled generation with three prompts.The case is presented in Fig. A2.

A.4. Implementation on Different Foundation Models · B. Further Details on Experimental Analysis

MIGA adapts frame-level autoregressive generation to VideoCrafter2 and Wan2.1 by modifying their noise prediction and sampling processes. Its generalizability depends on cross-attention-based interaction between noise, text conditions, and latents, while migration to CogVideoX can produce bad cases.

  • A.4. Implementation on Different Foundation Models: VideoCrafter2 serves as an early, representative backbone for existing train-free long video generation methods, and MIGA equips it with frame-level autoregressive generation.The adaptation follows VideoCrafter2’s original denoising inference code while modifying its noise prediction model and sampler.
  • A.4. Implementation on Different Foundation Models: MIGA changes VideoCrafter2’s noise prediction model and DDIM sampler so inference handles latents with different noise levels across steps.Originally, the model receives identical noise levels and the sampler applies the same denoising operation to all latents.
  • A.4. Implementation on Different Foundation Models: MIGA is migrated to Wan2.1, reflecting the need to adapt train-free long video generation frameworks to newer text-to-video foundation models.The Wan2.1 adaptation likewise adjusts the noise prediction model and sampler, with UniPC used by Wan2.1.
  • A.4. Implementation on Different Foundation Models: MIGA transfers to VideoCrafter2 and Wan2.1 because both models use cross-attention for interactions between latents, noise conditions, and text conditions.Latents can encode noise timestep conditions through frame indices, while text conditions are applied collectively across latents.
  • A.4. Implementation on Different Foundation Models: The adopted frame-level autoregressive framework inherently requires foundation models to process latents whose noise levels vary across frames.This requirement motivates the method’s model-specific modifications to noise prediction and denoising processes.
  • A.4. Implementation on Different Foundation Models: Migrating frame-level autoregressive generation to CogVideoX, which uses an MMDiT architecture, can result in a bad case.The supplied figure illustrates this migration failure case.

B.1. Implementation Details of Ablation Studies · B.2. Qualitative Comparison between VideoCrafter2-based MIGA and Wan2.1-based MIGA

The ablation studies evaluate MIGA’s Two-Stage Training-Inference Alignment (TTA) and Dual Consistency Enhancement (DCE) through controlled settings and hyperparameter tests. Qualitative comparison shows that VideoCrafter2-based MIGA can outperform Wan-2.1-based MIGA in subject and background consistency on VBench because of their differing video styles.

  • B.1. Implementation Details of Ablation Studies: The ablation studies use VideoCrafter2-based MIGA on VBench to evaluate the proposed mechanisms against FIFO-Diffusion.Detailed implementation information is provided for each ablation setting.
  • B.1. Implementation Details of Ablation Studies: TTA adds Stage 1 zigzag iterative denoising with Lzig = 4 and Stage 2 unified-noise-level denoising with τe = 10 to FIFO-Diffusion.The TTA-only setting includes both stages, while the separate TTA study evaluates Stage 1 and Stage 2 incrementally.
  • B.1. Implementation Details of Ablation Studies: TTA’s hyperparameter study varies Lzig and τe on a subset containing 50% of randomly sampled evaluation prompts because long-video generation is computationally expensive.FIFO-Diffusion with Lzig = 1 serves as the subset baseline.
  • B.1. Implementation Details of Ablation Studies: Stage 1 builds latent connections through autoregressive early denoising for semantic and spatial consistency, while Stage 2 completes denoising at the training-time noise span of 1.The two stages therefore provide complementary early latent coordination and final denoising behavior.
  • B.1. Implementation Details of Ablation Studies: DCE combines self-reflection with long-range frame guidance, with ablations varying the adjustment threshold δadju and the number of guiding frames mguid.The baseline uses δadju = 0.07 with no extended search, and guiding-frame settings progressively increase mguid.
  • B.1. Implementation Details of Ablation Studies: The computational costs introduced by self-reflection and long-range frame guidance are discussed separately in Sec. B.3.The DCE ablations report overall-score effects for δadju and model-performance effects for mguid.
  • B.2. Qualitative Comparison between VideoCrafter2-based MIGA and Wan2.1-based MIGA: On VBench, Wan-2.1-based MIGA shows weaker subject and background consistency than VideoCrafter2-based MIGA despite using the later foundation model.The passage attributes this counterintuitive result mainly to VideoCrafter2’s focus on animation-style videos, where maintaining long-term consistency is easier.
  • B.2. Qualitative Comparison between VideoCrafter2-based MIGA and Wan2.1-based MIGA: The qualitative comparison includes a music-studio sequence in which a cymbal changes from golden to silver, bronze, black, green, and finally red.The associated material also identifies VBench evaluation results.

B.3. Computational Efficiency Analysis · B.4. Comparison with Training-Based Methods · B.5. Human Evaluation Experiments

MIGA preserves near-FIFO-Diffusion computational efficiency through two-stage alignment and sliding-window denoising, while its optional consistency mechanisms trade additional computation for improved video quality. The paper also situates MIGA against training-based approaches and validates it through a 48-prompt, eight-annotator human study.

  • B.3. Computational Efficiency Analysis: TTA preserves the original FIFO-Diffusion computational efficiency when introduced alone.Each latent requires the same number of denoising steps as in FIFO-Diffusion.
  • B.3. Computational Efficiency Analysis: Sliding-window denoising keeps foundation-model processing per window constant, preventing significant additional memory overhead despite Stage 2’s growing maintained queue.Stage 2 denoises all long-video frames to a unified noise level before final denoising.
  • B.3. Computational Efficiency Analysis: Increasing TTS computational cost improves model performance, with its degree flexibly controlled through n_adju and n_samp.The adjustment parameter n_adju can be modified through δ_adju.
  • B.3. Computational Efficiency Analysis: +0.05 s is the marginal inference-time increase reported for MIGA without DCE relative to FIFO-Diffusion, while maintaining comparable computational efficiency and evaluating quality with VBench Overall Score.Efficiency is measured by average time per generated frame, M_t.
  • B.4. Comparison with Training-Based Methods: Training-based long-video methods achieve extended generation by designing specialized training strategies and model architectures, unlike MIGA’s train-free objective.The comparison discusses frame-level autoregressive methods including CausVid and Self-forcing.
  • B.5. Human Evaluation Experiments: The human study compares MIGA with improved FIFO-Diffusion using 48 randomly sampled prompts, 48 video pairs, and eight annotators.Pairwise comparisons assess subject consistency, background consistency, motion smoothness, and temporal flicker.

B.6. More Qualitative Results

MIGA produces approximately one-minute long videos across Wan2.1-1.3B and VideoCrafter2, with additional qualitative cases shown in the appendix. Under default frame rates, the Wan2.1- and VideoCrafter2-based variants generate 1000 and 600 frames, respectively.

  • Qualitative Results: MIGA generates three long-video cases with Wan2.1-1.3B, presented in Fig. 1.Additional Wan2.1-1.3B cases appear in Fig. A6.
  • Qualitative Results: Additional MIGA generations using Wan2.1-1.3B and VideoCrafter2 are shown in Figs. A6 and A7.All generated videos are approximately one minute long.
  • Qualitative Results: 1000 frames: Wan2.1-based MIGA produces long videos under its default fps settings.The videos are approximately one minute long.
  • Qualitative Results: 600 frames: VideoCrafter2-based MIGA generates long videos under its default fps settings.The videos are approximately one minute long.

C. Limitations and Future Work

MIGA effectively extends the generated length of existing foundation models without training, but longer videos increase the risk of unintended model behaviors. One example shows prompt-aligned motion degrading over time into a cat head-and-tail reversal.

  • MIGA provides a train-free approach for extending the generated length of existing foundation models.
  • Longer video duration creates more space for content creation but increases the risk of unintended model behaviors.
  • In one case, a cat initially follows the text prompt by walking left to right, but later its head and tail suddenly switch places.The passage presents this phenomenon as a hallucination-related failure.
Loading 2605.18233v1…