Source-linked AI summary
FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching
Jangho Park, Geon Yeong Park, Gihyun Kwon, Jong Chul Ye
TL;DR
Generating videos substantially longer than pretrained models’ native horizons remains challenging. FlowLong addresses this with an inference-time, architecture-agnostic framework that blends overlapping windows and synchronizes their sampling trajectories, consistently outperforming training-free and autoregressive baselines across video, audio-video, and text-to-3DGS generation.
Problem
Pretrained video diffusion models produce high-quality clips only within their trained chunk length, limiting generation of substantially longer videos.
Method
FlowLong blends predicted clean samples across overlapping windows with Tweedie matching and synchronizes trajectories through stochastic early-phase sampling before deterministic ODE sampling.
Results
FlowLong consistently outperforms training-free and autoregressive baselines across text-to-video, audio-video joint generation, and text-to-3DGS without fine-tuning.
Takeaways & Limitations
The framework extends pretrained flow-based video models beyond their native horizon without architectural modification or additional training.
Takeaways & Limitations
FlowLong assumes a pretrained encoder-decoder latent flow model for its formulation.
Abstract
from arXiv · showhide
Extending the generation horizon of video diffusion models to long sequences remains a long-standing and important challenge. Existing training-free approaches fall into two categories: extensions of bidirectional models, which are tightly coupled to specific architectures and suffer from quality degradation over long horizons, and autoregressive models, which accumulate drift errors due to exposure bias and tend to produce repetitive motion patterns. To address these issues, we propose a novel but simple inference-time approach for long video generation that is architecture-agnostic and requires no additional training. Our method generates long videos via overlapping sliding windows, where predicted clean samples from adjacent windows are blended via \emph{Tweedie matching} to enforce both \textbf{manifold constraint and temporal consistency} across overlap regions. \emph{Stochastic early-phase sampling} then synchronizes per-window trajectories by injecting fresh noise after each Tweedie matching correction in the high-noise phase, before transitioning to deterministic ODE sampling to preserve fine-grained visual fidelity. Applied to various video generation models, our method generates videos several times longer than the native window length while outperforming both training-free and autoregressive baselines in temporal consistency and visual quality, and further extends to audio-video joint generation and text-to-3DGS without any fine-tuning.
1 Introduction
FlowLong is a training-free, model-agnostic inference framework that extends pretrained flow-based video diffusion models beyond their native horizon. It aligns overlapping chunk trajectories through Tweedie matching and stochastic early-phase sampling, improving long-horizon consistency across multiple generation tasks without fine-tuning or architectural changes.
- Motivation: Long video generation is challenging because bidirectional extensions degrade in consistency and accumulate artifacts over longer horizons while requiring architecture-specific modifications.These methods extend pretrained bidirectional diffusion models without additional training but remain tightly coupled to model architectures.
- Motivation: Autoregressive approaches accumulate exposure bias and temporal drift through KV-cache reuse, while also tending toward repetitive motion patterns over long horizons.The supplied passage identifies error accumulation, temporal drift, and reduced motion diversity as key limitations.
- Method: FlowLong reformulates long video generation as an inverse problem that aligns multiple chunk-sampling trajectories toward a coherent sequence.Its geometric formulation regularizes each chunk’s denoising path to promote smooth, manifold-constrained generation.
- Scope and validation: FlowLong applies without fine-tuning or architectural modification to text-to-video, audio-video joint generation, and text-to-3DGS, outperforming training-free and autoregressive baselines.The framework is described as uniformly applicable across these tasks and validated with qualitative and quantitative evaluations.
- Method: Tweedie matching blends predicted clean samples across overlapping segments to enforce manifold constraint and temporal consistency.The method operates on overlapping windows and uses clean-sample blending to coordinate adjacent segments.
- Method: Stochastic early-phase sampling injects noise in the high-noise regime to break per-window trajectory inertia before switching to deterministic ODE sampling.This combines early stochastic synchronization with later deterministic sampling for fine-grained generation.
2 Related Work
Related work on long video generation primarily follows bidirectional or autoregressive paradigms. Bidirectional extensions rely on backbone-specific interventions, while autoregressive methods face accumulated drift and global-context challenges beyond trained lengths.
- Bidirectional video diffusion: Bidirectional video diffusion models generate fixed-length windows using full spatio-temporal attention.Recent models adopt a bidirectional architecture for fixed-length frame windows.
- Bidirectional video diffusion: Training-free extensions of bidirectional models use backbone-specific interventions to generate longer sequences.Examples include FIFO-Diffusion’s first-in-first-out denoising queue and RIFLEx’s reduced rotary-embedding frequency.
- Autoregressive video diffusion: Autoregressive video diffusion enables fast and robust generation through an autoregressive process, including extensions of distribution matching distillation to videos.The passage identifies pioneer works that extend DMD to video generation.
- Autoregressive video diffusion: Beyond the trained length, autoregressive generation accumulates errors, causing drift and difficulty maintaining global context coherence.These limitations motivate approaches that better preserve consistency over extended sequences.
3 Preliminaries
This section introduces flow matching as transport from Gaussian noise to data along a straight path, then describes ODE sampling and denoised/noisy estimates used by the method. It also specifies text conditioning and the latent-space video representation.
- Flow model: Flow matching transports samples from a simple source distribution p1 to a target distribution p0 over R^d along a straight path.Rectified flow linearly interpolates between data x0 ∼ p0 and noise x1 ∼ N(0, I).
- Flow model: A neural network vθ(xt, t) approximates the velocity field dxt/dt that transports noise x1 back to data x0 through conditional flow matching.The learned velocity field is trained using a conditional flow matching objective.
- Sampling: Sampling starts from x1 ∼ N(0, I) and generates x0 ∼ p0 by solving the learned ODE from t = 1 to t = 0.An Euler step provides a discrete approximation from time t to an earlier time s < t.
- Sampling: The formulation defines denoised and noisy estimates, with the denoised estimate equivalently derived from Tweedie’s formula.These estimates allow an Euler step to be reformulated as interpolation between denoised and noisy states.
- Text conditioning: For text-guided flow models, the velocity field and denoised estimate are conditioned on textual embedding c.The paper omits c when it does not create notational ambiguity.
- Latent representation: The paper uses a latent flow model with a pretrained encoder-decoder and denotes encoded video latents by x.The encoder and decoder are represented as Eϕ and Dψ, respectively.
4 FlowLong: Inference-time Long Video Generation
FlowLong extends pretrained flow-matching video models beyond their fixed native horizon without fine-tuning by generating overlapping chunks and harmonizing their shared regions. Tweedie matching enforces manifold-constrained temporal consistency, while stochastic early-phase renoising synchronizes trajectories before deterministic sampling preserves visual fidelity.
- 4 FlowLong: Inference-time Long Video Generation: Pretrained video diffusion models generate high-quality clips within F frames but cannot natively extend videos beyond that trained chunk length.FlowLong instead samples K overlapping chunks to form a coherent sequence of N > F frames, invoking the pretrained model independently on each chunk.
- Tweedie Matching: Tweedie matching blends adjacent chunks’ denoised overlap estimates, leaving non-overlap frames unchanged while enforcing manifold constraint and exact boundary consistency.A symmetric per-frame schedule interpolates overlap frames toward neighboring estimates, producing smooth blending across the shared O-frame window.
- Prompt Conditioning: Shared global prompts maintain stylistic and semantic coherence across multi-shot transitions, while per-chunk prompts provide local content and fine-grained diversity.When all chunks share one prompt, the guidance loss enforces temporal coherence under a single scene description.
- Stochastic Early-Phase Sampling: Stochastic early-phase renoising remixes chunk trajectories after each Tweedie correction, overcoming deterministic ODE inertia from independently initialized noise or distinct prompts.The schedule uses full stochastic renoising for t ≥ t∗, then switches to deterministic ODE sampling for t < t∗ to preserve fine-grained visual fidelity.
- Broader Applications: The overlap-based framework extends beyond visual-video temporal expansion to audio-video joint generation and text-to-3D generation without fine-tuning.It applies Tweedie matching to aligned audio-video streams and to overlapping video latents converted into 3D Gaussian splats.
5 Experiments
Experiments show that FlowLong improves long-video generation over bidirectional, autoregressive, and sliding-window baselines, while extending text-to-3DGS generation beyond the pretrained model’s context window. Ablations attribute these gains to Tweedie matching and stochastic early-phase sampling, which balance temporal consistency and visual quality.
- Long Video Generation: Beyond 30 seconds, bidirectional and autoregressive baselines lose meaningful motion and exhibit progressively saturated pixels from accumulated error drift.These models also continuously cache key-value pairs from previous frames, while FlowLong samples from independent initial points to preserve motion diversity and reduce drift.
- Long Video Generation: FlowLong achieves the best overall VBench score against training-free bidirectional models, including superior Dynamic Degree, and surpasses a sliding-window baseline for LTX2 across most metrics.VBench evaluates aesthetic quality, imaging quality, background consistency, subject consistency, motion smoothness, dynamic degree, and temporal flickering on 30s and 60s videos.
- Text-to-3DGS Generation: FlowLong extrapolates text-to-3DGS generation beyond VIST3A’s fixed native context window, producing substantially wider viewpoint coverage.The extension uses Tweedie matching and stochastic early-phase renoising to generate longer videos that translate into broader scene-level 3D coverage.
- Text-to-3DGS Generation: 1.64× more Gaussians per scene are generated than the VIST3A baseline, while top-30% confidence filtering retains 2.47M Gaussians on average.Mean confidence logit rises from 26.27 for the baseline to 41.52 for FlowLong, and the 0.7-quantile logit increases from 30.47 to 46.28.
- Ablations: Full-SDE preserves temporal consistency across chunks but degrades visual quality, whereas full-ODE produces sharper results at the cost of exposure bias and temporal inconsistency.The ablation indicates that the sampling strategy must balance consistency and visual fidelity across chunks.
- Ablations: Tweedie matching in predicted clean-sample space outperforms arbitrary-noise-level overlap blending on Dynamic Degree, Consistency, and Quality.The ablation studies Tweedie matching for overlapping regions and separately examines stochastic early-phase sampling against full-SDE and full-ODE strategies.
6 Conclusion
FlowLong is a training-free, architecture-agnostic framework for long video generation centered on Tweedie matching and stochastic early-phase sampling. Together, these components enforce temporal consistency across overlapping windows and synchronize per-window trajectories before deterministic ODE sampling.
- 6 Conclusion: Our method achieves the best overall performance in the ablation study.The table highlights the first and second values.
- 6 Conclusion: FlowLong combines Tweedie matching with stochastic early-phase sampling to address key failure modes of prior long-video generation approaches.Tweedie matching blends predicted clean samples across overlapping windows, while stochastic sampling injects noise during the high-noise regime before deterministic ODE sampling.
A Tweedie Matching: Implementation Details
This appendix details the practical implementation of Tweedie matching, covering latent-space window geometry, per-frame blending, aggregation, and audio–video joint generation.
- Implementation Details: The implementation specifies latent-space window geometry for Tweedie matching.This is covered in §A.1.
- Implementation Details: It defines the explicit per-frame blending schedule λ_j.The schedule is described in §A.2.
- Implementation Details: It establishes equivalence between iterating Eq. (15) over chunk pairs and a single weighted aggregation.This equivalence is developed in §A.3.
- Implementation Details: It describes the aggregation algorithm used in the implementation.The algorithm is presented in §A.4.
- Implementation Details: It extends the corresponding geometry to audio–video joint generation.This geometry is discussed in §A.5.
A.1 Window geometry in latent space
The method defines overlapping video-generation windows entirely in VAE latent space, with window length, blending-zone size, stride, chunk count, and total-buffer length determining the long-video geometry. The overlap is chosen so adjacent chunks predict the same global latent positions, enabling Tweedie matching across the blending zone.
- Latent-space setup: All window operations occur in video-VAE latent space, whose temporal stride is r = 8 for LTX-2.The latent representation is the basis for defining chunk and overlap geometry.
- Window parameters: F, O, S, K, and N denote latent chunk length, blending-zone length, chunk stride, chunk count, and total buffer length N = F + (K −1) S.The blending zone comprises the last O latent frames of each chunk.
- Overlap alignment: Adjacent chunks must predict every blending-zone frame at the same global temporal position, preventing guidance targets from referencing frames unseen by the next chunk.This alignment ensures chunk k’s blending zone is also predicted by chunk k + 1.
- Practical configuration: For LTX-2 with pixel window W = 121 and overlap start w = 64, the resulting latent geometry is (F, O, S) = (16, 8, 7), used throughout text-to-video experiments.This configuration satisfies the overlap condition with one frame of slack.
- Indexing: Global latent index g = (k −1) S + j maps local frame j in chunk k to the shared buffer, whose blending zone is Ωk = { F −O, ..., F −1 }.The blending zone corresponds to the indicator vector 1Ωk used in the guidance loss.
A.2 The per-frame blending schedule λj
The blending zone uses a linear per-frame convex combination between adjacent clean estimates, ensuring exact seam consistency, symmetric aggregation, and uniform transitions.
- The per-frame blending schedule λj: The blending zone linearly combines adjacent clean estimates using weight λj.On the overlap, Eq. (15) becomes a per-frame convex combination with weight λj.
- The per-frame blending schedule λj: Boundary consistency sets λF−O = 0 and λF−1 = 1, eliminating discontinuities at both overlap boundaries.The leftmost frame comes entirely from chunk k, and the rightmost entirely from chunk k + 1.
- The per-frame blending schedule λj: With local index i = j − (F − O), the schedule satisfies λj = i/(O − 1) and the mirror identity λj + λjmir = 1.Symmetric updates therefore produce the same convex combination at corresponding global frames.
- The per-frame blending schedule λj: Because λj is linear in frame index, transitions across the blending zone are uniform.Preliminary experiments found no additional gains from smoother schedules such as raised-cosine windows, while the linear form supports simple aggregation equivalence.
A.3 Multi-chunk pairwise updates collapse to a single weighted aggregation
Under the symmetric linear schedule, iterating pairwise updates across K overlapping chunks collapses to one weighted aggregation pass. Each global frame is written from one adjacent pair’s convex combination, with rightmost-pair resolution when blending zones overlap.
- Single-pass aggregation: For K chunks, all adjacent pairwise modulated Euler updates collapse into a single pass computing one adjacent pair’s weighted average for each global frame.This follows under the symmetric linear schedule (23).
- Single-pass aggregation: Within blending zone Bk, chunk k and chunk k + 1 are combined with weights 1 − λj and λj, respectively.The symmetric update yields the same convex combination, allowing one global-buffer write.
- Overlap resolution: When S < O, overlapping consecutive blending zones are resolved using the rightmost pair, so each overlap frame is stored exactly once.For example, (F, O, S) = (16, 8, 7) produces Bk and Bk+1 overlap by O − S frames.
A.4 Aggregation in the long-video buffer · A.5 Audio–video joint geometry
A.4 aggregates overlapping denoising chunks into a long-video buffer through direct, single-pass writes, then re-slices the updated state for the next step. A.5 aligns audio and video chunk geometry by temporal stride and applies aggregation independently to both modalities.
- A.4 Aggregation in the long-video buffer: The long-video buffer is constructed with a single pass over chunks, writing each global index exactly once.This reduction allocates the buffer and avoids repeated accumulation during aggregation.
- A.4 Aggregation in the long-video buffer: The leading prefix is copied from chunk 1, while each adjacent blending zone uses position-dependent interpolation between neighboring clean predictions.For blending zone B_k, the write is (1 − λ_j) x̂^(k)_0|t[j] + λ_j x̂^(k+1)_0|t[j − S].
- A.4 Aggregation in the long-video buffer: Interior gaps caused by S > O are copied from the next chunk, and trailing suffix frames are copied from chunk K.Direct assignment eliminates weight accumulation and re-normalisation.
- A.4 Aggregation in the long-video buffer: When S < O creates overlapping blending zones, conflicting writes are resolved by last-writer-wins.The rule applies during direct buffer assignment.
- A.4 Aggregation in the long-video buffer: After aggregation and the next-state update, the resulting X_s is re-sliced into K overlapping windows for the next denoising step.The update uses deterministic Euler in the low-noise phase and stochastic early-phase sampling as specified in the surrounding method.
- A.5 Audio–video joint geometry: LTX-2 jointly denoises video and audio latents with different temporal rates, so audio geometry is chosen to match the video stride in seconds.The video rate is fps_v/r frames per second, while the audio rate is ρ_a latents per second, with ρ_a = 25 for LTX-2.
- A.5 Audio–video joint geometry: For (W, w, fps_v, ρ_a) = (121, 64, 24, 25), the audio geometry is (F_a, O_a, S_a) = (126, 67, 59), satisfying O_a ≥ S_a.The total audio length is N_a = F_a + (K − 1) S_a.
- A.5 Audio–video joint geometry: Continuous-time rounding can misalign audio blending by at most one latent, approximately 40 ms for LTX-2; clamping the offset to zero had no observable phase-locking effect.Aggregation then runs independently on the video and audio buffers using their respective (F, O, S).