Source-linked AI summary
Pathwise Test-Time Correction for Autoregressive Long Video Generation
Xunzhi Xiang, Zixuan Duan, Guiyu Zhang, Haiyu Zhang, Zhe Gao, Junta Wu, Shaofeng Zhang, Tengfei Wang, Qi Fan, Chunchao Guo
TL;DR
Long-horizon autoregressive video generation suffers from accumulated drift, while TTO is limited by difficult reward design and distilled-model sensitivity. TTC addresses this by correcting intermediate stochastic states against the initial-frame context without retraining. The method extends generation to 30 seconds with slight overhead and competitive visual quality across distilled models.
Problem
Long-horizon autoregressive video generation accumulates temporal drift, while TTO struggles with long-range reward design and optimization sensitivity in distilled models.
Method
TTC performs reference-based correction along the stochastic sampling trajectory, using the initial-frame context without modifying model parameters or retraining.
Results
TTC consistently improves long-horizon stability across multiple distilled models, extending generation length to 30 seconds with slight computational overhead and competitive visual quality.
Takeaways & Limitations
Test-time intervention alone can stabilize distilled autoregressive diffusion models while preserving the original generation behavior.
Takeaways & Limitations
The approach assumes correction occurs at a sampling stage where global layout and object structure have stabilized.
Abstract
from arXiv · showhide
Distilled autoregressive diffusion models facilitate real-time short video synthesis but suffer from severe error accumulation during long-sequence generation. While existing Test-Time Optimization (TTO) methods prove effective for images or short clips, we identify that they fail to mitigate drift in extended sequences due to unstable reward landscapes and the hypersensitivity of distilled parameters. To overcome these limitations, we introduce Test-Time Correction (TTC), a training-free alternative. Specifically, TTC utilizes the initial frame as a stable reference anchor to calibrate intermediate stochastic states along the sampling trajectory. Extensive experiments demonstrate that our method seamlessly integrates with various distilled models, extending generation lengths with negligible overhead while matching the quality of resource-intensive training-based methods on 30-second benchmarks.
1 Introduction
Long-horizon autoregressive video generation is limited by cascading errors, while existing test-time optimization is unstable on distilled models. TTC instead corrects stochastic sampling states using the initial frame as a reference, extending stable generation beyond 30 seconds with slight overhead.
- Extended diffusion video generation remains difficult because longer contexts increase computation and latency while temporal coherence degrades over long horizons.
- Autoregressive generation compounds inaccuracies from prior outputs, causing temporal drift and long-horizon degradation.
- TTO fails on long-horizon distilled generation because reward design is difficult and infinitesimal gradients can trigger reward collapse.
- TTC replaces parameter optimization with sampling-space intervention that rectifies stochastic intermediate states toward the initial global context.
- TTC delays correction until global structure stabilizes, then uses reference-conditioned denoising and explicit re-noising to avoid sink collapse.
- Over 30 seconds, TTC extends stable generation with slight computational overhead and visual quality comparable to training-based methods across multiple architectures.
2 Related Work
Video generation spans bidirectional diffusion and autoregressive formulations. Autoregressive methods support streaming with low latency but accumulate errors over long sequences, motivating stabilization strategies.
- Bidirectional diffusion denoises all frames jointly with spatiotemporal attention, leveraging global temporal context for coherent, high-fidelity videos.
- Discrete autoregressive models make deterministic single-step predictions, while multi-step diffusion follows a deterministic ODE trajectory.
- Few-step distilled diffusion replaces deterministic ODE solvers with stochastic sampling that injects noise at intermediate steps.
- Autoregressive video generation enables streaming with low initial latency but is susceptible to error accumulation, temporal drift, and degraded long-video coherence.
3 Test-time Optimization for Distilled Models
Few-step distilled autoregressive diffusion generates sequential chunks through stochastic denoise–re-noise trajectories. Test-time optimization targets reward-aligned distributions, but long-video rewards and distilled-model sensitivity produce degenerate or ineffective outcomes.
- Autoregressive video generation factorizes the sequence distribution into context-conditional next-chunk predictions based on previously generated frames or chunks.
- Few-step distilled models inject intermediate noise, producing a stochastic trajectory rather than a single deterministic mapping or ODE path.
- Each sampling transition denoises a noisy latent and re-injects scheduled noise, advancing generation to the next noise level.
- TTO defines a reward-weighted target distribution proportional to the base model distribution multiplied by exp(r(x)).
- Its objective trades off reward maximization against deviation from the original model distribution through KL divergence.
- Long-video reward design is difficult because temporal drift couples semantic, appearance, and motion errors that one handcrafted objective may not capture.
- LoRA test-time optimization either collapses later frames toward the initial frame, destroying motion, or fails to reduce long-horizon drift.
4 From Test-Time Optimization to Test-Time Correction
The paper shifts from unstable parameter-space test-time optimization to training-free, trajectory-aware correction in sampling space. TTC uses the initial frame as a reference, applies sparse corrections after structural stabilization, and re-noises corrected states to preserve smooth stochastic transitions.
- Motivation: TTO for long-horizon generation is limited by reward-design difficulty and distilled models’ sensitivity to test-time optimization.Temporal drift combines semantic, appearance, and motion errors, while optimization can collapse toward degenerate solutions.
- Pathwise correction: At selected steps, TTC denoises with the initial-frame context, re-noises the corrected state to the current noise level, and resumes denoising.This keeps the intervention compatible with the expected noise distribution and the evolving sampling path.
- From TTO to TTC: TTC avoids model updates by intervening directly in the stochastic sampling trajectory.The method treats intermediate states as rectifiable latent states rather than fixed predictions.
- Correction timing: TTC delays a small number of corrections until global structure stabilizes, preventing sink-collapse during generation.The correction schedule preserves early structural formation before applying reference-guided intervention.
- Pathwise correction: Path-wise correction avoids the flickering and temporal discontinuities caused by directly replacing a latent at one denoising step.The corrected state is integrated through valid diffusion transitions before denoising continues.
5 Experiments
Experiments evaluate TTC on distilled autoregressive baselines and 30-second generation benchmarks using VBench and temporal-drift measures. Across qualitative and quantitative comparisons, TTC improves long-horizon consistency while retaining motion and visual quality, and path-wise correction outperforms single-point intervention.
- Settings: TTC is evaluated on CausVid and Self-Forcing, with additional comparisons against Rolling Forcing and LongLive in 30-second generation.The baselines generate 5-second clips at 16 FPS and 832 × 480 resolution, while standard evaluations use 128 MovieGen prompts.
- Qualitative results: TTC maintains stable temporal coherence and visual fidelity over 30-second sequences, including videos with complex motion and appearance changes.Qualitative comparisons report reduced error accumulation relative to Self-Forcing and quality comparable to or better than training-based methods.
- Quantitative results: TTC consistently improves long-horizon quality across prompts and scenes while enhancing dynamic degree without sacrificing motion smoothness or imaging quality.The evaluation also reports improved subject and background consistency and more stable appearance evolution over time.
- Ablations: Path-wise correction achieves higher temporal consistency and lower t-LPIPS than single-point correction.Single-point intervention frequently introduces flickering, temporal instability, and degraded consistency metrics.
- Short-video results: TTC also outperforms the baseline across most metrics for 5-second video generation, indicating effectiveness beyond long-horizon settings.The short-video comparison supports robustness across temporal horizons.
6 Conclusion
Test-Time Correction is a training-free, reference-based intervention that stabilizes distilled autoregressive diffusion models during long-horizon video generation. It suppresses temporal drift and extends generation length with slight computational overhead and competitive visual quality.
- TTC stabilizes distilled autoregressive diffusion models through training-free, reference-based correction along the stochastic sampling process.
- Corrected predictions are smoothly inherited by subsequent denoising steps without modifying model parameters or requiring additional training.
- TTC consistently improves long-horizon stability across multiple distilled video generation models.
- 30 seconds is the achievable generation length, with slight computational overhead and competitive visual quality.
A Details on Samplers.
The appendix contrasts stochastic few-step sampling with deterministic ODE sampling and describes how distilled causal models perform discrete-time denoising updates. Stochastic transitions inject fresh Gaussian noise, whereas ODE trajectories are fixed once initialization and discretization are set.
- Rectified Flow interpolates between a clean latent x0 and isotropic Gaussian terminal state xTmax.
- Differentiating the interpolation yields a constant velocity vt = x0 − xTmax, which the flow network is trained to predict with mean squared error.
- Few-step causal inference uses a small discrete timestep set and predicts denoised latents with the RF update form.
- Stochastic transitions re-sample Gaussian noise between adjacent timesteps, while ODE sampling uses deterministic numerical integration.
- Under ODE sampling, fixing the initial state and discretization scheme fully determines the generated trajectory.
B Details on Evaluations
The evaluation appendix measures boundary continuity, representation-space drift, and distributional fidelity for long-horizon autoregressive videos. It also describes fixed-budget inference-time scaling protocols that select among stochastic trajectories or stepwise candidates.
- Boundary Continuity (t-LPIPS): Boundary Continuity, or t-LPIPS, averages LPIPS over frame pairs spanning consecutive generated chunk boundaries.
- Color Shift: Color Shift compares the color histograms of the first and last generated frames.
- JEPA Consistency: JEPA evaluation uses a frozen encoder to estimate intrinsic density through local Jacobian volume changes and semantic drift relative to the initial frame.
- JEPA Consistency: JEPA-Std measures representation-drift volatility, while JEPA-Diff measures accumulated long-range semantic deviation.
- Test-time Scaling Configuration: With N = 5, Best-of-N selects the highest-reward completed segment, whereas Search-over-Path selects candidate transitions at each denoising step.
C Details on Methods.
The TTO appendix describes gradient-based test-time adaptation at every sampling step. Each latent chunk is decoded into pixel space before computing optimization losses.
- TTO uses AdamW with a learning rate of 1 × 10^-4 for gradient-based optimization at each sampling step.
- For each latent chunk, the prediction is decoded into pixel space through a pretrained VAE decoder.
- The optimization computes Mean Squared Error and CLIP score on the decoded image.
D Further Quantitative Results.
Further quantitative evaluations show that the method improves overall and semantic quality while preserving dynamic motion and temporal coherence.
- Unlike baselines that reduce motion magnitude for stability, the method sustains a superior dynamic degree while preserving temporal coherence.Dynamic preservation is assessed using perceptual variation between temporally strided frames with LPIPS, SSIM, and PSNR.
E Further Qualitative Results.
Qualitative results across diverse scenarios show consistently high visual quality and temporal coherence, with the method outperforming Self-Forcing in 30-second generation.
- Additional examples across diverse scenarios consistently exhibit high visual quality and temporal coherence.The results are presented in Figures 11–13 and reinforce robustness across different prompts and settings.
- In 30-second long-horizon comparisons, the method significantly outperforms Self-Forcing.The comparison includes Self-Forcing, Rolling Forcing, and LongLive.
- The method achieves temporal coherence and visual quality comparable to training-based methods.This comparison is reported for 30-second long-horizon video generation.