PaperlayerView exact revision on arXiv

Source-linked AI summary

RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPO

Yanzuo Lu, Ronglai Zuo, Jiankang Deng

arXiv:2605.15190v1cs.CV

TL;DR

Autoregressive video distillation struggles to supervise the generated histories that future chunks depend on. RAVEN aligns training with inference-time extrapolation, while CM-GRPO optimizes the consistency transition directly; together, they surpass recent baselines across quality, semantic, and dynamic metrics.

  • Problem

    Autoregressive video distillation either trains on histories unlike inference or leaves rollout-history representations outside end-to-end supervision.

  • Method

    RAVEN repacks self rollouts into clean historical endpoints and noisy denoising states, while CM-GRPO applies policy optimization directly to the consistency transition kernel.

  • Results

    Together, RAVEN and CM-GRPO surpass recent causal video distillation baselines across quality, semantic, and dynamic dimensions, achieving the strongest results on every metric considered.

  • Takeaways & Limitations

    The combined framework provides a stronger causal video distillation approach across the evaluated quality, semantic, and dynamic metrics.

  • Takeaways & Limitations

    The bidirectional teacher cannot currently be sampled through the consistency interface, so the closed-form reference regularizer remains impractical and is left for future work.

Abstract

from arXiv · show

Causal autoregressive video diffusion models support real-time streaming generation by extrapolating future chunks from previously generated content. Distilling such generators from high-fidelity bidirectional teachers yields competitive few-step models, yet a persistent gap between the history distributions encountered during training and those arising at inference constrains generation quality over long horizons. We introduce the Real-time Autoregressive Video Extrapolation Network (RAVEN), a training-time test framework that repacks each self rollout into an interleaved sequence of clean historical endpoints and noisy denoising states. This formulation aligns training attention with inference-time extrapolation and allows downstream chunk losses to supervise the history representations on which future predictions depend. We further propose Consistency-model Group Relative Policy Optimization (CM-GRPO), which reformulates a consistency sampling step as a conditional Gaussian transition and applies online Reinforcement Learning (RL) directly to this kernel, avoiding the Euler-Maruyama auxiliary process adopted in prior flow-model RL formulations. Experiments demonstrate that RAVEN surpasses recent causal video distillation baselines across quality, semantic, and dynamic degree evaluations, and that CM-GRPO provides further gains when combined with RAVEN.

1 Introduction

Real-time video generation motivates causal autoregressive diffusion models, but distillation suffers from a mismatch between training and inference histories and insufficient supervision of reused history representations. RAVEN addresses this gap by repacking self rollouts into interleaved clean and noisy states for direct history supervision, while the introduction situates CM-GRPO within recent diffusion and flow-model reinforcement-learning work.

  • Motivation: Bidirectional video diffusion achieves high fidelity but is poorly suited to continuous real-time streaming because it requires bidirectional context and many denoising steps.
  • Problem: Autoregressive distillation faces a history supervision gap: training histories may differ from inference, while detached rollout caches receive no end-to-end supervision from later chunk losses.Teacher Forcing optimizes under a history distribution that does not match inference, whereas Self Forcing uses self rollouts but detaches the historical cache.
  • RAVEN: RAVEN repacks few-step causal-generator self rollouts into interleaved clean historical endpoints and noisy denoising states to directly supervise history construction during extrapolation.Clean rollout chunks provide causal history for subsequent predictions, while noisy states are included in the interleaved sequence.
  • CM-GRPO: Recent reinforcement-learning work has extended policy optimization to diffusion and flow models, including Flow-GRPO’s ODE-to-SDE conversion followed by Euler-Maruyama discretization.

2 Related Work

Related work spans autoregressive video diffusion distillation, alternative rollout and conditioning designs, and online RL methods for diffusion, flow, autoregressive, and world models. The paper distinguishes training paradigms by how historical states are formed, attended to, and supervised.

  • Autoregressive Video Diffusion Distillation: Autoregressive video generation explores longer prediction windows and conditioning on intermediate noisy latents rather than fully denoised historical outputs.The paper’s implementation instead conditions on clean historical endpoints.
  • Attention Mask Configuration: Teacher Forcing and Diffusion Forcing use data-driven historical states, creating a training distribution that differs from inference.Self Forcing shifts history toward inference but reuses the historical cache as detached context, whereas RAVEN repacks each self rollout.
  • Online RL in Diffusion Model: Online RL for diffusion and flow models evolved from reward-guided image-generation optimization into policy-optimization methods tailored to diffusion and flow trajectories.The approach has also extended to autoregressive generators and world models for preference alignment and preserving pretrained behavior.

3 Methodology

RAVEN aligns autoregressive video-distillation training with inference by reusing self-rollout states as interleaved clean histories and noisy denoising inputs. CM-GRPO performs online policy optimization directly on the stochastic Gaussian transitions of a consistency sampler, avoiding Flow-GRPO’s auxiliary stochastic process.

  • RAVEN: Unlike Diffusion Forcing and Self Forcing, RAVEN brings cache construction inside end-to-end supervision by matching training context to inference-time extrapolation.Diffusion Forcing perturbs ground-truth prefixes, whereas Self Forcing reuses detached cache representations treated as fixed context.
  • RAVEN: RAVEN repacks each self rollout into interleaved clean endpoints and noisy states, enabling later noisy states to attend to encoded clean histories in one causal forward pass.For sampled timestep u, clean endpoints form hRAVEN_t = H(ˆx<t), while noisy states serve as supervised denoising targets.
  • RAVEN: RAVEN alternates fake-score critic updates on perturbed self-rollout samples with generator updates using reverse-KL score gradients from teacher and critic evaluations.The causal student is distilled from a frozen bidirectional teacher, while the fake-score critic is learned from autoregressive self rollouts.
  • RAVEN: RAVEN scales chunk losses using a future participation score whose profile decreases monotonically toward later chunks, then applies a predefined weighting function to obtain normalized per-chunk weights.Earlier chunks contribute supervised elements from themselves and all subsequent chunks, reflecting their richer downstream participation.
  • CM-GRPO: CM-GRPO formulates policy optimization directly on the consistency sampler’s stochastic Gaussian transition kernel, avoiding Flow-GRPO’s auxiliary Euler-Maruyama stochastic process.Consistency transitions arise through the predicted clean endpoint, whereas Flow-GRPO converts a deterministic ODE into an auxiliary SDE because inference uses deterministic ODE sampling.
  • CM-GRPO: The closed-form CM-GRPO regularizer is derived for completeness but is not practically applied because the bidirectional teacher lacks a compatible consistency interface.Its practical use is deferred until a compatible reference consistency model is available.

4 Experiments

Experiments use Wan2.1-T2V-1.3B with VBench-based evaluation and show that RAVEN improves the quality–motion trade-off over prior causal baselines. CM-GRPO adds further gains, while ablations support the roles of training-time history supervision and chunk-wise loss scaling.

  • Implementation and Evaluation: Experiments build on Wan2.1-T2V-1.3B with 3 latent frames per chunk, initialize RAVEN as in Causal Forcing, and apply CM-GRPO from the RAVEN checkpoint.Evaluation reports VBench Total, Quality, and Semantic Scores, and uses UnifiedReward-32B for dynamic degree across 6,220 generated videos.
  • Quantitative Comparisons: RAVEN surpasses every prior baseline across total, quality, semantic, and dynamic dimensions, with its largest margin on dynamic degree.The results indicate that supervising cached history alleviates rather than redistributes the quality–motion trade-off.
  • Quantitative Comparisons: Adding CM-GRPO to Causal Forcing yields a smaller motion-focused gain, while combining it with RAVEN produces the leading entry on every dimension.The comparison suggests that RAVEN and CM-GRPO are complementary and that policy updates benefit from an inference-aligned generator.
  • Qualitative Comparisons: Qualitatively, RAVEN with CM-GRPO avoids Causal Forcing’s motion-related structural failures, including distorted bodies, detached heads, facial distortion, and unnatural oversaturation.Figure 3 compares Causal Forcing and RAVEN with and without CM-GRPO across animal, urban, and human prompts.
  • Effect of Training-time Test: Training-time-test ablations show that TF favors motion, SF favors semantic alignment but weakens motion, and self-rollout endpoints in DF recover motion close to TF while eroding quality.All variants share ODE-distilled initialization and chunk-wise loss scaling, isolating history formation and supervision.
  • Effect of Chunk-wise Loss Scaling: Chunk-wise loss-scaling profiles centered near rollout midpoints or biased toward early chunks fall below uniform on total and quality, whereas later-chunk emphasis is evaluated through shifted schedules.The shift parameterization uses α = 1 for early-chunk bias, α = 0 for uniform weighting, and negative α for reversed-coordinate emphasis on later chunks.

5 Conclusion

The paper presents RAVEN, which propagates supervision through cached histories during autoregressive extrapolation, and CM-GRPO, which optimizes the inference-time consistency transition kernel directly. Together, the methods surpass recent causal video distillation approaches.

  • RAVEN: RAVEN repacks each self rollout into interleaved clean historical endpoints and noisy denoising states.This training-time test framework enables supervision to propagate through the cached history used during autoregressive extrapolation.
  • CM-GRPO: CM-GRPO formulates the policy objective directly on the inference-time consistency transition kernel.It avoids basing the objective on an auxiliary Euler-Maruyama process.
  • Combined contribution: Together, RAVEN and CM-GRPO surpass recent causal video distillation approaches.

A Algorithm Formulations

The algorithms formalize RAVEN training through self rollouts, interleaved-history supervision, and cached causal representations, then extend the resulting checkpoint with CM-GRPO. CM-GRPO evaluates grouped consistency rollouts and updates the policy on sampled transitions using group-relative advantages and chunk-wise weighting.

  • RAVEN training iteration: RAVEN training performs self rollouts under the consistency sampler, updates the critic every iteration, and updates the generator every r iterations on an interleaved sequence.A single causal forward pass routes later-chunk gradients through the cached RAVEN history used during extrapolation.
  • CM-GRPO training iteration: CM-GRPO initializes from a RAVEN checkpoint and draws G independent consistency rollouts under a shared text condition.The rollout stage runs without gradient through the consistency generator and maintains a KV cache across chunks.
  • Reward evaluation and advantages: Each CM-GRPO rollout receives composite reward scores across dimensions, which are group-normalized into a group-relative advantage and clipped to the prescribed range.Reward evaluation computes each dimension as R_m(ˆx_i, c) before normalization and clipping.
  • Policy update on sampled transitions: CM-GRPO samples one transition u → s from each cached rollout and applies stop-gradient regression to the predicted clean endpoint of the consistency kernel.The sampled transition index is restricted to k_i ∈ {1, . . . , K−1}.
  • Policy update on sampled transitions: The policy loss is split by chunk and weighted using future participation scores, preserving RAVEN’s alignment between supervision and the autoregressive horizon.Chunk weights satisfy w_t ∝ g_η(p_t).

B More Implementation Details

RAVEN and CM-GRPO use VidProM text prompts under the Self Forcing protocol, with real-video ablations drawing on temporally upsampled OpenVidHD clips. Training largely follows prior causal-video settings, while CM-GRPO adds parameter-efficient adaptation and uses rewards based on motion and temporal coherence.

  • Dataset: Both methods train exclusively on filtered and LLM-extended VidProM text prompts, while real-video ablations use OpenVidHD-0.4M clips temporally upsampled with RIFE.The data protocol follows Self Forcing.
  • Training Details: Most RAVEN settings inherit Self Forcing and Causal Forcing, disable weight decay, and reduce the TTUR critic-to-generator update ratio from 5 to 2.CM-GRPO is optimized parameter-efficiently on the RAVEN checkpoint with LoRA.
  • Training Details: The Causal Forcing + CM-GRPO ablation lowers learning rate to 2 × 10−6, reinstates weight decay at 0.01, and raises Dynamic Degree reward weight from 0.35 to 2.35.These settings isolate the policy objective from RAVEN itself and compensate for weaker motion in the Causal Forcing checkpoint.
  • Training Details: RAVEN and CM-GRPO consume approximately 70 and 170 NVIDIA H200 GPU hours, respectively.The reported budgets apply to the two methods’ training runs.
  • Reward Composition: Reward composition captures temporal dynamics through Dynamic Degree and motion smoothness, using RAFT optical-flow peaks and AMT dropped-frame reconstruction error.Dynamic Degree uses the top-5% mean optical-flow magnitude across consecutive frame pairs, while lower AMT reconstruction error indicates greater temporal coherence.

C User Study

The user study compares RAVEN with four causal video-generation baselines using randomized pairwise evaluations across 100 long, detailed prompts. Each method produces four samples per prompt, and users rate clips on Quality, Semantic, and Overall criteria.

  • Study Design: The study uses 100 long and detailed prompts drawn from existing baselines’ qualitative showcases.Each method generates 4 samples per prompt.
  • Compared Methods: RAVEN is compared against CausVid, Self Forcing, Reward Forcing, and Causal Forcing.These four baselines are designed for 5-second short video generation.
  • Evaluation Criteria: Individual users evaluate randomized RAVEN–baseline sample pairs on Quality, Semantic, and Overall dimensions.Each RAVEN clip is presented against its baseline counterpart in randomized order.

D Discussion

The discussion argues that RAVEN and CM-GRPO extend beyond the causal autoregressive video-distillation setting evaluated in the experiments. RAVEN permits flexible cached-history representations, while CM-GRPO applies to few-step generators using stochastic consistency transitions across generator structures and modalities.

  • RAVEN: RAVEN’s supervised forward pass does not restrict the form of cached history, allowing arbitrary rollout-derived representations to replace clean historical chunks.Its current interleaved sequence construction treats clean chunks as historical context.
  • CM-GRPO: CM-GRPO depends only on the conditional Gaussian transition induced by the consistency sampler, not on the generator’s autoregressive structure.This makes the objective applicable to few-step generators that sample through a stochastic consistency step.
  • CM-GRPO: CM-GRPO therefore encompasses bidirectional video models and generators in other modalities distilled through consistency-based stochastic sampling.The passage describes this as an extension beyond the evaluated autoregressive setting.

E Prompts

This section lists the text prompts used for qualitative comparisons, organized by their positions within figure grids. The prompts cover animated animals, people, action scenes, nature photography, and a scenic train departure.

  • Figure-grid prompts: The top-left prompt depicts a cheerful Corgi running through a neon-lit futuristic park at sunset in a cyberpunk cityscape.The scene emphasizes the dog’s playful expression, wagging tail, grass, skyscrapers, and holographic advertisements.
  • Figure-grid prompts: The top-right prompt shows a majestic African lion releasing a small antelope in a dense savanna after gripping it in its jaws.The lion has golden fur, green eyes, and a flowing mane, while the antelope runs away through tall grass and acacia trees.
  • Figure-grid prompts: The bottom-left and bottom-right prompts feature a smiling woman in wind-blown clothing and a joyful boy sprinting through rainy puddles.The woman appears outdoors among trees and flowers, while the boy runs barefoot under heavy rain with blurred umbrella-carrying figures behind him.
  • Figure-grid prompts: The left, middle, and right prompts depict a crab entering its burrow, a surfer accelerating on a wave, and a steam train departing across Scotland’s Glenfinnan Viaduct.These scenes emphasize camera motion, dynamic water action, and a picturesque train journey framed by greenery and mountains.

F Broader Impacts

RAVEN and CM-GRPO improve the practical viability of real-time autoregressive video generation, bringing both positive and negative societal implications. Potential benefits include creative, educational, accessibility, and interactive simulation applications, alongside reduced computational demands and lower barriers to research.

  • RAVEN and CM-GRPO advance the practical viability of real-time autoregressive video generation, with both positive and negative societal implications.
  • Real-time lowlatency video synthesis could support creative tools, education, accessibility applications, and interactive simulation.
  • The approach could reduce video-generation compute footprints and lower barriers for downstream research.
Loading 2605.15190v1…