Source-linked AI summary
Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generation
Bin Wu, Mengqi Huang, Shaojin Wu, Weinan Jia, Yuxin Wang, Zhendong Mao, Yongdong Zhang
TL;DR
Existing DMD distillation weights every rollout, frame, and pixel equally despite variation in supervision reliability and improvement potential. Stream-R1 uses reward-guided reweighting at rollout and spatiotemporal levels, consistently improving visual quality, motion quality, and text alignment without added inference cost.
Problem
Existing DMD methods weight every rollout, frame, and pixel equally, overlooking reliability differences across rollouts and perplexity differences across spatiotemporal elements.
Method
Stream-R1 derives rollout reliability and spatiotemporal saliency weights from one pretrained video reward model, with adaptive balancing across quality dimensions.
Results
Stream-R1 consistently improves visual quality, motion quality, and text alignment over DMD baselines on standard streaming video generation benchmarks without architectural modification or added inference cost.
Takeaways & Limitations
Stream-R1 achieves the highest overall VBench score among compared methods and improves long-video imaging quality and temporal stability.
Abstract
from arXiv · showhide
Distillation-based acceleration has become foundational for making autoregressive streaming video diffusion models practical, with distribution matching distillation (DMD) as the de facto choice. Existing methods, however, train the student to match the teacher's output indiscriminately, treating every rollout, frame, and pixel as equally reliable supervision. We argue that this caps distilled quality, since it overlooks two complementary axes of variance in DMD supervision: Inter-Reliability across student rollouts whose supervision varies in reliability, and Intra-Perplexity across spatial regions and temporal frames that contribute unequally to where quality can still be improved. The objective thus conflates two questions under a uniform weight: whether to learn from each rollout, and where to concentrate optimization within it. To address this, we propose Stream-R1, a Reliability-Perplexity Aware Reward Distillation framework that adaptively reweights the distillation objective at both rollout and spatiotemporal-element levels through a single shared reward-guided mechanism. At the Inter-Reliability level, Stream-R1 rescales each rollout's loss by an exponential of a pretrained video reward score, so that rollouts with reliable supervision dominate optimization. At the Intra-Perplexity level, it back-propagates the same reward model to extract per-pixel gradient saliency, which is factored into spatial and temporal weights that concentrate optimization pressure on regions and frames where refinement yields the largest expected gain. An adaptive balancing mechanism prevents any single quality axis from dominating across visual quality, motion quality, and text alignment. Stream-R1 attains consistent improvements on all three dimensions over distillation baselines on standard streaming video generation benchmarks, without architectural modification or additional inference cost.
1 Introduction
Streaming video diffusion enables causal, sequential generation but still requires effective distillation to inherit a multi-step teacher’s fidelity. Stream-R1 addresses indiscriminate DMD supervision by modeling rollout-level reliability and within-rollout spatiotemporal perplexity with reward-guided reweighting.
- Motivation: Streaming video diffusion reduces the cost and fixed-window limitation of multi-step denoising by generating frames sequentially with support for unbounded video generation.Distillation-based acceleration is presented as necessary to make this paradigm practical.
- Limitation: Existing distillation methods assign equal weight to every rollout, frame, and pixel, implicitly treating all supervision as equally reliable.This indiscriminate treatment is identified as a shared limitation across otherwise different approaches.
- Conceptual contribution: DMD supervision varies along Inter-Reliability across student rollouts and Intra-Perplexity across spatial regions and temporal frames within each rollout.Inter-Reliability concerns supervision reliability, whereas Intra-Perplexity reflects how much further refinement can improve quality.
- Technical contribution: Stream-R1 adaptively reweights DMD at both rollout and spatiotemporal-element levels through a single reward-guided mechanism.At the rollout level, a pretrained video reward model produces an exponential loss rescaling for each rollout.
- Technical contribution: Stream-R1 derives both reliability and perplexity weights from one pretrained video reward model, with adaptive balancing across visual quality, motion quality, and text alignment.The framework reports consistent improvements over DMD-based baselines on standard streaming video generation benchmarks without architectural modification.
2 Related Work
Video diffusion models deliver strong visual synthesis but face efficiency and temporal-scalability limits from multi-step denoising over fixed windows. Related work addresses these constraints through autoregressive streaming formulations and reward- or preference-based optimization for visual generation.
- Video generation: Video diffusion models achieve remarkable visual synthesis but require multi-step denoising over fixed-length temporal windows, limiting inference efficiency and temporal scalability.This limitation motivates alternative formulations for scalable video generation.
- Video generation: Autoregressive diffusion enables streaming, frame-by-frame video synthesis that can in principle extend to arbitrary temporal horizons.Pyramidal-Flow reduces long-sequence computation with multi-scale flow matching, while SkyReels-V2 uses diffusion forcing and structural planning for scalable synthesis.
- Reward-based optimization: Reinforcement learning optimizes non-differentiable objectives and aligns generative models with human preferences, with growing adoption in visual generation.VideoReward, VideoScore, and VisionReward develop multidimensional quality scoring for visual content.
3 Methodology
Stream-R1 reweights DMD supervision along two axes: across rollouts according to supervision reliability and within rollouts according to reward-gradient saliency. It combines quality-specific saliencies, factors them into temporal and spatial weights, and balances optimization across quality dimensions.
- Inter-Reliability Score Extraction: Stream-R1 uses a pretrained video reward model to aggregate per-dimension rewards into an overall score that proxies rollout-level supervision reliability.High-scoring rollouts lie nearer the teacher’s high-quality mode, where DMD gradients more faithfully reflect reliable supervision.
- Inter-Reliability Score Extraction: The rollout loss is rescaled by an exponential function of the overall reward, causing reliable rollouts to dominate the gradient signal.The temperature β > 0 controls reweighting sharpness, and monotonicity in rfinal favors accurate score estimates over low-quality refinements.
- Adaptive Gradient-Saliency Combination: Per-pixel absolute reward gradients provide saliency maps identifying spatial regions and temporal frames where targeted refinement most affects each quality dimension.The saliency computation requires one backward pass through the reward model per dimension, adding negligible overhead relative to diffusion-model passes.
- Adaptive Gradient-Saliency Combination: Saliency maps are adaptively combined to prioritize lower-scoring quality dimensions while retaining signals from all dimensions.As τ → 0, the combination selects the worst-performing dimension; as τ → ∞, it becomes uniform averaging.
- Spatiotemporal Saliency Decomposition: The combined saliency volume is decomposed into separately normalized temporal and spatial weights, then multiplied and globally mean-normalized.Per-frame spatial normalization preserves within-frame contrast even when a frame’s overall saliency is low, while temporal weights modulate whole-frame contributions.
4 Experiments
Experiments show that Stream-R1 improves short- and long-video quality over distillation baselines across automated metrics, VLM evaluation, and human preference, while retaining substantial speed advantages. Its gains extend to temporal consistency, semantic alignment, and long-horizon stability.
- Short video generation: Compared with Reward Forcing, Stream-R1 improves Total, Quality, and Semantic scores by +0.27, +0.30, and +0.12, respectively.The Quality gain closes over 60% of the gap between Reward Forcing (84.84) and Wan2.1 (85.30), without additional inference cost.
- Short video generation: As a 4-step model, Stream-R1 surpasses Wan2.1 in Total (84.40 vs. 84.26) and Semantic (81.44 vs. 80.09) while running at 30× higher inference speed.These results indicate that reward-guided distillation can improve quality beyond the multi-step teacher while accelerating inference.
- Long video generation: Stream-R1 outperforms Reward Forcing on all six VBench metrics at every evaluated long-video duration, with the performance gap widening as duration increases.It achieves higher subject consistency, background consistency, imaging quality, motion smoothness, and aesthetic quality, while maintaining lower drift.
- Human preference evaluation: Stream-R1 is preferred over Reward Forcing on all five human-evaluation dimensions, with the largest margins on Dynamic Reasonableness (63.0%) and Visual Quality (60.0%).The study covers 50 long videos of 60 seconds using anonymized A/B comparisons.
5 Conclusion
Stream-R1 uses reward-model gradient backpropagation to factor scalar rewards into spatial and temporal saliency, concentrating optimization on deficient regions and frames without added inference cost. It achieves the highest overall VBench score among compared methods and the best long-video imaging quality with the lowest drift.
- Method: Stream-R1 decomposes scalar reward signals into spatial and temporal saliency through reward-model gradient backpropagation.This concentrates optimization intensity on quality-deficient regions and frames.
- Efficiency: Stream-R1 concentrates optimization on quality-deficient regions and frames at no additional inference cost.
- Results: Stream-R1 achieves the highest overall VBench score among all compared methods, including the multi-step bidirectional teacher Wan2.1.
- Results: On long video generation, Stream-R1 attains the best imaging quality and lowest drift.These results demonstrate superior temporal stability.