PaperlayerView exact revision on arXiv

Source-linked AI summary

FlashVSR+: Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching

Ruchang Yao, Shi Guo, Junhao Zhuang, Mingde Yao, Xin Cai, Jiahao Zhan, Bin Fu, Xinyuan Chen, Yu Qiao, Tianfan Xue

arXiv:2610.17552v1cs.CVcs.AI

TL;DR

High-resolution VSR remains costly, and one-step distillation is difficult to align with human preferences. FlashVSR+ combines hybrid attention with Ry-DMD, using reward-weighted teacher rollouts without reward gradients. It improves the quality-efficiency trade-off, enables spatially untiled 4K inference on one 24GB RTX 4090, and demonstrates Ry-DMD gains in VSR and text-to-image generation.

  • Problem

    High-resolution video attention remains costly, while standard one-step DMD does not explicitly favor human-preferred outputs and existing reward methods are ill-suited to deterministic generators with black-box rewards.

    Source: §1 · p. 2

  • Method

    FlashVSR+ combines local attention with sparse global blocks and uses Ry-DMD to reweight teacher rollouts by black-box rewards, correcting the DMD teacher term without reward gradients.

    Source: §1 · p. 2

  • Results

    FlashVSR+ improves the quality-efficiency trade-off and enables spatially untiled 4K inference on one 24GB RTX 4090; Ry-DMD raises OCR score from 0.087 to 0.241 and SD3.5-M OCR accuracy from 0.501 to 0.824 without increasing NFE.

    Source: §4.4 · p. 10, §5 · p. 11

  • Takeaways & Limitations

    The framework demonstrates efficient high-resolution VSR and black-box reward optimization in both VSR and text-to-image generation.

    Source: §5 · p. 11

    Reward tilting remains within the teacher distribution's original support, reweighting teacher-supported outputs rather than expanding that support.

    Source: §3.3.1 · p. 5

Abstract

from arXiv · show

Recent video super-resolution (VSR) models achieve efficient one-step generation through diffusion distillation. However, high-resolution inference remains costly, and distilled models are difficult to align with human preferences. These challenges call for improvements in both model architecture and training strategy. We present \textbf{FlashVSR+} to address both limitations. On the architecture side, we introduce hybrid attention based on the strong locality observed in pretrained VSR models, while retaining sparse global interaction for long-range consistency. It achieves up to $2\times$ DiT speedup and 45\% lower peak memory, enabling spatially untiled 4K VSR on a single 24GB RTX 4090. On the training side, we propose Reward-Yield Distribution Matching Distillation (Ry-DMD) for preference-aligned one-step generation. Ry-DMD uses reward-weighted teacher rollouts to incorporate black-box rewards into distribution matching, without backpropagating through the reward model. We further show that Ry-DMD generalizes beyond VSR to T2I generation. Together, FlashVSR+ delivers stronger restoration quality and substantially improves high-resolution efficiency, with consistent gains in user preference. We release our code, models, and training data at \href{https://www.flashvsr.tech}{\textcolor{FlashBlue}{\textbf{www.flashvsr.tech}}}.

1 Introduction

FlashVSR+ targets the high memory and computation costs of high-resolution generative VSR and the difficulty of aligning one-step models with human preferences. It combines hybrid attention with Ry-DMD, while experiments show that longer DMD training alone does not yield consistent gains.

1 Introduction

  • High-resolution attention costs can make consumer-GPU inference require spatial tiling, which limits global context and may introduce boundary artifacts.Standard DMD also cannot explicitly favor outputs aligned with human preferences, while existing reward-guided methods are ill-suited to deterministic one-step generators with black-box rewards.Source: §1 · p. 2
  • Up to 2× DiT acceleration and nearly half the peak memory enable spatially untiled 4K VSR on one 24GB RTX 4090 while maintaining comparable restoration quality.Hybrid attention replaces most attention blocks with local spatiotemporal attention while retaining sparse global blocks.Source: §1 · p. 2
  • Ry-DMD uses reward-weighted teacher rollouts to guide one-step generation toward higher-reward outputs without differentiating through black-box rewards.The difference between reward-weighted and original teacher predictions supplies a preference direction added to standard DMD.Source: §1 · p. 2
  • Longer DMD training does not yield consistent gains; student initialization and teacher capacity matter more for final performance.Source: §1 · p. 2

2 Related Work

Related work spans reconstruction-based and generative VSR, efficient attention for visual generation, and reward optimization for diffusion models. FlashVSR+ addresses remaining challenges in high-resolution one-step VSR efficiency and reward-guided training.

2.1 Video Super-Resolution

  • Conventional VSR aggregates information across frames but can over-smooth ambiguous textures; generative VSR uses learned priors, and recent work increasingly targets one-step restoration.Approaches include direct fine-tuning, adversarial post-training, and distillation [Liu et al., 2025; Chen et al., 2026d; Wang et al., 2026; Sun et al., 2026; Zhuang et al., 2026; Lv et al., 2026].Source: §2.1 · pp. 2–3

2.2 Efficient Attention for Visual Generation

  • Visual-generation attention methods use sparsity, locality, linear attention, or hybrid backbones; FlashVSR+ instead exploits locality observed in a pretrained VSR model.It replaces most global-attention blocks with spatiotemporal local attention while retaining a small number of global blocks.Source: §2.2 · p. 3

2.3 Reward Optimization for One/Few-Step Generative Models

  • Reward optimization is challenging for one-step generators because they lack intermediate trajectory signals; Ry-DMD introduces a reward-tilted teacher target within DMD for black-box rewards.Prior approaches include trajectory-based optimization, differentiable rewards, inference-time optimization, and differentiable surrogates [Zhu et al., 2026].Source: §2.3 · p. 3

3 Method

FlashVSR+ uses a three-stage pipeline to produce a one-step VSR model with hybrid attention and reward-guided DMD. Ry-DMD shifts teacher targets using scalar-reward-weighted rollouts, without reward backpropagation.

3.1 Overview

  • The three-stage pipeline trains a 14B full-attention multi-step teacher, initializes a hybrid-attention student, then distills it into a one-step model using distribution matching and pixel loss.Ry-DMD injects scalar reward feedback into the teacher term without reward backpropagation.Source: §3.1 · p. 3, Figure 2 · p. 4

3.2 Efficient One-Step VSR with Hybrid Attention

  • Hybrid attention replaces most blocks with local spatiotemporal attention and retains sparse global blocks; the final configuration is (3, 1, 25), with one global block every five DiT blocks.After hybrid training, retained global blocks show reduced local attention mass, indicating a shift toward non-local interactions.Source: §3.2 · pp. 4–5, Figure 3 · p. 4

3.3 Reward-Yield Distribution Matching Distillation

  • Ry-DMD constructs a reward-tilted teacher target and converts its shift into a correction to the DMD teacher term, preserving the overall DMD framework.It supports black-box objectives such as OCR and is demonstrated on VSR and text-to-image generation.Source: §3.3 · p. 5

3.3.1 Reward-Tilted Distribution Matching

  • Reward tilting reweights the clean teacher distribution toward higher-reward samples while preserving its support; as β approaches zero, it recovers standard teacher matching.The reweighting is also described as the solution to a KL-regularized expected-reward objective.Source: §3.3.1 · p. 5

3.3.2 Local Reward-Tilted Teacher Target

  • At each noisy student state, stochastic K-step teacher rollouts approximate the conditional endpoint distribution; standardized scalar rewards weight rollouts to estimate the local target shift without reward gradients.The estimated shift is derived entirely from teacher samples and scalar reward evaluations.Source: §3.3.2 · pp. 5–6

3.3.3 Reward-Yield DMD Update

  • The estimated endpoint displacement becomes a correction to the DMD teacher term; λ_RY controls its strength and ε_σ stabilizes the update at small noise levels.Source: §3.3.3 · p. 6

4 Experiments

The experiments assess FlashVSR+ on VSR quality and efficiency, hybrid-attention configurations, OCR-guided post-training, text-to-image transfer, and DMD scaling. Results report user preference for Ours-Base, improved high-resolution efficiency, and reward-guided gains across VSR and text-to-image settings.

4.1 Comparison with State-of-the-Art VSR Methods

  • Ours-Base is preferred over all compared methods in a blinded pairwise study, while Ours-Turbo remains close despite higher inference efficiency.The evaluation also reports strong overall restoration quality across the tested VSR benchmarks.Source: Table 1 · p. 6, §4.1 · p. 6
  • FlashVSR+ substantially reduces inference latency and peak memory versus the base model, enabling 4K VSR on one RTX 4090 with additional memory optimizations.Figure 6 evaluates 89-frame videos at 1080p, 2K, and 4K; its RTX 4090 wall-clock measurement includes CPU KV-cache offloading and asynchronous prefetching overhead.Source: Figure 6 · p. 7, §4.1 · p. 8

4.2 Efficient VSR with Hybrid Attention

  • After adaptation, (3, 1, 25) is the default: more localization reduces latency and memory while preserving quality, but (3, 1, 30) causes a quality drop.Without adaptation, direct replacement of several more restrictive configurations introduces structural artifacts.Source: Figure 7 · p. 8, §4.2 · p. 9

4.3 Reward-Guided VSR Post-Training

  • Ry-DMD achieves the strongest OCR improvement while preserving restoration quality; qualitative comparisons show clearer text restoration with fewer artifacts.The evaluation compares it with the pretrained baseline, continued standard DMD training, and DMDR, which uses a differentiable OCR surrogate.Source: §4.3 · p. 9, Figure 8 · p. 9, Table 3 · p. 9

4.4 Generalization to Text-to-Image Generation

  • On SDXL, Ry-DMD raises OCR score from 0.087 to 0.241, surpassing the 50-step teacher while maintaining or improving complementary quality metrics.On four-step SD3.5-M, OCR accuracy rises from 0.501 to 0.824 without increasing NFE.Source: §4.4 · p. 10, Table 4 · p. 10, Figure 9 · p. 10

4.5 Rethinking DMD Scaling for One-Step VSR

  • DMD metrics improve rapidly early, then become non-monotonic and can decline, so increasing update count alone does not reliably improve the final model.The evaluation uses a fixed held-out set drawn from multiple synthetic and real-world benchmarks.Source: Figure 10 · p. 10, §4.5 · pp. 10–11
  • A stronger student initialization and a larger teacher each improve the final student, with their combination performing best.These results indicate that effective DMD scaling depends on the overall distillation setup, not optimization duration alone.Source: Table 5 · p. 11, §4.5 · p. 11

5 Conclusion

FlashVSR+ combines hybrid spatiotemporal attention with Ry-DMD for efficient, reward-guided one-step VSR. It improves the quality-efficiency trade-off and enables spatially untiled 4K inference on a single 24GB RTX 4090.

5 Conclusion

  • FlashVSR+ improves the quality-efficiency trade-off while enabling spatially untiled 4K inference on a single 24GB RTX 4090.The framework combines hybrid spatiotemporal attention with Ry-DMD for efficient, reward-guided one-step VSR.Source: §5 · p. 11
  • Ry-DMD enables black-box reward optimization without backpropagating through the reward model.Its benefits are demonstrated in both VSR and text-to-image generation.Source: §5 · p. 11
  • FlashVSR+ combines hybrid spatiotemporal attention with reward-guided one-step VSR.The framework pairs its attention architecture with Ry-DMD.Source: §5 · p. 11

A Training Details

Training proceeds through teacher training, hybrid-attention student training, and one-step distillation. The Ry-DMD configuration uses β = 0.1, λRY = 0.2, J = 4, and K = 3.

A Training Details

  • The 14B full-attention teacher is trained for 4 days with LoRA, then the hybrid-attention student is trained for 2 days from a pretrained FlashVSR stage-2 checkpoint.Teacher training uses rank 384 and batch size 16; student training uses full-parameter training, batch size 32, and EMA.Source: A Training Details · p. 16
  • One-step distillation uses the 14B teacher for 600 steps with full-parameter training, batch size 16, and EMA.The optimizer uses AdamW with (β1, β2) = (0, 0.999) and a learning rate of 1 × 10−5.Source: A Training Details · p. 16
  • Ry-DMD uses β = 0.1, λRY = 0.2, J = 4, and K = 3, alongside a filtered coreset and text-SFT augmentation.Training uses AdamW in the teacher and student stages, with the specified stage-specific learning rates and batch sizes.Source: A Training Details · p. 16

B RTX 4090 Implementation

The RTX 4090 implementation adapts hybrid attention with Triton and reduces persistent GPU memory use by offloading global-attention KV caches. It processes each frame at full spatial resolution for untiled 4K inference on one 24GB RTX 4090.

B RTX 4090 Implementation

  • A lightweight Triton adaptation based on FlashAttention-2 supports the hybrid-attention computation on Ada GPUs.The H200-equivalent implementation's attention operators are not directly supported on RTX 4090 GPUs.Source: B RTX 4090 Implementation · p. 16
  • Selective offloading of retained global-attention KV caches to pinned CPU memory enables spatially untiled 4K inference on a single 24GB RTX 4090.Caches are asynchronously transferred on a dedicated CUDA stream and prefetched before the corresponding layer executes in the next streaming chunk.Source: B RTX 4090 Implementation · p. 16
  • The offloading adaptation changes KV-cache residency, not spatial processing: each frame is processed at full spatial resolution.Most transfers overlap with computation, reducing persistent GPU memory usage.Source: B RTX 4090 Implementation · p. 16

C Ry-DMD Hyperparameter Analysis

The SDXL analysis varies four Ry-DMD hyperparameters individually from a stated default configuration. Performance is stable under moderate variations, while overly strong reward weighting or correction degrades it.

C Ry-DMD Hyperparameter Analysis

  • Ry-DMD on SDXL is stable under moderate variations of β, λRY, J, and K, but overly strong reward weighting or correction degrades performance.The analysis varies one parameter at a time while keeping the others fixed.Source: C Ry-DMD Hyperparameter Analysis · p. 16
  • The SDXL hyperparameter analysis uses defaults β = 0.1, λRY = 0.2, J = 4, and K = 3.Table 6 varies one hyperparameter at a time from this default configuration.Source: C Ry-DMD Hyperparameter Analysis · p. 16, Table 6 · p. 16
  • Overly strong reward weighting or correction degrades Ry-DMD performance on SDXL.The analysis concerns reward temperature β and correction weight λRY, as well as rollout count J and depth K.Source: C Ry-DMD Hyperparameter Analysis · p. 16

D Extending Ry-DMD to Few-Step Generation

Ry-DMD is evaluated for four-step SD3.5-M with OCR-guided post-training. It improves OCR accuracy while preserving the sampling budget and improving general image-quality and preference metrics.

D Extending Ry-DMD to Few-Step Generation

E Additional Derivations for Ry-DMD

The derivations connect reward tilting to a KL-regularized objective, estimate its local effect from teacher rollouts, and translate the estimated endpoint shift into the DMD teacher correction.

E.1 Derivation of the Reward-Tilted Target

  • The KL-regularized expected-reward objective yields an exponential tilt of the clean teacher distribution that raises higher-reward sample density while preserving teacher support.As β approaches zero, the tilted distribution returns to the original teacher distribution.Source: §E.1 · p. 17

E.2 Local Reward-Tilted Mean Estimation

  • Local reward tilting shifts the conditional teacher endpoint mean, estimated in practice from independent stochastic teacher rollouts.The finite-rollout estimator uses sampled endpoints and rewards to estimate the reward-induced displacement.Source: §E.2 · pp. 17–18
  • Group-standardized rewards produce an adaptive tilt that reduces sensitivity to reward-function scale while preserving within-group ordering.The small-β shift follows variations in x0 that covary positively with reward.Source: §E.2 · p. 18

E.3 From Endpoint Shift to the DMD Teacher Correction

  • Ry-DMD adds the rollout-estimated endpoint displacement to the clean teacher target, with a stability adjustment at small noise levels.The remaining DMD optimization is unchanged and requires sampled teacher endpoints and scalar rewards, not reward-function gradients.Source: §E.3 · p. 19

F Synthetic Text-OCR VSR Dataset

The synthetic text-focused VSR dataset pairs procedurally rendered text targets with degraded low-quality inputs and retains annotations for OCR supervision.

F Synthetic Text-OCR VSR Dataset

  • The dataset is built from the VSR video corpus by rendering synthetic text onto decoded high-quality frames and degrading them with the standard VSR pipeline.Procedural rendering provides exact text content and spatial annotations.Source: F Synthetic Text-OCR VSR Dataset · p. 19
  • Each sample contains one or two multilingual text instances with randomized appearance and temporally coherent motion.Text may remain static, translate, change scale or rotation, or move partly outside the frame; per-frame locations and visibility are annotated.Source: F Synthetic Text-OCR VSR Dataset · p. 19
  • Invalid or insufficiently visible renderings are filtered, and OCR rewards use visible annotated regions from a small number of sampled frames.Compared post-training methods share the same source videos, targets, degradation, and filtering rules.Source: F Synthetic Text-OCR VSR Dataset · p. 19

Related papers

Loading 2610.17552v1…