PaperlayerView exact revision on arXiv

Source-linked AI summary

OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning

Yunyang Ge, Xianyi He, Zezhong Zhang, Bin Lin, Bin Zhu, Xinhua Cheng, Li Yuan

arXiv:2605.28691v1cs.CV

TL;DR

Long video token sequences and full attention make high-quality video generation inefficient. OSP-Next combines sparse attention, sequence parallelism, 8-bit quantization, and reinforcement learning, achieving an 83.73% VBench score and substantial speedups across NVIDIA H200 and Ascend 950PR hardware.

  • Problem

    Long spatio-temporal sequences and full attention limit the speed and achievable resolution of video generation.

  • Method

    OSP-Next combines Skiparse-2D sparse attention, Sparse Sequence Parallelism, HiF8 quantization, and Mix-GRPO post-training for text-to-video generation.

  • Results

    83.73% VBench total score surpasses Wan2.1, while OSP-Next reaches up to 1.64× single-GPU and 1.52× eight-GPU speedups on NVIDIA H200 GPUs.

  • Takeaways & Limitations

    OSP-Next provides high-quality, accelerated sparse video generation across NVIDIA H200 and Ascend 950PR hardware platforms.

Abstract

from arXiv · show

Diffusion Transformers achieve strong video generation quality, but the quadratic cost of full attention limits efficiency. We introduce OSP-Next, an efficient text-to-video generation model that integrates sparse attention, parallelism, quantization, and reinforcement learning. OSP-Next uses a hybrid full-sparse attention architecture, where the sparse component is implemented with Skiparse-2D Attention. This fixed-pattern mechanism applies token-wise and group-wise sparse attention along spatial dimensions, leveraging locality while maintaining native compatibility with FlashAttention kernels. Based on the local equivalence of rearrangement in Skiparse-2D Attention, we further propose Sparse Sequence Parallelism (SSP), which partitions subsequences across ranks and switches sparse patterns through a single All-to-All communication. Compared with Ulysses Sequence Parallelism (SP), SSP provides a native parallel strategy for sparse attention and reduces communication volume by 75%. OSP-Next also incorporates HiF8 quantization to enable stable joint training with 8-bit quantization and sparse fine-tuning, and applies Mix-GRPO post-training to improve the performance of the sparse model. Experiments show that OSP-Next achieves a VBench total score of 83.73%, surpassing the Wan2.1 baseline. Under the 5-second 720P and 5-second 768P settings, OSP-Next achieves up to 1.64$\times$ single-GPU speedup and over 1.52$\times$ eight-GPU speedup on NVIDIA H200 GPUs. In addition, with only a 0.4% drop in VBench total score, OSP-Next-HiF8 achieves 1.69$\times$ and 2.27$\times$ speedups under the two settings on a single Ascend 950PR, demonstrating the efficiency and performance of OSP-Next across hardware platforms.

1 Introduction

OSP-Next addresses the speed and resolution limits of video Diffusion Transformers by integrating sparse attention, parallelism, HiF8 quantization, and reinforcement learning. It combines these techniques to retain FlashAttention compatibility, improve sparse-model quality, and accelerate video generation across NVIDIA and Ascend hardware.

  • Motivation: 30 minutes to one hour are required to generate a 5-second 720p video with HunyuanVideo or Wan2.1 on a single NVIDIA A100.The passage identifies DiT computational inefficiency and long video token sequences as limits on speed and achievable resolution.
  • OSP-Next: OSP-Next integrates sparse attention, parallelism, quantization, and reinforcement learning while maintaining native compatibility with FlashAttention kernels.The model is designed to mitigate prior performance and efficiency issues while supporting parallel computation.
  • Sparse Attention: Skiparse-2D Attention applies fixed-rule token-wise and group-wise sparse patterns separately along height and width to better align with spatial locality.This extends Skiparse-1D Attention and approximates 3D full attention without dynamic token selection.
  • Sparse Sequence Parallelism: OSP-Next introduces Sparse Sequence Parallelism by assigning rearranged subsequences to different ranks, enabling parallel sparse attention through Skiparse Rearrange’s local equivalence.The supplied passage states that Skiparse Rearrange partitions the original sequence into subsequences concatenated along the batch dimension.
  • Results: 83.73% is OSP-Next’s VBench total score, outperforming the Wan2.1 full-attention baseline.OSP-Next achieves 1.53× and 1.64× speedups on a single NVIDIA H200, 1.42× and 1.52× on eight NVIDIA H200 GPUs, while OSP-Next-HiF8 reaches 1.69× and 2.27× on a single Ascend 950PR with a 0.4% VBench drop.

2 Related Work

Prior work addresses video-generation efficiency through sparse attention, parallel execution, fine-grained quantization, and reinforcement-learning post-training, but sparse attention remains insufficiently integrated with sequence-parallel execution. Existing quantization and RL approaches also trade numerical or optimization benefits against system overhead and computational cost.

  • Sparse video generation: Sparse video-generation models reduce Diffusion Transformer cost through sparse attention patterns or training-free acceleration of pretrained models.Open-Sora Plan introduces Skiparse Attention by reorganizing tokens into sparse subsequences while preserving spatio-temporal modeling; Sparse VideoGen accelerates pretrained video DiTs without training.
  • Parallelism: Sequence-parallel methods partition sequences or circulate key-value blocks to support efficient long-sequence attention across devices.Ulysses-style sequence parallelism uses All-to-All communication, whereas Ring Attention circulates key-value blocks while overlapping communication with computation.
  • Parallelism: Sparse video generation and sequence parallelism remain underexplored because direct integration can cause workload imbalance and irregular communication.Different devices may receive different numbers of active tokens or attention blocks, motivating sparse models naturally compatible with sequence-parallel execution.
  • Quantization: Fine-grained quantization reduces error through adaptive scaling but adds scale computation, storage, loading, and broadcasting overhead.Prior work uses per-token activation scales, per-channel or per-block weight scales, and MXFP8 microscaling; HiF8 instead supports coarse per-tensor quantization through a wider dynamic range.
  • Reinforcement-learning post-training: RL post-training improves diffusion-video preference alignment using rewards for visual quality, motion consistency, and text-video alignment, but full-trajectory optimization is costly.FlowGRPO optimizes denoising trajectories with GRPO-style updates, while DiffusionNFT avoids approximate diffusion transition probabilities used for importance sampling or policy-ratio estimation.

3 Method · 3.1 Skiparse-2D Attention

Skiparse-2D Attention extends Skiparse-1D Attention to preserve two-dimensional locality in image and video modalities. It alternates token-wise and group-wise sparse patterns along spatial dimensions to reduce attention computation while enabling interactions between any two tokens within two operations.

  • 3.1 Skiparse-2D Attention: Flattening video latents into one dimension disrupts spatial locality, restricting interactions among adjacent tokens and misaligning global interactions with two-dimensional structure.
  • 3.1 Skiparse-2D Attention: For arbitrary video resolutions, the method pads dimensions to multiples of k^2 and assigns tokens at matching spatial positions to the same subfigure.
  • 3.1 Skiparse-2D Attention: Skiparse-2D Attention applies Skiparse-1D Attention separately along the width and height dimensions to construct a two-dimensional interaction pattern.
  • 3.1 Skiparse-2D Attention: The token-wise interaction pattern resembles pixel unshuffle, adapting sparse attention to the spatial organization of image and video modalities.
  • 3.1 Skiparse-2D Attention: Token-wise and group-wise sparse attention reduce the sequence dimension and increase the batch dimension, lowering the computation of each attention operation.
  • 3.1 Skiparse-2D Attention: Alternating token-wise and group-wise sparse attention uses the same skip interval in both operations and lets any two tokens interact within at most two attention operations.

3.2 Any Resolution Strategy

OSP-Next supports arbitrary spatial resolutions by padding dimensions to preserve Skiparse Rearrange’s local equivalence and unchanged non-padding interactions. Its masking strategy uses no mask for compatible resolutions and a one-dimensional flattened-sequence mask otherwise, while retaining native FlashAttention support.

  • Any Resolution Strategy: When H or W is not divisible by k^2, naive Skiparse Rearrange is invalid because subfigures cannot form the smallest repeatable unit.The smallest repeatable unit is a k^2 × k^2 subfigure.
  • Any Resolution Strategy: Padding the ends of both H and W preserves local equivalence and keeps interactions among non-padding tokens unchanged.The method constructs a corresponding attention mask after padding.
  • Any Resolution Strategy: When both H and W are exact multiples of k^2, Skiparse Attention uses no attention mask; otherwise, it uses a 1D flattened-sequence mask.This provides the masking rule for arbitrary resolutions.
  • Any Resolution Strategy: Skiparse Attention natively supports FlashAttention kernels and avoids FlexAttention’s compilation overhead without requiring a complex 2D mask.The local-equivalence design enables this implementation behavior.

3.3 Sparse Sequence Parallel (SSP)

Sparse Sequence Parallel (SSP) shards equal-length subsequences across ranks for balanced parallel computation and uses Skiparse Rearrange’s local-equivalence property to switch between sparse patterns. It is also compatible with Ulysses Sequence Parallel by repartitioning the enlarged batch dimension while preserving Skiparse Rearrange’s structure.

  • Sparse Sequence Parallel (SSP): In the non-parallel setting, transforming the original pattern into either sparse pattern and converting between sparse patterns each use a single rearrange operation.This operation is described in Algorithm 1.
  • Sparse Sequence Parallel (SSP): SSP shards Skiparse Rearrange’s equal-length subsequences across ranks, naturally maintaining load balance during parallel computation.Equal-length subsequences enable balanced distribution across ranks.
  • Sparse Sequence Parallel (SSP): Skiparse Rearrange’s local-equivalence property enables switching between the two sparse patterns without an All-G communication step.The supplied passage truncates after “All-G,” so the complete communication claim is not recoverable.
  • Sparse Sequence Parallel (SSP): SSP is compatible with Ulysses Sequence Parallel because it repartitions the enlarged batch dimension after Skiparse Rearrange.When both strategies are enabled, hidden states are partitioned at the boundaries of the smallest repeatable units before the main blocks.

3.4 Model Structure

OSP-Next adopts a spindle-shaped architecture to balance performance and efficiency. Like Open-Sora Plan v1.3, it places full-attention layers at the outer ends and sparse-attention layers in the middle.

  • 3.4 Model Structure: OSP-Next uses a spindle-shaped structure similar to Open-Sora Plan v1.3, combining full attention in outer layers with Skiparse Attention in middle layers.Open-Sora Plan v1.3 uses full attention in the first and last 2n layers, while OSP-Next adopts the same overall structure for performance-efficiency balance.

3.5 HiF8 Quantization

HiF8 is an 8-bit floating-point format that dynamically allocates bits between exponent and mantissa to balance dynamic range and precision. Its design supports wide exponent coverage, tapered precision, redundancy-free encoding, and mixed-precision training with adaptive scaling.

  • Format Design: HiF8 dynamically allocates bits between exponent and mantissa through a variable-width Dot field, mitigating the trade-off between dynamic range and representational precision.Unlike E4M3 and E5M2, HiF8 uses a tapered precision design tailored to neural-network value distributions.
  • Wide Dynamic Range: 38 distinct exponent values are covered by HiF8, approaching FP16’s 40-exponent coverage and reducing overflow or underflow risks in DiT attention layers.HiF8 extends the effective exponent range to [−22, 15], while FP16 covers [−24, 15].
  • Tapered Precision: HiF8 progressively reduces mantissa width as value magnitude increases, preserving higher precision in frequently occurring central value regions.This tapered allocation addresses the opposing precision and dynamic-range compromises of E4M3 and E5M2.
  • Redundancy-Free Encoding: HiF8 uses sign-magnitude exponent encoding with an implicit leading 1, covering the exponent space without gaps or duplicate representations.This avoids redundant code points present in conventional IEEE 754-derived FP8 formats.
  • Mixed-Precision Training: HiF8max is set to 15 for forward computation and 224 for backward computation when scaling BF16 and FP32 tensors into the representable range.The scaling factor is derived from each data block’s maximum absolute value and a small numerical-stability constant.

3.6 Mix-GRPO For the Sparse-Quant Model

Mix-GRPO uses hybrid SDE-ODE denoising for efficient preference alignment of the sparse-quant model. It limits stochastic exploration and optimization to selected early timesteps, reducing rollout costs while recovering quality lost through sparsification and quantization.

  • Mix-GRPO For the Sparse-Quant Model: Mix-GRPO aligns the fine-tuned sparse model with video-generation preferences while preserving OSP-Next’s efficiency advantages.The method addresses the quality gap introduced when converting a full-attention model to sparse attention.
  • Mix-GRPO For the Sparse-Quant Model: Mix-GRPO applies SDE sampling during initial denoising steps and deterministic ODE sampling during later stages.This shortens the effective MDP horizon and confines stochastic exploration to optimized timesteps.
  • Mix-GRPO For the Sparse-Quant Model: Only a subset of timesteps contributes to the GRPO loss, reducing optimization overhead and rollout cost for video generation.ODE portions are excluded from the reinforcement-learning objective, avoiding unnecessary stochastic sampling.
  • Mix-GRPO For the Sparse-Quant Model: Mix-GRPO helps recover performance losses from sparse fine-tuning and quantization while remaining compatible with Skiparse-2D Attention, SSP, and HiF8 simulation.The post-training stage provides stable gains for sparse video generation and brings OSP-Next closer to the full-attention model.
  • Mix-GRPO For the Sparse-Quant Model: VideoAlign supplies visual quality, motion quality, text alignment, and overall-score rewards, with the overall score used for Mix-GRPO optimization.Its video-native training reduces the evaluation gap relative to image-trained models such as CLIP.

4 Experiment · 4.1 Training Setup · 4.2 Main Results

OSP-Next is trained on filtered 81-frame, 720 × 1280 videos and evaluated with VBench, visual comparisons, and cross-platform inference-speed measurements. It surpasses Wan2.1 in VBench while preserving visual quality and delivering substantial acceleration, including HiF8 gains on Ascend 950PR.

  • 4.1 Training Setup: The internally collected training set is filtered for overly static or low-resolution videos and fixed at 81 frames and 720 × 1280 resolution.
  • 4.1 Training Setup: OSP-Next initializes from Wan2.1-T2V-14B and combines eight full-attention blocks with 32 middle Skiparse Attention layers at sparse ratio 2.Training uses SSP with parallelism degree 4, FSDP, and FSDP-EMA.
  • 4.1 Training Setup: Inference uses a Discrete Euler sampler with 50 steps, shift value 7.0, and classifier-free guidance 5.0.
  • 4.2.1 Vbench Scores: VBench is used to quantify model generation quality across the OSP-Next series and Wan2.1 baseline.
  • 4.2.1 Vbench Scores: 83.73% is OSP-Next’s total VBench score, surpassing Wan2.1; OSP-Next-HiF8 reaches 83.29%, keeping the gap from the baseline below 0.4%.The results support joint training of HiF8 precision with the sparse model while preserving pretrained capability.
  • 4.2.2 Visual Results: OSP-Next shows no noticeable visual-quality difference from Wan2.1, while OSP-Next-HiF8 shows no clear degradation compared with OSP-Next.The visual results support sparse-model fine-tuning combined with 8-bit quantization.
  • 4.2.3 Inference Speed: Inference speed is evaluated on NVIDIA H200, Ascend 910C, and Ascend 950PR using comparisons across single- and multi-GPU/NPU settings.
  • 4.2.3 Inference Speed: 1.53× is the single-GPU/NPU speedup OSP-Next achieves over Wan2.1 under both padding and no-padding settings, while OSP-Next-HiF8 reaches up to 2.27× on Ascend 950PR.At 2- and 4-GPU/NPU settings, Skiparse blocks use SSP; at 8-GPU/NPU, they use SSP and SP, while full-attention blocks use Ulysses SP.

4.3 Ablation Study

The ablations show that OSP-Next’s efficiency benefits from sparse attention and HiF8 quantization, while full-attention blocks and Mix-GRPO mitigate the capability gap from adapting pretrained full-attention models.

  • Inference speed: More than 1.42× speedup is achieved across hardware models in both 81-frame 720P and 768P settings, with better performance without padding.The 720P setting includes padding and attention masks, whereas 768P does not.
  • Inference speed: 2.27× speedup is achieved by OSP-Next-HiF8 on Ascend 950PR, demonstrating the efficiency advantage of HiF8 quantization.This result is reported for the evaluated inference-speed settings.
  • Initialization gap: All-Skiparse sparse attention fails to generate valid videos in zero-shot inference, revealing a transfer gap from pretrained full attention.The gap is evaluated by varying the number of full-attention blocks in the spindle-shaped architecture.
  • Initialization gap: OSP-Next reduces this initialization gap with a spindle-shaped architecture that places full-attention blocks on both sides of the sparse-attention blocks.Increasing the number of full-attention blocks progressively brings zero-shot sparse-model results closer to the baseline.
  • Reinforcement learning: Mix-GRPO post-training substantially improves spatial relationships, text-video alignment, and aesthetics in the sparse model.These gains indicate reinforcement learning is effective during sparse-model post-training.

5 Conclusion

OSP-Next introduces a sparse video model built around Skiparse-2D Attention and a sequence-parallel strategy natively aligned with its sparse pattern. Sparse Sequence Parallelism reduces communication volume by 75% versus Ulysses Sequence Parallelism and can be used jointly with it.

  • 5 Conclusion: OSP-Next uses Skiparse-2D Attention to better leverage locality in image and video modalities.The model is described as a sparse model designed around the locality inherent in these modalities.
  • 5 Conclusion: Sparse Sequence Parallelism is natively aligned with the sparse attention pattern of Skiparse-2D Attention.SSP is developed from the properties of Skiparse-2D Attention.
  • 5 Conclusion: 75%: SSP reduces communication volume versus Ulysses Sequence Parallelism and can be used jointly with Ulysses SP.The conclusion explicitly reports a 75% communication-volume reduction compared with Ulysses SP and joint usability with it.
Loading 2605.28691v1…