PaperlayerView exact revision on arXiv

Source-linked AI summary

Uniform Discrete Diffusion with Metric Path for Video Generation

Haoge Deng, Ting Pan, Fan Zhang, Yang Liu, Zhuoyan Luo, Yufeng Cui, Wenxuan Wang, Chunhua Shen, Shiguang Shan, Zhaoxiang Zhang, Xinlong Wang

arXiv:2510.24717v1cs.CV

TL;DR

Discrete visual generation lags continuous methods and faces error accumulation and long-context consistency challenges. URSA iteratively refines discrete tokens using metric-path designs and asynchronous scheduling; it outperforms existing discrete methods and reports competitive results against continuous diffusion models.

  • Problem

    Discrete approaches lag continuous visual generation and face error accumulation and long-context consistency challenges, especially for video.

    Source: §1 · p. 2

  • Method

    URSA generates images and videos through iterative global refinement of discrete tokens, using a linearized metric path, resolution-dependent timestep shifting, and asynchronous temporal scheduling.

    Source: §1 · p. 2

  • Results

    URSA consistently outperforms existing discrete methods and achieves highly competitive results against state-of-the-art continuous diffusion models; its text-to-video score is 82.4 on VBench.

    Source: §1 · p. 3, §5 · p. 11

  • Takeaways & Limitations

    The authors frame URSA as a step toward unifying discrete and continuous generative modeling paradigms.

    Source: §5 · p. 11

Abstract

from arXiv · show

Continuous-space video generation has advanced rapidly, while discrete approaches lag behind due to error accumulation and long-context inconsistency. In this work, we revisit discrete generative modeling and present Uniform discRete diffuSion with metric pAth (URSA), a simple yet powerful framework that bridges the gap with continuous approaches for the scalable video generation. At its core, URSA formulates the video generation task as an iterative global refinement of discrete spatiotemporal tokens. It integrates two key designs: a Linearized Metric Path and a Resolution-dependent Timestep Shifting mechanism. These designs enable URSA to scale efficiently to high-resolution image synthesis and long-duration video generation, while requiring significantly fewer inference steps. Additionally, we introduce an asynchronous temporal fine-tuning strategy that unifies versatile tasks within a single model, including interpolation and image-to-video generation. Extensive experiments on challenging video and image generation benchmarks demonstrate that URSA consistently outperforms existing discrete methods and achieves performance comparable to state-of-the-art continuous diffusion methods. Code and models are available at https://github.com/baaivision/URSA

1 INTRODUCTION

URSA uses iterative global refinement of discrete tokens to address the quality and long-context challenges facing discrete visual generation. Its metric-path designs and asynchronous scheduling target scalable, multi-task generation, and experiments report strong benchmark performance.

Motivation

  • Discrete visual methods lag continuous approaches amid error accumulation and difficulty maintaining long-context consistency, particularly in video generation.The introduction notes low visual quality and unnatural object motions in masked diffusion models.Source: §1 · p. 2

URSA

  • URSA generates images and videos by iteratively refining global discrete spatiotemporal tokens rather than fixing tokens as they are produced.It starts from categorical noise and refines along a metric-guided probability path toward the target data.Source: §1 · p. 2
  • A linearized metric path and resolution-dependent timestep shifting adapt perturbations to long visual sequences, supporting high-resolution images and long videos with fewer inference steps.Asynchronous timestep scheduling independently samples timesteps for each frame and supports minute-level videos and multiple tasks in one model.Source: §1 · p. 2

Results

  • 82.4 on VBench for text-to-video, outperforming discrete and continuous baselines, is URSA’s reported benchmark result.URSA also reports 86.2 on VBench for image-to-video and 86.0 on DPG-Bench for text-to-image.Source: §1 · p. 3

Contributions

  • The paper presents URSA as a framework bridging the gap to continuous diffusion and enabling scalable video generation.Its stated contributions include state-of-the-art performance on VBench, DPG-Bench, and GenEval.Source: §1 · p. 3

2 RELATED WORKS

Related work contrasts the progress of continuous-space generation with discrete visual approaches, including autoregressive and masked diffusion methods. It positions URSA as discrete diffusion using iterative global refinement.

2.1 CONTINUOUS-SPACE VISUAL GENERATION

  • Continuous-space diffusion models have achieved strong image and video generation results, while URSA shares their global iterative-refinement approach over discrete tokens.The passage cites diffusion advances among the developments behind this progress.Source: §2.1 · p. 3

2.2 DISCRETE-SPACE VISUAL GENERATION

  • Autoregressive visual models predict tokens sequentially from prior context, a design associated with slow inference and error accumulation.The passage discusses image and video applications, citing Sun et al. (2024a) and Wang et al. (2024b), among others.Source: §2.2 · p. 3
  • Masked diffusion enables parallel generation and global-context modeling, but long-sequence tasks such as high-fidelity long-form video remain challenging.URSA is presented as uniform discrete diffusion that performs iterative global refinement from categorical noise.Source: §2.2 · p. 3

3 METHODOLOGY

URSA uses discrete flow matching as a foundation for global refinement of image and video tokens. Its metric path, resolution-dependent timestep shifting, and asynchronous frame scheduling address long-sequence modeling and support unified multitask generation.

3.1 PRELIMINARY: DISCRETE FLOW MATCHING

  • Discrete flow matching defines a time-dependent probability path from a source distribution to a target, whose evolution is governed by a learned CTMC transition-rate field.The velocity field describes how the current state moves toward the target over time.Source: §3.1 · p. 4

3.2 UNIFORM DISCRETE DIFFUSION WITH METRIC PATH

  • URSA combines a linearized metric path, resolution-dependent timestep shifting, and frame-wise independent perturbation scheduling for image and video generation.The framework is presented as bridging discrete and continuous approaches for scalable video generation.Source: §3 · p. 3, §3.2 · p. 4

3.2.1 METRIC PROBABILITY PATH FOR LONG SEQUENCE DATA

  • The linearized metric path derives token transition probabilities from embedding distances, with distance zero exactly when generated and target tokens match.Its scheduler has boundary values β_0 = 0 and β_1 = ∞; the path aims to make token distance vary linearly with time, from uniform noise to the target. Figure 3 depicts refinement from global semantics toward fine details.Source: §3.2.1 · pp. 4–5, Figure 3 · p. 5
  • Resolution-dependent timestep shifting adjusts perturbation to sequence resolution: λ > 1 gives higher resolutions stronger perturbations, while λ < 1 makes perturbations more gradual.The shift modulates the linear relationship between time and token distance.Source: §3.2.1 · p. 5

3.2.2 ASYNCHRONOUS TIMESTEP SCHEDULING

  • Asynchronous scheduling assigns each video frame an independent uniform timestep, enabling one model to train for text-to-video, image-to-video, extrapolation, and start–end-frame control.Decoupled frame noise levels balance local frame reconstruction with global temporal coherence during training.Source: §3.2.2 · p. 5

3.2.3 TRAINING AND SAMPLING

  • Training encodes clips as discrete frame-token sequences, perturbs frames using the metric path, and predicts clean tokens from text and noisy tokens.The objective is expected cross-entropy; sampling starts from uniform vocabulary noise and iteratively refines predictions with an Euler solver.Source: §3.2.3 · p. 6

4 EXPERIMENT

Experiments evaluate URSA across text-to-video, image-to-video, and high-resolution image generation, alongside ablations of its sampling, path, model-size, and timestep choices. URSA matches or exceeds Sora-like text-to-video models, performs strongly on image-to-video and image benchmarks, and shows distinct semantic and fidelity effects across ablations.

4.2 MAIN RESULTS

  • URSA matches or exceeds Sora-like text-to-video models on VBench, particularly on semantic metrics, despite using a discrete video tokenizer.The comparison includes Vchitect, Pyramid Flow, LuminaVideo, OpenSora, and OpenSoraPlan.Source: §4.2 · p. 7, Table 1 · p. 7
  • URSA excels in camera control and subject movement for image-to-video generation and achieves a leading DPG-Bench score with dense prompts.The reported image-to-video evaluation uses VBench++; performance on GenEval remains strong with rewritten prompts.Source: §4.2 · p. 7, Table 2 · p. 8, Table 3 · p. 8
  • At high resolutions, URSA reduces inference steps versus autoregressive and masked diffusion approaches while preserving fine-grained detail.Source: §4.2 · p. 8

4.3 ABLATION STUDY

  • Across identical training iterations, reducing inference steps without iterative refinement lowers masked-diffusion GenEval scores, while video generation requires stepwise error correction for temporal coherence and fidelity.The comparison evaluates text-to-image and text-to-video sampling performance across inference steps.Source: §4.3 · p. 8, Figure 4 · p. 9
  • Path choice depends on c and α; calibrating them to make timestep and noisy-to-clean embedding distance strongly linear helps reach model-performance limits for each tokenizer.Figure 5 reports Pearson correlations of -0.995, -0.921, -0.997, and -0.949 for the evaluated paths.Source: §4.3 · p. 9, Figure 5 · p. 9
  • Increasing model size from 0.6B to 4B improves semantic performance across image and video evaluations but does not significantly improve generation quality.The authors suggest discrete tokenizer representation capacity may constrain output fidelity.Source: §4.3 · p. 9, Figure 6 · p. 10
  • After about 30K iterations, timestep embedding or prompting provides no measurable GenEval benefit, and increasing embedding variance may disrupt token embeddings and training stability.The study compares three timestep-conditioning strategies across training iterations.Source: §4.3 · pp. 9–10, Figure 7 · p. 10
  • For video generation, Esser et al.'s timestep-shifting strategy is effective for uniform diffusion and enables URSA to match continuous counterparts.The study trains four text-to-video models with different timestep shifts and evaluates their VBench performance.Source: Figure 8 · p. 10, §4.3 · p. 11

5 CONCLUSION

URSA combines a linearized metric path and resolution-dependent timestep shifting to control perturbations in long sequences, with asynchronous temporal scheduling for multi-task video generation. Experiments show stronger results than existing discrete methods and competitive results against state-of-the-art continuous diffusion models.

5 CONCLUSION

  • URSA consistently outperforms existing discrete methods and achieves highly competitive results against state-of-the-art continuous diffusion models.The paper frames the framework as a step toward unifying discrete and continuous paradigms for scalable, versatile, and efficient video generation.Source: §5 · p. 11
  • A linearized metric path and resolution-dependent timestep shifting provide fine-grained control over perturbations for long sequences.The conclusion presents these as URSA’s two key innovations.Source: §5 · p. 11
  • Asynchronous temporal scheduling enables multi-task video generation in one model.Source: §5 · p. 11
  • The work was supported in part by the National Natural Science Foundation of China, and the authors acknowledge contributors to implementation, design feedback, discussions, and editing.Source: §5 · p. 11

APPENDIX

The appendix provides code and pretrained models, and organizes implementation details, experiments, and qualitative results across three listed topics.

APPENDIX

  • The authors report publishing code and pretrained models to improve interpretability and ensure reproducibility.Source: APPENDIX · p. 18
  • Appendix materials include implementation details, experiments, and qualitative results.Source: APPENDIX · p. 18
  • The listed topics are training and sampling details, video extrapolation, and start-end frame control experiments.Source: APPENDIX · p. 18

A TRAINING AND SAMPLING DETAILS

URSA training encodes visual data as discrete latent tokens and learns predictions from metric-path-noised sequences. Its sampling procedure iteratively predicts and updates tokens using a time-shifted schedule and a geometry-aligned velocity field.

Training and Sampling Details

Sampling

  • Sampling starts from a uniformly drawn vocabulary token sequence, then iteratively predicts a target and updates tokens along a velocity field.The velocity field permits transitions from z to x only when x is closer to the target x1 than z; the algorithm returns the generated discrete sample.Source: A TRAINING AND SAMPLING DETAILS · p. 18

Training

  • Training encodes images and videos as discrete latent tokens, noises them along the metric path, and minimizes cross-entropy between ground-truth visual tokens and predicted distributions.Text prompt embeddings and noised visual tokens are concatenated into a unified sequence; training samples a timestep from [0, 1].Source: A TRAINING AND SAMPLING DETAILS · p. 18

B VIDEO EXTRAPOLATION EXPERIMENTS

URSA video extrapolation extends a text-to-video segment through sliding-window generation, with historical-frame resampling used to mitigate sampling errors.

B VIDEO EXTRAPOLATION EXPERIMENTS

C START-END FRAME CONTROL EXPERIMENTS

URSA is evaluated on start–end frame control by conditioning generation on frames sampled from a video. The reported setup aims to preserve coherent object and camera motion and spatial relationships.

C START-END FRAME CONTROL EXPERIMENTS

  • Frames sampled every four seconds are placed at the beginning and end of the context window to condition start–end frame control.The setup is intended to preserve coherent object and camera motion and spatial relationships.Source: C START-END FRAME CONTROL EXPERIMENTS · p. 18
  • Figure 9 extends a four-second text-to-video result to 40 seconds in zero-shot video extrapolation.Source: Figure 9 · p. 19
  • Figure 10 renders the start–end frames with transparency.Source: Figure 10 · p. 19

Related papers

Loading 2510.24717v1…