PaperlayerView exact revision on arXiv

Source-linked AI summary

T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback

Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sugato Basu, Wenhu Chen, William Yang Wang

arXiv:2405.18750v2cs.CV

TL;DR

Diffusion-based T2V sampling is slow, while video consistency models trade quality for faster inference. T2V-Turbo integrates mixed reward feedback into consistency distillation, and its four-step generations outperform SOTA methods on VBench and are preferred to 50-step teacher DDIM samples.

  • Problem

    Diffusion-based T2V models sample slowly, while video consistency models enable faster inference at the cost of sample quality.

    Source: §1 · p. 2

  • Method

    T2V-Turbo integrates image-text and video-text reward feedback into consistency distillation and optimizes rewards on single-step generations.

    Source: §1 · pp. 2–3

  • Results

    Four-step generations from both T2V-Turbo variants outperform SOTA methods on VBench, including Gen-2 and Pika, and humans prefer them to 50-step teacher DDIM samples.

    Source: §1 · p. 3, §6 · pp. 9–10

  • Takeaways & Limitations

    Human preference over 50-step teacher samples alongside four-step generation represents over tenfold inference acceleration with improved video quality.

    Source: §1 · p. 3, §6 · p. 10

    Because no open-sourced video-text reward model trained to reflect human preferences on video-text pairs is available, the approach uses video foundation models as its video-text reward model.

    Source: §6 · p. 10

Abstract

from arXiv · show

Diffusion-based text-to-video (T2V) models have achieved significant success but continue to be hampered by the slow sampling speed of their iterative sampling processes. To address the challenge, consistency models have been proposed to facilitate fast inference, albeit at the cost of sample quality. In this work, we aim to break the quality bottleneck of a video consistency model (VCM) to achieve $\textbf{both fast and high-quality video generation}$. We introduce T2V-Turbo, which integrates feedback from a mixture of differentiable reward models into the consistency distillation (CD) process of a pre-trained T2V model. Notably, we directly optimize rewards associated with single-step generations that arise naturally from computing the CD loss, effectively bypassing the memory constraints imposed by backpropagating gradients through an iterative sampling process. Remarkably, the 4-step generations from our T2V-Turbo achieve the highest total score on VBench, even surpassing Gen-2 and Pika. We further conduct human evaluations to corroborate the results, validating that the 4-step generations from our T2V-Turbo are preferred over the 50-step DDIM samples from their teacher models, representing more than a tenfold acceleration while improving video generation quality.

1 Introduction

Diffusion-based T2V sampling is slow, while video consistency models trade quality for speed; prior reward optimization also faces costly iterative gradients and limited temporal assessment. T2V-Turbo adds mixed reward feedback to VCM distillation and reports strong four-step benchmark and human-evaluation results.

1 Introduction

  • Earlier video consistency models enable plausible generation in 4–8 steps but inherit teacher quality limits, while iterative reward gradients are memory-intensive and image-only rewards miss video dynamics.InstructVideo limits gradient calculation to the final DDIM step and evaluates a limited prompt set, mostly involving animals.Source: §1 · p. 2
  • T2V-Turbo integrates image-text and video-text reward feedback into consistency distillation, optimizing rewards on single-step generations rather than backpropagating through iterative sampling.The video-text reward model evaluates temporal dynamics and transitions, while the image-text model aligns individual frames with human preference.Source: §1 · pp. 2–3, Figure 2 · p. 2
  • At four steps, both T2V-Turbo variants outperform SOTA methods on VBench, including Gen-2 and Pika, and humans prefer them to 50-step teacher DDIM samples.The human evaluation used 700 EvalCrafter prompts; the four-step generations represent over tenfold inference acceleration.Source: §1 · p. 3
  • The authors identify three contributions: mixed reward feedback including a video-text model, four-step VBench results surpassing proprietary systems, and human preference over 50-step teacher generations.The human comparison represents over tenfold inference acceleration with quality improvement.Source: §1 · p. 3

2 Preliminaries

The preliminaries describe diffusion-based T2V generation and consistency distillation as a route from slow sequential sampling to direct prediction from points on an ODE trajectory.

2 Preliminaries

  • Diffusion models perturb data with Gaussian noise in the forward process, then recover data from prior noise through reverse-time dynamics modeled by the Probability Flow ODE.The noise schedule uses α(t) and β(t), while μ(t) and σ(t) denote drift and diffusion coefficients.Source: §2 · p. 3
  • T2V models denoise video latents conditioned on text, and Classifier-Free Guidance combines conditional and unconditional noise predictions using the guidance scale.The generated latent is decoded into a video by the VAE decoder.Source: §2 · p. 3
  • A consistency model maps points on a Probability Flow ODE trajectory to its origin, and the video consistency model applies this mapping to text-conditioned video latents.The parameterization uses cskip(t) and cout(t) to ensure the output equals the input at the small fixed endpoint.Source: §2 · p. 4
  • Consistency distillation trains a consistency model from a pretrained T2V diffusion model, updating the target parameters by exponential moving average and using DDIM as the ODE solver.The augmented solver estimates an earlier latent using a skipping interval and classifier-free guidance.Source: §2 · p. 4

3 Training T2V-Turbo with Mixed Reward Feedback

T2V-Turbo adds mixed reward feedback to video consistency distillation by optimizing the decoded single-step generation, avoiding gradients through an iterative sampling process.

3 Training T2V-Turbo with Mixed Reward Feedback

  • The training pipeline applies multiple differentiable reward models to the decoded video from a single-step generation arising in consistency distillation.This avoids backpropagating reward gradients through iterative sampling.Source: §3 · p. 4

3.1 Optimizing Human Preference on Individual Video Frames

  • The image-text reward model is optimized on randomly sampled frames from the decoded video to align individual frames with human preference.The method maximizes the image-text model's scores on the sampled frames.Source: §3.1 · p. 4

3.2 Optimizing Video-Text Feedback Model

  • Because image-text reward models cannot assess inter-frame dynamics and transitions, the method adds a video-text reward model to evaluate generated videos.The passage identifies motion dynamics and transitions as temporal aspects requiring video-level assessment.Source: §3.2 · pp. 4–5

3.3 Summary

  • The total training loss linearly combines consistency-distillation loss with weighted image-text and video-text reward objectives, while training optimizes LoRA weights rather than the full model.The LoRA weights are merged after training so per-step inference cost remains identical to the teacher model.Source: §3.3 · p. 5

4 Experimental Results

The experiments evaluate T2V-Turbo’s 4–8-step video generation on VBench and through human comparisons, then examine the contributions of its reward models. The reported results cover automatic benchmark performance, human preferences, and reward-model choices.

4 Experimental Results

  • The study distills T2V-Turbo from VideoCrafter2 and ModelScopeT2V and trains the models on WebVid10M using eight NVIDIA A100 GPUs for 10K gradient steps.HPSv2.1 supplies image-text feedback; InternVid2 S2 is used for VC2 and ViCLIP for MS.Source: §4 · p. 5

4.1 Automatic Evaluation on VBench

  • Both 4-step T2V-Turbo variants surpass all baselines on VBench Total Score, including Gen-2 and Pika.The paper also reports that T2V-Turbo outperforms its teacher T2V model and improves over the baseline VCM.Source: §4.1 · p. 6, Table 1 · p. 6

4.2 Human Evaluation with 700 EvalCrafter Prompts

  • 4-step T2V-Turbo generations are favored over 50-step teacher samples, while 8-step generations are favored over 4-step generations on all three human-evaluation metrics.The evaluation compares videos from 700 EvalCrafter prompts; Figure 3 reports the human-evaluation results.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 Ablation Studies

  • Adding either image-text or video-text reward feedback improves over baseline VCM, and adding video-text feedback can further improve VBench Semantic Score.The study evaluates four-step generations; the table compares reward-model ablations for models with the same teacher.Source: §4.3 · p. 8, Table 2 · p. 8
  • T2V-Turbo (VC2) performs decently with either ViCLIP or InternVid2 S2, while T2V-Turbo (MS) performs better with ViCLIP but still surpasses VCM (MS) + Rimg with InternVid2 S2.The table reports the effect of using different video-text reward models.Source: §4.3 · p. 8, Table 3 · p. 8

5 Related Work

Related work covers diffusion-based text-to-video generation, methods for faster diffusion sampling, vision-and-language reward models, and alignment from human or AI feedback. The paper positions its approach as extending reward-based alignment to video with both image-text and video-text feedback.

Diffusion-based T2V Models

  • Diffusion-based T2V models use large-scale image data or pretrained text-to-image weights, while high-quality videos and aligned captions remain limited.Prior work uses image-video joint training or separates motion from appearance to address data constraints.Source: §5 · p. 9

Accelerating inference of Diffusion Models

  • Sampling acceleration includes numerical ODE solvers and distillation; consistency models map points on an ODE trajectory to its origin.Li et al. [2024] augment consistency distillation with image-text reward optimization, which this work extends to T2V using image-text and video-text rewards.Source: §5 · p. 9

Vision-and-language Reward Models

  • Image-text reward models are trained on human preferences, while the paper uses HPSv2.1 and video foundation models ViCLIP and InterVid S2 for reward feedback.The passage notes that no video-text reward model mirroring human preference on text-video pairs had been publicly released.Source: §5 · p. 9

Learning from Human/AI Feedback

  • Prior alignment work includes reinforcement learning and reward fine-tuning for images, while InstructVideo uses image-text rewards without evaluating video transitions.This work combines image-text and video-text reward feedback for T2V generation.Source: §5 · p. 9

6 Conclusion and Limitations

T2V-Turbo combines mixed reward feedback with consistency distillation, and its four-step generations outperform VBench baselines and are preferred to teacher-model samples in human evaluation. The authors note that its video-text reward model relies on video foundation models because an open-source model trained for human preferences was unavailable.

6 Conclusion and Limitations

  • Both four-step T2V-Turbo variants outperform VBench baselines, and human evaluators prefer T2V-Turbo generations over 50-step DDIM samples from their teachers.The reported comparison represents more than tenfold inference acceleration with improved video generation quality.Source: §6 · pp. 9–10
  • T2V-Turbo integrates mixed reward feedback into the consistency-distillation process of a teacher text-to-video model.Source: §6 · p. 9
  • The video-text reward uses ViCLIP and InternVid S2 because no open-source video-text reward model trained on human preferences was available.The authors suggest that exploring a more advanced video-text reward model could further improve performance.Source: §6 · p. 10

A Experiment and Hyperparameter (HP) Details

The appendix specifies the distillation setup for two teacher models, including training data, reward models, hyperparameters, and the DDIM solver used for sampling.

A Experiment and Hyperparameter (HP) Details

  • Qualitative head-to-head video comparisons use the same random seed, and the sampling setup uses DDIM as the ODE solver.The appendix presents its solver formula for the step from t_n+k to t_n and identifies the teacher's noise-prediction model.Source: A Experiment and Hyperparameter (HP)… · p. 16
  • Distillation uses VideoCrafter2 at 512x320 with FPS fixed at 16, and ModelScopeT2V at 256x256 with 8 FPS.Source: A Experiment and Hyperparameter (HP)… · p. 16
  • Both models are trained on WebVid10M for 10K gradient steps, using six to eight A100 GPUs and one 16-frame video per GPU batch.Source: A Experiment and Hyperparameter (HP)… · p. 16
  • HPSv2.1 supplies image-text rewards, while InternVideo2 or ViCLIP supplies video-text rewards; six frames are sampled for the image-reward objective.InternVideo2 is used when distilling from VideoCrafter2, and ViCLIP when distilling from ModelScopeT2V.Source: A Experiment and Hyperparameter (HP)… · p. 16
  • The appendix specifies learning rate 1e-5, guidance scales from 5 to 15, skipping step 20, and reward weights that differ by teacher.Image/video reward weights are 1/2 for T2V-Turbo (VC2) and 2/3 for T2V-Turbo (MS).Source: A Experiment and Hyperparameter (HP)… · p. 16

B Psudoe-codes for Training our T2V-Turbo

The appendix outlines the T2V-Turbo training loop and describes how VBench constructs its scores. On VBench, both four-step variants surpass the listed baselines in Total Score.

B Psudoe-codes for Training our T2V-Turbo

  • The training pseudocode adds image-text and video-text reward terms to the consistency-distillation loss and updates the model parameters iteratively.It takes a text-video dataset, initial parameters, solver, reward models, and related training settings as inputs.Source: B Psudoe-codes for Training our… · p. 17

C Further Details about VBench

  • VBench Quality Score combines seven dimensions, while Semantic Score averages nine normalized dimensions; Total Score is a weighted sum of the two scores.The Quality Score gives Dynamic Degree a weight of 0.5 and each other quality dimension a weight of 1.Source: C Further Details about VBench · p. 18
  • Both T2V-Turbo variants with four inference steps surpass all listed baselines in VBench Total Score, including Gen-2 and Pika.Source: Table 4 · p. 19

D Human Evaluation Details

The human evaluation compares generated videos across T2V-Turbo variants, teacher samples, and a baseline VCM using labeler judgments on video comparisons.

D Human Evaluation Details

  • The evaluation uses 700 EvalCrafter prompts and 16-frame videos, comparing variant-specific step counts with 50-step teacher samples and four-step generations with baseline VCM.VC2 is tested at four and eight steps; MS at two and four steps.Source: D Human Evaluation Details · p. 20
  • Labelers are recruited from English-speaking countries and complete comparison tasks as paid Mechanical Turk HITs.Each task takes about 30 seconds and pays $0.20, equivalent to about $24 per hour.Source: D Human Evaluation Details · p. 20
  • The project’s data annotation was classified as exempt by the Human Subject Committee under IRB protocols.Source: D Human Evaluation Details · p. 20
  • Figure 5 depicts the interface displayed to labelers during the human evaluation.Source: Figure 5 · p. 20

E Additional Ablation Studies

Image-text reward feedback and video-text reward choices both affect the ablation results: the tested image-text rewards improve on the baseline VCM, while the preferred video-text reward varies by teacher model.

E Additional Ablation Studies

  • The three tested image-text rewards improve 4-step video quality over baseline VCM (VC2), though CLIP-based rewards may contribute to that improvement.HPSv2.1 and PickScore are fine-tuned from CLIP with human preference data.Source: E Additional Ablation Studies · p. 21
  • VideoCrafter2 achieves decent VBench performance with either ViCLIP or InternVid2 S2 feedback, while ModelScopeT2V achieves a better result with ViCLIP.Source: Table 5 · p. 21

F Qualitative Results

The qualitative results compare T2V-Turbo with the baseline VCM and teacher models, and show 4-step improvements from several image-text rewards. The broader-impact discussion notes potential misuse risks and planned usage guidelines.

F Qualitative Results

  • The paper provides additional qualitative comparisons of T2V-Turbo with the baseline VCM and their teacher T2V models, reported in Figures 7, 8, 9, and 10.Source: F Qualitative Results · p. 21

G Broader Impact

  • Realistic synthetic videos may be misused for misinformation, defamation, or fraud; the authors plan to require users to follow usage guidelines.They describe safeguards and ethical guidelines as responses to these risks.Source: G Broader Impact · p. 21
  • The authors characterize T2V-Turbo as making high-quality video generation more accessible and practical at high speed.Source: G Broader Impact · p. 21
  • The three tested image-text rewards each improve 4-step generation quality over baseline VCM (VC2).Source: Figure 6 · p. 22

H Comparing videos generated by T2V-Turbo and VCM + Rimg

This section presents qualitative comparisons of videos generated by T2V-Turbo and VCM + Rimg, alongside documentation of the human-subject review and participant materials. It also reports the checklist’s code and model release information and notes the absence of error bars because training was run only once.

H Comparing videos generated by T2V-Turbo and VCM + Rimg

  • Across examples, VCM + Rimg’s right-side videos show prompt mismatches: a seated panda, backward-moving motorcycle, and extra horse or vase.These examples contrast with prompts asking for a panda standing on a surfboard, a motorcycle accelerating, and one galloping horse or one black vase.Source: H Comparing videos generated by… · pp. 27–29
  • Other right-side examples fail to show guitar playing, make a squirrel appear to hold a burger, move background hills, or show a dog that does not look happy.The prompts ask for a raccoon playing electronic guitar, a squirrel eating a burger, a moving Mars rover, and a happy dog; the comparison notes the guitar failure, burger appearance, moving hills, and unhappy-looking dog, respectively.Source: H Comparing videos generated by… · pp. 27–29

NeurIPS Paper Checklist

  • The checklist says the paper contains no theoretical results and addresses its limitations in Section 6.Source: NeurIPS Paper Checklist · pp. 30–31
  • The authors say experimental settings are detailed in Sections 4 and Appendix A, and that code and models are released.The checklist also says code is included in the supplementary materials and points to the project repository.Source: NeurIPS Paper Checklist · pp. 31–32
  • The authors report no error bars because computational resources allowed only one training run.The checklist answers No to reporting error bars or other statistical significance information.Source: NeurIPS Paper Checklist · p. 32
  • The paper reports its human evaluation in Sec. 4.2 and provides participant instructions and screenshots in Appendix D.Source: NeurIPS Paper Checklist · p. 35
  • The data annotation was classified as exempt by the Human Subject Committee under IRB protocols.Source: NeurIPS Paper Checklist · p. 35

Related papers

Loading 2405.18750v2…