PaperlayerView exact revision on arXiv

Source-linked AI summary

FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation

Jibin Song, Mingi Kwon, Jaeseok Jeong, Youngjung Uh

arXiv:2512.24724v1cs.CV

TL;DR

Video diffusion generation is computationally costly, and existing acceleration approaches often assume uniform model capacity across denoising steps. FlowBlending assigns large and small models to different stages; across two video diffusion models, it reports faster inference with quality comparable to the large models.

  • Problem

    Video diffusion generation is costly, while common acceleration approaches generally assume the same model capacity across denoising timesteps.

    Source: §1 · pp. 1–2

  • Method

    FlowBlending uses large models in capacity-sensitive early and late stages and a small model in the intermediate stage, with stage boundaries guided by similarity and velocity divergence.

    Source: §1 · p. 2, §7 · p. 8

  • Results

    Across LTX-Video and WAN 2.1, FlowBlending achieves up to 1.65× faster inference with 57.35% FLOPs while preserving large-model visual fidelity, temporal coherence, and semantic alignment.

    Source: §7 · p. 8

  • Takeaways & Limitations

    The small model can handle intermediate denoising while large-model quality is maintained across the evaluated LTX-Video and WAN 2.1 settings.

    Source: §1 · p. 2, §7 · p. 8

    Stage boundaries often need to be re-estimated when the diffusion model changes.

    Source: §7 · p. 8

Abstract

from arXiv · show

In this work, we show that the impact of model capacity varies across timesteps: it is crucial for the early and late stages but largely negligible during the intermediate stage. Accordingly, we propose FlowBlending, a stage-aware multi-model sampling strategy that employs a large model and a small model at capacity-sensitive stages and intermediate stages, respectively. We further introduce simple criteria to choose stage boundaries and provide a velocity-divergence analysis as an effective proxy for identifying capacity-sensitive regions. Across LTX-Video (2B/13B) and WAN 2.1 (1.3B/14B), FlowBlending achieves up to 1.65x faster inference with 57.35% fewer FLOPs, while maintaining the visual fidelity, temporal coherence, and semantic alignment of the large models. FlowBlending is also compatible with existing sampling-acceleration techniques, enabling up to 2x additional speedup. Project page is available at: https://jibin86.github.io/flowblending_project_page.

1. Introduction

Video diffusion offers improved fidelity and temporal coherence, but iterative denoising and large models make generation computationally costly. FlowBlending allocates model capacity by stage, using large models where they matter and small models in intermediate steps.

1. Introduction

  • Prior acceleration methods reduce sampling steps or distill the process, but generally assume uniform model capacity across timesteps.The paper identifies this as a gap for video diffusion acceleration.Source: §1 · p. 2
  • Model capacity is most consequential in early and late denoising, while small models produce nearly identical outputs to large models in the intermediate stage.Early steps shape global structure and semantic alignment; late steps refine details and remove artifacts.Source: §1 · p. 2
  • FlowBlending uses the large model in early and late stages and the small model for most intermediate steps, without additional training, distillation, or architectural modification.The approach is complementary to existing acceleration techniques.Source: §1 · p. 2
  • Semantic similarity between latents, a fine-detail quality indicator, and velocity-divergence analysis support stage-boundary selection.The velocity-divergence analysis suggests a principled way to identify early and late boundaries.Source: §1 · p. 2
  • Across LTX-Video and WAN 2.1, FlowBlending achieves up to 1.65× faster inference with 57.35% FLOPs while preserving large-model visual fidelity, temporal coherence, and semantic alignment.It is also complementary to existing acceleration techniques.Source: §1 · p. 2

2. Related work

The related work describes flow matching as a framework for generative modeling and reviews sampling acceleration through solver improvements and step distillation. It also distinguishes prior multi-model sampling for image diffusion from this paper’s video-focused analysis.

Background: flow matching

  • Flow matching learns a velocity field that transfers samples from a source distribution toward a target distribution through an ordinary differential equation.The described conditional flow matching setup forms intermediate latents from source and target samples.Source: §2 · p. 2

Acceleration of diffusion models

  • Sampling acceleration commonly reduces function evaluations with numerical solvers or compresses multi-step sampling through distillation; video extensions may require costly retraining.The paper frames model-capacity switching as orthogonal to step reduction and retraining.Source: §2 · p. 2

Multi-model sampling

  • Prior multi-model allocation studies concern image diffusion, whereas this work finds large models crucial early in video diffusion for structure and coherent motion.Earlier image-diffusion work includes Yang et al. [47], Liu et al. [26], and Pan et al. [33].Source: §2 · p. 3

3. Stage-aware multi-model sampling

FlowBlending treats model choice as a timestep-dependent capacity-allocation problem: the large model establishes early structure and motion and refines late details, while the small model handles intermediate steps. The section motivates this schedule through stage-wise comparisons.

3. Stage-aware multi-model sampling

  • The capacity-allocation problem is to replace sampling steps with the small model where its updates closely match the large model, without degrading structure, motion, or identity consistency.The section frames this as reusing existing large and small models rather than modifying the sampling algorithm or retraining by distillation.Source: §3 · p. 3

3.1. Early structure formation

  • Using the large model only in the earliest steps yields structure, motion coherence, and semantic consistency nearly identical to large-only sampling; small-only or small-then-large schedules struggle with semantics.The comparisons use WAN-2.1 schedules, with L denoting 14B and S denoting 1.3B models.Source: §3.1 · p. 3
  • Early large-model use establishes coarse structure and motion, after which the small model can take over with minimal perceptual degradation.The early stage is described as capacity-sensitive.Source: §3.1 · p. 3

3.2. Late refinement

  • LSS retains coarse structure but may leave spatial distortions or temporal flicker; reintroducing the large model late suppresses artifacts and restores fine detail.The late-stage role is artifact correction and detail refinement, rather than initial structure formation.Source: §3.2 · pp. 3–4

3.3. Stage-aware multi-model sampling

  • The proposed FlowBlending schedule uses the large model in early and late stages and the small model during the capacity-tolerant intermediate stage.The goal is to reduce computation while retaining large-model performance.Source: §3.3 · p. 4

4. Experiment

Across LTX-Video and WAN 2.1, the experiments test stage boundaries and compare schedule quality and efficiency. The selected LSL schedule approaches large-only quality with reduced computation, while analyses support using the large model early and late.

4.1. Experimental setup

  • The evaluation uses LTX-Video (2B/13B) and WAN 2.1 (1.3B/14B), with PVD and VBench assessments and runtime and DiT-block FLOPs measurements.FID and FVD use 284 generated samples; four VBench metrics use 355 samples.Source: §4.1 · p. 4

4.2. Analysis of the early stage

  • Early-stage LSS remains substantially closer to LLL than SLL or SSS, supporting the large model’s role in establishing global structure and semantic alignment.The early boundary is selected just before a sharp DINO-similarity decline, typically above 96%, where global structure and motion remain nearly indistinguishable from LLL.Source: §4.2 · pp. 4–5, Figure 4 · p. 5

4.3. Analysis of the late stage

  • With the early boundary fixed, the late-stage FID curve has a V-shaped minimum; moderate intermediate small-model use followed by large-model refinement improves detail without excess artifacts.The pattern occurs for both WAN and LTX-Video; excessive small-model use raises FID and can introduce artifacts or temporal flicker.Source: Figure 5 · p. 5, §4.3 · p. 5

4.4. Quantitative results

  • LSL achieves quality nearly indistinguishable from LLL across video-quality metrics while accelerating inference by up to 1.65× with 57.35% FLOPs.Table 2 compares schedules across two models; LSS preserves global structure but leaves late-stage artifacts, while SSS degrades across most metrics.Source: §4.4 · p. 5, Table 2 · p. 6

5. Extended analyses

Extended analyses find that the selected LSL schedule offers a favorable quality-compute trade-off and that its stage boundaries align with velocity-divergence patterns. Schedule comparisons also show that beginning with the large model and choosing the boundaries carefully matter.

5.1. Velocity divergence across sampling steps

  • Velocity divergence is lowest in the intermediate stage, while early-stage variance and late-stage mean divergence support assigning the large model to both ends of sampling.The authors summarize the allocation principle as using the large model when divergence is high and the small model when it is low.Source: §5.1 · pp. 6–7
  • Across LTX-Video and WAN 2.1, empirically selected stage boundaries coincide with boundaries inferred from cosine-distance curves, suggesting velocity divergence as a convenient proxy.A similar pattern appears for ℓ2 distance and unconditioned velocities; the authors speculate that rising late-stage variance marks the late boundary.Source: §5.1 · pp. 6–7

5.2. Extensive schedule comparison

  • Schedules beginning with the small model have worse FID and Motion Smoothness than those beginning with the large model; even one initial large-model segment performs better than a schedule omitting it.The schedule comparison divides sampling into 12.5% segments and evaluates FID and Motion Smoothness against FLOPs.Source: §5.2 · p. 7
  • The proposed LSL schedule attains the lowest FID among the compared variants while maintaining an excellent FID–FLOPs trade-off and near-indistinguishable outputs from LLL.The selected schedule lies on the Pareto frontier with LLL-level quality at lower FLOPs.Source: §5.2 · p. 7, Figure 7 · p. 7
  • At the same FLOPs budget, shifted LSL boundaries perform worse than the selected configuration, and adding or removing a boundary segment also worsens FID and Motion Smoothness.Yellow LSL schedules using the identified early boundary generally have a more favorable FID–FLOPs trade-off than alternatives.Source: §5.2 · p. 7

6. Compatibility with other acceleration methods

FlowBlending is compatible with both reduced-step solvers and distilled models. With DPM++ or an intermediate-stage distilled model, it retains outputs comparable to the large-only baseline while reducing compute.

Compatibility overview

  • FlowBlending is presented as orthogonal to both sampling-step reduction and distillation-based training.Table 3 summarizes compatibility with reduced-step solvers and distilled backbones.Source: §6 · p. 7, Table 3 · p. 8

Compatibility with sampling-step reduction

  • DPM++ combined with LSL reduces total TFLOPs by about 2× with negligible quality loss, while reproducing results comparable to the large-only baseline.At half the number of function evaluations, LSL avoids artifacts seen with the small-only schedule.Source: §6 · p. 7, Table 3 · p. 8

Compatibility with distilled models

  • Replacing the intermediate-stage small model with LTX-Video’s distilled variant in LDL yields results comparable to LLL while reducing total FLOPs to nearly half.The distilled model is used for four intermediate sampling steps; DDD does not reproduce LLL results.Source: §6 · p. 8, Table 3 · p. 8

7. Discussion and conclusion

FlowBlending assigns large-model capacity to early and late denoising stages and uses the small model in between. The paper reports maintained video quality at lower sampling cost, while noting that stage boundaries often need re-estimation for a different model.

Compatibility with distillation

  • In the LSL framework, the distilled model works as an intermediate-stage replacement, whereas the all-distilled DDD configuration does not reproduce LLL results.The LDL configuration is reported to reproduce results comparable to the large-only baseline.Source: Figure 9 · p. 8

Discussion

  • The early stage governs global structure and motion, while the late stage refines high-frequency details and suppresses artifacts; both are capacity-sensitive.The intermediate stage is delegated to the small model in FlowBlending.Source: §7 · p. 8

Conclusion

  • Across LTX-Video and WAN 2.1, FlowBlending achieves up to 1.65× faster inference with 57.35% FLOPs while preserving video quality.It allocates the large model to early and late stages and the small model to the intermediate stage.Source: §7 · p. 8

Boundary analysis

  • Empirically selected stage boundaries coincide with regions of increasing variance in velocity divergence between the large and small models.The paper proposes DINO similarity and FID-based trade-off analysis for selecting the boundaries.Source: §7 · p. 8

Limitation

  • FlowBlending’s stage boundaries often need to be re-estimated when the diffusion model changes.The paper identifies automatic boundary detection or model-agnostic criteria as ways to improve usability and generality.Source: §7 · p. 8

A. Experimental Details

The experiments use LTX-Video and WAN 2.1 under their default sampling configurations, with the PE Video Dataset as the primary evaluation set. Early-boundary ablations compare hybrid schedules with the large-only baseline and select a boundary before similarity drops sharply.

A. Experimental Details

  • The main evaluations use 284 PVD samples for FID and FVD and 355 VBench-prompt samples for VBench metrics; runtime and DiT-block FLOPs are also measured.LTX-Video runtime is measured on an A6000 GPU and WAN 2.1 on an A100 GPU.Source: A. Experimental Details · p. 12
  • PE Video is the primary benchmark because its richer, more dynamic prompts better reflect the quality differences under study than many short, simple VBench prompts.The authors report broadly similar trends across PE Video and VBench.Source: A. Experimental Details · p. 12

B.1. Early stage ablation

  • The evaluation covers LTX-Video and WAN 2.1 using 40 and 50 denoising steps, respectively, with 355 VBench-prompt videos for the early-boundary ablation.The boundary is varied in increments of five timesteps for LTX-Video and ten for WAN 2.1.Source: §B.1 · p. 12
  • Across DINO, CLIP, LPIPS, PSNR, and SSIM, similarity to LLL drops sharply beyond a point; the chosen boundary is just before that drop.Figures A1 and A2 plot the small-model introduction point against similarity; DINO and CLIP align most closely with the authors’ observations.Source: §B.1 · p. 13, Figure A1 · p. 13, Figure A2 · p. 13
  • Introducing the small model at 40% of denoising preserves appearance and motion nearly identical to LLL, whereas delaying the switch to 60% changes motion.The authors select 40% as the balance between quality and efficiency.Source: §B.1 · p. 13

B.2. Late stage ablation

The late-boundary ablations vary only the point where the large model returns, holding the early boundary fixed. Across both model families, the authors select boundaries using FID and FVD trends and report a 20% reintroduction point as a quality-efficiency balance.

Experimental details

  • The late-boundary ablation fixes the early boundary and varies only the late one, evaluating 284 PVD videos with 40 LTX-Video steps and 50 WAN 2.1 steps.The late boundary is shifted in five-timestep intervals for LTX-Video and ten for WAN 2.1.Source: §B.2 · pp. 13–14

Boundary selection

  • The early-boundary algorithm uses DINO similarities at candidate small-model introduction points, computes local slopes, and selects the first index whose slope falls below a fraction of the reference slope.The reference is the initial slope; the threshold is α.Source: §B.2 · p. 14

Results

  • For LTX-Video, the selected late boundary yields the best FID and FVD scores, best video-quality-related metrics, and near-best temporal flickering and motion smoothness.The selected boundary’s image-quality score is lower than with more large-model use, but the observed visual difference is negligible.Source: §B.2 · p. 14, Figure A3 · p. 15
  • WAN 2.1 shows trends consistent with LTX-Video across metrics, and its optimal late boundary is identified where FID and FVD are minimized.Figure A4 plots VBench metrics, FID, and FVD.Source: §B.2 · p. 14, Figure A4 · p. 15

Qualitative results

  • Reintroducing the large model at the final 20% preserves fine-grained textures without noticeable artifacts; moving the point to 0% produces visible artifacts, including a severe dog-eye artifact.The authors identify 20% as the quality-efficiency balance.Source: §B.2 · pp. 14–15

C.1. Experimental details

The analysis measures divergence between small- and large-model velocity predictions across denoising steps and finds the least divergence in the intermediate stage.

C.1. Experimental details

  • The study measures divergence between the small and large models’ velocity fields across denoising timesteps.Source: §C.1 · p. 15
  • The large-model sampling trajectory supplies intermediate outputs to both models, whose velocity predictions are compared at each timestep using cosine distance and L2 distance.The divergence measurement is applied to LTX-Video and WAN 2.1.Source: §C.1 · p. 16
  • Velocity divergence is lowest in the intermediate stage, while early-stage variance and late-stage mean divergence are higher, supporting large-model use at both ends.The early and late patterns correspond to structure formation and detail refinement, respectively.Source: §C.1 · p. 16, Figure A5 · p. 16

D. Comprehensive schedule sweep

Comprehensive schedule sweeps report strong LSL performance across PV and VBench evaluations. The supplementary material includes videos for main-paper figures, while the related-work discussion distinguishes FlowBlending’s aim from Pan et al. [33].

D. Comprehensive schedule sweep

E. Qualitative results

F. More Related Work

  • Prior acceleration work includes adversarial distillation and step-wise feature reuse; Pan et al. [33] is closest, but targets image quality rather than preserving large-model video fidelity while reducing cost.The paper notes that early-stage motion misalignment in video cannot be corrected by a small model later, motivating its stage-aware analysis.Source: F. More Related Work · p. 17

D. Comprehensive schedule sweep

Loading 2512.24724v1…