Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching

1 INTRODUCTION

• Static caching reuses computations at fixed intervals, but can fail to align with changing output variations across timesteps.The introduction contrasts this rigidity with dynamic caching approaches.Source: §1 · p. 1, Figure 1 · p. 1
• TeaCache uses polynomial fitting and offline profiling of dataset-specific priors to estimate cache intervals, but its profiling and tuning make it sensitive to dataset distributions.The authors also note that dataset priors may misalign caching with model dynamics and compromise fidelity, especially under aggressive feature reuse.Source: §1 · pp. 1–2
• The authors identify relative stability in DiT transformation rates during denoising as the basis for estimating intermediate outputs from previously computed transformations.This motivates EasyCache’s training-free feature caching framework.Source: §1 · p. 2
• EasyCache’s online decisions use lightweight first-order dynamics, without retraining, dataset-specific tuning, or architectural modification.The method monitors relative transformation rates and reuses previously computed transformation vectors when local stability is estimated, subject to an accumulated local error indicator.Source: §1 · p. 2
• On HunyuanVideo, EasyCache improves speedup over TeaCache by 36%, boosts PSNR by 37% and SSIM by 14%, and reaches 3.3× speedup with a 0.3 PSNR drop alongside efficient attention.The combined acceleration result is reported for already accelerated efficient attention.Source: §1 · p. 2

2.1 Diffusion Model

• Diffusion models support image, 3D, and video synthesis, while video generation has drawn attention for dynamic content creation.The section describes diffusion models as a dominant paradigm for high-quality content synthesis.Source: §2.1 · p. 2
• DiT-based video models such as Sora, HunyuanVideo, and Wan2.1 model complex spatiotemporal relationships, but iterative denoising contributes to inference latency.The section describes video generation as challenging because of temporal dynamics and computational overhead with long sequences.Source: §2.1 · p. 2

2.2 Training-free Diffusion Inference Acceleration

• Training-free methods reduce inference costs without resource-intensive retraining; SVG uses 3D full-attention sparsity to reduce per-step cost.The section presents efficient attention as one approach to training-free acceleration.Source: §2.2 · p. 2
• Feature caching reuses computation across denoising steps: static methods use predetermined rules, while TeaCache uses a fitted polynomial and external dataset prior.The related work notes that TeaCache’s offline profiling can hinder generalizability, motivating caching based on runtime inference dynamics.Source: §2.2 · p. 2

3.1 A Revisit of Diffusion Modeling with Transformer

• DiT-based video generation requires iterative denoising and substantial computation, motivating training-free acceleration that preserves generation quality.The passage gives an example cost of approximately 2 H20 hours to generate a 5-second, 720P video with HunyuanVideo or Wan2.1-14B.Source: §3.1 · p. 3

3.2 Investigation of Feature Dynamics

• The relative transformation rate k stabilizes after an initial period of sharp fluctuation, motivating runtime-adaptive computation reuse.Unlike the input and output L1 norms, which show different growth and U-shaped patterns, k remains relatively stable later in denoising.Source: §3.2 · p. 3, Figure 2 · p. 3

3.3 Runtime-Adaptive Criterion

• EasyCache approximates outputs in stable diffusion phases by reusing a previously computed transformation vector, ∆i, instead of running a full forward pass.The transformation vector is defined as ∆t := vt − xt.Source: §3.3 · p. 3
• Reuse is governed by accumulated local stability indicators: ∆i is reused while Et < τ, and a full computation updates the cache when the threshold is reached or exceeded.The workflow also mandates full computation during the initial R warm-up steps and at the final step, then resets Et after updating the cache.Source: §3.3 · p. 4

3.4 An Intuitive Perspective on the Relative Stability

• Under flow matching, the paper relates k's stability to a near-linear velocity relationship in later generation and a fixed or slowly varying scheduler step.The authors connect this relative stability to identifying redundant computation steps at runtime while maintaining visual fidelity.Source: §3.4 · p. 4

4.1 Experiment Setup

• The evaluation measures latency and speedup alongside PSNR, SSIM, LPIPS, and VBench for text-to-video, and FID-30k and CLIP Score for text-to-image.Principal video evaluations use VBench default prompts, five samples with varying seeds per prompt, and NVIDIA A800 GPUs.Source: §4.1 · pp. 4–5

4.2 Main Results

• On Wan2.1-1.3B, EasyCache reaches 2.54× speedup and improves PSNR 11.8% versus TeaCache; on HunyuanVideo, it improves PSNR 36.9% versus TeaCache.On HunyuanVideo, it also reports 69.2% lower LPIPS than TeaCache and a 6.09dB PSNR and 57.9% speedup improvement over SVG.Source: Table 1 · p. 5, §4.2 · p. 5
• On 50-step FLUX.1-dev, EasyCache reports approximately 40% speedup over TeaCache, 10% better FID, and a 0.1 CLIP score improvement over FLUX.1-dev.The study uses this text-to-image evaluation to assess generalization beyond video generation.Source: Table 2 · p. 5, §4.2 · p. 6

4.3 Qualitative Results

• In single-sample comparisons on Wan2.1-14B and HunyuanVideo, EasyCache outputs more closely match the original videos than prior caching methods.On Wan2.1-14B, the reported speedup is 2.63× versus TeaCache’s 1.46×; the authors caution that single-sample comparisons may vary more than aggregated metrics.Source: §4.3 · p. 6, Figure 4 · p. 6

4.4 Ablation Study

• On Wan2.1-1.3B, τ = 5% gives 2.54× speedup and PSNR 24.74; τ = 2% gives 1.61× speedup and PSNR 30.73.Raising τ to 10% improves speedup by 21.7% over τ = 5%, but degrades visual quality.Source: Table 3 · p. 6, §4.4 · p. 6
• At similar ∼2.5× speedups, EasyCache’s adaptive reuse criterion best balances efficiency and fidelity, while its local k update balances speed and visual quality.Fixed-probability reuse and reuse without correction degrade visual retention; averaging k achieves higher quality but slower speedup, while kEMA slightly improves PSNR at reduced efficiency.Source: Table 5 · p. 7, §4.4 · p. 7, Table 6 · p. 7

4.5 Compatibility with Other Acceleration Techniques

• On HunyuanVideo, combining EasyCache with SVG reaches 3.33× speedup with a 1.1% PSNR drop, versus 3.18× speedup and a 17.8% PSNR reduction for SVG with TeaCache.For a 5s 720P video, the reported generation time falls from nearly two hours to 33 minutes with EasyCache and SVG.Source: Figure 5 · p. 7, §4.5 · p. 7

• EasyCache dynamically reuses computations using a lightweight runtime-adaptive criterion, avoiding costly offline profiling and external dataset priors.Source: §5 · p. 7
• 2.1–3.3× speedup is reported for EasyCache, described as state-of-the-art while preserving visual fidelity.Source: §5 · pp. 7–8
• A real-time generation gap remains, which the authors attribute mainly to the large-scale models needed for high-quality video generation.Source: §5 · p. 8