PaperlayerView exact revision on arXiv

Source-linked AI summary

RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance

Yiming Liu, Ben Wan, Tongxuan Liu, Ao Wang, Yuqi Xiong, Fan Zhang, Hui Chen, Guiguang Ding

arXiv:2609.36433v1cs.CV

TL;DR

CFG caching criteria based on separate branch errors do not capture how errors combine into the guided prediction or how their effects vary across timesteps. RA-CFGCache composes branch-wise proxies and rescales guided risk for propagation, then uses threshold-based control for joint branch reuse. It improves the efficiency–fidelity trade-off over evaluated training-free caching baselines on text-to-image and text-to-video diffusion transformers.

  • Problem

    Branch-local reuse criteria do not account for how conditional and unconditional cache errors combine under CFG or how local guided errors’ final impact varies by timestep.

    Source: §1 · p. 2

  • Method

    RA-CFGCache combines branch-wise proxies using CFG structure and calibrated cross-branch alignment, rescales guided risk by timestep-dependent propagation, and controls joint branch reuse with an online threshold.

    Source: §1 · p. 3, §3.4 · p. 6

  • Results

    RA-CFGCache consistently improves the efficiency–fidelity trade-off over evaluated training-free caching baselines on text-to-image and text-to-video diffusion transformers.

    Source: §1 · p. 3, §4.2 · p. 9, §5 · p. 10

  • Takeaways & Limitations

    The approach controls caching without changing the sampling schedule or guidance rule.

    Source: §1 · p. 3

    The accumulated-risk score is an interval-level control quantity, not an exact estimate or formal bound on final output deviation.

    Source: §3.4 · p. 6

Abstract

from arXiv · show

Diffusion models enable high-quality visual generation, but iterative denoising remains computationally expensive, especially under classifier-free guidance (CFG), which requires both conditional and unconditional evaluations. Training-free caching reduces this cost by reuse of previously computed features or predictions. However, existing branch-local reuse criteria do not explicitly account for how cache errors combine under CFG or how local perturbations affect the final output. We identify two misalignments in cache control: a branch-guided mismatch, where guided error depends on both the magnitudes and alignment of branch errors, and a local-final mismatch, where the downstream impact of a local error varies across timesteps. We propose RA-CFGCache, a Risk-Aligned Caching framework under CFG that incorporates both factors while keeping the sampling schedule and guidance rule fixed. CFG-aware Guided-Risk Composition combines existing branch-wise proxies using CFG coefficients and offline-calibrated cross-branch alignment. Propagation-Aware Rescaling further weights the resulting guided-risk estimate with a timestep-dependent propagation prior calibrated from isolated reuse perturbations. An online threshold controller then determines when to jointly refresh or reuse both branches. Experiments on FLUX.1-dev, Wan2.1-T2V-1.3B, and CogVideoX-2B demonstrate improved efficiency--fidelity trade-offs over evaluated training-free caching baselines. Moreover, RA-CFGCache is compatible with diverse base proxy families, including TeaCache-, DiCache-, and MagCache-style estimators, and consistently improves fidelity at nearly unchanged latency. Code is available at https://github.com/yiming-l21/RA-CFGCache.git.

1 Introduction

RA-CFGCache addresses two CFG caching mismatches: branch errors do not directly represent guided error, and a local error’s final impact varies by timestep. It combines guided-risk composition with propagation-aware rescaling to control joint branch reuse.

1 Introduction

  • Training-free caching reuses computed features or predictions when estimated changes are sufficiently small, but branch-local criteria do not account for interactions after CFG composition.Diffusion inference is costly because it repeats network evaluations across denoising steps, and explicit CFG evaluates both conditional and unconditional predictions.Source: §1 · pp. 1–2
  • Branch-wise error magnitudes alone do not fully characterize the guided error that drives the denoising update, which also depends on branch-error directions and guidance scale.The guided-error curve can differ substantially from branch-wise error curves, especially early in sampling.Source: §1 · p. 2
  • Similar local guided errors can produce substantially different final deviations depending on the timestep at which the perturbation is introduced.The single-step study applies reuse at one timestep while using full computation at all other steps.Source: §1 · pp. 2–3
  • CFG-aware guided-risk composition combines branch-wise proxies using CFG structure and cross-branch alignment, while propagation-aware rescaling accounts for timestep-dependent downstream impact.An online threshold controller governs joint refresh and reuse of both branches without changing the sampling schedule or guidance rule.Source: §1 · p. 3
  • The paper characterizes branch–guided mismatch and local–final mismatch as distinct sources of misalignment in CFG cache control.The first concerns guided error versus branch-wise error; the second concerns timestep-varying downstream impact.Source: §1 · p. 3
  • RA-CFGCache improves the efficiency–fidelity trade-off over evaluated training-free caching baselines under CFG.Experiments and ablations on diffusion transformer pipelines support the proposed analysis and control strategy.Source: §1 · p. 3

2 Related Work

Related work spans predefined reuse, adaptive reuse, prediction-based caching, and CFG acceleration or guidance optimization. RA-CFGCache combines CFG-specific branch-error interaction with downstream propagation while keeping the sampling schedule and guidance rule unchanged.

Training-free diffusion caching

  • Training-free caching includes predefined reuse schedules, adaptive reuse signals, and prediction through extrapolation, interpolation, or empirical correction.Examples include DeepCache Ma et al. [2023], TeaCache Liu et al. [2025a], and TaylorSeer Liu et al. [2025b].Source: §2 · p. 3
  • Prior work addresses downstream cache effects through final-output-aware scheduling or error-propagation correction, while RA-CFGCache combines these concerns with CFG-specific branch-error composition.LeMiCa Gao et al. [2025] uses final-output impact for global scheduling, and ERTACache Peng et al. [2026] analyzes propagation and introduces correction mechanisms.Source: §2 · p. 3

CFG-aware acceleration and guidance optimization

  • CFG acceleration methods exploit cross-branch redundancy, modify guidance, or jointly optimize guidance and caching; RA-CFGCache instead controls joint branch reuse without changing the sampling schedule or guidance rule.Examples include FasterCache Lv et al. [2025], CFG Schedulers Wang et al. [2024], and OUSAC Sun et al. [2025].Source: §2 · pp. 3–4

3 Method

The method frames CFG caching around the guided prediction entering the sampler, then addresses how branch errors combine and how local perturbations affect the final sample. RA-CFGCache combines branch-wise proxies, rescales guided risk by a timestep-dependent propagation prior, and uses that estimate for online refresh decisions.

3.1 Preliminaries

  • Caching replaces a fresh prediction with a reused-computation prediction, while CFG sends a guided combination of unconditional and conditional branches to the sampler.Thus, cache perturbations relevant to sampling are measured after guidance composition, not on either branch alone.Source: §3.1 · p. 4

3.2 Branch–Guided Mismatch

  • Under CFG with s > 1, guided error depends on both branch-error magnitudes and alignment: positive alignment partially cancels error, while weak or negative alignment can amplify it.The alignment varies across timesteps, with weaker cancellation in earlier steps and stronger cancellation in later ones.Source: §3.2 · pp. 4–5

3.3 Local–Final Mismatch

  • Similar local guided errors can produce different final deviations depending on their timestep, motivating a timestep-dependent propagation prior.The study isolates a perturbation at one timestep while using full computation at all other steps.Source: §3.3 · p. 5

3.4 RA-CFGCache

  • RA-CFGCache combines nonnegative branch-wise proxy estimates using CFG coefficients and offline-calibrated cross-branch alignment to form a guided-risk estimate.The resulting proxy is a cache-control estimate, not an exact estimate of true guided-error magnitude.Source: §3.4 · p. 6
  • The method rescales guided risk with a timestep-dependent propagation prior and triggers refresh when accumulated risk crosses a threshold, resetting the score after refresh.The accumulated score is an interval-level control quantity, not an exact estimate or formal bound on final-output deviation.Source: §3.4 · p. 6

Wan2.1-T2V-1.3B

Joint reuse is the default action, letting the controller account for both CFG branches’ combined proxy risk. It can save more latency than one-sided reuse, though parallel execution may limit one-sided savings.

Wan2.1-T2V-1.3B

  • Joint reuse of both CFG branches is the default, so the controller can account for their combined proxy risk.It also skips recomputation of cached components in both branches.Source: Wan2.1-T2V-1.3B · p. 7
  • When branch errors are positively aligned and s > 1, their contributions can partially cancel under CFG composition.Source: Wan2.1-T2V-1.3B · p. 7
  • Under parallel branch execution, one-sided reuse may provide limited latency savings because the guided prediction still waits for the recomputed branch.Source: Wan2.1-T2V-1.3B · p. 7

4 Experiments

Across the evaluated text-to-image and text-to-video pipelines, RA-CFGCache improves efficiency–fidelity trade-offs over evaluated caching baselines. Results cover the main comparisons and component, reuse-policy, guidance-scale, risk-validation, and calibration-robustness analyses.

4.1 Experimental Setup

  • The evaluation covers FLUX.1-dev, Wan2.1-T2V-1.3B, and CogVideoX-2B, comparing training-free acceleration baselines with shared models, prompts, schedules, and hardware unless specified otherwise.Fidelity is assessed against vanilla outputs using LPIPS, SSIM, and PSNR; efficiency uses latency and speedup.Source: §4.1 · pp. 7–8

4.2 Main Results

  • Across evaluated pipelines, RA-CFGCache offers favorable efficiency–fidelity trade-offs, with lower LPIPS than MagCache and DiCache at comparable latency across operating points.The trade-off improvement is not limited to a single tuned threshold.Source: §4.2 · p. 9
  • On FLUX.1-dev, RA-CFGCache better preserves textures, structures, and semantic details than TeaCache, MagCache, and DiCache at comparable speedups.The baselines show visible scene changes or local structural artifacts in highlighted regions.Source: §4.2 · p. 9

4.3 Ablation Studies

  • Each proposed component improves fidelity at similar latency, and combining guided-risk composition with propagation-aware rescaling performs best in the evaluated setting.Source: §4.3 · p. 9
  • Under a matched refresh budget, joint branch reuse gives the best fidelity among evaluated actions and larger latency savings by skipping recomputation in both branches.A separate parallel-execution study reports lower latency for joint control than independent branch-wise control at matched branch-evaluation counts.Source: §4.3 · p. 9
  • Across tested CFG scales, RA-CFGCache remains among the best methods in trade-off, with stable speedup and strong fidelity relative to competing baselines.Source: §4.3 · p. 10
  • On 37,400 reuse events, full guided-risk composition improves correlation and high-risk discrimination while reducing tail errors under the same 30% refresh budget.Oracle-assisted variants use true branch-error magnitudes and/or alignment for diagnosis only and are unavailable online.Source: §4.3 · p. 10
  • Propagation weighting raises Spearman correlation with mean final LPIPS from 0.6571 to 0.9429; tested calibration-set variations also yield relatively stable downstream fidelity and speedup.Calibration robustness applies to the tested random and outlier prompt subsets and calibration-set sizes.Source: §4.3 · p. 10

5 Conclusion

RA-CFGCache frames training-free caching under CFG as guided-error control, aligning reuse decisions with guided prediction error and its timestep-dependent propagation. It combines these controls with threshold-based scheduling and improves efficiency–fidelity trade-offs over the evaluated baselines.

5 Conclusion

  • The conclusion frames training-free diffusion caching under CFG as guided-error control rather than branch-local reuse control.Source: §5 · p. 10
  • RA-CFGCache aligns reuse decisions with guided prediction error and rescales risk by its timestep-dependent propagation to the final sample.Source: §5 · p. 10
  • Across evaluated text-to-image and text-to-video diffusion transformers, RA-CFGCache improves efficiency–fidelity trade-offs over the evaluated training-free caching baselines.Source: §5 · p. 10

A Limitations

RA-CFGCache has limits in how it models error propagation, controls sequential reuse, and transfers calibration across inference configurations.

A Limitations

  • The propagation gain is an empirical timestep-dependent surrogate, not a complete model of how local guided error affects the final output.It is estimated from isolated single-step perturbations and omits higher-order interactions among errors from consecutive reuse decisions.Source: A Limitations · p. 14
  • The cumulative scheduler approximates the sequential control problem and neither optimizes long-horizon reuse interactions nor provides a formal optimality guarantee.Source: A Limitations · p. 14
  • Alignment and propagation statistics are calibrated for an inference configuration, so transfer to substantially different models, samplers, schedules, or guidance settings may require validation or recalibration.Improving transferability while retaining lightweight online control is identified as future work.Source: A Limitations · p. 14

B.1 Details of the Compared Methods

The compared methods use specified implementations and operating settings; RA-CFGCache reuses their branch-wise control signals, then applies calibrated alignment and propagation statistics. Calibration is offline, while the added test-time control uses no additional diffusion-model forward pass.

B.1 Details of the Compared Methods

  • The comparison includes TeaCache, TaylorSeer, DiCache, MagCache, HiCache, FasterCache, and a 25-step vanilla baseline, with method-specific settings.The vanilla baseline halves the denoising steps from 50 to 25; other baselines use the same models, prompts, seeds, and hardware as the main experiments.Source: §B.1 · pp. 14–15

B.2 Branch-wise Proxy Definitions Used by RA-CFGCache

  • RA-CFGCache uses each base method’s signal as an independent branch-wise proxy rather than introducing a new branch-local error estimator.TeaCache- and DiCache-style proxies use online feature signals, while the MagCache-style proxy uses fixed offline-calibrated magnitude-ratio statistics.Source: §B.2 · pp. 15–16
  • TeaCache-style proxies retain relative feature-change and polynomial rescaling, while DiCache-style proxies use shallow-feature change and MagCache-style proxies use calibrated magnitude ratios.TeaCache and DiCache signals are sample-dependent; the MagCache-style signal is not instance-adaptive under fixed settings.Source: §B.2 · p. 15

B.3 Calibration Details

  • Offline calibration estimates anchor–target cross-branch alignment from residual drift and a timestep-dependent propagation gain from isolated one-step prediction perturbations.The fitted propagation gains are lower-bounded at zero when used for risk rescaling.Source: §B.3 · p. 16
  • For 100-prompt FLUX.1-dev calibration at T = 50, alignment and propagation calibration require about 0.60 and 8.3 GPU-hours, respectively, totaling 8.9 GPU-hours.Separate ablations found stable performance with 50 alignment-calibration prompts or 10 propagation-calibration prompts in the evaluated setting; the joint 50/10 setting was not established.Source: §B.3 · p. 17
  • At test time, RA-CFGCache adds constant-time lookup and scalar composition/rescaling beyond the base proxy, without an additional diffusion-model forward pass.The online overhead is included in reported end-to-end latency.Source: §B.3 · p. 17

C.1 CFG Error Geometry under Classifier-Free Guidance

Guided-risk control changes where refreshes occur because CFG error depends on cross-branch alignment, while downstream effects of local errors vary with timestep. The propagation prior is an empirical, single-perturbation approximation rather than an exact model of repeated-reuse effects.

C.1 CFG Error Geometry under Classifier-Free Guidance

  • CFG-relevant cache error is the error after combining conditional and unconditional branches, so branch-error magnitudes alone do not characterize the perturbation driving the denoising update.The identity’s temporal-drift interpretation is exact for direct prediction reuse; for intermediate-feature or residual caching, it is only a proxy.Source: §C.1 · p. 17
  • Mean refresh step fell from 16.87 to 10.03, and first-20%-step refreshes rose from 56.1% to 75.0% with guided composition at a matched average refresh count.The replay isolates scheduling behavior and does not measure end-to-end generation quality.Source: Figure 8 · p. 18, §C.1 · p. 18
  • Correction norm decreases over sampling while adjacent-step variation is large early, smaller mid-sampling, and rises near the end; these dynamics alone do not establish increased alignment.Relative drift magnitudes and directions also matter.Source: §C.1 · p. 18
  • The calibrated cross-term shifts refresh allocation in replay, but that comparison does not establish whether the redistribution improves output fidelity.Source: §C.1 · p. 18

C.2 Propagation-aware Rescaling under Diffusion Dynamics

  • A first-order analysis motivates timestep-dependent propagation weighting because a local guided prediction perturbation’s effect on the final latent depends on subsequent sampler dynamics.The analysis approximates the final perturbation through ordered effective Jacobians along the unperturbed trajectory.Source: §C.2 · pp. 18–19
  • The fitted propagation gain is an empirical surrogate from complete rollouts of isolated one-step perturbations, not a direct Jacobian estimate or an exact decomposition under repeated reuse.The accumulated control statistic is not a bound on final output deviation.Source: §C.2 · p. 19

C.3 Reuse Object Selection: Why Joint Branch Reuse Is a Strong Default

The preferred reuse object varies with CFG scale and timestep, but joint branch reuse is a strong default in the evaluated settings. Its error per saved branch evaluation is best overall in the offline diagnostic, which does not establish an optimal online policy.

C.3 Reuse Object Selection: Why Joint Branch Reuse Is a Strong Default

  • In direct-prediction analysis, reuse-both saves two branch evaluations, whereas reuse-∆, reuse-uncond, and reuse-cond each save one.For intermediate caching, savings depend on cached components and computations retained for proxy evaluation.Source: §C.3 · p. 20
  • The offline action oracle finds reuse choice is phase-dependent: reuse-both is selected more often as CFG scale increases, while reuse-∆ is mainly competitive late and reuse-cond is rarely preferred.At low CFG scale, reuse-uncond remains competitive over a larger portion of the trajectory.Source: §C.3 · p. 20, Figure 9 · p. 20
  • Reuse-both performs best overall in guided error per saved branch evaluation across the evaluated CFG scales, but this auxiliary diagnostic does not establish an optimal online policy.Source: §C.3 · p. 20
  • Joint branch reuse is a strong practical default in the evaluated settings and is frequently preferred at moderate-to-large CFG scales.Source: §C.3 · p. 20

D.1 Comparison with CFG-aware Guidance and Acceleration Methods

These additional evaluations examine RA-CFGCache against guidance-aware methods, across proxy and calibration variants, and under parallel CFG execution. They report fidelity-oriented acceleration of a fixed CFG sampler alongside timestep-dependent effects and limits on transferring calibration or latency findings.

D.1 Comparison with CFG-aware Guidance and Acceleration Methods

  • 3.95× speedup versus OUSAC’s 3.44× came with LPIPS 0.1484 vs 0.5530, SSIM 0.8340 vs 0.5092, and PSNR 22.75 vs 11.74, with comparable CLIPScore.RA-CFGCache-Slow offers a more fidelity-oriented point: LPIPS 0.1137, SSIM 0.8680, and PSNR 24.02 at 3.19× speedup. RA-CFGCache preserves the 50-step sampler and fixed CFG rule; its claim is sampler fidelity, not universal superiority in absolute generation quality.Source: §D.1 · p. 21

D.2 Generality of Propagation-aware Rescaling

  • Propagation-aware rescaling improves the final efficiency–fidelity trade-off across tested non-CFG caching baselines, reducing LPIPS and raising SSIM and PSNR at nearly identical speedup.The results support adjusting local reuse scores for timestep-dependent downstream propagation before using them for decisions.Source: §D.2 · p. 22, Table 10 · p. 22

D.3 Compatibility with Different Base Proxy Estimators

  • Across Tea-style, Mag-style, and DiCache-style proxies, RA-CFGCache improves fidelity at nearly unchanged latency; the gains come from correcting proxy composition and accumulation, not stronger proxy signals.The underlying proxy signals remain unchanged in the comparisons.Source: §D.3 · pp. 22–23, Table 11 · p. 22

D.4 Additional Calibration Visualizations

  • Figure 10’s structured alignment heatmap motivates retaining anchor–target dependence; measured propagation gains vary by timestep, and a fitted power law provides compact online control.The heatmap alone does not establish that a full pairwise matrix is necessary.Source: §D.4 · p. 23, Figure 10 · p. 23

D.5 Cross-model Alignment and Propagation Patterns

  • Across evaluated models and CFG settings, alignment often strengthens later and propagation effects generally diminish later; similar patterns do not establish parameter transfer.The findings are consistent with model-specific alignment calibration and a shared low-dimensional functional family for the propagation prior.Source: §D.5 · p. 24, Figure 11 · p. 24, Figure 12 · p. 24

D.6 Why a Power-law Surrogate for Propagation Gain

  • The power-law surrogate yields best LPIPS and SSIM with competitive PSNR at comparable speedup; omitting rescaling gives weakest fidelity, and the fit is a scheduling prior, not a strict law.The comparison distinguishes approximation-quality metrics R2 and NMAE from downstream fidelity metrics LPIPS, SSIM, and PSNR.Source: Table 12 · p. 25, §D.6 · p. 25

D.7 Parallel Consistency under CFG Parallelism

  • At matched branch-evaluation counts, joint control takes 4.21 s versus 6.81 s for independent control, avoiding one-sided reuse in the tested parallel setup.Independent control used one-sided reuse at 53.8% of steps and joint reuse at 35.1%; the latency advantage is not established for other hardware or backends.Source: Table 13 · p. 25, §D.7 · p. 25

E Broader impacts

  • RA-CFGCache’s efficiency improvements can lower deployment cost and energy use, but the method adds no generative capabilities, datasets, or pretrained models.The paper identifies no negative societal impacts unique to the caching algorithm and says deployment should follow the underlying models’ safety and content-moderation policies.Source: E Broader impacts · p. 25

F Additional Qualitative Results

The additional qualitative comparisons cover FLUX.1-dev, CogVideoX-2B, and Wan2.1-T2V-1.3B examples.

F Additional Qualitative Results

  • Figure 13 provides an additional qualitative comparison on FLUX.1-dev.Source: Figure 13 · p. 26

A polar bear is playing guitar

An astronaut flying in space.

  • Figures 16 and 17 present qualitative comparisons for Wan2.1-T2V-1.3B, labeled Part I and Part II respectively; the cited captions do not identify prompts or describe outcomes.Source: Figure 16 · p. 29, Figure 17 · p. 30
Loading 2609.36433v1…