Rethinking What to Cache in Few-Step Diffusion Transformers: Solver-Aware Target Selection

1 Introduction

• Step distillation reduces sampling steps, but the remaining forward passes are costly and larger solver-step gaps make reused tensors less reliable.The passage cites prior work on distillation and caching: (Salimans & Ho, 2022; Luo et al., 2023) and (Ma et al., 2024b; Kahatapitiya et al., 2025).Source: §1 · p. 1
• Most cache methods fix the tensor type, although different types can produce different sampling trajectories; the choice depends on the model, solver, and reuse schedule.Source: §1 · p. 1
• For linear-flow Euler sampling, velocity and a clean endpoint yield the same trajectory when converted at each new step, while a stored update can differ if step size changes.Source: §1 · p. 2
• AutoTarget measures candidate reuse errors in uncached runs and selects the lowest-error target when the candidates are sufficiently separated, without training a predictor.It compares candidates after solver conversion and frozen VAE decoding; all candidates use the same DiT evaluations and solver sequence.Source: §1 · p. 2
• The analysis explains error accumulation and when cached forms give the same Euler trajectory or differ as step sizes change.Source: §1 · p. 2
• Across image and video DiTs, AutoTarget saves DiT calls and retained storage while keeping generation quality near uncached runs.PixArt-LCM selects velocity at 512 px but denoised/clean at 1024 px when only resolution changes.Source: §1 · p. 2

2 Related Work

• Step reduction and cache reuse both target repeated transformer passes in DiT pipelines.The cited foundations include diffusion and score-based modeling (Ho et al., 2020; Song et al., 2021b), latent-space generation (Rombach et al., 2022), and transformer denoisers (Peebles & Xie, 2023).Source: §2 · p. 2
• Step distillation compresses diffusion sampling into fewer steps, but wider gaps between solver coordinates can make reuse errors more likely to change the trajectory.The cited recipes include progressive distillation (Salimans & Ho, 2022), consistency training (Luo et al., 2023), and distribution-matching or adversarial objectives (Yin et al., 2024b;a; Sauer et al., 2024).Source: §2 · pp. 2–3
• Caching methods reuse internal features, layer outputs, blocks, attention results, or tokens, while other methods adapt reuse timing or predict stale features; the representation is generally fixed.Examples and citations include deep or block features (Ma et al., 2024b; Wimbauer et al., 2024), skip routers (Ma et al., 2024a), and predictors (Liu et al., 2025b; Zou et al., 2026).Source: §2 · p. 3
• Recent flexible cache policies choose internal features, reuse solvers, blocks, or schedules, whereas AutoTarget chooses which tensor passed to the sampling solver to cache.Source: §2 · p. 3
• AutoTarget groups candidates with identical solver transitions and measures local reuse error on exact calibration runs; its selected target depends on model, resolution, and solver.The method trains no predictor.Source: §2 · p. 3

3 Method

• Given a model, solver, and fixed DiT evaluation schedule, AutoTarget compares supported cache targets using reuse errors from uncached runs.It deploys the lowest-error target on new prompts only when its error is sufficiently below the runner-up’s.Source: §3 · p. 3

3.1 DIT SAMPLING AND CACHE TARGETS

• A cache target is stored at an anchor step, when the full DiT runs and refreshes the cache, then converted into the solver input at reuse steps.The solver transition maps the current latent and prediction to the next latent.Source: §3.1 · p. 3
• AutoTarget includes only pipeline-supported targets with defined conversions, groups analytically or numerically equivalent transitions, and compares one representative per group.Candidates share data type, tensor shape, anchor steps, storage size, and solver coefficients; a target with an undefined conversion at any evaluated step is omitted.Source: §3.1 · p. 4

3.2 EQUIVALENT CACHE TARGETS UNDER EULER SAMPLING

• Under deterministic Euler sampling, held velocity and clean or noise endpoints produce the same cached trajectory when converted at each current state.This equivalence assumes cache reuse replaces selected DiT evaluations without changing the solver-coordinate sequence; running only at anchor steps is a different discretization.Source: §3.2 · p. 5
• A stored Euler displacement retains the anchor step size, so it can produce a different transition when the current step size changes.Source: §3.2 · p. 5
• Changing only PixArt-LCM resolution reverses the preferred target: velocity beats denoised-residual by 1.634 dB at 512 × 512, while denoised/clean beats velocity by 2.499 dB at 1024 × 1024.The selected target has higher PSNR on all 20 held-out prompts at both resolutions.Source: §3.2 · p. 5

3.3 SELECTING A CACHE TARGET FROM EXACT RUNS

• Calibration risk averages squared decoded RGB error over prompts, reuse steps, and RGB values; AutoTarget deploys the lowest-risk candidate only when the runner-up-to-winner risk ratio is at least 1.10.Otherwise it makes no target selection, leaving uncached sampling or a separately validated policy to the application.Source: §3.3 · p. 6
• Exact calibration runs cost |P|T DiT evaluations, while testing additional candidates afterward requires conversions and VAE decoding but no more DiT evaluations.Held-out complete cached runs assess whether calibration rankings persist as reuse errors accumulate.Source: §3.3 · p. 6

4.1 EXPERIMENTAL SETUP

• The evaluation uses matched model, solver, coordinates, precision, prompts, and seeds within each few-step model comparison, with paired measures separating task quality from reference fidelity.Overall speedup is relative to a full-step reference, while cache-only speedup isolates acceleration over the matching uncached few-step sampler; Figure 4 depicts calibration followed by reuse on new prompts.Source: §4.1 · pp. 6–7, Figure 4 · p. 7

4.2.1 TEXT-TO-IMAGE GENERATION

• On FLUX.1-schnell, caching adds 1.289× speedup; on PixArt-LCM, it adds 1.277× with 23.15 dB PSNR and 0.7518 SSIM.AutoTarget retains 0.500 MiB on FLUX.1-schnell and 0.125 MiB on PixArt-LCM, versus baseline ranges of 48–5814 MiB and 18–1512 MiB, respectively.Source: §4.2.1 · pp. 7–8, Table 1 · p. 8

4.2.2 TEXT-TO-VIDEO GENERATION

• On HunyuanVideo, AutoTarget runs in 150.4 seconds with 7.55× end-to-end and 1.539× cache-only speedup, scoring 70.9/80.9/78.9 on VBench Semantic/Quality/Total.DisCa runs in 151.3 seconds and scores 69.9/81.0/78.8; the 50-step reference takes 1136.2 seconds, while uncached 20-step sampling takes 231.5 seconds and scores 75.5 Total.Source: §4.2.2 · p. 8, Table 2 · p. 8

4.3 ABLATION STUDY

• At 1024 × 1024, denoised/clean reaches 23.155 dB PSNR, above velocity at 18.340 dB, denoised residual at 12.844 dB, and ϵ at 10.007 dB.Across 1,000 sampled calibration subsets of sizes 1, 2, 4, 8, 16, and 32 prompts, every trial selects denoised/clean; Appendix D.1 also reports held-out rankings and controls.Source: §4.3 · p. 9

4.4 DISCUSSION AND LIMITATIONS

• Changing only PixArt’s resolution reverses the velocity versus denoised/clean ranking, and solver changes alter valid candidates and lower-ranked targets.The results indicate that the preferred target depends on sampling configuration even when candidate tensors come from the same model prediction.Source: §4.4 · p. 9
• Calibration error gives an empirical ranking that must be checked on held-out prompts; recalibration is required when the model, solver, schedule, or precision changes.Each candidate also needs a valid conversion to solver input; after calibration, deployment retains only the accepted tensor and needs no candidate comparisons, external quality model, or learned predictor.Source: §4.4 · p. 9
• In shifted LTX-Video, the calibration winner beats the alternative on only 55% of held-out prompts, and its 7.36% regret leads AutoTarget to abstain under the 1.10 risk-ratio threshold.The application can then retain uncached sampling or a previously validated cache policy.Source: §4.4 · p. 9

• AutoTarget compares candidate targets on uncached runs for a fixed reuse schedule and deploys the lowest-risk target when calibration clearly separates it from the runner-up.The procedure accounts for solver conversion and needs no learned predictor.Source: §5 · p. 9
• Experiments on distilled image and video models show that the preferred target changes with sampling configuration; target selection adds acceleration over uncached few-step sampling with a small cache.Held-out tests assess calibration rankings during complete cached runs, and AutoTarget declines selection when candidates are insufficiently separated, leaving the fallback policy to the application.Source: §5 · p. 9

Declaration of AI Assistance

• AI-assisted tools supported academic writing, language editing, code development, debugging, and experiment implementation; the authors reviewed and verified the content and retain responsibility for the work.The authors state they revised AI-assisted content as necessary to ensure accuracy and consistency with research objectives.Source: Declaration of AI Assistance · p. 10

A AutoTarget Procedure

• Calibration tests all candidates from the same exact states, while deployment keeps only the selected target using the same model, solver, coordinate sequence, and full-evaluation steps.Source: A AutoTarget Procedure · p. 12

A.1 CANDIDATE CONSTRUCTION

• Candidates are quantities the pipeline produces or can convert exactly; each has an encoder for storage and a decoder for later use, and candidates with undefined conversions are omitted.The controlled Euler example uses velocity, clean endpoint, and noise endpoint; under equal storage budgets, candidates share tensor shape and data type.Source: §A.1 · pp. 12–13
• At each reuse step, every candidate is decoded from the same anchor and exact current state, then passed through the original solver to compare its next latent or expected intermediate latent.For PixArt-LCM, denoised prediction and predicted clean endpoint are grouped only when both fixed numerical tolerances pass on every calibration trace.Source: §A.1 · p. 13

A.2 SELECTION AND DEPLOYMENT

• AutoTarget averages decoded RGB error over prompts and reuse steps, then deploys the lowest-error target; calibration needs no additional DiT evaluations to test more candidates.Exact runs cost |P|T DiT evaluations; subsequent candidate conversions and VAE decoding add no further DiT evaluations.Source: §A.2 · p. 13
• In the measured PixArt-LCM setting, calibration costs 10.457 seconds and each later generation saves 0.0688 seconds, recovering calibration cost after about 152 generations.The calibration total excludes the separately measured 7.988-second model load.Source: §A.2 · p. 13

B.1 EQUIVALENCE UNDER EULER SAMPLING

• For Euler sampling, velocity, clean-endpoint, and noise-endpoint representations provide the same velocity at reuse steps and produce the same trajectory across refreshes.The equivalence holds under the stated Euler parameterization and reuse schedule.Source: §B.1 · pp. 13–14
• Directly reusing an anchor Euler displacement differs from velocity reuse, with the discrepancy vanishing for that transition when anchor and current step sizes match.The stored displacement is the anchor velocity multiplied by the anchor step size.Source: §B.1 · p. 14

B.2 HOW LOCAL ERRORS ACCUMULATE

• The accumulated complete-state error bound applies to discrete solvers whose transitions satisfy the local stability assumption.The reference evaluates the DiT at every step but refreshes its auxiliary cache only at anchors, matching the deployed schedule's anchor values.Source: §B.2 · p. 14

B.3 COMPARING TARGETS AFTER DECODING

• Because candidate-specific solver-state norms are not directly comparable, AutoTarget ranks candidates using decoded RGB error from the same frozen VAE.The theoretical bound measures complete solver-state error, whereas decoded VAE error is the practical selection score.Source: §B.3 · p. 15

C Experimental Details

• Within each model setting, methods share weights, solver, coordinates, prompts, seeds, and precision, while baseline reuse rules may yield different full-DiT schedules.The compared image settings include FLUX.1-dev at multiple step counts, four-step FLUX.1-schnell and PixArt-LCM, and a 50-step versus distilled 20-step HunyuanVideo comparison.Source: C Experimental Details · p. 15, §C.1 · p. 15

C.1 MODELS AND SAMPLING CONFIGURATIONS

• The controlled PixArt-LCM target ablation uses four steps at 1024 × 1024 with three full DiT evaluations and measures fidelity on 200 DrawBench prompts.The resolution comparison keeps model, solver, steps, guidance, arithmetic, prompts, and seeds fixed, changing only resolution.Source: §C.1 · p. 15
• Image quality uses ImageReward and CLIP, video quality uses VBench components, and image PSNR and SSIM are measured against paired reference trajectories.Table blocks use different fidelity references, while each cached output and reference share the same prompt and starting noise.Source: §C.1 · p. 15

C.2 METRICS AND MEASUREMENT

• Image quality is measured with ImageReward, CLIP, PSNR, and SSIM; video quality uses VBench Semantic, Quality, and Total components.Overall speedup uses the full-step model, while cache-only speedup uses the matching few-step uncached model.Source: §C.2 · p. 15
• Retained cache counts tensors kept between steps, including required history, and is distinct from peak runtime memory that includes temporary workspace.Figure 5 reports ranking agreement in verified settings; the fixed γ ≥1.10 rule abstains on shifted LTX-Video.Source: §C.2 · p. 15, Figure 5 · p. 16

C.3 BASELINE CONFIGURATIONS

• The image baselines are TeaCache, DiCache, TaylorSeer, and HiCache; video also includes DisCa, and the PixArt-LCM threshold sweep uses 50 paired prompts.The sweep reports mean full-DiT evaluations; four full evaluations means no cache acceleration.Source: §C.3 · p. 16

D.1 ADDITIONAL CONTROLS

• Calibration selects denoised/clean for PixArt-LCM and velocity for FLUX.1-schnell, and each selected target wins on all 20 held-out prompts.Figure 6 shows paired 95% bootstrap intervals; in both tested 20-step solver settings, ϵ remains the best target although solver choice changes target validity and the ordering of the two lower-ranked candidates.Source: §D.1 · p. 16, Figure 6 · p. 17
• In the PixArt-LCM 512² calibration, velocity has the lowest error and the second-lowest-to-lowest error ratio is γ = 1.442.Table 3 reports mean squared decoded RGB error over four prompts and one reuse step.Source: Table 3 · p. 17
• The reduced-step controls match the number of DiT evaluations and compare anchor-only solver runs with the standard three-step PixArt-LCM schedule.The baseline sweep covers TeaCache and DiCache thresholds on paired prompts and reports average full-DiT evaluations.Source: Table 4 · p. 17, Table 5 · p. 17

D.2 HOW THE CACHE TARGET AFFECTS REUSE

• Cache-target effects differ across models: FLUX.1-schnell settings are similar, PixArt-LCM target changes produce larger fidelity differences, and HunyuanVideo AutoTarget and DisCa have similar latency and overall VBench scores.These results motivate per-setting target selection but do not show AutoTarget is always best in task quality or speed.Source: §D.2 · p. 18
• On 20 matched HunyuanVideo prompts, the selected native target retains 288 KiB of cache.The cache-size figure excludes temporary runtime memory.Source: Figure 7 · p. 18

D.3 TASK QUALITY AND TRAJECTORY FIDELITY

• Velocity has higher fidelity than next-latent reuse on every held-out prompt, but paired CLIPScore differences are inconclusive and do not follow the PSNR gains.The section reports trajectory fidelity and prompt–image alignment separately because neither metric replaces the other.Source: Figure 8 · p. 18, §D.3 · p. 19

D.4 CALIBRATION STABILITY

• Every tested PixArt-LCM calibration-set size selects the denoised/clean target, which also has the highest fidelity in the target comparison.The calibration-size experiment starts from 32 recorded runs and tests set sizes of 1, 2, 4, 8, 16, and 32.Source: §D.4 · p. 19, Figure 9 · p. 19
• The target comparison uses 200 DrawBench prompts, while stability is assessed by recomputing rankings for 1,000 sampled subsets at each set size.The 200 prompts belong to the target comparison; the 1,000 subset trials belong to the calibration-size experiment.Source: §D.4 · p. 19, Figure 9 · p. 19

E Scope and Limitations

• The general analysis applies to discrete solvers whose complete-state transition satisfies the local stability condition; closed-form target equivalence is shown for Euler with linear-flow parameterization.For other solvers, target rankings must be measured from that solver’s transitions.Source: E Scope and Limitations · p. 19
• Calibration error applies to the tested model, solver, coordinate sequence, and reuse schedule, without guaranteeing the same ranking for other settings or every prompt distribution.The practical comparison uses decoded VAE error, whereas the theoretical bound concerns a state containing cache and solver history.Source: E Scope and Limitations · p. 19
• For tested anchors, cache reuse is nearly as faithful as the four-step reference, while the standard three-step PixArt-LCM schedule is much less faithful.The comparison applies only to the tested schedules and solvers.Source: E Scope and Limitations · p. 19