Source-linked AI summary
DriveCache: Action-Aware Caching for Driving World Model Inference
Jianchun Yang, Jian Liang, Xianda Guo, Pinhan Fu, Yanlun Peng, Conglang Zhang, Wenke Huang, Mang Ye
TL;DR
Driving video generators face inference-cost constraints, while existing cache controllers omit pre-generation driving signals. DriveCache uses planned ego motion to schedule feature reuse, improving the quality–efficiency trade-off across three evaluated generators, including a 2.036 dB PSNR gain over TeaCache at approximately 2× speedup on Wan2.2 A14B.
Problem
Diffusion driving generators incur costly repeated backbone evaluations, while existing cache controllers omit pre-generation signals such as ego speed and planned trajectories.
Method
DriveCache allocates scene-level reuse from planned motion and places it across denoising steps using dynamic programming with causal drift correction.
Results
Across three generator configurations, DriveCache improves quality–efficiency trade-offs over cache baselines, including 2.036 dB higher PSNR than TeaCache at approximately 2× speedup on Wan2.2 A14B.
Takeaways & Limitations
The results support planned ego translation and rotation as pre-generation control signals for action-aware diffusion caching in driving video generation.
Abstract
from arXiv · showhide
Driving video generation models support autonomous-driving development by predicting controllable future scenes for simulation, planning evaluation, and offline data generation. Diffusion-based driving generators repeatedly evaluate large backbones across denoising steps, which limits generation throughput. Existing diffusion acceleration methods reduce this cost, but general-purpose designs omit driving signals available before generation, such as ego speed and planned trajectories. Experiments across driving motions show that cache tolerance varies with ego translation and rotation, denoising progress, and consecutive reuse length. We propose DriveCache, a training-free, action-aware controller that uses planned motion to allocate reuse across scenes and dynamic programming to place it across denoising steps under a calibrated response budget. A causal drift check refreshes features and replans the remaining schedule when generation departs from calibration. Across three generator configurations, DriveCache improves the overall fidelity-efficiency trade-off over evaluated cache methods. Our code will be publicly available.
Introduction
Driving video generators provide controllable future scenes for simulation, planning, and data generation, but iterative diffusion inference constrains online latency and offline scale. DriveCache addresses this bottleneck with a training-free, action-aware caching controller that uses planned motion, dynamic programming, and causal drift correction.
- Motivation: Driving video generation models predict controllable future observations under ego actions for simulation, policy training, planning evaluation, and offline data generation.These predictions also enable candidate-plan comparison and diverse scenario generation.
- Motivation: Iterative denoising repeatedly evaluates large backbones, making inference latency a constraint for online planning cycles and large-scale offline generation.Cost increases with model capacity, resolution, and video length.
- Limitations of Existing Methods: Existing feature-cache controllers reuse intermediate features but generally rely on fixed schedules or signals observed only after denoising begins.Feature caching leaves the generator unchanged, unlike acceleration methods that may require new weights, retraining, calibration data, or specialized kernels.
- Design Requirements: Planned ego translation and rotation provide a pre-generation motion prior, but cache error also depends on denoising position, cache age, architecture, and generated content.A driving-aware controller must combine scene-level motion information with a denoising-level response model and causal correction.
- DriveCache: DriveCache interpolates terminal reuse responses from low-motion and moving-turn anchors, then uses exact dynamic programming to place reuse under one budget.A pre-reuse drift veto rejects out-of-support decisions and replans the unexecuted suffix.
Related Work
Related work spans diffusion and video-generation foundations, computational acceleration, training-free caching, and driving-specific generative systems. Existing cache controllers range from static schedules to runtime-adaptive reuse, while driving generators incorporate histories, maps, layouts, and ego trajectories.
- Diffusion Generation and Acceleration: Diffusion models define reverse transport from noise to data, with deterministic samplers, continuous-time formulations, spatiotemporal video modeling, and scalable Diffusion Transformers.Token merging and low-precision attention reduce per-step cost, while training-based acceleration requires optimized weights and model-specific training.
- Training-Free Acceleration: Training-free acceleration preserves pretrained weights through fast solvers and cache controllers that reuse internal computation.Static routing appears in DeepCache, PAB, FORA, and Learning-to-Cache, whereas TeaCache, AdaCache, and EasyCache adapt reuse to runtime changes.
- Driving Systems: Driving-system research covers occupancy, sensor fusion, planning, safety-critical generation, scene representations, and surround-view spatial reasoning.Surround-view studies specifically characterize cross-view depth and spatial reasoning.
- Driving Systems: Driving video generators forecast observations from histories, maps, layouts, and ego trajectories.This line of work connects generative scene forecasting with driving-specific inputs and motion information.
Methodology
DriveCache makes caching a causal, action-aware decision before each backbone evaluation. It calibrates motion-dependent reuse responses, optimizes legal placements with dynamic programming, and refreshes and replans when runtime checks detect departure from calibration.
- Causal cache decisions: DriveCache separates each denoising step into an expensive reusable backbone and an inexpensive interface, deciding full evaluation or reuse from the backbone input before execution.The first step always uses full computation, and reuse substitutes the most recently computed backbone output.
- Schedule optimization: The dynamic program maximizes feasible reuse under response budget τ while enforcing first evaluation, cache-age, backbone-boundary, and native no-cache constraints.The recurrence exactly optimizes the calibrated response objective in O(K3) time and O(K2) rolling memory.
- Motion-aware calibration: Planned ego translation and accumulated rotation define a two-anchor calibration space that interpolates motion-dependent run responses without scene-specific coefficients.The low-motion anchor minimizes standardized motion, while the moving-turn anchor maximizes standardized motion among supported candidates; unsupported scenes use full inference.
- Motion-aware calibration: Run response combines signed terminal-latent perturbations propagated by frozen-sampler Jacobian-vector products, while denoising position and cache age determine reuse cost and tolerance.Normalization by terminal-latent energy enables cross-step comparison, and signed responses preserve cancellation and amplification effects.
- Runtime correction: A causal drift-and-charge veto accepts reuse only within calibrated thresholds and budget, otherwise evaluates the backbone, refreshes the cache, and replans the remaining suffix without altering the executed prefix.Calibration uses no optimizer or parameter update, and runtime inference leaves generator weights and sampling steps unchanged.
Experiments
Experiments evaluate DriveCache across Wan2.2 5B, A14B, and Epona under matched-compute, multi-seed protocols. Results show improved latency–fidelity trade-offs, motion-conditioned reuse, robustness to pose noise, and ablation support for joint placement and runtime correction.
- Models and protocol: Evaluation spans 2,220 DrivingGen samples across 222 scenes and all 150 nuScenes validation scenes for Epona, using ten fixed seeds and frozen reuse targets.Wan uses 40 denoising steps, Epona uses 100 visual denoising steps, and calibration uses 32 disjoint scenes.
- Overall results: 1.84× speedup with 34.744 dB PSNR on 5B and 1.97× with 32.778 dB on A14B make DriveCache fastest or tied for fastest across configurations.At comparable speed, it improves over SeaCache by 1.884 dB PSNR on 5B and 1.276 dB on A14B, with gains in trajectory quality and DTW.
- Motion adaptation and robustness: 22.4 reuses for stationary clips versus 18.7 for turns confirm motion-conditioned allocation, while anchors cover 96.4% of scenes.Planned ego motion sets each schedule before denoising, and the drift check evaluates generated states before planned reuse.
- Motion adaptation and robustness: 1.57× speedup with a 0.213 dB loss under 1.0 m/5 degree pose noise shows that causal correction complements the motion prior.The drift check refreshes features and corrects schedule drift during inference.
- Transfer to Epona: 1.89× speedup and 0.140 LPIPS on 150 Epona scenes outperform DeepCache’s 1.68× at comparable aggregate and turning fidelity.Larger stationary gains reflect more reuse on tolerant scenes and conservative behavior on turns.
- Ablations and calibration: Joint propagation captures start-step and cache-age effects, while fixed placement causes the largest loss and the drift veto guards against schedule drift.Predicted response tracks realized degradation, two to eight anchors change PSNR by only 0.062 dB, and moving reuse into the high-noise A14B expert sharply reduces quality.
Conclusion
DriveCache is a training-free, action-aware controller that uses planned ego motion to allocate cache reuse across scenes and denoising steps. It calibrates responses with anchor traces and uses causal drift checks to refresh features and replan when generation departs from calibration.
- Method: DriveCache predicts scene-level cache tolerance from planned ego translation and rotation before denoising.This enables action-aware reuse allocation across evaluated driving motions.
- Method: DriveCache allocates reuse across scenes and places it across denoising steps using exact dynamic programming.The controller is training-free and action-aware.
- Method: Two anchor traces calibrate joint run responses, while a causal drift check refreshes the cache and replans the unexecuted suffix after calibration departure.The refresh-and-replan mechanism responds when generation departs from calibration.