Source-linked AI summary
WorldCache: Content-Aware Caching for Accelerated Video World Models
Umair Nawaz, Ahmed Heakl, Ufaq Khan, Abdelrahman Shaker, Salman Khan, Fahad Shahbaz Khan
TL;DR
DiT-based video world models are costly to run, and existing training-free caching can mishandle dynamic scenes. WorldCache adapts when and how features are reused through motion-, saliency-, approximation-, and phase-aware mechanisms, achieving 2.3× speedup with 99.4% baseline-quality retention on Cosmos-Predict2.5-2B.
Problem
Sequential denoising and spatio-temporal attention make DiT-based world-model rollouts computationally expensive, while zero-order-hold caching can produce ghosting, blur, and motion inconsistencies in dynamic scenes.
Method
WorldCache is a training-free caching framework that adapts skip thresholds to motion and saliency, and approximates skipped features through blending, warping, and phase-aware scheduling.
Results
2.3× speedup with 99.4% baseline-quality retention is achieved on Cosmos-Predict2.5 (2B) evaluated on PAI-Bench.
Takeaways & Limitations
WorldCache provides training-free acceleration without architectural changes and transfers across model scales and conditioning modalities.
Takeaways & Limitations
The saliency-weighted drift design relies on channel variance as an indicator of information-rich regions where caching errors are perceptually visible.
Abstract
from arXiv · showhide
Diffusion Transformers (DiTs) power high-fidelity video world models but remain computationally expensive due to sequential denoising and costly spatio-temporal attention. Training-free feature caching accelerates inference by reusing intermediate activations across denoising steps; however, existing methods largely rely on a Zero-Order Hold assumption i.e., reusing cached features as static snapshots when global drift is small. This often leads to ghosting artifacts, blur, and motion inconsistencies in dynamic scenes. We propose \textbf{WorldCache}, a Perception-Constrained Dynamical Caching framework that improves both when and how to reuse features. WorldCache introduces motion-adaptive thresholds, saliency-weighted drift estimation, optimal approximation via blending and warping, and phase-aware threshold scheduling across diffusion steps. Our cohesive approach enables adaptive, motion-consistent feature reuse without retraining. On Cosmos-Predict2.5-2B evaluated on PAI-Bench, WorldCache achieves \textbf{2.3$\times$} inference speedup while preserving \textbf{99.4\%} of baseline quality, substantially outperforming prior training-free caching approaches. Our code can be accessed on \href{https://umair1221.github.io/World-Cache/}{World-Cache}.
1 Introduction
DiT-based world models offer physically consistent visual prediction but incur high rollout latency. WorldCache addresses caching failures in dynamic scenes by adapting reuse decisions and approximations, achieving substantial speedup while retaining quality.
- Motivation: World-model rollouts are computationally expensive because they generate many frames through sequential transformer blocks across dozens of denoising steps.This latency obstructs interactive world simulation and closed-loop deployment.
- Motivation: Training-free caching skips expensive layers when estimated feature drift falls below a threshold, reusing cached activations instead of recomputing every block.Existing methods include FasterCache’s fixed schedule and DiCache’s adaptive shallow-layer probes.
- WorldCache: WorldCache replaces zero-order hold caching with a perception-constrained dynamical approximation designed for DiT-based world models.Its modules target both when to skip computation and how to approximate skipped features.
- WorldCache: WorldCache combines motion-adaptive thresholds, saliency-weighted drift, least-squares blending, and motion-compensated warping to reduce stale or inaccurate reuse.These components align caching decisions with latent motion and perceptually important regions.
- Results: 2.3× speedup is achieved on Cosmos-Predict2.5 (2B) while preserving 99.4% of baseline quality on PAI-Bench.The reported result outperforms DiCache and FasterCache in the speed–quality trade-off.
2 Related Work
Prior work accelerates diffusion and video models through cheaper sampling, activation caching, and motion- or perception-aware reuse. WorldCache builds on these directions while retaining the base model and denoising schedule and targeting internal activation reuse.
- Video diffusion and world simulators: Video generation models are increasingly studied as world simulators, with Cosmos-Predict and PAI-Bench targeting physical-AI simulation and evaluation.These systems are evaluated for physical consistency and action-relevant prediction.
- Efficient diffusion inference: Efficient diffusion inference reduces sampling cost through fewer denoising steps, cheaper steps, alternative samplers, fast solvers, or distillation.WorldCache follows a different axis by reusing internal activations while keeping the base model and schedule.
- Caching and reuse in diffusion transformers: Caching methods exploit redundancy across diffusion timesteps and guidance passes, including high-level feature reuse, attention-feature reuse, and adaptive probing.FasterCache also caches conditional and unconditional redundancy to reduce guidance overhead, while DiCache makes reuse adaptive with an online probe.
- Motion-compensated and perception-aware reuse: Motion-compensated propagation, multirate schedules, optical flow, perceptual metrics, and multiscale detail measures motivate alignment-aware and perception-aware feature reuse.These prior directions frame accuracy–efficiency trade-offs that WorldCache incorporates for video world models.
3 Method
WorldCache replaces fixed-threshold, static feature reuse with motion-, saliency-, and phase-aware decisions plus dynamical approximation for cached deep outputs. Its pipeline probes shallow blocks, selectively skips deep blocks, and improves cache-hit reconstruction through interpolation and motion compensation.
- Foundation: Probe-Then-Cache: WorldCache inherits a probe-then-cache skeleton but replaces DiCache’s fixed skip criterion and reuse mechanism.The probe evaluates shallow blocks and estimates deep-layer change before deciding whether to reuse cached states.
- Causal Feature Caching: CFC tightens the skip threshold during fast dynamics and relaxes it during slow dynamics using a latent two-step motion proxy.The proxy anchors to the most recent fully computed input, while a ping-pong buffer keeps reuse tied to recent fully computed states.
- Saliency-Weighted Drift: SWD emphasizes drift in perceptually important regions, triggering recomputation for salient foreground changes while permitting reuse when only background regions change.Its saliency map derives from channel-wise variance, with high variance indicating structurally complex regions where errors are more visible.
- Optimal Feature Approximation: OFA reconstructs skipped deep outputs through least-squares optimal blending and optional motion-compensated warping instead of verbatim copying.Its vector-projection formulation preserves directional information and attenuates extrapolation when feature trajectories curve.
- Adaptive Threshold Scheduling: ATS tightens thresholds during early structure formation and relaxes them during late detail refinement to increase safe reuse.With t=2, T=35, and βd=4.0, the multiplier is approximately 1.2 early and reaches approximately 4.6 at t=32; the ablation reports less than 0.6% quality reduction relative to baseline.
4 Experiments
Experiments evaluate WorldCache across Cosmos-Predict2.5 scales, Image2World and Text2World settings, WAN2.1 transfer, qualitative rollouts, and incremental ablations. Across these evaluations, WorldCache delivers strong speed–quality trade-offs, while its modules separately improve motion safety, salient-region decisions, approximation fidelity, and late-stage acceleration.
- Main results: WorldCache achieves the best Image2World speed–quality trade-off on Cosmos-Predict2.5 at both 2B and 14B scales.It delivers up to 2.3× speedup at 2B and 2.18× at 14B with negligible quality degradation across domain and quality dimensions.
- Main results: 2.1× speedup reduces Cosmos-2B Text2World latency from 54.34 s to 26.28 s while preserving ∼99.6% of baseline average quality.On Cosmos-14B, WorldCache reaches 2.14× speedup and improves Avg. to 0.771 from the 0.769 unaccelerated baseline.
- Main results: 2.3× speedup reduces Cosmos-2B Image2World latency from 55.04 s to 24.48 s while retaining an Avg. score of 0.798 versus the 0.803 baseline.At 14B, WorldCache reaches 2.18× speedup with Avg. 0.813 versus the 0.814 baseline.
- Qualitative comparison: WorldCache preserves more coherent object appearance, scene layout, and trajectories than DiCache in qualitative Image2World rollouts while also accelerating inference.The comparison shows DiCache ghosting, deformation, and inconsistent motion under dynamic scene evolution.
- Ablation study: CFC yields 1.52× speedup with essentially unchanged overall score, SWD raises speed to 1.67×, and OFA reaches Overall 0.8035 while reducing net speedup to 1.49×.These ablations respectively indicate benefits from motion-adaptive thresholds, salient-region drift estimation, and higher-quality approximation.
- Ablation study: OFA trades throughput for approximation fidelity, whereas ATS unlocks the largest acceleration by relaxing thresholds late in denoising after stabilizing effects from CFC, SWD, and OFA.Late denoising refinement updates are small, allowing more aggressive threshold relaxation.
5 Conclusion
WorldCache is a training-free, architecture-preserving framework for accelerating DiT-based video generation through perception-constrained dynamical caching. It combines motion-aware decisions, saliency-aligned drift estimation, improved approximation, and denoising-phase scheduling for deployable world-model inference acceleration.
- Conclusion: WorldCache replaces Zero-Order Hold caching with motion-aware decisions, saliency-aligned drift estimation, improved approximation operators, and denoising-phase scheduling.Its modules are CFC, SWD, OFA, and ATS.
- Conclusion: WorldCache requires no training and no architectural changes, making it immediately deployable for accelerating next-generation video world models.
Overview
The paper evaluates quality and efficiency across PAI-Bench and EgoDex-Eval using perceptual, semantic, physical, fidelity, latency, and speedup metrics.
- Evaluation scope: PAI-Bench-G evaluates 1,044 Text-to-World and Image-to-World samples using Quality, Domain, Overall, Latency, and Speedup metrics.Quality comprises perceptual sub-metrics, while Domain measures semantic and physical plausibility.
- Quality metrics: Subject Consistency measures primary-subject identity stability using DINO-feature cosine similarities across initial and adjacent frames.The metric combines long-range consistency with local frame-to-frame stability.
- Quality metrics: Background Consistency analogously measures background stability using CLIP image features across initial and adjacent frames.
- Quality metrics: Motion Smoothness uses frame interpolation after removing odd-indexed frames, with lower reconstruction error corresponding to smoother motion.The raw error is normalized and inverted so higher values indicate smoother motion.
- Quality metrics: Aesthetic Quality, Imaging Quality, and Overall Consistency measure visual appeal, low-level fidelity, and video–text semantic alignment, respectively.Aesthetic Quality uses LAION, Imaging Quality uses MUSIQ, and Overall Consistency uses ViCLIP.
- Additional evaluation: EgoDex-Eval reports reconstruction-style PSNR, SSIM, and LPIPS alongside efficiency, with higher PSNR/SSIM and lower LPIPS indicating closer reference agreement.
B Implementation Details and Runtime Setup
WorldCache is a training-free, runtime-toggleable method evaluated across multiple DiT backbones under fixed hardware and generation settings, with benchmark-scale runtime savings.
- Implementation: WorldCache changes neither model weights nor training data and can toggle cache decisions and cache-hit approximation at runtime.
- Runtime setup: Experiments cover Cosmos-Predict2.5, WAN2.1, and DreamDojo backbones, including multiple parameter scales and world-model generation settings.
- Runtime setup: All experiments use a single NVIDIA H200 except Domain-score computation, which uses four H200 GPUs for the Qwen3-VL judge.
- Evaluation protocol: PAI-Bench-G evaluation uses the full 1,044-sample suite under both T2W and I2W, reporting benchmark quality and end-to-end efficiency metrics.
- Runtime impact: On Cosmos-2B I2W, processing the full benchmark takes approximately 7.1 hours with WorldCache versus approximately 16 hours for baseline, saving approximately 9 hours.
- Robotics evaluation: On WAN2.1-14B EgoDex-Eval, WorldCache generates 81-frame rollouts conditioned on the first frame and reports PSNR, SSIM, LPIPS, latency, and speedup.
- Baseline comparison: Across Cosmos-2B PAI-Bench comparisons, existing caching baselines cluster around approximately 1.3× speedup, while WorldCache exceeds 2× within the same quality band.
D Evaluation Results on Robotic Manipulation: EgoDex-Eval
EgoDex-Eval shows that WorldCache accelerates robotics video prediction while remaining close to baseline fidelity across WAN2.1, Cosmos, and DreamDojo, with motion alignment adding limited overhead.
- EgoDex-Eval results: 2.30× speedup on WAN2.1-14B reduces latency from 391.9 s to 171.6 s while retaining approximately 99.2% PSNR quality.PSNR is 13.19 versus 13.30 baseline, SSIM is 0.498 versus 0.503, and LPIPS is 0.460 versus 0.459.
- EgoDex-Eval results: WorldCache exceeds DiCache on WAN2.1-14B, which reaches 1.88× speedup with larger fidelity gaps across PSNR, SSIM, and LPIPS.
- Cross-backbone transfer: On Cosmos-Predict2.5-2B, WorldCache reaches 0.518 PSNR, equal to baseline, and 0.466 SSIM versus 0.455 baseline and 0.445 DiCache.
- Cross-backbone transfer: On DreamDojo-2B, WorldCache achieves 1.90× speedup while preserving PSNR at 23.69 versus 23.63 baseline.SSIM and LPIPS change from 0.775 to 0.737 and from 0.226 to 0.251, respectively.
- Cross-backbone transfer: Across WAN2.1, Cosmos, and DreamDojo, WorldCache consistently outperforms DiCache with higher speedups and lower quality degradation.
- Feature alignment: OFA estimates motion on spatially downsampled features, then upsamples and rescales displacement vectors for final warping.The design focuses flow estimation on macroscopic motion and reduces the feature-map area used by the solver to 1/25.
- Feature alignment: The flow-based alignment mechanism adds less than 3% computational overhead per cached step.
- Adaptive threshold scheduling: ATS relaxes the caching threshold during denoising, reaching an approximately 5.0 scaling multiplier near the final step and enabling aggressive late-stage reuse.
F.1 Hyperparameter Selection
Hyperparameter studies select moderate motion, saliency, threshold, and warping settings, while increasing denoising steps still leaves WorldCache substantially faster than baseline.
- Skip-decision sensitivity: CFC selects α=2 because increasing motion sensitivity helps until α=2, after which performance drops.
- Skip-decision sensitivity: SWD performs best at βs=0.12, indicating that moderate saliency weighting improves skip decisions while stronger weighting reduces robustness.
- Skip-decision sensitivity: ATS selects βd=4 because larger values over-relax reuse and degrade both Domain and Quality through excessive late-denoising skipping.
- Approximation sensitivity: OSI+Warp provides the strongest overall fidelity among tested approximation operators, while warp scale 0.5 balances alignment benefits and noise sensitivity.
- Denoising step budget: At 35 denoising steps, WorldCache latency is 25.0 s versus 57.0 s for baseline; at 140 steps, it is 66.0 s versus 199.1 s.Baseline latency scales roughly linearly with denoising steps, while cache reuse reduces effective per-step cost.
- Qualitative behavior: The qualitative PAI-Bench comparison reports unstable pedestrian appearance and position for DiCache in a dynamic crossing scene.
G Additional Qualitative Results
WorldCache preserves temporal coherence in challenging dynamic and contact-rich rollouts, maintaining stable scene structure, object geometry, and robot motion across extended sequences.
- Image2World examples: WorldCache maintains coherent identities and trajectories for salient moving pedestrians while preserving global scene layout.In manipulation scenes, it also preserves object boundaries and hand-pose consistency around fast, articulated motion.
- Image2World examples: WorldCache preserves object boundaries and hand poses across frames in manipulation scenes involving fast articulated motion and salient carried objects.
- Additional qualitative results: Additional rollouts show stable scene layout, robot pose, and object geometry as actions unfold across refrigerator, tabletop, arm–human, and toolbox scenarios.
- Robotics-centric rollouts: WorldCache maintains stable scene structure and coherent robot or object motion over long-horizon robotics rollouts, including contact-rich interactions and cluttered environments.These examples show no introduced drift in object geometry or kinematic consistency as rollouts advance.
H Limitations and Future Work
WorldCache remains conservative in difficult regimes, and abrupt scene changes can reduce cache hit rates; future work targets adaptive policies and stronger motion-aware approximation.
- Limitations: Extremely abrupt scene changes, including rapid viewpoint jumps and heavy occlusions, can occasionally reduce cache hit rates.Motion- and saliency-aware constraints make caching conservative in difficult regimes.
- Future work: Future work will explore online caching policies, stronger motion estimation, and uncertainty-aware warping for high-speed dynamics and occlusions.Proposed policies include lightweight predictors and per-layer or per-token reuse budgets.