Source-linked AI summary
An Empirical Study of World Model Quantization
Zhongqian Fu, Tianyi Zhao, Kai Han, Hang Zhou, Xinghao Chen, Yunhe Wang
TL;DR
World-model planning is costly because repeated inference over long horizons increases computational and memory demands, while PTQ effects in iterative world-model rollouts remain poorly understood. The paper systematically evaluates PTQ for DINO-WM across methods, precision settings, granularities, visual-planning tasks, and horizons up to 50 iterations, finding distinct rollout and planning failures under low-bit quantization. These results identify module-specific sensitivities and provide deployment guidance under computational constraints.
Problem
Repeated world-model inference makes long-horizon planning computationally and memory intensive, while PTQ behavior in iterative world-model planning remains poorly understood.
Method
The study evaluates diverse PTQ methods for DINO-WM under weight-only and joint weight-activation settings across bit-widths, granularities, visual-planning tasks, and horizons up to 50 iterations.
Results
Quantization effects extend beyond standard accuracy trade-offs, with group-wise weights stabilizing some low-bit rollouts, encoder quantization proving more damaging than predictor quantization, and aggressive precision reduction misaligning planning with task success.
Takeaways & Limitations
Quantized world-model deployment requires accounting for long-horizon rollout dynamics, module asymmetry, and task-dependent failure modes rather than relying only on standard accuracy measurements.
Takeaways & Limitations
DINO-WM's reliance on pretrained features introduces a fixed bottleneck, and maintaining high-resolution latent representations remains challenging for real-time deployment.
Abstract
from arXiv · showhide
World models learn an internal representation of environment dynamics, enabling agents to simulate and reason about future states within a compact latent space for tasks such as planning, prediction, and inference. However, running world models rely on hevay computational cost and memory footprint, making model quantization essential for efficient deployment. To date, the effects of post-training quantization (PTQ) on world models remain largely unexamined. In this work, we present a systematic empirical study of world model quantization using DINO-WM as a representative case, evaluating diverse PTQ methods under both weight-only and joint weight-activation settings. We conduct extensive experiments on different visual planning tasks across a wide range of bit-widths, quantization granularities, and planning horizons up to 50 iterations. Our results show that quantization effects in world models extend beyond standard accuracy and bit-width trade-offs: group-wise weight quantization can stabilize low-bit rollouts, activation quantization granularity yields inconsistent benefits, and quantization sensitivity is highly asymmetric between encoder and predictor modules. Moreover, aggressive low-bit quantization significantly degrades the alignment between the planning objective and task success, leading to failures that cannot be remedied by additional optimization. These findings reveal distinct quantization-induced failure modes in world model-based planning and provide practical guidance for deploying quantized world models under strict computational constraints. The code will be available at https://github.com/huawei-noah/noah-research/tree/master/QuantWM.
1 Introduction
World models support predictive simulation and iterative visual planning, but repeated inference makes long-horizon deployment computationally and memory intensive. This study examines how PTQ methods, granularities, and module sensitivities affect planning reliability in DINO-WM.
- World models provide internal predictive representations that let agents simulate environment dynamics for planning and decision-making.
- Repeated inference over candidate trajectories makes computational cost and memory footprint scale with the planning horizon, limiting practical deployment.FP16 reduces overhead but remains insufficient when inference runs tens of times per episode.
- PTQ is attractive for world models because it compresses pretrained systems without requiring retraining, but its effects in iterative planning remain poorly understood.Quantization errors can compound across recurrent latent rollouts and affect long-horizon performance.
- DINO-WM exhibits activation outliers and non-uniform scale distributions in its encoder and predictor, increasing susceptibility to quantization noise.Activation smoothing can partially mitigate these numerical effects, but its interaction with long-horizon rollout dynamics remains unclear.
- The study evaluates RTN, OMSE, AWQ, SmoothQuant, and OmniQuant across weight-only and joint weight-activation settings, varying bit-widths, granularities, tasks, and horizons up to 50 iterations.Experiments use DINO-WM on two embodied visual planning environments.
- The resulting empirical analysis aims to guide deployment of DINO-WM and similar world models under strict computational constraints.
2 Related Works
Related work frames world models as compact predictive environment representations and describes DINO-WM's visual-planning design. It also introduces quantization and PTQ as efficiency techniques whose application to world-model planning remains insufficiently characterized.
- World Models: World models learn compact predictive environment representations for efficient planning and decision-making in reinforcement learning.Earlier systems combined variational autoencoders with recurrent or deep latent-variable models, while later work incorporated transformers and multimodal observations.
- World Models: DINO-WM uses pretrained self-supervised visual features to support zero-shot planning in unseen environments without task-specific reward functions.
- World Models: High-resolution latent representations create a computational bottleneck for real-time deployment, while high-fidelity visual environments require substantial resources.
- Model Quantization: Quantization improves inference efficiency by converting full-precision parameters into low-precision integer representations, reducing computation and memory use.The paper formalizes quantization with scaling, rounding, zero-point adjustment, and clipping.
- Model Quantization: PTQ uses a small calibration dataset to optimize quantization parameters while avoiding retraining, and has been applied broadly to CNNs, Vision Transformers, and Language Transformers.Its behavior in iterative world-model planning remains poorly understood.
- Model Quantization: Table 1 reports 3 to 8-bit weight-only PTQ results on Wall, with #W, #A, #G, and AQG denoting weight bits, activation bits, group size, and activation quantization granularity.
3 Empirical Study
The study evaluates PTQ for DINO-WM across quantization settings, granularities, modules, and planning horizons, focusing on long-horizon visual planning reliability. Results show that group-wise weights can stabilize some low-bit rollouts, activation granularity is inconsistent, and encoder quantization is more damaging than predictor quantization.
- Experiment Settings: Experiments evaluate DINO-WM with representative PTQ methods, weight-only and weight-activation settings, and planning horizons up to 50 iterations.The study uses a common pretrained checkpoint, calibration data separated from evaluation, and multiple weight and activation granularities.
- Weight Quantization: At 8-bit weight precision, evaluated methods remain comparable to FP32 across planning horizons, whereas 4-bit quantization degrades short-horizon performance.The 4-bit degradation appears under both per-channel and per-group weight quantization.
- Weight Quantization: Under 4-bit OmniQuant with per-group grouping (#G = 128), Wall success improves from 0.20 at 0 iterations to 0.94 at 50 iterations.This recovery closely matches the FP32 baseline and is stronger than the recovery observed with per-channel grouping.
- Weight Quantization: At 3-bit precision, success rates collapse near zero across datasets, methods, and grouping strategies even as planning iterations increase.The stabilizing effect of group-wise quantization largely disappears when quantization noise overwhelms the learned transition dynamics.
- Activation Quantization: Per-token activation quantization does not consistently outperform per-tensor quantization, with lower precision and longer horizons producing additional variability.The results suggest that globally consistent activation scaling can matter more than finer-grained representational flexibility in iterative planning.
- Module Sensitivity: Encoder quantization causes rapid, persistent degradation, while moderate predictor quantization is more recoverable as planning horizons increase, especially on Wall.PushT can retain visually plausible reconstructions despite sharp success-rate drops, indicating that task failure may reflect latent dynamic bias rather than immediate visual collapse.
- Planning Loss: Aggressive quantization can prevent planning loss from decreasing, weakening alignment between the optimization objective and task success.This misalignment makes additional optimization increasingly ineffective as precision is reduced.
4 Key Insights of World Model Quantization
The study identifies distinct quantization effects across rollout stability, module sensitivity, task failure modes, and planning-objective alignment. These findings show that low-bit quantization affects world-model planning beyond ordinary accuracy degradation.
- Group-wise weight quantization stabilizes 4-bit rollouts, but loses its benefit when extreme precision reduction overwhelms learned transition dynamics.Moderate grouping partially alleviates scale mismatch, whereas 3-bit Wall quantization remains dominated by quantization noise.
- Per-token activation quantization provides limited and inconsistent gains over per-tensor quantization, especially across rollout lengths and low-bit settings.More fine-grained scaling can help at moderate bit-widths, but may introduce instability when precision is low.
- Encoder quantization is more damaging than predictor quantization because representation distortions propagate through subsequent rollouts and cannot be corrected by additional planning.Predictor quantization mainly affects temporal consistency and can sometimes be partially mitigated by longer planning horizons.
- PushT primarily exhibits planning-level geometric misalignment, whereas Wall exhibits representation-level collapse under low-bit quantization.PushT retains visually plausible reconstructions despite degraded success, while Wall can show severe reconstruction degradation from the initial frame.
- Aggressive low-bit quantization can make the planning loss increase during optimization, producing a mismatch between the planning objective and true task success.Under severe precision constraints, additional planning iterations can yield diminishing or negative returns because optimization becomes ineffective.
5 Conclusion
The paper presents a systematic study of post-training quantization for planning world models using DINO-WM across methods, granularities, bit-widths, and planning horizons. It finds that quantization interacts with latent rollout dynamics and planning objectives, producing component- and task-dependent failures that constrain aggressive low-bit deployment.
- The study evaluates post-training quantization for planning-based world models using DINO-WM across quantization methods, granularities, bit-widths, and planning horizons.
- Quantization effects extend beyond standard accuracy degradation to latent rollout dynamics, planning objectives, component sensitivity, and task-dependent failure modes.
- Aggressive low-bit precision has limits in long-horizon planning, motivating quantization strategies that explicitly account for planning dynamics.
- The findings provide practical guidance for deploying quantized world models under strict computational constraints.