PaperlayerView exact revision on arXiv

Source-linked AI summary

QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Models

Jingxuan Zhang, Yunta Hsieh, Zhongwei Wan, Haokun Lin, Xin Wang, Ziqi Wang, Yingtie Lei, Mi Zhang

arXiv:2602.20309v4cs.LG

TL;DR

VLA models face substantial compute and memory demands as their language and action components scale, while existing efficiency methods do not directly address the tightly coupled DiT action head. QuantVLA introduces training-free, architecture-preserving PTQ with selective quantization and scale calibration for language and action pathways. Across LIBERO evaluations, it maintains or exceeds full-precision performance while reducing memory use on quantized components.

  • Problem

    Large VLA models create substantial computational and memory overhead, while quantization remains difficult under tightly coupled language and DiT action pathways.

  • Method

    QuantVLA selectively quantizes language and DiT feed-forward linear layers while retaining attention projections in floating point and calibrating attention temperature and residual-output energy.

  • Results

    QuantVLA maintains or exceeds full-precision task performance across representative LIBERO VLA evaluations while substantially reducing memory usage.

  • Takeaways & Limitations

    QuantVLA provides a training-free, architecture-preserving path toward low-bit VLA deployment under constrained memory and power budgets.

Abstract

from arXiv · show

Vision-language-action (VLA) models unify perception, language, and control for embodied agents but face significant challenges in practical deployment due to rapidly increasing compute and memory demands, especially as models scale to longer horizons and larger backbones. To address these bottlenecks, we introduce QuantVLA, a training-free post-training quantization (PTQ) framework that, to our knowledge, is the first PTQ approach for VLA systems and the first to successfully quantize a diffusion transformer (DiT) action head. QuantVLA incorporates three scale-calibrated components: (1) a selective quantization layout that integerizes all linear layers in both the language backbone and the DiT while keeping attention projections in floating point to preserve the original operator schedule; (2) attention temperature matching, a lightweight per-head scaling mechanism that stabilizes attention logits and is folded into the dequantization scales at inference; and (3) output head balancing, a per-layer residual interface calibration that mitigates post-projection energy drift. The framework requires no additional training, uses only a small unlabeled calibration buffer, and supports integer kernels for low-bit weights and activations while leaving the architecture unchanged. Across representative VLA models on LIBERO, QuantVLA exceeds the task success rates of full-precision baselines, achieves about 70% relative memory savings on the quantized components, providing a practical pathway toward scalable low-bit embodied intelligence under strict compute, memory, and power constraints.

1. Introduction

VLA models connect visual perception, language reasoning, and executable control, but their growing reasoning and action modules create substantial deployment costs. QuantVLA addresses this gap with training-free, architecture-preserving PTQ that quantizes both language and diffusion-action pathways while retaining strong task performance.

  • Motivation: Large VLA models incur substantial computational and memory overhead in downstream reasoning and sequential control, limiting embedded and mobile deployment.Profiling attributes a substantial portion of overhead to reasoning and control rather than visual perception.
  • Motivation: Existing efficiency methods primarily optimize vision encoders, leaving the language backbone and tightly coupled DiT action head largely unchanged.This leaves major reasoning and action-generation costs underexploited.
  • Research gap: Most prior frameworks redesign blocks or add routing and caching, whereas few directly apply PTQ to the sensitive DiT action head without retraining.The paper identifies low-bit quantization of the action head as an important unused opportunity for reducing memory and bandwidth.
  • Approach: QuantVLA is introduced as a scale-calibrated, training-free PTQ framework specifically designed for VLA models.Its analysis targets quantization sensitivity in the DiT-based action head and guides selective quantization and calibration.
  • Outcome: About 70% relative memory savings on quantized modules accompanied QuantVLA's success rates exceeding the full-precision baseline on LIBERO.The reported result combines low-bit deployment efficiency with improved benchmark performance.

2. Related Work

Related VLA efficiency work reduces cost through compact architectures, new training pipelines, or inference-time pruning, routing, and caching. QuantVLA differs by applying post-training numerical quantization to both language and diffusion-action components while preserving the original architecture and execution order.

  • VLA architectures: VLA systems include encoder–decoder policies, pretrained language or vision-language models, diffusion transformers, and hybrid language–diffusion architectures.Hybrid systems tightly couple semantic reasoning with low-level actuation through diffusion-based control.
  • Efficient and compact VLA models: Compact VLA approaches reduce deployment cost through smaller backbones, lightweight architectures, specialized training pipelines, or asynchronous inference stacks.QuantVLA is presented as orthogonal to these designs because it operates after training.
  • Inference optimization: Inference-framework methods improve runtime through language-layer pruning, compact visual tokens, intermediate-representation reuse, key–value caching, and mixture-of-layers routing.These methods retain unchanged numerical precision while reducing computation during rollouts.
  • Post-training quantization: QuantVLA directly changes numerical precision after training, quantizing both the language backbone and diffusion-based action head without modifying execution order or adding routing logic.This distinguishes it from architectural and inference-framework efficiency methods.
  • Post-training quantization: PTQ research commonly targets weight-only or weight–activation quantization, but VLA control remains an open stability problem under tight multimodal–diffusion coupling.The paper positions its contribution around low-bit efficiency for coupled VLA control.

3. Method

QuantVLA is a training-free PTQ framework for diffusion-based VLA models that preserves the original architecture and operator schedule while calibrating quantization-sensitive attention and residual pathways.

  • VLA preliminaries: The DiT policy head conditions on fused visual-language representations, robot proprioception, and a diffusion timestep before iteratively refining an action latent.After T refinement steps, the final latent is decoded into the action.
  • Quantization sensitivity: Quantization perturbs upstream inputs and can propagate linearly through attention even when attention weights remain in floating point.The first-order analysis models the quantized input as XQ = XT + εup and tracks its effect on attention.
  • QuantVLA framework: QuantVLA integerizes LLM linear layers and selected DiT layers while retaining Q, K, V, and O projections in floating point.This selective layout avoids compounding errors at the most sensitive attention and residual interfaces while preserving the operator schedule.
  • Quantization sensitivity: DiT quantization causes attention-temperature drift and post-projection energy drift, altering attention entropy and residual injection behavior across layers.Variance changes in Q and K shift softmax temperature, while output projection changes modify residual-stream amplitude.
  • QuantVLA framework: Attention Temperature Matching rescales logits with a calibrated per-head factor, while Output Head Balancing rescales residual-path activations using per-layer energy matching.Both corrections use safe ranges and neutrality bands; their scalar factors are folded into existing dequantization scales without additional inference GEMMs.
  • QuantVLA framework: QuantVLA combines selective quantization, ATM, and OHB to stabilize the DiT action head under low-bit widths without retraining.The calibration mechanisms introduce no new operators or activations and require only one-time scalar folding during calibration.

4. Experiment

QuantVLA is evaluated across two DiT-based VLA policies and LIBERO task settings, using selective low-bit quantization with lightweight calibration. It preserves performance while reducing memory, and remains robust across precision and denoising conditions.

  • Experimental Setup: Evaluation covers OpenPI π0.5 and GR00T N1.5 on four LIBERO task suites, with W4A8 implementation using a small unlabeled calibration buffer.Scales are folded into dequantization at inference, with ATM and OHB clipping applied for stability.
  • Selective Quantization Layout: Quantizing all LLM linear layers and DiT MLP blocks while keeping Q, K, V, and O floating point remains closest to baseline performance.Full action-head or full-stack quantization produces the largest degradation, especially on long-horizon tasks.
  • Calibration Effects: ATM reduces attention-logit standard-deviation mismatch, while OHB aligns post-projection attention-output RMS with the floating-point teacher.The calibrated curves narrow the teacher gap across blocks, particularly in deeper layers.
  • LIBERO Results: 97.6% average success and 1.28 GB memory on π0.5 compare with 4.27 GB baseline, while GR00T N1.5 reaches 88.0% average accuracy at 0.91 GB versus 2.02 GB.QuantVLA matches or exceeds baseline task performance while reducing memory on both models.
  • Efficiency: Reduced memory consumption supports long-horizon policy generation, longer temporal contexts, extended input horizons, or parallel control policies within the same hardware budget.The efficiency benefit is attributed to the selective quantization layout and lightweight calibration.
  • Robustness and Generalization: 95.3% average success at W4A4 on π0.5 and 88.0% average success at eight denoising steps on GR00T N1.5 show robustness across inference settings.The study also evaluates applicability beyond DiT-based policies using OpenVLA.

5. Conclusion

QuantVLA is a training-free PTQ framework that quantizes VLA language and action pathways while preserving architecture and stabilizing low-bit inference. It reduces memory usage and improves accuracy without additional training.

  • QuantVLA integerizes the language backbone and diffusion-transformer feedforward blocks while keeping attention projections in floating point.This selective layout preserves the original operator schedule.
  • Two lightweight calibration scalars align attention temperature and restore output energy after quantization.These mechanisms stabilize low-bit inference at attention and residual interfaces.
  • Post-training quantization reduces memory footprint and accelerates inference without additional training.
  • Activations use per-token unsigned quantization, while weights use per-output-channel signed quantization in a bias-free linear layer.Activation scales are estimated from a small unlabeled calibration buffer, and dequantization reconstructs floating-point approximations.

B. DuQuant Implementation Details

The DuQuant implementation smooths activations and factorizes layers with orthogonal rotations and permutations before integerization. The inverse transform is folded into weights to preserve functional equivalence.

  • Per-channel smoothing with a diagonal matrix balances activation difficulty against the relative ease of quantizing weights.
  • The transformed layer is further factorized using block orthogonal rotations and a permutation.
  • The activation-side transform is quantized before integerization, while its inverse is folded into weights to preserve equivalence.Integer matrix multiplication then uses the corresponding dequantization scales at the selected activation and weight bit widths.

C. How logits transfer from the language backbone to DiT in a VLA

In DiT attention, dequantization scales determine both attention sharpness and residual-stream energy. QuantVLA therefore calibrates the scale transfer from quantized queries, keys, values, and output projections.

  • Quantized queries, keys, and values are dequantized with separate scales before attention computation.The formulation uses ˆQ = s_q ˜Q, ˆK = s_k ˜K, and ˆV = s_v ˜V.
  • The output projection is dequantized as ˆW_o = s_o ˜W_o before concatenated attention heads produce the block output.
  • The ratio d/(s_qs_k) controls attention sharpness, while s_vs_o primarily determines energy injected into the residual stream.

D. QuantVLA Parameters

QuantVLA uses low-bit integerization with shared DuQuant and statistical-calibration settings across the evaluated models. Attention and output interfaces receive separate scalar corrections fitted from unlabeled data.

  • The evaluated models use 4-bit weights and 8-bit activations to reduce memory and bandwidth while keeping accuracy stable.
  • Block size is 64 for both inputs and outputs, with channel permutation and row rotations applied to improve quantization suitability.The rotations preserve the real-valued function while reducing the impact of large channels and outliers.
  • Attention temperature matching learns one scalar α per head, while output head balancing learns one scalar β per layer.The β calibration is limited to the diffusion-transformer head, and both scalars are fitted from a small unlabeled calibration buffer.

E. Comparison with other PTQ Method

QuantVLA maintains stable performance under more aggressive W4A8 quantization than SmoothQuant, including on long-horizon tasks. Its average W4A8 success rate is slightly higher than the floating-point baseline.

  • QuantVLA achieves comparable or slightly better results than SmoothQuant under W4A8, despite SmoothQuant performing reasonably at W8A8.QuantVLA therefore operates at a lower bitwidth while maintaining stable performance across the evaluated task suites.
  • QuantVLA improves performance on the long-horizon task, where low-precision inference typically accumulates greater drift over sequential generation.
  • The average success rate under W4A8 is slightly higher than the floating-point baseline.
  • QuantVLA sustains task performance in long-sequence scenarios and provides a more favorable accuracy–efficiency trade-off for low-bit VLA inference.

F. Extended Benchmark Evaluation

On the Pick-and-Can manipulation benchmark, QuantVLA better preserves task performance than SmoothQuant under W4A8, although it does not fully match the floating-point baseline.

  • Under W4A8, QuantVLA substantially narrows the performance gap to FP16 and maintains a higher success count than SmoothQuant.SmoothQuant exhibits a noticeable drop compared with the FP16 baseline under the same precision setting.
  • QuantVLA better preserves task performance under aggressive quantization, suggesting reduced sensitivity of the action head to quantization noise in manipulation scenarios.

G. Applicability Beyond DiT-Based VLA Models

QuantVLA also applies beyond the DiT-based VLA setting evaluated in the main experiments. On OpenVLA, whose architecture differs substantially, QuantVLA matches the reported performance.

  • QuantVLA matches OpenVLA performance on LIBERO-Spatial despite OpenVLA using a deeper language backbone and a non-DiT action head.The DiT-specific ATM and OHB mechanisms are not directly applicable to OpenVLA because its language–action coupling differs.
Loading 2602.20309v4…