PaperlayerView exact revision on arXiv

Source-linked AI summary

AGQ: Action-Guided Ultra-Low-Bit Quantization for Vision-Language-Action Models

Xuhong Qiu, Zhuoyu Wu, Qiawei Zheng, Wenhui Ou, Wenqi Fang, Raphaël C. -W. Phan

arXiv:2610.33074v1cs.CVcs.AI

TL;DR

VLA models can perform language-directed robotic tasks, but their large weight footprint challenges deployment on memory-constrained onboard platforms, and ultra-low-bit quantization risks task performance. AGQ allocates matrix bitwidths using action sensitivity and measured recovery cost, alongside per-channel calibration, grouped lossless coding, and block-wise recovery. On LIBERO, it reaches 92.7% success at W3 and 70.5% at W2.5; its asymmetric W1.5* configuration achieves 6.10× complete-model compression while retaining 25.5% success.

  • Problem

    VLA models have large weight footprints that challenge onboard deployment, while ultra-low-bit quantization risks degrading action prediction and closed-loop task outcomes.

    Source: §I · p. 1

  • Method

    AGQ allocates matrix bitwidths using action sensitivity and measured recovery cost, with per-channel calibration, grouped lossless coding, and block-wise recovery.

    Source: §I · pp. 1–2

  • Results

    92.7% overall success at W3 exceeds AWQ and GPTQ by 1.2 and 3.7 percentage points, respectively; AGQ achieves 70.5% at W2.5 on LIBERO.

    Source: §I · p. 2, C. Task Success at Ultra-Low Bitwidths · p. 4

  • Takeaways & Limitations

    The W1.5* W4-vision/W1.5-language configuration achieves 6.10× complete-model compression relative to BF16 while retaining 25.5% task success.

    Source: §I · p. 2, §IV · p. 4

Abstract

from arXiv · show

Vision-language-action (VLA) models enable robots to perform tasks specified through natural-language instructions, but their large weight footprint challenges deployment on memory-constrained onboard platforms. Ultra-low-bit quantization reduces this footprint, yet preserving task performance and controlling weight recovery costs remain challenging. We propose Action-Guided Quantization, which combines action sensitivity with a measured proxy for weight recovery cost to allocate matrix bitwidths under storage budgets. Per-channel calibration compensates for quantization error, while a uniform bitwidth within each matrix preserves regular layouts. Bit-dependent grouped lossless coding reduces resident weight storage, and block-wise recovery limits temporary buffers. On LIBERO, AGQ achieves 92.7% task success at W3, exceeding AWQ and GPTQ by 1.2 and 3.7 percentage points, respectively, and retains 70.5% success at W2.5. An asymmetric configuration, denoted W1.5$^{*}$, uses W4 vision weights and a parameter-weighted average of 1.5 bits for language weights. With lossless coding, it achieves $6.10\times$ complete-model compression relative to BF16 while retaining 25.5% task success. Code and experimental results are available at https://github.com/Qiuxu00/AGQ

I. INTRODUCTION

VLA models bring language-directed robotics to memory-constrained platforms, but ultra-low-bit quantization risks degrading closed-loop task performance. AGQ allocates matrix bitwidths using action sensitivity and recovery cost, and combines calibration and coding strategies to balance performance with storage.

I. INTRODUCTION

  • VLA models enable robots to perform natural-language-specified tasks, but their large weight footprint challenges resource-constrained onboard deployment.Quantization-induced action errors can affect subsequent observations and task outcomes in closed-loop control.Source: §I · p. 1
  • AGQ achieves a better tradeoff between task success and target-weight compression than QVLA in Fig. 1.The figure distinguishes task-success lines from target-weight compression bars relative to BF16.Source: Figure 1 · p. 1, §I · p. 2
  • AGQ combines action sensitivity with measured weight-recovery cost to allocate matrix bitwidths under storage budgets.A common W4 probe supplies an action-sensitivity prior; each matrix keeps a uniform bitwidth.Source: §I · pp. 1–2
  • Per-channel quantizer learning reduces quantization error, while grouped lossless coding and block-wise recovery reduce resident and temporary weight storage.Uniform bitwidths within matrices preserve regular layouts.Source: §I · pp. 1–2
  • In closed-loop LIBERO tasks, AGQ achieves 92.7% overall success at W3 and 70.5% at W2.5; asymmetric W4-vision/W1.5-language achieves 25.5% success with 6.10× complete-model compression relative to BF16.Source: §I · p. 2

A. Overview

AGQ separates offline allocation and calibration from compressed inference. It assigns one bitwidth per matrix using action sensitivity and recovery overhead, then recovers active BF16 weight blocks during inference.

A. Overview

  • Inference keeps encoded weights GPU-resident and decodes, unpacks, and dequantizes only the active block to BF16, releasing it on exit.Allocation and parameter learning remain offline.Source: A. Overview · p. 2

B. Action-Guided Bit Allocation

  • A common W4 perturbation supplies a lower-bit allocation prior, while one bitwidth per matrix preserves regular decoding.One probe per matrix replaces scans at every candidate bitwidth without assuming identical sensitivity rankings.Source: B. Action-Guided Bit Allocation · p. 2
  • Matrix sensitivity is measured by quantizing that matrix alone to W4 while keeping all other parameters in BF16 and measuring same-input action deviation.The overview identifies calibration observations and backbone storage budgets as inputs to the preparation stages.Source: B. Action-Guided Bit Allocation · p. 2, A. Overview · p. 2
  • The allocation uses action sensitivity and measured recovery overhead, with candidate-format storage—including metadata and padding—constrained by backbone budget.The quadratic penalty protects matrices with greater action sensitivity or measured recovery overhead; W4 timing sets priority but does not predict latency at each precision.Source: A. Overview · p. 2, B. Action-Guided Bit Allocation · p. 3

C. Per-Channel Calibration for Regular Execution

Per-channel clipping bounds determine quantization scales and zero points, which are learned through block reconstruction before fused unpacking and dequantization at deployment.

C. Per-Channel Calibration for Regular Execution

D. Bit-Dependent Grouped Coding and Execution

Bit-dependent grouped lossless coding compresses quantized weights, while independently decodable segments and block-wise recovery manage decoding dependencies and temporary storage.

D. Bit-Dependent Grouped Coding and Execution

  • Entropy coding compresses quantized weights, with bit-dependent grouping balancing codebook size against the number of values recovered per symbol.Source: D. Bit-Dependent Grouped Coding and… · p. 3
  • Four adjacent W2 values or two W3/W4 values form joint Huffman symbols, while W1 uses raw packing.The six- or eight-bit symbols cover at most 256 symbols and recover multiple values per decode.Source: D. Bit-Dependent Grouped Coding and… · p. 3
  • Per-matrix encoding chooses the smaller representation, including codebooks and block indexes, with raw packing as fallback and no added quantization error.Fixed-size blocks are split into independently decodable segments, and prefix lookup reduces bit-by-bit traversal.Source: D. Bit-Dependent Grouped Coding and… · p. 3
  • During inference, block entry restores packed codes and block exit releases temporary BF16 weights, bounding dense temporary storage by the largest active block.Encoded weights and metadata remain GPU-resident.Source: D. Bit-Dependent Grouped Coding and… · p. 3

A. Experimental Setup

The evaluation tests OpenVLA-OFT in closed-loop LIBERO tasks and measures success, storage, memory, and latency on an RTX 3090.

A. Experimental Setup

  • OpenVLA-OFT is evaluated on four LIBERO suites, with some weights calibrated on A100 and all evaluations run on RTX 3090.Source: A. Experimental Setup · p. 3
  • Ten configurations each run 600 closed-loop episodes, with 150 episodes per suite and 6,000 episodes total.Source: A. Experimental Setup · p. 3
  • Metrics include overall and per-suite success, complete-model storage, peak GPU memory, and mean latency over ten fixed inputs; timing excludes observation preprocessing and action postprocessing.Each call predicts eight actions.Source: A. Experimental Setup · p. 3

B. Effectiveness of Joint Bit Allocation

At W2.5, guided allocation’s best setting reaches 70.5% overall success, outperforming calibrated random and QVLA allocation; probe results show substantial matrix-wise differences in sensitivity and recovery overhead.

B. Effectiveness of Joint Bit Allocation

C. Task Success at Ultra-Low Bitwidths

AGQ retains strong LIBERO task success at ultra-low precisions, including an asymmetric W1.5* configuration. Storage savings come with configuration-dependent latency costs.

C. Task Success at Ultra-Low Bitwidths

  • At W3, AGQ reaches 92.7% task success, exceeding AWQ, GPTQ, and QVLA by 1.2, 3.7, and 55.3 percentage points; it retains 70.5% at W2.5 and 35.0% at W2 versus QVLA’s 0.0%.At W1.5*, it achieves 25.5% success; at W4, AGQ reaches 97.7% versus 96.9% for BF16.Source: C. Task Success at Ultra-Low Bitwidths · p. 4

D. Storage and Inference Efficiency

  • At W1.5*, quantization reduces complete-model storage from 15.457 to 2.656 GB, and lossless coding reduces it further to 2.534 GB (6.10× compression); peak GPU memory falls from 16.082 to 3.537 GB (78.0%).Ratios include all model parameters and quantization metadata.Source: Table III · p. 4, D. Storage and Inference Efficiency · p. 4
  • Uncoded W4–W1.5* configurations take 243.6–254.4 ms versus 187.3 ms for BF16, consistent with execution dominated by recovery and BF16 matrix multiplication rather than packed weight size.Coding raises W1.5* latency from 250.5 to 310.4 ms and W2.5 latency from 245.2 to 473.9 ms.Source: D. Storage and Inference Efficiency · p. 4
  • W1.5* has the smallest footprint at 310.4 ms and 25.5% success; coding saves 0.122 GB for 59.9 ms additional latency, versus 0.204 GB and 228.7 ms at W2.5.The authors conclude that storage gains should be assessed jointly with decoding cost.Source: D. Storage and Inference Efficiency · p. 4

IV. CONCLUSION

AGQ allocates matrix bitwidths using action sensitivity and a measured proxy for weight recovery cost. On LIBERO, it outperforms compared baselines at shared precisions below four bits, while W1.5* compresses the complete model 6.10× and retains 25.5% success.

IV. CONCLUSION

  • AGQ guides matrix-wise bit allocation with action sensitivity and a measured proxy for weight recovery cost, alongside per-channel calibration, grouped lossless coding, and block-wise recovery.The allocation is for ultra-low-bit VLA deployment.Source: §IV · p. 4
  • On LIBERO, AGQ outperforms the compared baselines at shared precisions below four bits and achieves 70.5% task success at W2.5.Source: §IV · p. 4
  • W1.5* achieves 6.10× complete-model compression while retaining 25.5% task success.Source: §IV · p. 4
Loading 2610.33074v1…