PaperlayerView exact revision on arXiv

Source-linked AI summary

ActionCodec: What Makes for Good Action Tokenizers

Zibin Dong, Yicheng Liu, Shiduo Zhang, Baijun Ye, Yifu Yuan, Fei Ni, Jingjing Gong, Xipeng Qiu, Hang Zhao, Yinchuan Li, Jianye Hao

arXiv:2602.15397v1cs.ROcs.AI

TL;DR

The paper addresses the limited understanding of how action-tokenizer design affects VLA optimization beyond reconstruction fidelity. It derives information-theoretic design principles and implements them in ActionCodec, which improves training efficiency and VLA performance across simulated and real-world benchmarks. The work positions these principles and the tokenizer as a roadmap for future VQ-based action representation.

  • Problem

    Existing action-tokenizer research primarily evaluates reconstruction or generative fidelity, leaving which properties optimize VLA training unresolved.

  • Method

    The paper analyzes action tokenization from information-theoretic and representation-learning perspectives, then builds ActionCodec around four design principles.

  • Results

    ActionCodec improves training efficiency, success rates, and resistance to overfitting, achieving state-of-the-art performance across simulated and real-world benchmarks.

  • Takeaways & Limitations

    The identified principles and ActionCodec provide a systematic roadmap for developing more effective VQ-based action representations.

  • Takeaways & Limitations

    ActionCodec is currently pretrained on a limited selection of large-scale robotic datasets, constraining broader in-the-wild transfer across robotic embodiments.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models leveraging the native autoregressive paradigm of Vision-Language Models (VLMs) have demonstrated superior instruction-following and training efficiency. Central to this paradigm is action tokenization, yet its design has primarily focused on reconstruction fidelity, failing to address its direct impact on VLA optimization. Consequently, the fundamental question of \textit{what makes for good action tokenizers} remains unanswered. In this paper, we bridge this gap by establishing design principles specifically from the perspective of VLA optimization. We identify a set of best practices based on information-theoretic insights, including maximized temporal token overlap, minimized vocabulary redundancy, enhanced multimodal mutual information, and token independence. Guided by these principles, we introduce \textbf{ActionCodec}, a high-performance action tokenizer that significantly enhances both training efficiency and VLA performance across diverse simulation and real-world benchmarks. Notably, on LIBERO, a SmolVLM2-2.2B fine-tuned with ActionCodec achieves a 95.5\% success rate without any robotics pre-training. With advanced architectural enhancements, this reaches 97.4\%, representing a new SOTA for VLA models without robotics pre-training. We believe our established design principles, alongside the released model, will provide a clear roadmap for the community to develop more effective action tokenizers.

1. Introduction

VLA research has made action tokenization central to autoregressive control, but existing evaluation emphasizes reconstruction fidelity rather than VLA optimization. The paper identifies four optimization-oriented design principles and introduces ActionCodec, which improves training efficiency and performance across benchmarks.

  • Action tokenization has become a critical bottleneck as VLA models increasingly represent actions as discrete tokens within pretrained VLMs.
  • Existing VQ tokenizers are mainly evaluated by generative fidelity, leaving their effects on downstream VLA optimization insufficiently understood.
  • The paper identifies four desiderata: temporal overlap, low vocabulary redundancy, high multimodal mutual information, and token independence.
  • ActionCodec integrates these design choices, then uses residual quantization and embodiment-specific soft prompts to improve fidelity and cross-platform transfer.
  • ActionCodec improves VLA training efficiency, mitigates overfitting, and achieves state-of-the-art performance across simulated and real-world benchmarks.

2. Related Works

VLA models differ in how they represent actions, with continuous and discrete approaches offering distinct trade-offs. Existing discretization and VQ methods leave unresolved questions about which tokenizer properties support VLA optimization.

  • Vision-Language-Action Models: VLA architectures primarily diverge between continuous and discrete action-prediction frameworks.Continuous approaches can require structural changes to the native VLM architecture.
  • Action Tokenization Schemes: Uniform quantization can reduce training efficiency because it ignores the inherent structure of the action space.Parallel decoding and string-based representations address some issues but do not resolve the broader limitations described for action discretization.
  • ActionCodec: ActionCodec uses a Perceiver-like transformer to model diverse token relations and encode variable-length action sequences.The architecture is presented as flexible for handling different token relationships and sequence lengths.
  • Action Tokenization Schemes: VQ tokenizers learn flexible discrete latent representations but are often treated as black boxes in VLA systems.The specific VQ properties that facilitate or obstruct VLA optimization remain insufficiently understood.

3. Preliminaries

The policy maps visual observations and language instructions to action sequences, which are discretized into vocabulary tokens for compatibility with autoregressive VLMs. A VQ tokenizer assigns latent vectors to a learnable codebook and reconstructs actions from the selected codebook entries.

  • Policy and Action Representation: The policy πθ(A|V, L) maps visual observations V and language instructions L to an action sequence A.The action sequence has dimensions T × D.
  • Policy and Action Representation: The action sequence is discretized into a proxy token sequence C whose elements belong to a vocabulary of size S.This discretization enables use of the autoregressive paradigm of pre-trained VLMs.
  • Vector Quantization: Each latent vector is quantized by selecting the nearest learnable codebook entry under squared Euclidean distance.The selected codebook entries are then passed to the decoder to reconstruct the action sequence.
  • Vector Quantization: The reconstructed action sequence is denoted as  = G(e_c1, . . . , e_cn).The tokenizer optimization uses a stop-gradient operator.

4. A VLA Perspective on Action Tokenization

The paper frames action tokenization as optimization of the VLA supervision signal, identifying information-theoretic design criteria and testing them through LIBERO-Goal policy performance. Results support high temporal overlap, controlled capacity, multimodal alignment, and independent tokens as effective choices.

  • Optimization perspective: The expected NLL decomposes into model-fitting error and supervisory ambiguity, making tokenizer design part of VLA optimization.The ambiguity term is H(C|V, L), which the paper links to conflicting gradients and noisy supervision.
  • Overlap rate: Higher overlap rate improves training efficiency, convergence, and robustness; 70% OR reaches 33.4% success at 500 steps versus 8.2% for naive VQ-VAE and 2% for FAST.High overlap also forms distinct, more stable latent clusters earlier in training.
  • Capacity and vocabulary: Token budget and vocabulary size control capacity through n log2 S, with token budget exerting the stronger influence on overfitting resilience.Excessive reduction harms reconstruction, while S = 2048 and n = 16 are identified as a practical trade-off.
  • Vision-language alignment: TCL and CLIP objectives improve overlap and latent structure, while CLIP encourages instruction-relevant attention and TCL favors dataset-specific demonstration patterns.The comparison spans attention maps and t-SNE visualizations across 40 LIBERO tasks.
  • Residual grammar: Independent tokens outperform SA and Causal-SA because autoregressive dependencies can amplify early errors and reduce sensitivity to real-time visual feedback.Perturbation experiments show lower, more stable reconstruction error for independent tokenization, which also yields higher task success and faster early convergence.

5. ActionCodec

ActionCodec instantiates the paper’s action-tokenization design principles and adds practical mechanisms for multi-embodiment transfer and reconstruction fidelity. Its RVQ post-training is designed to improve fidelity without sacrificing the temporal stability needed for VLA training.

  • ActionCodec integrates the identified optimal action-tokenization design choices into a high-performance tokenizer.
  • Embodiment-specific Soft-prompts: Embodiment-specific soft prompts separate shared robotic priors from embodiment-specific mechanical constraints and control frequencies.
  • RVQ Post-training: Standard RVQ can reduce overlap rate below 20%, destabilizing VLA training despite improving reconstruction fidelity.
  • RVQ Post-training: ActionCodec uses RVQ post-training after a stable single-layer VQ model to recover fidelity while preserving overlap-oriented stability.

6. Experiments

Experiments evaluate ActionCodec across simulated and real-world benchmarks, tokenizer baselines, VLA architectures, efficiency metrics, and ablations. Across these settings, the passages report faster optimization, strong success rates, broad architectural compatibility, and transfer to robotic platforms.

  • The experiments address tokenizer efficiency and success rate, architectural integration, real-world performance, and the effects of individual design choices.
  • Comparison with Mainstream Action Tokenizers: 89.5% success within 5K training steps compares with 38.6% for FAST, while ActionCodec also reaches the highest peak success rate.
  • Efficiency: ActionCodec achieves the highest action throughput while maintaining strong task performance, unlike Binning and String, which have prohibitive latency and low throughput.
  • Integration with Prevailing VLA Paradigms: ActionCodec integrates with Parallel Decoding, Knowledge Isolation, and Block-wise Autoregressive paradigms, consistently yielding performance gains.
  • Integration with Prevailing VLA Paradigms: 97.4% average success rate is achieved by ActionCodec-BAR on LIBERO without robotics pre-training, establishing a new SOTA under that setting.
  • Real-world Evaluation: On SO100-ShapeSorter, co-training supports corrective recovery behaviors that task-only models fail to reproduce, especially during placement.
  • Real-world Evaluation: 82.5% success rate is achieved by ActionCodec with pre-training on xArm-PickVeg, alongside lower reconstruction error and higher OR than without pre-training.
  • Ablations: Ablations identify embodiment-specific soft prompts as important for cross-platform knowledge transfer, while RVQ further reduces reconstruction error.

7. Conclusion

The paper concludes that ActionCodec and its action-tokenization design principles improve performance across diverse simulated and real-world benchmarks. It identifies broader embodiment coverage and architectural and alignment refinements as future directions.

  • ActionCodec achieves superior performance across diverse simulation and real-world benchmarks by integrating the identified action-tokenization best practices.
  • Current pre-training uses a limited selection of large-scale robotic datasets, constraining the scope of prospective in-the-wild transfer.
  • Future work targets broader robotic embodiments, optimized neural architectures, and refined vision-language alignment schemes.

A.1. Benchmarks

The benchmark suite spans simulated and real-world manipulation settings that test instruction following, generalization, semantic recognition, and fine-grained control. Each benchmark uses task-specific demonstrations, training procedures, and evaluation protocols.

  • LIBERO: LIBERO contains four suites—Goal, Spatial, Object, and Long—with 10 tasks and 50 demonstrations per task, evaluating instruction following, spatial reasoning, object recognition, and long-horizon completion.
  • SimplerEnv-WidowX: SimplerEnv-WidowX evaluates zero-shot generalization on four out-of-domain desktop manipulation tasks after BridgeData fine-tuning.
  • SO100-ShapeSorter: SO100-ShapeSorter requires language-guided identification and insertion of numbered blocks, testing semantic recognition and fine-grained manipulation.
  • xArm-PickVeg: xArm-PickVeg is a multi-task vegetable-picking benchmark using language instructions, randomized object positions, and a binary success metric.

A.2. Details of Validation Experiments

The validation experiments isolate how action-tokenizer design affects VLA optimization using controlled overlap, architecture, and training configurations. They use a native autoregressive VLM setup and evaluate learning dynamics on LIBERO-Goal.

  • Tokenizer and architecture: Validation uses a Perceiver-based Transformer whose cross-attention-only variant produces mutually independent action tokens, while self-attention variants introduce inter-token dependencies.The comparison includes Independent, self-attention, and causal self-attention architectures.
  • VLA implementation: The VLA predicts action tokens by full-parameter fine-tuning of SmolVLM2-256M without robotics-specific architectural modifications.The vocabulary is expanded with S special action tokens, and invalid sequences are replaced with zero-action chunks during inference.
  • Baseline handling: FAST requires soft decoding with zero-padding or truncation because autoregressive outputs frequently violate its required decoded sequence length.Without this adjustment, FAST-trained models show negligible success rates through most of training.
  • Benchmarking: LIBERO-Goal evaluation monitors variants at 500, 1k, 5k, 10k, and 20k steps, with convergence typically reached by 20k steps.Performance below 500 steps is indistinguishable across variants.
  • Overlap-rate validation: Controlled overlap-rate variants use 26%, 40%, and 70% OR, with higher overlap significantly improving VLA training efficiency.The variants are created by augmenting the VQ-VAE objective with an InfoNCE contrastive loss, although TCL or CLIP-style losses provide higher OR and better stability in later validation.
  • Token-dependence analysis: The perturbation experiment shuffles one predicted action token during autoregressive generation and then evaluates the resulting decoded action sequence.This procedure probes how token dependencies affect downstream action prediction.

A.3. Details of Tokenizer Comparison Experiments

The tokenizer comparison experiments evaluate ActionCodec and mainstream baselines under a common SmolVLM2-2.2B autoregressive setup across LIBERO and additional benchmark configurations. Evaluation tracks convergence and uses the official LIBERO trial protocol.

  • Tokenizer training: Action tokenizers are pretrained on LIBERO, BridgeData, and DROID using 8,192-sample batches, a 2 × 10^-4 learning rate, and 100k training steps.Each tokenizer encodes a fixed 1-second temporal window with timestamp-based positional encoding for heterogeneous control frequencies.
  • Baselines: Baselines include 1,000-bin Binning and raw Python-list String representations, with Binning limited to T = 8 because token count grows linearly with action dimensionality and horizon.For 7-DoF LIBERO at 20Hz, an unconstrained 1-second Binning chunk would require 140 tokens.
  • VLA implementation: All tokenizers are evaluated with full-parameter autoregressive fine-tuning of SmolVLM2-2.2B without structural modifications.The accompanying configurations cover tokenizer comparisons and Simpler-WidowX, SO100, and xArm experiments.
  • Benchmarking: A single VLA model is trained across all LIBERO task suites, with empirical convergence observed by 30k steps.Results follow the official protocol using 50 trials per task from 50 predefined environment states.
  • Prompt format: The VLA prompt format combines state and task text with an assistant action sequence containing special action tokens.The provided message structure includes system instructions, user state-task inputs, and assistant action-token outputs.

A.4. Details of Paradigm Adaptation Experiments

The paradigm-adaptation experiments extend ActionCodec to parallel decoding, knowledge isolation, and block-wise autoregressive action experts. These designs target attention-mask constraints, inference efficiency, and compatibility with diverse robotic embodiments.

  • Action-expert adaptation: A 300M-parameter action expert is introduced to avoid specialized action-token attention masks that prevent Flash Attention and reduce training efficiency.The expert receives the VLM key-value cache from the backbone.
  • Parallel Decoding: Parallel Decoding predicts n action-token logits in one forward pass using bidirectional attention and n learnable [BOS] tokens.Argmax decoding performs comparably to probabilistic sampling, consistent with ActionCodec's condensed semantic representations and low perplexity.
  • Inference efficiency: Parallel Decoding reduces inference latency from n forward passes to one because ActionCodec makes autoregressive modeling of inter-token correlations redundant.This equivalence follows from the tokenizer's highly condensed action representation.
  • Knowledge Isolation: Knowledge Isolation uses a bidirectional flow-matching action expert, but its performance is slightly lower than the naive autoregressive baseline.It remains superior to Knowledge Isolation training with FAST, while the expert-to-VLM gradient is severed.
  • Knowledge Isolation: The authors suggest Knowledge Isolation may be better suited to large-scale VLA pre-training than fine-tuning on specific benchmarks.The stated rationale is that pre-training can enrich the VLM with broad robotic priors.
  • Block-wise Autoregressive: Block-wise Autoregressive prediction groups tokens by RVQ codebook level, allowing within-block visibility and attention only to preceding blocks.Additional passes refine predictions, while RVQ's residual structure provides error damping.
  • Block-wise Autoregressive: BAR achieves the highest performance and establishes a new LIBERO SOTA for models without robotics-specific pre-training.With one pass, BAR is equivalent to Parallel Decoding; additional passes iteratively refine accuracy.
  • Embodiment adaptation: Soft prompts assign learnable embodiment embeddings, while Fourier-embedded timestamps encode control frequency and action duration across platforms.This design supports multi-embodiment integration and cross-platform knowledge transfer.

C. Details of RVQ Post-training

RVQ post-training improves reconstruction fidelity without changing the original action-tokenization procedure. The method preserves the first codebook's tokens while refining the decoder with reconstruction and RVQ objectives.

  • Post-training rationale: RVQ post-training reduces reconstruction error at no additional cost while maintaining high Overlap Rate and vision-language alignment.The procedure is applied after training a VQ-based ActionCodec.
  • Procedure: The RVQ variant is initialized with n = 16 tokens and residual depth L = 3, inheriting and freezing the VQ encoder and primary codebook.Post-training uses only reconstruction loss and RVQ objectives.
  • Token preservation: Because the encoder and first codebook remain frozen, the RVQ model's first-codebook tokens are identical to those of the original VQ model.This preserves the action-tokenization procedure while allowing decoder refinement.
  • Decoder transfer: Copying the post-trained RVQ decoder weights back to the VQ model enhances reconstruction fidelity without altering action tokenization.ActionCodec-BAR uses the full RVQ tokenizer, whereas other models use the VQ tokenizer with the post-trained RVQ decoder.
  • Artifact Entropy: Artifact Entropy evaluates discrete-action-space stability by measuring token uncertainty induced by infinitesimal local perturbations in the action space.The definition accounts for real-world control noise even though deterministic VQ encoding gives H(C|A) = 0 for a fixed action.
  • Artifact Entropy: High Artifact Entropy indicates that negligible input fluctuations can switch the tokenizer between disparate discrete codes, adding uncertainty to VLA supervision.The metric therefore probes sensitivity of quantization boundaries to sensor noise or control jitter.
Loading 2602.15397v1…