PaperlayerView exact revision on arXiv

Source-linked AI summary

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin, Yichao Du, Zhirui Zhang, Laurence T. Yang, Kai Chen

arXiv:2609.18487v1cs.ROcs.AIcs.CLcs.CV

TL;DR

Action tokenizers can preserve low pointwise reconstruction error while distorting the physical ordering of context-dependent action adjustments. The paper introduces PRC and ActionPiece to supervise relational structure during representation learning and quantization, with ActionPiece achieving strong success across four benchmarks under a shared policy setup.

  • Problem

    Mean squared error does not fully capture whether action tokenization preserves physical relationships and context-dependent adjustments between demonstrations.

  • Method

    ActionPiece jointly applies physical rank preservation and quantization regularization to learned features and codeword assignments, alongside reconstruction.

  • Results

    ActionPiece outperforms compared tokenizers, reaching 94.8% on LIBERO, 68.8% on unseen LIBERO-Plus, 71.9% on SimplerEnv, and 51.5% across VLA-Arena L0-L2.

  • Takeaways & Limitations

    The results support incorporating physical relationships into tokenizer learning for autoregressive robot control.

Abstract

from arXiv · show

Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted tokens. Their fidelity is commonly evaluated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. We introduce physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction. Evaluating decoded actions provides a common reference across token vocabularies and decoder architectures, complementing pointwise accuracy with a measure of relational fidelity. We further present ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical rank preservation supervises near-far ordering in encoder and quantized feature distances, while quantization regularization applies the same ordering to codeword assignment distributions. Both objectives augment reconstruction, producing discrete action tokens for standard autoregressive policy learning and execution through a frozen decoder. Under the same Qwen3-VL-4B policy training setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2. Component ablations show that the two objectives jointly improve PRC and policy success, demonstrating the value of physical relationship supervision for action tokenization.

1 Introduction

ActionPiece reframes action tokenization around preserving physical relationships between demonstrations, not only reconstructing individual actions. It introduces PRC and jointly supervises feature geometry and codeword assignments, achieving strong results across four benchmarks.

  • 1 Introduction: The motivation is that MSE can keep individual actions close while failing to preserve context-dependent adjustments needed for precise robot control.Compression may attenuate, distort, or reverse relative action differences even when individual reconstruction errors are small.
  • 1 Introduction: ActionPiece supplies discrete targets for standard autoregressive policy learning, and its frozen decoder converts predicted tokens into executable action chunks.The frozen encoder and quantizer provide policy targets after tokenizer training.
  • 1 Introduction: 94.8% LIBERO success and 68.8% unseen LIBERO-Plus success exceed the strongest baselines by 1.1 and 4.5 percentage points, respectively.Under the shared Qwen3-VL-4B policy training setup, ActionPiece outperforms all compared action tokenizers; it also reaches 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2.
  • 1 Introduction: PRC measures whether local physical distance rankings survive action reconstruction, complementing pointwise reconstruction accuracy across tokenizer designs.It evaluates decoded actions in a common physical reference space.
  • 1 Introduction: ActionPiece jointly supervises physical relationships in learned features and codeword assignment distributions while optimizing reconstruction.Physical rank preservation guides encoder and quantized feature distances, while quantization regularization applies the ordering to codeword assignment distributions.

2 Related Work

Prior VLA action interfaces span discrete, continuous, and hybrid designs. Discrete tokenizers range from coordinate bins and frequency compression to learned latent quantization, trading compactness and autoregressive compatibility against representation choices.

  • 2 Related Work: Discrete VLA interfaces include coordinate-bin tokenization, frequency-domain compression, and use of a VLM’s natural-language vocabulary.RT-2 and OpenVLA discretize coordinates, FAST compresses action chunks, and VLM2VLA uses existing language tokens.
  • 2 Related Work: Continuous approaches predict real-valued action chunks, with regression, diffusion, and flow-matching policies supporting fine-grained or multimodal action generation.Examples include OpenVLA-OFT, RDT, π0, and the GR00T family.
  • 2 Related Work: Hybrid designs combine discrete supervision with continuous execution through flow-matching experts, gradient blocking, or jointly trained autoregressive and diffusion predictions.Examples include π0.5, Knowledge Insulation, and HybridVLA.
  • 2 Related Work: Learned latent tokenizers optimize encoders, decoders, and quantized representations to produce compact, decodable action tokens.FASTer and OAT add residual or finite scalar quantization with structured temporal, spectral, or coarse-to-fine organization.

3 Physical Structure in Action Tokenization

Pointwise reconstruction does not ensure that tokenization preserves the physical ordering of action adjustments across demonstrations. The section therefore motivates PRC, which compares local distance rankings before and after decoding in a common physical action space.

  • 3.1 Preserving Action Variations: Tokenization errors can attenuate, amplify, or reverse action differences, so small individual errors do not guarantee preservation of contextual adjustments.This matters for operations involving precise alignment, grasping, or contact.
  • 3.1 Preserving Action Variations: Decoded reconstruction errors can prevent correct token prediction from recovering the original context-dependent adjustments during execution.The decoder still produces the reconstructed action even when the policy predicts the demonstrated token sequence correctly.
  • 3.2 Physical Rank Consistency: The physical distance combines translation, shortest-geodesic rotation on SO(3), and gripper differences across an action chunk.This defines the physical action space used to assess relative differences.
  • 3.2 Physical Rank Consistency: PRC compares physical-distance rankings from each action chunk to the same original neighbors before and after reconstruction.The metric uses k = 32 nearest neighbors and Spearman correlation.
  • 3.2 Physical Rank Consistency: Higher PRC indicates better preservation of local distance ordering, whereas MSE measures fidelity of individual reconstructions.Decoded action space enables comparison across token vocabularies, sequence lengths, and decoder architectures.

4 ActionPiece

ActionPiece combines compact discrete action coding with physical-order supervision across representation learning and quantization, then freezes the decoder for autoregressive policy execution.

  • ActionPiece: ActionPiece jointly supervises physical relationships in learned representations and codeword assignments alongside reconstruction.Physical rank preservation acts on encoder and quantized feature distances, while quantization regularization applies the ordering to codeword assignment distributions.
  • Compact, Fully Decodable Tokens: Eight-step action chunks are compressed into discrete tokens and decoded back into complete executable chunks.The encoder produces latent slots, residual vector quantization represents them, and the decoder reconstructs the full chunk in one pass.
  • Physical Action Distance: Physical action distance combines translation, shortest-path rotation on SO(3), and gripper differences to define near and far training pairs.Chunk distance averages corresponding per-step distances, and the same physical ordering supports both training objectives and decoded-action evaluation.
  • Physical Rank Preservation: Physical rank preservation aligns representation distances before and after quantization with near–far physical ordering.The weighted distance combines encoder features with discrete representations used by the decoder, using differentiable codeword estimation for gradient transmission.
  • Quantization Regularization: Quantization regularization applies the same physical ordering to Jensen–Shannon divergences between codeword assignment distributions.Soft codeword probabilities are compared across corresponding slots, using the same near and far neighbors as the rank objective.
  • Training and Policy Integration: After training, the tokenizer is frozen, its indices enter the VLM vocabulary, and generated token sequences decode into executable action chunks.The VLA uses standard next-token prediction for policy learning before decoding the generated sequence.

5 Experiments

Experiments evaluate ActionPiece across matched tokenizer comparisons, transfer settings, and component ablations. Results show stronger physical-order preservation and downstream success across four benchmarks, with both objectives contributing.

  • 5.1 Experimental Protocol: The evaluation spans LIBERO, unseen LIBERO-Plus, real-to-sim SimplerEnv, and VLA-Arena conditions using matched policy protocols and complementary generalization settings.Controlled comparisons fix representation, normalization, execution, policy, optimization, budget, and evaluation settings.
  • 5.2 Relationship Between Tokenizer Fidelity and Policy Success: PRC correlates more strongly with policy success than reconstruction fidelity across 55 tokenizer–benchmark evaluations, with Spearman correlations of 0.681 versus 0.544.Both metrics show positive associations with success.
  • 5.3 Matched Action-Tokenizer Comparison: 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus make ActionPiece the strongest compared tokenizer, exceeding the strongest baseline by 1.1 and 4.5 points.All policies use the same Qwen3-VL-4B setup; LIBERO-Plus tests transfer under seven unseen perturbation types.
  • 5.4 Evaluation on SimplerEnv and VLA-Arena: 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2 are ActionPiece’s highest aggregate success rates among evaluated methods.The VLA-Arena result includes 82.2%/42.7%/29.5% on L0/L1/L2 and leads L1 and L2 averages despite training only on L0.
  • 5.5 Component Ablation: 94.8% and 68.8% are reached when physical rank preservation and quantization regularization are combined, exceeding either objective alone on both benchmarks.Starting from standard RVQ, PRP alone reaches 93.8% and 65.4%, while QR alone reaches 92.9% and 62.4%.
  • 5.5 Component Ablation: 0.953 PRC is achieved by the combined model, compared with 0.902 for standard RVQ, while PRP alone provides the larger individual gain.Adding QR further raises LIBERO-Plus success by 3.4 percentage points, and the combined model exceeds either component alone in six LIBERO-Plus categories.

6 Conclusion

The paper argues that action tokenization should preserve relationships among actions, not only individual reconstruction accuracy. It introduces PRC and ActionPiece, whose controlled evaluations and ablations improve physical structure and downstream success across diverse settings.

  • 6 Conclusion: ActionPiece combines physical rank preservation and quantization regularization to incorporate action relationships into tokenizer learning for control.The approach is evaluated through controlled comparisons and ablations spanning four benchmarks, action-data sources, and generalization settings.
  • 6 Conclusion: PRC evaluates physical distance ordering in decoded action space alongside reconstruction accuracy, providing a common reference across tokenizer representations.This distinguishes preservation of action relationships from fidelity of individual reconstructions.

A Alternative Training Objectives

The experiments compare objectives added to standard RVQ across reconstruction, physical structure, and policy success. ActionPiece combines complementary supervision to improve both physical rank consistency and policy performance.

  • A Alternative Training Objectives: SIGReg regularizes encoder features toward an isotropic Gaussian distribution.
  • A Alternative Training Objectives: Temporal supervision matches first differences for translation and gripper commands and relative quaternion rotations for rotation.It targets the demonstrated magnitude and direction of change rather than motion itself.
  • A Alternative Training Objectives: Neighborhood attracts each action representation toward its ten physically nearest minibatch actions using larger weights for closer neighbors.Physical rank preservation instead compares a nearby action with a farther one to supervise relative ordering.
  • A Alternative Training Objectives: Table 5 evaluates objective variants and component ablations on reconstruction, physical structure, and policy success.Each plus row adds only the named objective to standard RVQ; PRP and QR denote physical rank preservation and quantization regularization.
  • A Alternative Training Objectives: ActionPiece improves both PRC and policy success, whereas Neighborhood improves policy success while decreasing PRC.The comparison uses Qwen3-VL-4B policies trained for 30K steps.

B Implementation Details

The final ActionPiece configuration is shared across the evaluated environments, while alternative tokenizers retain settings from their original papers or codebases.

  • B Implementation Details: ActionPiece uses the same configuration for LIBERO, LIBERO-Plus, VLA-Arena, and SimplerEnv.The configuration and objective weights are summarized in Table 6.
  • B Implementation Details: Other tokenizers use configurations specified in their original papers or released codebases.
  • B Implementation Details: Table 6 summarizes the ActionPiece configuration and objective weights.

B.1 Differentiating Codeword Selection

With one quantization level, each latent slot selects a codeword using a hard forward assignment and a soft gradient path. This differentiable construction preserves hard execution while enabling representation-distance optimization.

  • B.1 Differentiating Codeword Selection: With Q = 1, each of the 16 latent slots selects one codeword.
  • B.1 Differentiating Codeword Selection: The straight-through estimator uses the selected hard code contribution for the forward value and soft assignment probabilities for gradient computation.No Gumbel noise is sampled and τq = 1.
  • B.1 Differentiating Codeword Selection: The reconstruction objective uses an identity straight-through estimator, while the codebook is updated by exponential moving average.

B.2 Multiple Quantization Levels

The tokenizer extends differentiable codeword selection to multiple residual quantization levels by averaging partial reconstructions. At Q = 1, the multi-level expression reduces to the single-level formulation.

  • B.2 Multiple Quantization Levels: Q = 1 and Q = 2 produce similar downstream policy performance, so the final ActionPiece configuration uses Q = 1.
  • B.2 Multiple Quantization Levels: For Q > 1, differentiable codeword selection is applied at each residual level.The cumulative contribution after the corresponding output projections is tracked for each level.
  • B.2 Multiple Quantization Levels: The extended expression averages partial reconstructions across all residual levels, using LayerNorm and unit-normalized representations.At Q = 1, this expression reduces to Eq. (6).

B.3 Physical-Neighbor Selection

Physical-neighbor selection chooses near and far action examples separately for each anchor from the other actions in its minibatch, using the 5th and 95th distance percentiles.

  • B.3 Physical-Neighbor Selection: For each anchor, the method excludes the anchor itself and selects the q0.05 action as near and q0.95 action as far by increasing physical distance.The percentile calculation is performed separately over the other actions in the anchor’s minibatch.
  • B.3 Physical-Neighbor Selection: Physical rank preservation and quantization regularization use the same near-far action pair.
Loading 2609.18487v1…