Source-linked AI summary
OAT: Ordered Action Tokenization
Chaoqi Liu, Xiaoshen Han, Jiawei Gao, Yue Zhao, Haonan Chen, Yilun Du
TL;DR
Autoregressive robot policies need action tokenizers that avoid long sequences and unstructured latent spaces while remaining compatible with next-token prediction. OAT combines compression, total decodability, and causal ordering, enabling prefix-based decoding; across simulation and real-world tasks, OAT consistently outperforms prior tokenization schemes and diffusion-based baselines. Its deployment currently uses a fixed autoregressive depth, leaving adaptive token allocation as an open problem.
Problem
Existing action tokenizers either produce prohibitively long sequences or unstructured, partially decodable representations that limit autoregressive generation.
Method
OAT uses transformer register tokens, finite scalar quantization, causal attention, and nested dropout to create compressed, causally ordered, prefix-decodable action tokens.
Results
Across 20+ simulation and real-world manipulation tasks, OAT consistently achieves superior performance and supports flexible prefix-based decoding compared with prior tokenization schemes and diffusion-based baselines.
Takeaways & Limitations
Prefix-based detokenization provides an anytime trade-off between inference computation and action fidelity, with later tokens progressively refining decoded actions.
Takeaways & Limitations
OAT uses a fixed autoregressive depth at deployment, although the required token count may vary with action complexity and precision needs.
Abstract
from arXiv · showhide
Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference. However, applying autoregressive modeling to continuous robot actions requires an effective action tokenization scheme. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences, or learned latent tokenizers that lack structure, limiting their compatibility with next-token prediction. In this work, we identify three desiderata for action tokenization - high compression, total decodability, and a left-to-right causally ordered token space - and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using transformer with registers, finite scalar quantization, and ordering-inducing training mechanisms. The resulting token space aligns naturally with autoregressive generation and enables prefix-based detokenization, yielding an anytime trade-off between inference cost and action fidelity. Across more than 20 tasks spanning four simulation benchmarks and real-world settings, autoregressive policies equipped with OAT consistently outperform prior tokenization schemes and diffusion-based baselines, while offering significantly greater flexibility at inference time.
I. INTRODUCTION
Autoregressive robot policies depend critically on how continuous actions are converted into discrete tokens. OAT addresses the trade-offs in existing tokenizers by combining compression, total decodability, and causal ordering.
- Action tokenization converts continuous control signals into discrete token sequences and shapes learning dynamics, model capacity, scalability, and downstream performance.
- Existing approaches trade off compression, autoregressive modelability, and decodability, producing sequences that may be long, unstructured, or only partially decodable.
- Effective action tokenization requires high compression, total decodability, and causal ordering aligned with left-to-right next-token prediction.
- OAT uses register-based transformers, finite scalar quantization, and nested dropout to produce compressed, causally ordered tokens whose prefixes decode to plausible action chunks.
- The paper reports that OAT satisfies all three desiderata and enables superior performance with flexible prefix-based decoding across 20+ simulation and real-world manipulation tasks.
II. RELATED WORK ON GENERATIVE POLICIES
Generative robot policies include autoregressive, diffusion, and flow-based approaches. Action chunking supports temporally consistent behavior, while continuous generative models provide expressive action synthesis within broader policy systems.
- Action chunking predicts multi-step action sequences, improving temporal consistency, reducing compounding error, and stabilizing long-horizon behavior compared with single-step prediction.
- Modelability describes how difficult it is for generative models to capture the distribution of a representation.
- Diffusion and flow models increasingly serve as action experts or continuous decoding heads that translate higher-level representations into executable actions.
- Autoregressive policies factorize action-sequence distributions into conditional predictions generated one element at a time.
III. ACTION TOKENIZATION PRELIMINARIES
Action tokenization maps continuous action chunks to discrete sequences for autoregressive policies, requiring compactness, total decodability, and causal ordering. OAT combines register-based encoding, FSQ, and nested dropout so prefixes remain valid and progressively refine decoded actions.
- Action tokenization maps continuous action chunks into discrete tokens and detokenizes generated sequences back into executable continuous action chunks.
- Effective tokenizers should combine high compression, total decodability, and a meaningful left-to-right causal ordering.
- Binning: Per-dimension binning is totally decodable but produces long serialized sequences that fail compression and causal-ordering requirements.
- Frequency-domain Transform: FAST provides compact, coarse-to-fine frequency ordering, but variable-length BPE sequences can make detokenization undefined for invalid token counts.
- Quantized Latents: Existing learned tokenizers achieve compact, decodable representations but generally lack consistent token ordering for autoregressive generation.
- OAT: OAT uses transformer register tokens, FSQ, causal register attention, and nested dropout to create ordered tokens whose prefixes decode into increasingly detailed action chunks.
V. EXPERIMENTS
The experiments compare autoregressive policies using multiple action tokenizers with non-autoregressive diffusion policies, evaluating both downstream performance and tokenizer design choices.
- The evaluation compares different action tokenization schemes within autoregressive policies against diffusion-based policies.
- Controlled ablations assess the impact of key tokenizer design choices alongside downstream policy performance.
1) Policy Implementation:
Policies predict 32-action chunks from two previous observations, execute half of each chunk, and compare binning, FAST, QueST, and OAT tokenization configurations.
- Policy setup: All policies predict action chunks with horizon Ha = 32 conditioned on Ho = 2 past observations.
- Policy setup: Execution uses the first 16 actions from each predicted chunk before re-inferring.
- Tokenizers: The study compares per-dimension binning, FAST, QueST, and OAT within an autoregressive policy framework.
- Simulation setups: Simulation setups span four robotic manipulation benchmarks covering object manipulation, tool use, and multi-stage interactions.
2) Evaluation Tasks:
Evaluation covers more than 20 tasks across four simulation benchmarks and real-world tabletop manipulation, with standardized rollout protocols and success-rate reporting.
- Simulation: Simulation experiments compare OAT, Bin, FAST, QueST, and diffusion policies across 20+ tasks from LIBERO, RoboMimic, MetaWorld, and RoboCasa.
- Simulation: Each simulation task uses 5 random seeds and 50 evaluation rollouts per seed, totaling 250 rollouts per task.
- Real-world evaluation: Real-world validation uses an ARX-5 arm and webcam on Pick & Place Ball and Stack Cups, with 200 demonstrations per task.
- Results reporting: Table I reports mean success rates with standard errors and shows OAT outperforming prior tokenization schemes while performance improves monotonically with decoded-token count.
B. Simulation Benchmarking
Across four simulation benchmarks, OAT outperforms prior tokenization schemes while preserving flexible prefix-based decoding. Its ordered tokens support steadily improving performance as more tokens are generated.
- OAT consistently outperforms prior action tokenization schemes and matches or exceeds the strongest baselines across four simulation benchmarks.
- Prefix-based decoding lets OAT reconstruct action chunks from the first K tokens, a capability unavailable to existing methods.
- Bin produces excessively long token sequences and high inference latency, whereas QueST uses fixed-length sequences with lower latency but limited decoding flexibility.
- OAT4 closes much of the gap to QueST and DP, while OAT8 achieves the best performance across all benchmarks.
C. Ablation and Analysis
The analyses show that OAT's ordering, horizon choices, and codebook capacity jointly govern autoregressive learning, compression, and execution stability. Ordered tokens and suitable latent capacity improve performance while preserving controllable inference trade-offs.
- Compression and latency: OAT provides a smooth trade-off among compression rate, inference latency, and policy performance, unlike QueST's fixed decoding length.
- Action and latent horizons: The LIBERO horizon analysis reports mean success rates across action horizons Ha and token horizons Hl, with standard error over five seeds and 50 rollouts per task.
- Token space ordering: Removing nested-dropout ordering consistently degrades performance, sometimes making OAT× worse than QueST across benchmarks.
- Action and latent horizons: Under practical execution, performance degrades as action horizon Ha increases for fixed Hl, while increasing Hl mitigates the degradation.
- Real-world validation: OAT is validated on Pick & Place Ball and Stack Cups using a fixed-base robotic arm with randomly placed tabletop objects.
- Action and latent horizons: The default Ha = 32 and Hl = 8 balances long-horizon expressivity, compression, and execution stability.
- Codebook size: Moderate codebook capacity improves performance, but excessively large codebooks degrade downstream autoregressive learning.
D. Real-world Results
In real-world tabletop manipulation, OAT achieves the highest success rates across both tasks, with performance improving as more tokens are decoded. Its ordered prefixes also yield progressively smoother and more precise action execution.
- The real-world trends support transferring the benefits of ordered, prefix-decodable action tokens from simulation to robotic control.
- OAT consistently achieves the highest success rates across both real-world manipulation tasks.The results are reported as mean success rates over 20 evaluation rollouts per task.
- Performance improves monotonically as the number of decoded tokens increases, providing a direct computation–fidelity trade-off.OAT_K denotes detokenization using the first K tokens.
- Early OAT tokens capture coarse action structure, while later tokens add fine-grained corrections needed for precise manipulation.With fewer than four tokens, the robot can reach visually close configurations but fail fine-grained insertions such as placing a ball fully into a cup.
- OAT produces smoother trajectories, and smoothness improves as more tokens are decoded.
A. Implementation Details
The evaluation uses shared policy and tokenizer settings across four simulation benchmarks, alongside analyses of tokenization and decoding behavior. These details expose why fixed-topology representations matter for reliable autoregressive control.
- All policies predict 32-step action chunks from two observations and execute 16 actions before re-inference.This receding-horizon strategy balances temporal consistency and responsiveness.
- The comparison includes Bin, FAST, QueST, and OAT within a shared autoregressive policy framework, with Bin sequences scaling with action horizon and dimensionality.Bin uses 256 uniform bins per action dimension, while FAST uses a vocabulary of 1024.
- OAT uses an eight-token, four-dimensional FSQ latent with levels [8] [5], giving an implicit codebook of approximately 1000 entries.Its encoder is a two-layer transformer, and its decoder architecture and latent dimensionality match QueST for fair comparison.
- Policies are evaluated on LIBERO, RoboMimic, MetaWorld, and RoboCasa using task-specific demonstrations and action dimensions.RoboCasa uses 50 human and 150 machine-generated demonstrations per task; its evaluation has five seeds and 250 episodes per task.
- FAST can generate variable-length coefficient sequences that fail the controller’s fixed topology, making reshaping mathematically impossible when the target count is unmet.Padding or truncation is unsafe because coefficient shifts can corrupt the frequency-domain action layout; OAT’s decoder reconstructs a fixed Ha × Da action chunk.
- OAT can use a rectified-flow decoder conditioned on quantized register latents when simple reconstruction struggles with severe compression.The decoder receives partially noised actions and quantized register tokens during training.
D. Simulation Benchmarking
Simulation benchmarking reports policy success, tokenizer compression, and inference latency across tasks, using standardized evaluation protocols and mean success rates with standard errors.
- Simulation results are reported in Table VI for policy performance, tokenizer compression rate, and inference latency on one NVIDIA A100.FAST uses mean token count because its generated sequences have variable length.
- Success rates are averaged across five seeds and 50 evaluation rollouts per seed per task, with standard error reported.
- OAT_K denotes detokenization using the first K tokens, while OAT× denotes tokenizer training without nested dropout.