PaperlayerView exact revision on arXiv

Source-linked AI summary

TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models

Keisuke Shirai, Tomohiro Motoda, Hanbit Oh, Ryoichi Nakajo, Roman Mykhailyshyn, Ryo Hanai, Shotaro Miwa, Yukiyasu Domae

arXiv:2610.00899v1cs.ROcs.AIcs.LG

TL;DR

Deterministic action tokenization can force a policy to reproduce one of several valid representations of the same motion. TOAST samples alternative tokenizations during training while preserving the action, and reports consistent improvements over deterministic tokenization, especially with fewer demonstrations.

  • Problem

    Deterministic tokenization supervises only one of multiple valid token sequences for the same quantized action, a concern when costly demonstrations are limited.

    Source: §I · p. 1

  • Method

    TOAST samples alternative tokenizations of the same action sequence during training, diversifying discrete supervision without additional robot trajectories.

    Source: §I · pp. 1–2

  • Results

    TOAST consistently improves over deterministic tokenization across simulation and real-world tasks, with larger benefits when fewer demonstrations are available.

    Source: §I · p. 2, §VII · p. 8

  • Takeaways & Limitations

    The results support stochastic action tokenization as a way to extract more useful supervision from limited robot demonstrations.

    Source: §I · p. 2, §VII · p. 8

    The experiments use one Franka embodiment and do not compare other stochastic tokenization schemes, leaving broader embodiment scope and the source of the benefit open.

    Source: §VI · p. 8

Abstract

from arXiv · show

Autoregressive Vision-Language-Action models often represent continuous robot actions as discrete token sequences, enabling action prediction with standard next-token objectives. FAST has substantially improved this representation by compactly encoding action containing diverse temporal frequencies into relatively few tokens. However, while such compression reduces the number of action tokens required for autoregressive prediction, it does not necessarily improve the efficiency of policy learning from limited demonstrations. In particular, FAST typically assigns a single deterministic tokenization to each quantized action sequence, although multiple token sequences can represent and decode to the same robot motion. We investigate whether exploiting this representational redundancy can improve policy learning. In this paper, we propose TOkenization of Action sequences with STochastic sampling (TOAST), a stochastic action tokenization method that samples alternative tokenizations of the same quantized action sequence during policy training. This diversifies the discrete supervision while preserving the underlying robot action and requires no additional demonstrations. Experiments on LIBERO show that TOAST consistently improves over its deterministic counterpart, with the improvement increasing as training data decreases, achieving a 6.8 point gain in success rate when only 1/16 of training data is available. Across four real-robot manipulation tasks, TOAST further improves mean success rate by 15.8 points over the deterministic counterpart. These results demonstrate the effectiveness of stochastic action tokenization for autoregressive robot policy learning, particularly when training data are limited.

I. INTRODUCTION

TOAST addresses the ambiguity of deterministic action tokenization by sampling alternative tokenizations of the same motion during training. The paper reports consistent gains over deterministic tokenization, with larger benefits when demonstrations are limited.

I. INTRODUCTION

  • Autoregressive VLAs predict action sequences as tokens, and action tokenizers can shorten those sequences through learned vector quantization or frequency-based compression.Source: §I · p. 1
  • A quantized action sequence can have multiple tokenizations that decode to the same motion, but deterministic training supervises only one representation.The paper identifies this single-representation supervision as a potential source of over-reliance on token patterns or segmentation boundaries.Source: §I · p. 1
  • TOAST samples equivalent tokenizations during training, diversifying discrete supervision while preserving the underlying motion and requiring no additional robot trajectories.The method is instantiated with a unigram tokenizer and dimension-major flattening for frequency-based compression.Source: §I · pp. 1–2
  • Stochastic tokenization consistently improves policy performance over deterministic tokenization, with larger relative benefits when fewer demonstrations are available.The reported gains span controlled simulation and real-world manipulation tasks.Source: §I · p. 2
  • The paper frames stochastic tokenization as representation-level regularization against reliance on a single discrete representation.Source: §I · p. 2

II. RELATED WORK

Prior work develops compact action representations, while stochastic segmentation has been studied in natural-language tokenization. This paper frames its method within that context and outlines the action-tokenization pipeline.

II. RELATED WORK

  • FAST applies DCT followed by BPE for training-free action tokenization, while TOAST focuses on stochastic tokenization rather than a new compression scheme.Related methods also include B-spline control points and learned vector-quantized representations.Source: §II · p. 2
  • Stochastic subword methods sample among valid segmentations and have supported efficient learning in low-resource settings, but their value for robot policies remained unexplored.Frequency-based action tokenizers such as FAST also admit segmentation ambiguity.Source: §II · p. 2

III. PRELIMINARIES

  • Action chunks are transformed and discretized, flattened, then represented as discrete tokens; DCT coefficients can make the sequence highly compressible.Scalar binning yields N = DH tokens, whereas FAST's BPE produces variable-length sequences with N ≤ DH.Source: §III · p. 2

IV. METHOD

  • TOAST constructs a unigram model, samples alternative tokenizations that preserve the action, and uses sampled sequences for policy training.Source: §IV · p. 2, Figure 2 · p. 3

A. Constructing Action Sequence Tokenizer

TOAST builds a unigram tokenizer over quantized, dimension-major flattened action sequences, then samples among valid tokenizations of each sequence. The sampled alternatives preserve the quantized action.

A. Constructing Action Sequence Tokenizer

  • The tokenizer builds a vocabulary of individual quantized values and contiguous subsequences, then optimizes token probabilities and prunes tokens to the target vocabulary size.The unigram model assigns a sequence probability from the probabilities of its constituent tokens, enabling multiple valid tokenizations of a flattened action sequence.Source: A. Constructing Action Sequence… · p. 3

B. Stochastic Tokenization of Action Chunks

  • TOAST obtains n-best candidate tokenizations by unigram-model scores and samples one according to an approximated probability distribution.A larger α makes sampling more deterministic, while a smaller α makes it more uniform; every sampled sequence decodes to the same quantized action.Source: B. Stochastic Tokenization of Action… · p. 3

C. Policy Training with TOAST

TOAST trains the policy on sampled action-token sequences while keeping its architecture and learning objective unchanged. Its experiments evaluate LIBERO simulation and four real-robot manipulation tasks using shared policy and evaluation protocols.

C. Policy Training with TOAST

  • At each training step, TOAST samples an action-token sequence as the target; inference uses ordinary autoregressive prediction and decoding.The policy architecture and learning objective remain unchanged.Source: C. Policy Training with TOAST · p. 3

V. EXPERIMENTS

  • The experiments ask whether stochastic tokenization helps with limited data, transfers to real-world manipulation, and remains effective across tokenizer design choices.The questions are addressed through LIBERO and real-robot experiments.Source: §V · p. 3, A. Experimental Setup · p. 3

A. Experimental Setup

  • LIBERO evaluation covers four suites, while real-robot actions use 8-dimensional joint-angle and absolute-gripper representations.LIBERO actions are 7-dimensional end-effector and absolute-gripper values; the real-robot tasks use a wrist camera and two scene cameras.Source: A. Experimental Setup · pp. 3–4
  • The real-robot evaluation uses Table Bussing, Grocery Bagging, Breakfast Setup, and Drawer Stowing, covering grasping, insertion, placement, and articulated-object interaction.The tasks use a Franka Research 3 at 20 Hz; demonstrations were collected by teleoperation.Source: A. Experimental Setup · pp. 3–4
  • All policies share a PaliGemma-3B backbone, identical demonstration subsets, and matched architecture and optimization settings.Policies are fine-tuned from vision-language-pretrained weights without prior action-policy training, and evaluation reports mean success across three seeds.Source: A. Experimental Setup · pp. 4–5
  • TOAST uses DCT quantization, dimension-major flattening, and a 512-token unigram vocabulary; its deterministic counterpart holds these settings fixed and selects one tokenization.The vocabulary is built from 1M DROID action chunks; stochastic sampling uses 64 candidates, with α=0.1 on LIBERO and α=0.5 on the real robot.Source: A. Experimental Setup · pp. 4–5

B. Simulation Evaluation across Data Scales

TOAST outperforms its deterministic counterpart across LIBERO training-data scales, with larger gains as demonstrations become scarcer. It leads the evaluated methods with one quarter or less of the demonstrations.

B. Simulation Evaluation across Data Scales

  • TOAST improves over its deterministic counterpart at every data scale, with wider gains as data decreases and significant gains with half or less of the data.With one quarter or less of demonstrations, TOAST achieves the highest performance among evaluated methods.Source: Table III · p. 5, B. Simulation Evaluation across Data… · p. 5
  • For each data scale, subsets are sampled uniformly per suite with a fixed seed, and all methods use identical subsets under an unchanged evaluation protocol.The tested scales are 1/k of available LIBERO demonstrations for k ∈ {1, 2, 4, 8, 16}.Source: B. Simulation Evaluation across Data… · p. 5
  • With sufficient training data, FAST (rebuilt) and BEAST perform comparably to TOAST, but their gap grows as data decreases.FAST (rebuilt) shares TOAST’s quantization, vocabulary-construction data, and vocabulary size, suggesting low-data gains are not due only to those factors.Source: B. Simulation Evaluation across Data… · p. 5

C. Real-Robot Evaluation

In real-robot manipulation, TOAST outperforms both its deterministic counterpart and FAST+. The evaluation uses identical decoding and execution pipelines across tokenizers, and observed gains include more reliable precise interactions.

C. Real-Robot Evaluation

  • The primary comparison isolates stochastic tokenization under identical configurations; TOAST (deterministic) serves as a real-robot proxy for FAST (rebuilt).FAST+ is included as an external deterministic reference, and FAST (rebuilt) and TOAST (deterministic) differ by at most 1.6 points across data scales.Source: C. Real-Robot Evaluation · p. 5
  • 50.8% success rate: TOAST improves over TOAST (deterministic) by 15.8 points and FAST+ by 20.0 points across real-robot tasks.The comparison uses 30 rollouts per task with three random seeds.Source: Figure 5 · p. 6, C. Real-Robot Evaluation · p. 6
  • All tokenizers use the same pipeline: greedy decoding, vocabulary lookup, inverse DCT, and open-loop execution of 20-step chunks.If a sequence cannot be decoded into a complete chunk, it is discarded and inference is rerun on the next observation; this rarely occurs.Source: C. Real-Robot Evaluation · p. 6
  • Larger gains occur in all tasks except Grocery Bagging; rollout observations associate them in part with more reliable precise interactions.Deterministic baselines frequently fail on precision-dependent actions such as grasping a plate rim or reaching a drawer handle.Source: C. Real-Robot Evaluation · p. 6

D. Analysis of Tokenization Design Choices

The design analyses test sensitivity to sampling, vocabulary corpus, flattening direction, equivalent tokenizations, and computational cost. TOAST’s gains persist across these choices, while its sensitivity to tokenization is reduced and sampling overhead is negligible.

Sensitivity to the sampling hyperparameter α

  • On LIBERO with 1/8 of training demonstrations, performance is stable for α from 0.05 to 1.0 and generally declines at larger α toward deterministic tokenization.The trend is not strictly monotonic; larger α concentrates sampling toward high-probability tokenizations.Source: D. Analysis of Tokenization Design… · p. 6, Figure 6 · p. 6

Effect of the vocabulary corpus

Effect of the flattening direction

  • At 1/16, dimension-major flattening achieves 6.8 points higher absolute performance than timestep-major flattening (42.8% vs 36.0%), while sampling improves both.At full data, the two flattenings are on par (91.4 vs. 91.3); sampling gains are significant at 1/4 data and below in both directions.Source: D. Analysis of Tokenization Design… · p. 7, Figure 7 · p. 7

Sensitivity to equivalent action tokenizations

  • Stochastic training reduces the held-out loss gap between canonical and alternative tokenizations to nearly zero, unlike deterministic training.TOAST also attains lower canonical-tokenization loss than its deterministic counterpart (1.146 vs. 1.252 at 1/8), and the same qualitative behavior appears on training trajectories.Source: D. Analysis of Tokenization Design… · pp. 7–8, Figure 8 · p. 7

Computational cost

  • Training steps take 0.417 s with TOAST, 0.417 s with TOAST (deterministic), and 0.415 s with FAST+, a 0.4% increase for TOAST.The authors report that n-best search runs in linear time and sampling cost is negligible.Source: D. Analysis of Tokenization Design… · p. 8

VI. LIMITATIONS

The paper identifies limits in tokenizer applicability, embodiment coverage, stochastic-tokenization comparisons, and hyperparameter evaluation.

VI. LIMITATIONS

  • TOAST relies on multiple valid segmentations of the same quantized sequence and does not directly apply to learned vector-quantized tokenizers.Extending stochastic tokenization to learned discrete representations is left for future work.Source: §VI · p. 8
  • Experiments cover a single Franka embodiment, with end-effector actions in simulation and joint-angle actions on the real robot.The paper does not study a shared vocabulary across embodiments.Source: §VI · p. 8
  • TOAST is instantiated with a unigram model and subword regularization; BPE-dropout and other stochastic schemes were not compared.Whether the benefit comes from the unigram model specifically or stochastic segmentation generally remains open.Source: §VI · p. 8
  • Vocabulary size 512 and candidate count n remain fixed; their sensitivity is not examined.Source: §VI · p. 8

VII. CONCLUSION

The paper presents TOAST, which samples alternative tokenizations of the same quantized action sequence during training. Experiments report gains on LIBERO and real-robot tasks, with larger LIBERO gains at lower demonstration counts.

VII. CONCLUSION

  • TOAST samples multiple tokenizations of the same quantized action sequence during training, diversifying discrete supervision without additional robot demonstrations.Source: §VII · p. 8
  • On LIBERO, TOAST consistently improves over its deterministic counterpart, with larger gains when fewer demonstrations are available; real-robot manipulation tasks also show consistent improvements.Source: §VII · p. 8
  • Analyses report gains robust to flattening order, sampling parameter, and vocabulary corpus, alongside substantially reduced sensitivity to alternative tokenizations of the same action sequence.The paper concludes that stochastic action tokenization is effective for learning autoregressive robot policies from limited demonstrations.Source: §VII · p. 8

APPENDIX

The appendix reports TOAST tokenizer and policy-training settings and defines the success criteria for four real-robot tasks.

A. Hyperparameters for Constructing TOAST Tokenizer

B. Hyperparameters for Policy Training

C. Task Evaluation Rubric

  • Table Bussing succeeds when three objects reach their designated destinations within 120 seconds; Grocery Bagging requires placing three objects in the bag without toppling it.Source: C. Task Evaluation Rubric · p. 9
  • Breakfast Setup requires both objects placed within 120 seconds, while lifting the toaster with the toast triggers a safety stop and failure; Drawer Stowing requires the object inside and drawer fully closed within 120 seconds.Source: C. Task Evaluation Rubric · p. 9

D. Successful Rollout Visualization on Real-Robot Tasks

Figure 9 presents keyframes from successful TOAST-policy rollouts on the real-robot tasks, with each sequence beginning at the environment's initial state.

D. Successful Rollout Visualization on Real-Robot Tasks

Loading 2610.00899v1…