PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Haoran Chen, Jingtian Ji, Samuel Wheeler, Kaylene Caswell Stocking, Matthew Walter

arXiv:2610.09170v1cs.RO

TL;DR

Action-token policies need tokenizers that do more than reconstruct actions, because policies must predict useful tokens and unseen predictions must still decode reasonably. ProAct adds tokenizer-only training interventions for predictability and robustness; it improves success across benchmarks, VLA policies, and real-world manipulation. The findings support balancing fidelity, predictability, and robustness in action tokenization.

  • Problem

    Reconstruction objectives alone do not characterize whether downstream policies can predict induced token targets or whether unseen policy-generated sequences decode into reasonable actions.

    Source: §1 · pp. 1–2

  • Method

    ProAct combines Token Balance Regularization and Token Perturbation, training tokenizers on action data to target predictability and robustness without downstream policies or observations.

    Source: §1 · p. 2

  • Results

    ProAct improves average success rate by up to 11.3 percentage points across Robomimic, LIBERO, and RoboTwin, by 21.8 points for a LIBERO VLA policy, and by 36.7 points on real-world manipulation tasks.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Effective action tokenization must balance fidelity, predictability, and robustness to the token sequences downstream policies actually produce.

    Source: §7 · p. 9

    Experiments focus on autoregressive imitation-learning policies and a limited set of benchmarks, robot embodiments, and training seeds; extension to reinforcement learning and broader task and embodiment distributions remains to be tested.

    Source: §7 · p. 9

Abstract

from arXiv · show

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful. It is also critical that the policy is able to predict the right tokens for new observations, and that unseen policy token predictions still decode into reasonable actions. These properties are downstream of tokenizer training and are not directly incentivized by a reconstruction objective alone. In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness. ProAct is policy-agnostic and uses only action datasets for training. Across the Robomimic, LIBERO, and RoboTwin benchmarks and a diverse set of tokenizer architectures, ProAct improves rollout success by an average of 11.3 percentage points. These improvements also translate to vision-language-action policies and real-world robotic manipulation, yielding average gains of 21.8 and 36.7 percentage points, respectively. These results suggest that effective action tokenization should be designed as a policy interface that balances fidelity, predictability, and robustness, rather than as a reconstruction problem alone.

1 INTRODUCTION

Action tokenization bridges continuous robot actions and autoregressive models’ discrete-token interface, but principles for designing effective learned tokenizers remain underexplored. The paper argues that reconstruction alone does not capture token predictability or the action quality of policy-generated sequences, and introduces ProAct to address these needs.

1 INTRODUCTION

  • Action tokenization connects continuous action chunks to discrete sequences, preserving autoregressive models’ next-token interface while allowing multiple valid action modes.The introduction also notes temporal compression and language-model-style policy post-training as uses of action tokenization.Source: §1 · p. 1
  • The design gap is that reconstruction objectives do not directly characterize whether policies can predict induced token targets or whether unseen predicted sequences decode to reasonable actions.The authors describe these as distinct downstream properties of the token space.Source: §1 · pp. 1–2
  • ProAct combines Token Balance Regularization for easier target prediction with Token Perturbation for robustness to token mismatches, using action data without downstream policies or observations.The study reports improved rollout performance across nearly all evaluated tokenizer configurations.Source: §1 · p. 2
  • Across 22 learned tokenizer configurations, sufficiently low reconstruction error is necessary for successful rollouts, but further error reductions do not reliably predict better downstream performance.The controlled study examines reconstruction fidelity, nominal bitrate, chunk size, and execution horizon.Source: §1 · p. 2
  • ProAct improves average success rate by up to 11.3 percentage points across Robomimic, LIBERO, and RoboTwin, by 21.8 points for a LIBERO VLA policy, and by 36.7 points on real-world manipulation tasks.The tokenizer-only method combines Token Perturbation and Token Balance Regularization.Source: §1 · p. 2

2 RELATED WORK

Related work spans action chunking and continuous- or token-based VLA outputs, alongside discrete representation learning. The paper positions its study around how to train action tokenizers as policy prediction targets without relying on task- or observation-specific supervision.

2 RELATED WORK

  • Action chunking predicts future action sequences and executes either the full chunk or a prefix before replanning, in imitation-learning policies such as ACT and Diffusion Policy.Recent theory studies when open-loop chunk execution can mitigate compounding imitation error under stability assumptions.Source: §2 · p. 2
  • Token-based VLA policies retain the native autoregressive next-token interface, while continuous-action policies predict action chunks directly through action heads.Examples include VQ-VLA, FASTer, and G0.5 for token prediction, and π0, π0.5, and GR00T N1 for continuous prediction.Source: §2 · pp. 2–3
  • Discrete representation methods used in tokenization include VQ, RVQ, NRVQ, FSQ, and reconstruction-conditioned codebooks such as QinCo.These methods are also used in vision and audio tokenizers.Source: §2 · p. 3
  • Earlier action tokenizers include per-dimension binning, FAST frequency-domain codes, BEAST B-spline coefficients, and learned methods for skills, prefix decodability, and autoregressive VLAs.Recent methods also incorporate vocabulary structure, robot state, or multimodal features.Source: §2 · p. 3
  • Auxiliary supervision can improve source-task alignment, but ties tokenizers to task- or observation-specific signals and may require realignment under distribution changes.This work instead trains on action chunks alone and jointly analyzes tokenizer design factors and policy-generated sequences.Source: §2 · p. 3

3 PRELIMINARIES

The setup freezes a learned action tokenizer before training an autoregressive policy to predict its demonstration-derived tokens. Evaluation holds policy and rollout settings fixed across tokenizer configurations.

3 PRELIMINARIES

  • A learned tokenizer maps an H-step action chunk through an encoder, quantizer, and decoder to create discrete tokens and reconstructed actions.The study spans 22 configurations combining transformer, Conv2D, or convolution–transformer backbones with several quantizer families and codebook designs.Source: §3 · p. 3
  • With a frozen tokenizer, the policy autoregressively predicts the encoded demonstration tokens from observation O, then detokenizes its sequence into an action chunk.The policy is trained with teacher-forced negative log-likelihood.Source: §3 · p. 3
  • Rollouts execute the first Hexec actions of each H-step chunk before replanning, with policy and rollout settings fixed across tokenizer configurations.Figure 1 places tokenizer validation reconstruction error against downstream rollout success and shows FM-policy performance under increasing pose noise.Source: §3 · p. 3, Figure 1 · p. 4

4 WHAT MATTERS FOR ACTION TOKENIZATION?

The section examines how reconstruction, bitrate, chunking, and policy-generated sequences relate to downstream control. It finds that fidelity is a necessary constraint, while token predictability, mismatch severity, and temporal choices also matter.

4 WHAT MATTERS FOR ACTION TOKENIZATION?

  • The section frames reconstruction, nominal bitrate, action horizon, and policy-generated sequences as coupled factors in downstream performance.Its empirical and theoretical analysis treats these factors together rather than evaluating reconstruction alone.Source: §4 · p. 4

4.1 LOW RECONSTRUCTION ERROR IS NECESSARY BUT NOT SUFFICIENT

  • High action MAE consistently accompanies poor policy performance, but among sufficiently accurate tokenizers, lower reconstruction error does not reliably indicate better rollout success.The relationship is examined on three Robomimic tasks; FM-policy noise experiments indicate that action-error tolerance differs by task.Source: §4.1 · p. 4

4.2 NOMINAL BITRATE AND ACTION CHUNKING

  • Moderate chunk and execution horizons provide the best rollout success; large chunks can raise reconstruction error above the approximately 0.04 level needed on Robomimic.Increasing bitrate may recover fidelity but does not make longer chunks easier to learn or better for control; the same execution-horizon trend appears for continuous FM policies.Source: §4.2 · p. 5

4.3 BEYOND EMPIRICAL TRAINING SUPPORT: A ONE-STEP ERROR ANALYSIS

  • Rollouts often generate token prefixes absent from encoded demonstrations, but leaving that empirical support does not necessarily mean control failure because unseen sequences may decode to valid actions.Figure 2 reports unobserved-prefix rate and reconstruction error versus chunk size, and success across horizon pairs on Robomimic.Source: Figure 2 · p. 4, §4.3 · p. 5
  • One-step errors are matches, benign mismatches, or uncontrolled mismatches; teacher forcing encourages exact matches but does not bound the action-space severity of mismatches.For matches, error reduces to reconstruction error; benign mismatches also depend on action similarity, while uncontrolled mismatches can be arbitrarily large relative to the reconstruction objective.Source: §4.3 · p. 5
  • Policy training mainly governs mismatch probability, whereas the tokenizer shapes both mismatch probability and the severity of decoded mismatches.This decomposition distinguishes how often mismatches occur from how harmful they are after decoding.Source: §4.3 · p. 5

5 LEARNING ROBUST AND PREDICTABLE ACTION TOKENIZERS

ProAct augments tokenizer training with Token Balance Regularization for target predictability and Token Perturbation for mismatch robustness, without access to the downstream policy or observations. Its design is motivated by the limits of reconstruction alone and by how token targets and mismatches affect policy learning.

5 LEARNING ROBUST AND PREDICTABLE ACTION TOKENIZERS

  • ProAct combines Token Balance Regularization for target predictability with Token Perturbation for mismatch robustness, using action data without downstream policies or observations.The paper reports that these tokenizer-only interventions consistently improve downstream performance.Source: §5 · p. 5

5.1 TOKEN BALANCE REGULARIZATION

  • Token Balance Regularization addresses ambiguity in token targets: the policy loss is lower-bounded by H(C | O), an irreducible tokenizer-dependent component.The bound applies even when an unrestricted policy matches the conditional distribution of token targets.Source: §5.1 · p. 6
  • For a linear encoder, covariance-aware preconditioning can enlarge token margins, reduce irreducible policy loss, and make correct token prediction easier.TBR promotes balanced feature learning using covariance-aware preconditioning for optimization updates, without changing the training objective.Source: §5.1 · p. 6

5.2 TOKEN PERTURBATION

  • Token Perturbation trains reconstruction from perturbed encoded token sequences, encouraging robustness to token mismatches.A straight-through estimator uses perturbed token embeddings in the forward pass while gradients flow through the original encoder latent.Source: §5.2 · p. 6

6 EXPERIMENTS

Across three manipulation benchmarks, ProAct improves rollout success across diverse tokenizer configurations, and the gains also appear with a pretrained VLA and on real-robot tasks. Its interventions improve token predictability and reduce the action-space severity of token mismatches, with a reconstruction trade-off for TBR.

6.1 EXPERIMENTAL SETTING

  • The study compares 22 tokenizer configurations with a fixed downstream autoregressive policy across Robomimic, LIBERO, and RoboTwin, covering single-arm and bimanual tasks.Reconstruction-only tokenizers serve as baselines; ProAct adds Token Balance Regularization and Token Perturbation.Source: §6.1 · pp. 6–7

6.2 SIMULATION RESULTS

  • ProAct raises average success by 10.4 points at Hexec = 4 and 11.3 points at Hexec = 8 across Robomimic, LIBERO, and RoboTwin.Gains are especially large for level-specific configurations (+12.3/+13.1 points) and QinCo variants (+20.2/+22.6 points).Source: Table 1 · p. 7, §6.2 · p. 7
  • On LIBERO, ProAct improves success in 7 of 8 tokenizer pairs, raising the average from 26.2% to 48.1%.The evaluation fine-tunes SmolVLM2-256M with an autoregressive action expert.Source: Table 2 · p. 7, §6.2 · p. 8

6.3 ABLATIONS AND ERROR ANALYSES

  • On Robomimic, Token Balance Regularization and Token Perturbation each independently improve rollout success, and using both yields the best performance.The ablation reports success rates at Hexec = 4 / 8, averaged by task and codebook sharing.Source: Table 3 · p. 7, §6.3 · p. 8
  • Token Balance Regularization lowers mean normalized NLL by 12.4% on Can, 10.0% on Lift, and 11.3% on Square, while increasing validation MAE by 7.2% on average.It also reduces within-tokenizer NLL variation across positions by 18.9–21.4%, with the strongest reduction at later autoregressive positions.Source: §6.3 · p. 8, Figure 3 · p. 8
  • As the number of sampled policy sequences increases, Token Perturbation makes the paired change in Max-M mismatch error increasingly negative despite a small increase in reconstruction error.This indicates reduced largest decoded errors under broader policy sampling.Source: §6.3 · pp. 8–9, Figure 4 · p. 9

6.4 REAL-ROBOT EXPERIMENTS

  • On two real-world UR5 tasks, ProAct improves success for every evaluated tokenizer configuration, with especially pronounced gains for level-specific variants on Cube Stacking.The tasks are Cube Stacking and Cup Hanging; the latter also shows substantial improvements across tokenizers.Source: Table 4 · p. 9, §6.4 · p. 9

7 CONCLUSIONS

The conclusion argues that effective action tokenization must balance reconstruction fidelity, predictability, and robustness, and reports ProAct improvements across simulation benchmarks and real-robot manipulation. The paper also limits its claims to the evaluated settings and describes its use of AI tools and reproducibility materials.

7 CONCLUSIONS

  • Figure 5 depicts the two real-robot manipulation tasks used in the study.Source: Figure 5 · p. 9
  • ProAct combines Token Balance Regularization and Token Perturbation to target token predictability and mismatch robustness alongside reconstruction.The conclusion reports improved downstream performance across Robomimic, LIBERO, RoboTwin, and real-robot manipulation.Source: §7 · p. 9
  • The experiments cover autoregressive imitation-learning policies and a limited set of benchmarks, embodiments, and training seeds.The authors identify reinforcement learning and broader task and embodiment distributions as next tests.Source: §7 · p. 9

AI USE STATEMENT

  • The authors report using AI tools for drafting, literature identification, implementation, analysis, and mathematical validation, but not for synthetic-data generation.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper documents tokenizer and rollout protocols, training settings, implementation details, simulation results, ablations, and the real-robot evaluation protocol in its appendices.Source: REPRODUCIBILITY STATEMENT · p. 10

A ANALYTICAL TOKENIZATION SCHEMES

The section motivates focusing the controlled study on learned tokenizers by showing that a short prefix of an analytical Bin code retains substantial success. It then describes the tested tokenizer backbones, quantizer families, and parameter-sharing variants.

A ANALYTICAL TOKENIZATION SCHEMES

  • Bin independently quantizes each scalar in an H = 8, seven-dimensional action chunk into 256 bins, producing 56 tokens; learned tokenizers represent the varying action structure with shorter codes.The authors therefore focus their controlled study on learned tokenizers.Source: A ANALYTICAL TOKENIZATION SCHEMES · p. 14
  • Predicting only eight tokens, Prefix-Bin retains 51.0% success at Hexec = 4, about three quarters of Bin’s, with a prediction horizon seven times shorter.At Hexec = 8, success is 28.1%; only 10 of 116,977 predicted prefixes appear in the training table at rollout.Source: A ANALYTICAL TOKENIZATION SCHEMES · p. 14, Table 6 · p. 14

B.1.1 TOKENIZER BACKBONES

  • The study uses Transformer, Conv2D, and Convolution–Transformer backbones, which differ in how they compress temporal and action-coordinate structure.Transformer supports multiple register latents and all five quantizer families; Conv2D and the hybrid emit one latent vector and pair with residual quantizers.Source: §B.1.1 · p. 15

B.1.2 QUANTIZER FAMILIES

  • The five quantizer families map encoder representations to discrete indices and quantized embeddings, with VQ and FSQ operating in parallel and RVQ, NRVQ, and QinCo quantizing sequentially.RVQ quantizes successive residuals, NRVQ uses normalized directions and learned scales, and QinCo conditions later codebooks on the accumulated representation.Source: §B.1.2 · pp. 15–16

B.1.3 SHARED AND LEVEL-SPECIFIC QUANTIZATION

  • Shared (S) and level-specific (L) variants reuse or independently parameterize the family-specific quantizer block across layers while retaining the same nominal bitrate.The suffixes identify parameter sharing, not a change in the number of token indices.Source: §B.1.3 · p. 17

B.2 ROLLOUT EVALUATION PROTOCOL

The section specifies rollout protocols and training settings for the simulation benchmarks and a LIBERO vision-language-action evaluation. Its additional analysis examines reconstruction-error relationships across a broad single-task tokenizer sweep.

B.2 ROLLOUT EVALUATION PROTOCOL

  • Simulation success is reported per execution horizon, with Robomimic evaluated over 300 episodes per task, LIBERO over 500 per suite, and RoboTwin over 150 per task.Benchmark-level cells macro-average task or suite success rates; the LIBERO benchmark equally averages its four suites.Source: §B.2 · p. 18

B.3 TRAINING HYPERPARAMETERS AND RANDOMNESS

  • Each simulation tokenizer configuration is trained once, so the sweep compares designs but does not estimate within-design tokenizer-training variance.Source: §B.3 · p. 19

TRAINING BUDGET

OBJECTIVE AND SELECTION

  • Across-training-seed rollout-success standard deviations were 4.3 percentage points for FSQ and 5.8 points for NRVQ on Square, while variation was described as modest on Can and Lift.The study uses this variability to contextualize individual pairwise differences, while relying on recurring trends across the sweep for aggregate conclusions.Source: OBJECTIVE AND SELECTION · p. 20

INPUTS AND PREPROCESSING

OPTIMIZATION

  • VLA optimization runs for 50,000 steps with AdamW, a warmup-and-cosine learning-rate schedule, gradient clipping, and EMA decay 0.999.Source: OPTIMIZATION · p. 21

EVALUATION

  • Sixteen LIBERO VLA policies share the same architecture, training recipe, and evaluation protocol, differing only in tokenizer and token targets; they are evaluated at Hexec = 8.The reported results use the final 50,000-step checkpoint because compute constraints prevented training to convergence.Source: EVALUATION · pp. 21–22

C ADDITIONAL ACTION-TOKENIZATION ANALYSIS

  • The additional analysis studies reconstruction error, bitrate, chunk size, training-support membership, and one-step error decomposition; the single-task correlation sweep retains 849 valid runs across Can, Lift, and Square.Within the RVQ, NRVQ, and QinCo subset, validation reconstruction-error correlations with rollout success are ρ = 0.45, 0.35, and 0.50 for Can, Lift, and Square.Source: C ADDITIONAL ACTION-TOKENIZATION… · p. 22, §C.1.1 · p. 22

C.1.2 CALIBRATING ACTION-ERROR TOLERANCE

The diagnostics calibrate how raw-action perturbations affect control, then separate tokenizer reconstruction distortion from autoregressive token-prediction difficulty. They find task-dependent error tolerance and show that reconstruction distortion alone does not explain the spread in policy success.

C.1.2 CALIBRATING ACTION-ERROR TOLERANCE

  • At ϵ = 0.30, Robomimic success falls to 17.3% on Can, 64.3% on Lift, and 2.7% on Square, while success remains near clean anchors through ϵ = 0.07.Across LIBERO and RoboTwin, degradation onset and rate also differ by suite or task; Figure 7 shows the joint-noise sweeps.Source: §C.1.2 · p. 24, Figure 7 · p. 24
  • At comparable noise amplitudes, rotation noise degrades Robomimic success more sharply than translation noise, though the response remains task dependent.Figure 8 varies translation and rotation noise independently, leaving the gripper unchanged.Source: §C.1.2 · p. 25, Figure 8 · p. 25

C.1.3 ISOLATING TOKENIZER DISTORTION

  • With token prediction absent, reconstruction error is strongly negatively associated with success on Can and Square, but less strongly on Lift, where success stays near ceiling across much of the error range.Figure 9 reports 273 tokenized conditions and isolates tokenizer-induced distortion.Source: §C.1.3 · p. 25, Figure 9 · p. 25

C.1.4 SEPARATING DISTORTION FROM TOKEN PREDICTION

  • Removing token prediction shifts nearly every tokenizer above the equal-success line, while tokenized-FM success stays in a comparatively narrow high band and tokenizer–AR success varies widely.The paired Figure 10 evaluations therefore indicate that reconstruction distortion alone does not account for the spread in tokenizer–AR performance.Source: §C.1.4 · pp. 25–26, Figure 10 · p. 26
  • The diagnostics support reconstruction as a feasibility condition: large distortion impairs control, but further error reductions in a sufficiently low regime do not reliably rank downstream performance.Source: §C.1.4 · p. 26

C.2 NOMINAL BITRATE ANALYSIS

Across these analyses, bitrate-driven reconstruction gains do not ensure better control, execution horizons have intermediate optima, and training-support constraints often worsen policy outcomes.

C.2 NOMINAL BITRATE ANALYSIS

  • Higher bitrate generally lowers validation reconstruction MSE, but rollout success is non-monotonic and depends on tokenizer design.Bitrate is a conditional capacity requirement: it must be sufficient for the chosen chunk size, but need not be maximized.Source: Figure 11 · p. 27, §C.2 · p. 27

C.3.1 CHUNK SIZE WITH FLOW-MATCHING POLICIES

  • Across Robomimic, LIBERO, and RoboTwin, flow-matching policies favor intermediate execution horizons, though each benchmark samples a different part of the rise-and-fall pattern.Robomimic shows both the rise and fall, LIBERO mainly the fall, and RoboTwin mainly the rise.Source: §C.3.1 · p. 28, Figure 12 · p. 28
  • Robomimic and LIBERO favor moderate execution, while RoboTwin benefits strongly from longer execution; Grab Roller success rises from 64% to 86% as Hexec increases from 2 to 4.The LIBERO short-horizon end is less completely evaluated, so its results do not establish a replanning penalty.Source: Figure 12 · p. 28, §C.3.1 · p. 29

C.3.2 CHUNK SIZE WITH TOKENIZERS AND TOKENIZER–AR POLICIES

  • Mean validation reconstruction error rises from 0.0168 at H = 4 to 0.0437 at H = 32, the first tested horizon above the approximately 0.04 Robomimic tolerance.H = 8 and H = 16 remain within that feasible regime.Source: Figure 13 · p. 30, §C.3.2 · p. 31
  • At H = 32, mean success falls from 63.3% to 50.9% at Hexec = 4 and from 61.2% to 49.2% at Hexec = 8, coinciding with reconstruction leaving the low-error regime.Performance is relatively stable from H = 8 to H = 16.Source: §C.3.2 · p. 31, Figure 14 · p. 31

C.4 TRAINING-SUPPORT MEMBERSHIP DIAGNOSTICS

  • The offline diagnostics compare greedy decoding with decoding constrained to the empirical support of encoded training chunks, isolating the effect of support membership.The cohort comprises 66 tokenizer–task conditions at H = Hexec = 8.Source: §C.4 · pp. 31–32

C.4.1 OFFLINE ANALYSIS

  • Excluding two degenerate tokenizers, exact target-sequence matches were at most 3.8%; support-constrained decoding raised MAE in 64 of 66 conditions.The mean constrained-minus-unconstrained MAE was 0.0302 on Can, 0.0227 on Lift, and 0.0335 on Square.Source: §C.4.1 · p. 32

C.4.2 ONLINE ANALYSIS

During rollout, constraining greedy decoding to the empirical training support usually reduced success; novel prefixes occurred across quantizer families.

C.4.2 ONLINE ANALYSIS

  • Support-constrained decoding reduced rollout success in 53 of 66 tokenizer–task conditions, with mean changes of −19.7, −14.7, and −17.6 percentage points on Can, Lift, and Square.It improved success in 11 conditions and tied in 2; the ties were degenerate conditions with zero success under both decoding rules.Source: §C.4.2 · p. 32, Figure 16 · p. 33
  • Rollout prefixes departed from the empirical training support across quantizer families, and the combined diagnostics show that support membership is neither necessary nor sufficient for a good prediction.Rollout diagnostics use greedy policies at the checkpoint with the best teacher-forced validation accuracy and Hexec = 8.Source: §C.4.2 · p. 32, Figure 16 · p. 33

C.5 ONE-STEP ERROR DECOMPOSITION

This section decomposes one-step action-chunk error into matched and mismatched predictions and examines how token predictability, conditional entropy, and token geometry shape mismatch error beyond what reconstruction and teacher-forced losses capture. It uses a linear model to motivate Token Balance Regularization, which applies activation preconditioning during tokenizer training to reduce noise-subspace sensitivity while preserving token assignments and leaving the model architecture unchanged.

C.5 ONE-STEP ERROR DECOMPOSITION

  • Matched predictions contribute reconstruction error, while mismatches contribute their probability multiplied by their conditional decoded error.The matched contribution is bounded by the average reconstruction loss; teacher forcing controls mismatch frequency, but not what mismatched sequences decode to. The analysis concerns one-step error under the offline expert data distribution.Source: §C.5 · pp. 34, 36
  • Teacher-forced likelihood bounds deployed match probability under sampling, mode-seeking, and truncated sampling, with a sharper sampling bound of 1 − e^-LAR.The bounds vary by decoding rule; for sampling, 1 − p̄ ≤ 1 − e^-LAR.Source: §C.5 · pp. 34–35
  • In-support mismatches can be bounded using a preimage action, but the bound requires worst-case reconstruction and action-range information beyond average reconstruction loss.A preimage-based bound depends on how close the corresponding action is to the target; a diameter bound may be much looser, and a small average reconstruction loss need not imply small worst-case error.Source: §C.5 · pp. 35–36
  • Off-support decoded errors have no finite upper bound from reconstruction and teacher-forced losses alone without an additional decoder range or regularity constraint.Finite-logit softmax sampling assigns positive probability to off-support sequences when the empirical encoded support is a proper subset of token space; mode-seeking or truncation may avoid them but do not supervise their decodings.Source: §C.5 · p. 36
  • A benign in-support mismatch has a nearby reconstruction target within the local tolerance region, but this is only a sufficient condition for compatibility, not an exhaustive account of acceptable predictions.Sequences encountered during tokenizer training may lie outside the final empirical support and decode acceptably in practice, but remain in the off-support contribution without a general error guarantee.Source: §C.5 · pp. 36–37

D.1.1 IRREDUCIBLE POLICY LOSS AND TOKEN GEOMETRY

  • Conditional token entropy is irreducible policy log-loss for fixed token targets, and margins bound it by limiting how often actions receive labels different from their center’s token.Entropy separates uncertainty about whether a demonstration keeps the center token from uncertainty about which alternative token it receives; the margin-based upper bound decreases as margins grow.Source: §D.1.1 · pp. 38–39

D.1.2 A LINEAR MODEL MOTIVATING BALANCED FEATURE LEARNING

  • Under a Gaussian-noise conditional-center model and a binary linear tokenizer, removing encoder sensitivity orthogonal to the center subspace increases margins and reduces conditional token entropy without changing center labels.The noise-subspace component changes sensitivity while the boundary’s cut through the center subspace—and thus the center-action token assignments—remains fixed.Source: §D.1.2 · p. 40
  • The reconstruction-guided linear-model analysis separates target-code choice from encoder fitting and states that its quadratic objective is exact for the regression problem.This setup examines whether reconstruction training penalizes the noise-subspace sensitivity that reduces token margins.Source: §D.1.2 · p. 41
  • For fixed reconstruction points in the signal subspace, the optimal assignment depends only on the signal component, and code-regression loss decomposes into optimum loss plus squared encoder deviation weighted by input covariance.The regression optimum lies in the signal subspace; the noise subspace has input covariance eigenvalue σ².Source: §D.1.2 · p. 41
  • With the stated boundary-density condition, fixed noise-subspace weight adds excess code-regression loss of order σ² but excess conditional entropy of order σ, allowing similar fitting losses with different token predictability.The comparison holds the signal-subspace component and threshold fixed as σ → 0.Source: §D.1.2 · pp. 42–43
  • For finite codebooks, projecting classifier weights onto the signal subspace cannot decrease margins; with multimodal actions, modewise bounds do not ensure lower total entropy because tokenizer changes can alter I(C; M | O).The modewise bounds control within-mode uncertainty and provide an upper bound on total conditional entropy.Source: §D.1.2 · pp. 44–45
  • In the controlled linear model, activation preconditioning removes noise-subspace sensitivity faster while preserving the clean-subspace partition, increasing margins and strictly reducing conditional token entropy and Bayes binary token-classification error.The comparison assumes matched fastest-direction speed and the stated strictness condition for the entropy ordering.Source: §D.1.2 · pp. 43–45

D.2 TOKEN BALANCE REGULARIZATION: IMPLEMENTATION

  • The implementation applies activation preconditioning only during tokenizer optimization, leaving architecture and parameter count unchanged; it covers eligible linear, attention, and convolutional layers using statistics re-estimated each step.Codebook embeddings, biases, and normalization parameters remain under AdamW alone; convolutional and wide layers use a rank-one-plus-diagonal surrogate.Source: §D.2 · pp. 45–46

D.3 TOKEN PERTURBATION: IMPLEMENTATION AND ALTERNATIVE SAMPLING STUDIES

Token Perturbation perturbs token encodings during reconstruction, while alternative sampling and selection rules yield mixed results. The accompanying E.1 material organizes comparisons around reconstruction error and task- or suite-level outcomes.

D.3 TOKEN PERTURBATION: IMPLEMENTATION AND ALTERNATIVE SAMPLING STUDIES

  • Token Perturbation perturbs clean token encodings and reconstructs the original actions; native-metric sampling replaces masked positions with nearby latent-space candidates.Perturbation probability and neighborhood size are annealed during training.Source: §D.3 · p. 46
  • MaxUp remains above Reconstruction-only on average, yet is 1.37–3.90 points below matched single-draw Token Perturbation.Spectral-metric perturbation performs inconsistently relative to Token Perturbation.Source: §D.3 · p. 47
  • MaxUp selects the highest-reconstruction-error candidate among eight, but these candidates are not selected using downstream-policy errors.The proposed explanations for its trade-offs remain hypotheses, not established family-level effects.Source: §D.3 · p. 47

E.1 PRIMARY COMPARISON: TASK-LEVEL RESULTS AND ROBUSTNESS CHECKS

  • The primary-comparison expansion includes task-level and suite-level outcomes alongside reconstruction-error results.The appendix also checks last-checkpoint and single-task comparisons on Robomimic.Source: §E.1 · p. 48

E.1.1 VALIDATION RECONSTRUCTION ERROR

  • The primary-comparison expansion reports validation reconstruction MAE in raw action units, separately by benchmark and independently of policy execution horizon.The table compares Reconstruction-only and ProAct for each benchmark and tokenizer pair.Source: §E.1.1 · p. 48, Table 16 · p. 48

E.1.2 TASK- AND SUITE-LEVEL RESULTS

  • Task- and suite-level tables compare Reconstruction-only with ProAct at both execution horizons across Robomimic, LIBERO, and RoboTwin.They report task or suite success with the stated episode-based confidence intervals or evaluation counts.Source: §E.1.2 · pp. 49–51, Table 17 · p. 49, Table 18 · p. 50

E.1.3 ROBUSTNESS TO CHECKPOINT AND DATA SETTING

Across the supplied analyses, ProAct improves real-robot success and policy-prediction metrics, while decoded mismatch-error reductions vary by tokenizer family. Robustness checks also report gains under last-checkpoint and single-task settings.

E.1.3 ROBUSTNESS TO CHECKPOINT AND DATA SETTING

  • Last-checkpoint success rose from 58.5% to 73.9% at Hexec = 4 and from 55.2% to 71.3% at Hexec = 8 with ProAct.For single-task policies, success rose from 68.2% to 78.0% at Hexec = 4 and from 63.8% to 76.7% at Hexec = 8.Source: §E.1.3 · pp. 51, 53, Table 20 · p. 52, Table 21 · p. 53

E.2 DETAILED ABLATION RESULTS

  • The detailed ablation tables report overall and family-level comparisons plus Can, Lift, and Square results for four training conditions; CT-NRVQ-S/L are excluded from the aggregate after collapsing under Reconstruction-only.The task-level tables cover all 22 configurations.Source: §E.2 · p. 54, Table 22 · p. 54, Table 23 · p. 54

E.3 DETAILED POLICY-PREDICTION ANALYSIS

  • Relative to w/o TBR, ProAct lowers mean validation loss by 0.655 and raises mean token accuracy by 1.88 percentage points; both improve for 14 of 22 tokenizers.Family-mean loss decreases for VQ, NRVQ, FSQ, and QinCo, while RVQ increases slightly.Source: Figure 19 · p. 57, §E.3 · p. 57, Table 26 · p. 57
  • Normalized per-position NLL profiles compare w/o TBR and ProAct by tokenizer and task; they measure induced prediction difficulty because treatments change encoded targets.The compared losses are not against identical labels.Source: §E.3 · pp. 57–58, Figure 20 · p. 58

E.4 DETAILED DECODED-ERROR ANALYSIS

  • Mean Greedy and Max-2048 decoded-error changes were −0.0046 and −0.0156 for R.O. → w/o TBR, and −0.0086 and −0.0313 for w/o TP → ProAct.Improvements occurred in 53/60 and 48/60 conditions for the first comparison, and 62/66 and 61/66 for the second; family effects varied.Source: Figure 21 · p. 59, §E.4 · p. 59, Figure 22 · p. 60

E.5 DETAILED REAL-ROBOT RESULTS

  • The real-robot study used one tokenizer and one policy across Cube Stacking and Cup Hanging, trained on 780 demonstrations.The shared dataset comprised 301 Cube Stacking and 479 Cup Hanging demonstrations.Source: §E.5 · p. 62
  • Real-robot successes rose from 56/300 (18.7%) with Reconstruction-only to 166/300 (55.3%) with ProAct across Cube Stacking and Cup Hanging.Each condition used the final policy checkpoint and 30 trials per task; initial object poses were randomized.Source: §E.5 · p. 62
Loading 2610.09170v1…