Whole-Body Humanoid Control via Stochastic Motion Tokenization

I. INTRODUCTION

• Whole-body humanoid motion admits many plausible continuations, while fixed gait generators and single-clip tracking reproduce only one trajectory or style well.Reward specification for RL and action-label collection for maximum-likelihood methods are also challenging, and maximum likelihood can average modes into muddied behaviors.Source: §I · p. 1
• DAPS is adopted as a stochastic discrete autoencoder whose autoregressively factorized categorical encoder has controlled entropy, and its motion reconstruction is consistently better at matched bottleneck capacity.The paper contrasts this design with FSQ and VQ-VAE, described as deterministic or limited to simplified mean-field distributions.Source: §I · p. 1
• The framework combines a stochastic discrete motion tokenizer, context-conditioned autoregressive TrajectoryGPT, and a low-level tracking policy that receives future latent chunks, including wildcards for transitions.The tracking policy acts as a decoder for physical motion, with domain randomization used for sim-to-real transfer.Source: §I · p. 2
• The paper reports a matched comparison of tokenizers on two datasets and deployment of simulation-trained tracking policies on a physical Unitree G1.Its stated contributions also include context-conditioned motion generation from scratch.Source: §I · p. 2

a) Discrete tokenizers

• Discrete motion representations include spline- and frequency-based targets, VQ-VAE and FSQ latents, and DAPS, whose stochastic categorical latents have controllable entropy.The paper notes that using discrete latents for motion synthesis by reusing a VAE decoder is relatively nascent in robotics.Source: §II · p. 2

b) Physics-based motion imitation

• Physics-based imitation methods use discriminator-derived rewards or tracking rewards, while related real-hardware systems track observed or teleoperated motion.The paper distinguishes its approach by generating an indefinite motion sequence with an autoregressive model.Source: §II · p. 2

c) Conditioned motion generation

• Conditioned motion generation includes diffusion and autoregressive methods, while improving motion quality under task information such as goals or language remains an ongoing problem.The cited examples span kinematics-only generation and generator–tracker systems, including interactive language-conditioned motion on humanoid hardware.Source: §II · p. 2

III. METHOD

• The three-stage framework trains a tokenizer, an autoregressive motion generator, and a low-level tracking policy using LAFAN1 and BONES-SEED motions retargeted to the 29-DoF Unitree G1.Motions are resampled to 30 Hz and divided into 24-frame, 0.8-second windows; robot state q is a 36-dimensional configuration.Source: §III · p. 3

A. Stage 1 — Motion Tokenization (DAPS)

• The tokenizer maps each window’s 162-dimensional joint-and-body-site feature vector to 16 tokens from a vocabulary of 1024, for a 160-bit representation.Temporal-transformer recognition and generative models jointly maximize a β-weighted ELBO under a uniform prior.Source: A. Stage 1 — Motion Tokenization (DAPS) · p. 3
• The decoder predicts joint configurations for all 24 frames, then applies differentiable forward kinematics to derive relative body-site positions and keep both feature components consistent.This design constrains the decoder output to the robot’s kinematic manifold.Source: A. Stage 1 — Motion Tokenization (DAPS) · p. 3

1) Encoder update as KL-regularized policy search

• The encoder update uses a REPS-based trust-region policy-search step, with reconstruction log-likelihood as per-sample return and a Monte Carlo baseline for the advantage.The passage introduces a Lagrangian solution for the update.Source: 1) Encoder update as KL-regularized… · p. 3
• The dual variable η is adapted online so self-normalized importance weights maintain a target effective sample size, serving as a distribution-space step-size controller.The weights are proportional to the ratio of the target distribution q* to the current encoder qθ.Source: 1) Encoder update as KL-regularized… · p. 3
• The new encoder is the weighted maximum-likelihood projection of the target distribution, without gradients through autoregressive sampling or a straight-through estimator; the decoder is updated in parallel.The decoder uses maximum-likelihood estimation on samples decoded from the current parametric encoder.Source: 1) Encoder update as KL-regularized… · p. 3

B. Stage 2 — Autoregressive TrajectoryGPT

• TrajectoryGPT predicts the next token chunk from a behavior label and the previous eight latent chunks.The context pairs the behavior label with latent history.Source: B. Stage 2 — Autoregressive… · p. 3
• Its objective fits the frozen Stage-1 posterior, including its entropy, rather than only predicting the posterior’s most likely tokens.The cross-entropy is computed under the encoder distribution and includes an entropy term constant in the generator parameters.Source: B. Stage 2 — Autoregressive… · p. 3
• Training sometimes replaces latent-history entries with a start token, allowing inference to begin without a motion prefix and continue in open-ended rollouts.A temperature on the logits controls tokenization determinism; low temperatures work well for capturing motions in the original dataset.Source: B. Stage 2 — Autoregressive… · p. 3

C. Stage 3 — RL Tracking

• The policy’s reward compares simulated state with the reference across joint, body-site, and planar-root-velocity feature groups.The comparison includes joint positions and velocities, root-relative site positions, orientations and velocities, and planar root velocity.Source: C. Stage 3 — RL Tracking · p. 3
• A rolling buffer gives the policy three latent chunks—2.4 s of intended motion in 480 bits—covering the current 0.8 s and the next 1.6 s.The policy also observes proprioception and a phase variable; it has no dedicated planar root-position term, so locomotion is propelled by physics rather than pinned to a global path.Source: C. Stage 3 — RL Tracking · p. 3
• During Stage 3 training, the rendered silver robot tracks pink generated references, and the renderings show true simulated robot states and kinematics.Training uses domain randomization for actuator gains, link masses, friction, and pushes, plus joint-acceleration and torque penalties to help reduce jitter and soften the response for sim-to-real transfer.Source: Figure 2 · p. 4, C. Stage 3 — RL Tracking · p. 4

IV. EXPERIMENTS

• The experiments cover BONES-SEED and LAFAN1 across multiple seeds, assessing tokenizers, conditional generation, and latent-code tracking in simulation and on hardware.The paper organizes these evaluations as tokenizer comparison, conditional generation, and low-level policy evaluation.Source: §IV · p. 4

A. Stage 1: Tokenizer Baseline at Matched Bits

• Token diversity rose from 0% changed positions at T=0 to 81.0% at T=1.25, while normalized reconstruction RMSE rose from 0.2476 to 0.2512.At T=1, 52.4% of token positions changed and normalized reconstruction RMSE was 0.2488; the reported relative RMSE increases were approximately 0.5% at T=1 and 1.4% at T=1.25.Source: A. Stage 1: Tokenizer Baseline at… · p. 4

B. Stage 2: Conditional Generation

• TrajectoryGPT autoregressively predicts DAPS chunks from behavior context and latent history, decoding each predicted chunk into 0.8 s of kinematics for tracking.The Stage-2 cross-entropy objective is not treated alone as a diagnostic of generated-motion quality.Source: B. Stage 2: Conditional Generation · p. 4
• ≈0.20 overall standardized RMSE summarizes DAPS reconstruction across representative whole-body channels over a 3.2 s G1 validation window.The channels include root state, joint angles, and relative body-site positions and rotations.Source: Figure 3 · p. 5

A. Stage 1: Tokenizer Baseline at Matched Bits

• 7.1 ± 0.3 versus 9.0±0.4 for FSQ and 11.5±1.7 for VQ-VAE validation joint position MSE on BONES-SEED; DAPS also had the lowest MSE on LAFAN1.The comparison uses matched tokenizer architectures and bottlenecks across five seeds; DAPS also attained the highest validation conditional log-likelihood on both datasets.Source: A. Stage 1: Tokenizer Baseline at… · p. 4, Figure 4 · p. 5

C. Stage 3: Tracking and Transfer

• Three latent-conditioned PPO policies test motion banks containing 25 generated motions, three base motions with nine transition pairings, or five motions.The Stage-3 subset comes from the LAFAN1-trained TrajectoryGPT model.Source: C. Stage 3: Tracking and Transfer · p. 5
• The five-motion policy completed generated dance, fight, and walk motions of 16.8 s each on the G1, and the stitching policy completed a 35.2 s fight-to-dance sequence.The stitched sequence includes a 1.6 s wildcard interval with no prescribed kinematic reference.Source: C. Stage 3: Tracking and Transfer · p. 5
• Fig. 5 plots token differences from deterministic encoding on the left and mean normalized reconstruction RMSE on the right.The temperature sweep shows increasing token diversity alongside a modest reconstruction penalty.Source: Figure 5 · p. 5

D. Implementation

• The stages train sequentially and freeze before the next begins; Stage 3 simulates 2048 MuJoCo environments on one GPU, with reference banks generated offline.Each stage trains on a single GPU with 24–80 GB of VRAM; TrajectoryGPT latency is excluded from tracking measurements, and deployment uses precomputed latent trajectories.Source: D. Implementation · p. 5

E. Hyperparameters

• Table I presents pipeline hyperparameters for both datasets, including a transferred DAPS ESS target, dataset-specific β annealing, and per-policy desired KL.For hardware transfer, the action-rate penalty is tuned to reduce jitter without excessively damping tracked motions.Source: E. Hyperparameters · p. 5, Table I · p. 6
• Fig. 6 tracks joint-position and joint-velocity tracking rewards, average episode length, and action-rate penalty over 2B environment steps.The action-rate penalty is part of the total environment reward.Source: Figure 6 · p. 6

• The shared discrete representation directly connects TrajectoryGPT to tracking, but DAPS can assign multiple similarly accurate token sequences to one motion.Changing T lets TrajectoryGPT train on either the most likely window encoding or sampled alternatives without changing the tokenizer or Stage 2 architecture.Source: §V · p. 6
• Each 0.8 s chunk carries 160 bits and a three-chunk buffer exposes 2.4 s of preplanned motion, but TrajectoryGPT does not condition on low-level robot state.Re-encoding measured robot history could close the loop without changing the architecture, although those histories could be out of distribution.Source: §V · p. 6
• A fight-to-dance sequence completed upright on the G1 through wildcard chunks with no reference-tracking objective, while regularization remained active for stability.The transition interval leaves the state-conditioned policy free to connect surrounding motions; supplying a prior motion could further shape the behavior.Source: §V · p. 6, Figure 7 · p. 7
• The discussion proposes varying repertoire size and transition difficulty, reporting per-motion tracking, and testing held-out latent-buffer sequences to separate skill reuse from schedule memorization.A proposed Stage 3 evaluation would introduce held-out buffer sequences only at evaluation.Source: §V · p. 6
• Fig. 8 compares measured robot state with source kinematics; because reference banks are fixed offline, residuals arise downstream of TrajectoryGPT.They include both policy-tracking and sim-to-real error, and neither TrajectoryGPT nor the DAPS decoder runs in the 50 Hz deployment loop.Source: §V · p. 6, Figure 8 · p. 7
• Chunks play at fixed 0.8 s timing, and tokens specify motion style rather than global path because planar root position is not explicitly tracked.The discussion proposes adapting phase to contact or tracking progress and adding goal- or path-conditioned context for route-following tasks such as navigation.Source: §V · p. 6

• At a matched 160-bit bottleneck, DAPS reconstructs G1 motion more accurately than FSQ and VQ-VAE while retaining temperature-controlled alternative encodings.TrajectoryGPT learns sequences in the compact token space, and a latent-conditioned policy tracks decoded motions without receiving raw reference poses.Source: §VI · p. 7
• The three-stage pipeline uses a discrete autoencoder as the shared interface between motion generation and whole-body control.The high-level model organizes behavior at the token level while the low-level policy remains reactive to environment dynamics.Source: §VI · p. 7
• A physical Unitree G1 executed individual generated motions and a full fight-to-dance sequence, with measured-state comparisons showing the policy followed source motion without explicit reference poses.The actor reads the token stream on the robot while the high-level model composes behavior one token at a time.Source: §VI · p. 7