MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation

1 Introduction

• Hierarchical humanoid policies give the upper body fine-grained joint targets but only coarse base commands to the lower body, leaving the legs focused on balance rather than task interaction.This split creates inconsistent action spaces and rules out foot interaction such as pedal stepping or kicking.Source: §1 · p. 2, Figure 2 · p. 2
• Iterative denoising over high-dimensional video-action latents makes existing WAMs too costly for real-time control, motivating a unified real-time humanoid approach.WAMs use future visual-state prediction to provide a dynamics prior for action prediction.Source: §1 · p. 2
• MotionWAM couples video and motion DiTs through intermediate denoising features and predicts unified whole-body motion tokens spanning locomotion, torso movement, height, foot placement, and hand manipulation.The unified action space gives the lower body task-driven behaviors such as stepping on a pedal or kicking an object.Source: §1 · pp. 2–3, Figure 2 · p. 2
• The three-stage training framework adapts the video model to egocentric visual dynamics, cross-embodiment action data, and target-task whole-body demonstrations.Stage 1 uses roughly 2,136 hours of egocentric human and humanoid video; Stage 2 trains on heterogeneous Unitree G1 data; Stage 3 fine-tunes on teleoperated target-task demonstrations.Source: §1 · pp. 2–3
• MotionWAM runs in real time across nine real-world tasks, outperforming fine-tuned VLA baselines and producing task-driven foot interaction and more coherent, human-like whole-body motion.Its foot interactions are beyond the reach of decoupled upper–lower policies.Source: §1 · p. 3

Imitation policies and VLAs for manipulation

• Imitation policies [23] [24] [25] and VLAs [27] [28] [29] [30] [31] [32] learn from demonstrations, but typically use static image–text inputs and arm-and-gripper actions; MotionWAM uses video-world-model features and whole-body control.The passage contrasts static image–text inputs with video features that condition the policy.Source: §2 · p. 3

World Action Models

• Prior WAM deployments [14] [15] [16] [17] [18] [19] [20] [21] [22] remain in short-horizon, fixed-base tabletop settings, with high-dimensional video-action denoising cited as a barrier to closed-loop speed.MotionWAM targets dynamically balancing humanoids and is described as the first real-time WAM for whole-body loco-manipulation, to the authors’ knowledge.Source: §2 · p. 3

Humanoid loco-manipulation

• Humanoid loco-manipulation work spans whole-body imitation [1] [2] [3] [4], command-conditioned controllers [5] [6] [7] [8] [9], and autonomous policies built on those controllers [10] [11] [30] [13].Hierarchical structures have become widely used.Source: §2 · pp. 3–4
• Autonomous hierarchical policies typically leave the legs to a low-level controller driven by base velocity, height, and orientation, whereas MotionWAM uses unified tokens so the legs can participate in the task.The decoupled action spaces limit legs to stabilizing the base.Source: §2 · p. 4

3.1 Problem Formulation

• The action representation combines a 64-dimensional discretized SONIC token for locomotion, torso, height, and foot intent with continuous dexterous end-effector commands.The motion latent is decoded into joint commands.Source: §3.1 · p. 4

3.2 Model Architecture

• A one-pass Video DiT exposes fixed-timestep hidden states to a Motion DiT, which combines them with proprioception and noisy motion tokens to predict the latent's velocity.The Video DiT operates near the pure-noise end of the schedule without denoising future frames, supporting real-time closed-loop control.Source: §3.2 · p. 4

3.3 Training Recipe

• Stage 1 pretrains only the Video DiT on about 2,136 hours of egocentric human and humanoid video for future-frame generation, while the VAE and text encoder remain frozen.This shifts the video world model toward first-person visual dynamics.Source: §3.3 · p. 5
• Stage 2 attaches the Motion DiT and jointly updates the video and action networks across heterogeneous Unitree G1 data, retaining video loss as a representation regularizer.Per-embodiment projectors wrap a shared Motion DiT trunk during multi-embodiment training.Source: §3.3 · pp. 5–6
• Stage 3 fine-tunes the full network end-to-end on 200 teleoperated whole-body episodes per task across nine real-world Unitree G1 tasks, without architectural changes.The model carries forward the joint loss and shared-trunk plus Unitree G1 projector configuration.Source: §3.3 · pp. 5–6

4.2 Comparison with the State-of-the-Art

• 76.1% overall success versus 43.9% for GR00T-N1.7, an over 32% absolute gain; MotionWAM wins every task.The comparison covers nine real-world tasks, with all policies finetuned on the same Stage 3 demonstrations.Source: Figure 5 · p. 7, §4.2 · p. 7

4.1 Experimental Setup

• Evaluation covers nine real-world G1 tasks, with 20 trials per task and five baselines finetuned on the same Stage 3 dataset.The task suite stresses waist control, height regulation, squatting locomotion, foot interaction, and body–hand coordination.Source: §4.1 · pp. 6–7

4.3 Ablation on the Three-Stage Training Framework

• Removing Stage 1 and Stage 2 leads to 11% and 28% absolute performance drops, respectively.Stage 3 remains enabled in the five-task ablation; the authors report that without Stage 2, performance collapses on every task.Source: §4.3 · pp. 7–8, Table 1 · p. 8

4.4 Real-Time Inference Frequency

• MotionWAM runs at 4.9 Hz versus 0.7 Hz for Cosmos Policy, which iteratively denoises future video before producing actions.The authors report that MotionWAM reads intermediate denoising features in a single pass.Source: §4.4 · p. 8

5 Conclusion

• MotionWAM couples a Video DiT and Motion DiT through intermediate denoising features and predicts whole-body motion tokens in a unified action space.It is trained with a three-stage egocentric-to-embodiment recipe.Source: §5 · p. 8
• MotionWAM outperforms the strongest VLA baseline by over 30% on nine real-world tasks.Source: §5 · p. 8
• The results point to a viable path from large-scale video pretraining to human-like whole-body humanoid policies.Source: §5 · p. 8

6 Limitations

• Stage 3 fine-tuning has been validated only on Unitree G1; transfer of the three-stage recipe to other humanoid platforms is unverified.Source: §6 · p. 8
• The evaluation lacks a controlled novel-object generalization study, and training and test objects share visual similarity.Success on strictly out-of-distribution objects is not reported.Source: §6 · p. 8
• MotionWAM can lose visual grounding and stall when an object leaves the camera view or the head-camera viewpoint drifts from training.Source: §6 · p. 8

A.1 Per-Task Language Prompts

• Table 3 lists the natural-language prompts for the nine real-world whole-body loco-manipulation tasks.Source: §A.1 · p. 13, Table 3 · p. 13

B.1 Hardware

• The Unitree G1 uses dual ALOHA 2 grippers on 7-DoF arms and a head-mounted Intel RealSense D435i RGB camera.The grippers provide a 16-DoF upper-body manipulation interface.Source: §B.1 · p. 13

B.2 VR-Based Teleoperation Pipeline

• VR teleoperation tracks the operator's ankles and hands, retargets a 24-joint SMPL pose to 29-DoF G1 joint angles, and records episodes at 50 Hz.The recorded streams include state, command, visual data, and the language goal.Source: §B.2 · p. 13

• The Video DiT starts from Cosmos-Predict2.5-2B [39], while its text and VAE encoders remain frozen throughout training.Source: C Model and Training Configuration · p. 13
• MotionDiT follows the DiT-B configuration from DiT4DiT [21], with enlarged action and state dimensions for the Stage 2 cross-embodiment mixture.Source: C Model and Training Configuration · p. 13
• All three stages reuse one flow-matching objective, with stage-specific modifications.Source: C Model and Training Configuration · p. 13

• Every baseline is fine-tuned on MotionWAM’s Stage 3 demonstrations and receives the same egocentric RGB observations, language goals, and proprioceptive states.Source: D Baseline Training Configurations · p. 13
• MotionWAM’s three stages share one flow-matching objective; the data mixture, learning rates, and trainable modules change across stages.Source: Table 4 · p. 14
• The baselines include diffusion-policy and action-chunking methods, generalist VLA models, and a parameter-matched Qwen3DiT variant.Source: D Baseline Training Configurations · pp. 14–15
• Diffusion Policy uses a pretrained ResNet-18 visual encoder and UNet action denoiser, with 100 iterative denoising steps at inference.Source: D Baseline Training Configurations · p. 14
• Diffusion Policy fails on most loco-manipulation tasks, which the authors attribute to insufficient visual capacity of its UNet model for the egocentric whole-body setting.Source: D Baseline Training Configurations · p. 15
• ACT is adapted to output a 66-dimensional Unitree G1 action vector, including two gripper states, using a chunk size of 100.Source: D Baseline Training Configurations · p. 15
• Qwen3DiT replaces MotionWAM’s Cosmos-Predict2.5 Video DiT with Qwen3-VL while retaining MotionDiT, the motion-latent action space, and the input interface.Source: D Baseline Training Configurations · p. 15

Stage 1 — egocentric video pretraining

• Stage 1 trains only the Video DiT on video streams from egocentric human and humanoid-robot sources, ignoring available action labels.Source: E Per-Stage Data Composition · p. 15
• Within each Stage 1 domain, source weights are allocated by episode count to prevent any single dataset from dominating.Source: E Per-Stage Data Composition · p. 15
• Stage 1 allocates its mixture budget across human videos (30%), G1-class humanoids (50%), and other real robots (20%).Source: Table 5 · p. 15

Stage 2 — cross-embodiment action post-training

• Stage 2 co-trains the Video DiT and Motion DiT on heterogeneous action-labelled humanoid datasets, using embodiment-specific projectors around a shared Motion DiT trunk.Source: E Per-Stage Data Composition · p. 15
• Stage 2 right-pads action vectors to 66 dimensions and masks invalid channels so embodiments with different action layouts can share the DiT trunk.Source: E Per-Stage Data Composition · pp. 15–16

Stage 3 — Unitree G1 whole-body fine-tuning

• Stage 3 fine-tunes the full network end-to-end on in-house teleoperated Unitree G1 data, with 200 episodes per task across nine tasks.Source: E Per-Stage Data Composition · p. 16

• Failures occur when the manipulated object leaves the head camera’s field of view, cutting off the visual reference for the task.Source: F Failure Cases · p. 16, Figure 6 · p. 16
• A head-camera viewpoint that drifts from the training distribution is another reported source of lost visual grounding.Source: F Failure Cases · p. 16, Figure 6 · p. 16
• When visual grounding is lost, the policy may stall or commit to an inaccurate whole-body trajectory.Source: F Failure Cases · p. 16