Source-linked AI summary
Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking
Zewei Zhang, Kehan Wen, Michael Xu, Junzhe He, Chenhao Li, Takahiro Miki, Clemens Schwarke, Chong Zhang, Xue Bin Peng, Marco Hutter
TL;DR
Whole-body humanoid locomotion needs coordinated control and terrain adaptation, but reward-shaped RL explores inefficiently and pure motion tracking mainly replays references. The paper combines diffusion-based reference generation with RL tracking and closed-loop tracker fine-tuning, achieving terrain-aware traversal across varied hardware terrains while remaining limited by perception quality.
Problem
Whole-body humanoid locomotion requires coordinated movement and terrain adaptation, while reward-shaped RL is difficult and pure motion tracking lacks reactivity to diverse environments.
Method
The framework trains a diffusion motion generator and RL reference tracker on retargeted human motions, then fine-tunes the tracker with the frozen generator in a closed loop.
Results
Hardware experiments demonstrate terrain-aware whole-body traversal over boxes, stairs, hurdles, and compound terrain combinations, including combinations unseen during RL fine-tuning.
Takeaways & Limitations
Online motion generation and tracker fine-tuning support generalized and robust perceptive whole-body locomotion without heavily engineered distillation strategies.
Takeaways & Limitations
Performance can deteriorate substantially when sensing noise degrades the LiDAR-based elevation-mapping pipeline.
Abstract
from arXiv · showhide
Whole-body humanoid locomotion is challenging due to high-dimensional control, morphological instability, and the need for real-time adaptation to various terrains using onboard perception. Directly applying reinforcement learning (RL) with reward shaping to humanoid locomotion often leads to lower-body-dominated behaviors, whereas imitation-based RL can learn more coordinated whole-body skills but is typically limited to replaying reference motions without a mechanism to adapt them online from perception for terrain-aware locomotion. To address this gap, we propose a whole-body humanoid locomotion framework that combines skills learned from reference motions with terrain-aware adaptation. We first train a diffusion model on retargeted human motions for real-time prediction of terrain-aware reference motions. Concurrently, we train a whole-body reference tracker with RL using this motion data. To improve robustness under imperfectly generated references, we further fine-tune the tracker with a frozen motion generator in a closed-loop setting. The resulting system supports directional goal-reaching control with terrain-aware whole-body adaptation, and can be deployed on a Unitree G1 humanoid robot with onboard perception and computation. The hardware experiments demonstrate successful traversal over boxes, hurdles, stairs, and mixed terrain combinations. Quantitative results further show the benefits of incorporating online motion generation and fine-tuning the motion tracker for improved generalization and robustness.
I. INTRODUCTION
Whole-body humanoid locomotion requires coordinated control across many degrees of freedom and adaptation to complex terrain. The proposed system combines diffusion-based motion generation with RL-based tracking to address these challenges.
- Humanoid locomotion is difficult because high degrees of freedom and morphological constraints make whole-body RL exploration and reward shaping challenging.
- Motion imitation can transfer coordinated whole-body skills, but pure trackers mainly replay choreographed trajectories and lack terrain adaptation.
- The proposed framework combines diffusion-based motion generation with RL-based motion tracking for real-time humanoid locomotion.
- Hardware experiments demonstrate directional goal-reaching and terrain-aware whole-body adaptation across diverse terrains, including terrain types unseen during training.
II. RELATED WORK
Prior work uses RL imitation to reproduce coordinated motion, but pure tracking remains limited when terrain conditions vary.
- RL-based motion imitation transfers dynamic maneuvers while keeping executed behavior consistent with physical constraints.
- Quadruped and humanoid controllers have imitated large motion datasets and supported real-time teleoperation, yet pure tracking remains poorly adaptable to varying terrains.
B. Motion Generation for Motion Control
Generative motion models offer scalable trajectory generation, while tracking policies can correct generated-motion artifacts. This work uses diffusion generation as a skill-composition mechanism and adds RL fine-tuning for terrain adaptation and robustness.
- Diffusion and other generative models can sample feasible motion trajectories from curated datasets, reducing engineering effort as motion repertoires expand.
- The framework uses a diffusion-based generator to produce environment-conditioned references for a low-level tracking policy instead of explicitly engineering multi-expert distillation.
- RL fine-tuning trains the tracker under randomized directions and diverse terrain configurations while the pretrained motion generator remains frozen.
III. METHOD
The method builds a whole-body motion dataset, pre-trains a diffusion generator and RL tracker, then fine-tunes the tracker with the generator frozen during receding-horizon deployment.
- The framework has three stages: collecting dynamically feasible robot trajectories, pre-training the generator and tracker, and fine-tuning the tracker on diverse terrains.
- The initial dataset contains approximately 5 minutes of clips covering box climbing, vaulting, box descent, stair traversal, and omnidirectional walking.
- Human motions are reconstructed from videos and retargeted to the humanoid using a contact-constrained inverse-kinematics solver.
- Retargeted trajectories are refined with a DeepMimic-style tracking policy to make training motions physically plausible for both learning components.
2) Motion Augmentation:
The system augments motion data through terrain-aware kinematic transformations, then trains a whole-body tracker to imitate the resulting references while respecting hardware constraints and using terrain observations.
- Motion Augmentation: Kinematics-based augmentation varies terrain geometry by scaling obstacle heights and inserting random small boxes along motion paths.The augmented motions are optimized to remain consistent with modified terrain using terrain-penetration and smoothness losses.
- Pre-training: Pre-training equips both the motion generator and tracker to react to different environments and produce motion references and joint actions.
- Whole-body Motion Tracker: The motion tracker is trained with PPO in IsaacLab on collected motion data using imitation rewards and regularization terms.These terms encourage imitation of all reference motions while respecting the physical constraints of the real hardware.
- Whole-body Motion Tracker: Tracker observations combine reference motion, proprioception, and terrain height scans, while actions are 23-dimensional target joint positions.Terrain information is included during pre-training because it can benefit later fine-tuning.
2) Diffusion-based Motion Generator:
The diffusion generator predicts short-horizon, terrain-conditioned whole-body references, while closed-loop fine-tuning makes the tracker more robust to imperfect generated motions and deployment disturbances.
- Diffusion-based Motion Generator: The diffusion model predicts 25 future frames over a 0.5-second horizon, conditioned on target heading, terrain height scans, and two past motion frames.Predicted features include root position and orientation, joint positions, and body-link positions.
- Diffusion-based Motion Generator: Training combines reconstruction with velocity, joint-consistency, and terrain-penetration losses for terrain-aware motion prediction.
- Limitations: Generated motion artifacts can cause failures when paired directly with a tracker trained mainly on high-quality references.The paper identifies limited robustness and reliability as consequences of simply combining the pretrained generator and tracker.
- RL Fine-tuning Stage: Closed-loop fine-tuning uses the frozen generator with robot-state conditioning, two denoising steps, noise injection, randomized headings, and a heading-tracking reward.Additional terrains include stairs, vaulting hurdles, climbing boxes, and pyramid steps with varied dimensions and orientations.
- RL Fine-tuning Stage: After fine-tuning, the tracker filters generated references and adjusts execution using terrain observations to suppress unsafe behavior across diverse terrains.Heading and terrain randomization supports behaviors beyond those present in offline data.
D. Details for Hardware Deployment
The full pipeline runs onboard the Unitree G1, using LiDAR-inertial state estimation, elevation mapping, sensor fusion, and accelerated receding-horizon diffusion inference.
- Perception: The system uses DLIO with Livox MID360 LiDAR and IMU for base-pose estimation, while elevation mapping reconstructs terrain height information.
- State Estimation: Torso-IMU pitch measurements are fused with DLIO orientation estimates to compensate for head movement caused by the passive neck joint.
- Real-time Deployment: TensorRT reduces generator inference time to approximately 0.02 s, and deployment updates the reference every 0.25 seconds.The generator predicts over a 0.5-second horizon and runs on a Jetson Thor mounted on the robot.
IV. EXPERIMENTS
Experiments evaluate terrain traversal and component contributions in simulation and on hardware. The system performs diverse whole-body behaviors, transitions across compound terrains, and benefits from online generation and tracker fine-tuning.
- Hardware Results: The robot climbs a 75 cm box, traverses stairs and vaulting hurdles, bypasses a box to reach a target, and crosses mixed terrain.
- Hardware Results: Hardware experiments evaluate box traversal, stair ascent and descent, continuous vaulting, and compound terrain sequences.The study also examines online motion generation and RL fine-tuning through quantitative simulation experiments.
- Hardware Results: Box traversal includes different approach directions and on-top reorientation, while vaulting crosses hurdles directly rather than stepping onto them.
- Quantitative Evaluation: Across five simulated terrain tasks, fine-tuning consistently improves success rates, with larger gains on more difficult terrain settings.The evaluation uses 500 robots per task with randomized initial poses and goal-directed traversal.
- Hardware Results: The controller transitions dynamically among motion styles on combined terrains, including stair ascent after a jump-down sequence unseen during RL fine-tuning.
- Local Navigation: After fine-tuning, the tracker may override generated references and bypass an obstacle from the side when the commanded traversal state is unsuitable.
B. Quantitative Results
The framework’s quantitative analysis tests whether online motion generation improves deployment adaptation and whether tracker fine-tuning remains necessary after coupling the generator and tracker.
- The study evaluates online motion generation for terrain adaptation and tracker fine-tuning after coupling the generator with the motion tracker.
1) Benefit of Online Motion Generation:
Online motion generation improves terrain-aware generalization beyond fixed-reference tracking, especially when terrain geometry changes substantially. Fixed-reference tracking retains limited local robustness but degrades under larger changes in obstacle height, orientation, or motion demands.
- The evaluation compares fixed-reference tracking with online generation on box climbing, vaulting, and stair ascent under modified terrain geometry.Test variations include obstacle-height changes and terrain yaw rotations.
- Tracker + Gen. achieves better generalization and higher average success rates across all three tasks than fixed-reference tracking.
- Fixed-reference tracking becomes brittle as box height increases, while online generation remains consistently robust across tested box-climbing settings.
- For vaulting and stair ascent, fixed-reference performance degrades under larger terrain-height or orientation changes, whereas the full system remains stable or nearly perfect.
- Fine-tuned trackers retain limited local terrain generalization, but substantial geometry changes requiring altered timing or motion style cause rapid degradation without adaptive references.
2) Impact of Fine-Tuning the Motion Tracker:
Fine-tuning the motion tracker consistently improves robustness when paired with online-generated references. The combined framework supports traversal across diverse and unseen terrain combinations, while its perception pipeline remains an important operational constraint.
- Fine-tuning consistently improves success rates across five terrain-traversal tasks, with the largest gains on more difficult terrain settings.Improvements become more apparent as box or stair height increases; vaulting remains comparatively easier.
- Fine-tuning adapts the tracker to imperfect online references and suppresses unsafe behaviors caused by distribution mismatch between offline training motions and deployment references.The frozen generator produces references conditioned on noisy state, target direction, and terrain observations.
- Combining motion generation with motion tracking enables perceptive whole-body locomotion over diverse terrain types and unseen terrain combinations.
- The system relies on a LiDAR-based elevation-mapping pipeline, and locomotion performance can deteriorate substantially when sensing noise degrades mapping quality.