Source-linked AI summary
Learning Humanoid Locomotion with Perceptive Internal Model
Junfeng Long, Junli Ren, Moji Shi, Zirui Wang, Tao Huang, Ping Luo, Jiangmiao Pang
TL;DR
Humanoid robots need accurate perception for stable locomotion over diverse terrain, but incorporating perception can add disturbances and costs. PIM uses an elevation map with HIM for perceptive state prediction, and the paper reports over 90% success when continuously climbing stairs. The authors present the framework as a potential foundation for future humanoid control algorithms.
Problem
Humanoid robots’ high degrees of freedom and unstable morphology make stable locomotion over diverse terrain difficult, while adding perceptual signals can introduce disturbances and costs.
Source: §I · p. 1Method
PIM adds elevation-map perception to HIM, training with ground-truth obstacle heights in simulation and using sampled map heights at inference.
Source: §I · p. 1Results
Over 90% success is reported for continuously climbing stairs, with evaluation across robots, challenging terrains, and sensor configurations.
Source: §I · p. 1Takeaways & Limitations
The authors propose PIM as a potential foundational algorithm for future humanoid control algorithms.
Source: §V · p. 7
Abstract
from arXiv · showhide
In contrast to quadruped robots that can navigate diverse terrains using a "blind" policy, humanoid robots require accurate perception for stable locomotion due to their high degrees of freedom and inherently unstable morphology. However, incorporating perceptual signals often introduces additional disturbances to the system, potentially reducing its robustness, generalizability, and efficiency. This paper presents the Perceptive Internal Model (PIM), which relies on onboard, continuously updated elevation maps centered around the robot to perceive its surroundings. We train the policy using ground-truth obstacle heights surrounding the robot in simulation, optimizing it based on the Hybrid Internal Model (HIM), and perform inference with heights sampled from the constructed elevation map. Unlike previous methods that directly encode depth maps or raw point clouds, our approach allows the robot to perceive the terrain beneath its feet clearly and is less affected by camera movement or noise. Furthermore, since depth map rendering is not required in simulation, our method introduces minimal additional computational costs and can train the policy in 3 hours on an RTX 4090 GPU. We verify the effectiveness of our method across various humanoid robots, various indoor and outdoor terrains, stairs, and various sensor configurations. Our method can enable a humanoid robot to continuously climb stairs and has the potential to serve as a foundational algorithm for the development of future humanoid control methods.
I. INTRODUCTION
Humanoid locomotion across difficult terrain needs perception, but perceptual signals can add cost and disturbance, while existing approaches struggle with challenging terrain. PIM uses elevation maps with HIM to support efficient perceptive locomotion, and the paper reports over 90% success while continuously climbing stairs.
I. INTRODUCTION
- Humanoid robots’ high degrees of freedom and unstable morphology make stable locomotion across diverse terrains difficult, and perceptual signals can add disturbances and costs.Prior approaches often use blind policies or multi-stage training, and can fail on more challenging scenarios such as continuous stairs.Source: §I · p. 1
- PIM uses onboard, continuously updated elevation maps centered around the robot, training on ground-truth obstacle heights and sampling the map during inference.It builds on HIM, which uses batch-level contrastive learning to optimize the simulated robot’s response.Source: §I · p. 1
- PIM avoids depth-image rendering during simulation and trains in 3 hours on an RTX 4090 GPU, while its larger local map captures terrain beneath the feet and is more robust to sensor movement and noise.The paper contrasts this with depth-map or raw-point-cloud encoding and notes the challenge of closing the simulation-to-real gap for depth maps.Source: §I · p. 1
- Over 90% success is reported for continuously climbing stairs, alongside evaluation across humanoid robots, challenging terrains, and sensor configurations.The robots include Unitree H1 and Fourier GR-1; terrains include indoor and outdoor settings.Source: §I · p. 1
II. RELATED WORK
Prior humanoid locomotion work commonly uses simulation training and sim-to-real transfer, with approaches addressing natural motion and state prediction in different ways. This paper introduces a policy that uses perceptive information for state prediction without additional training stages or explicit state decoding.
A. Learning Humanoid Locomotion
- Humanoid locomotion studies commonly use simulation training followed by sim-to-real transfer, while some use prior motion trajectories for natural whole-body movement.Research also reports that learning from scratch can produce coordinated whole-body movement, especially for challenging-terrain traversal.Source: A. Learning Humanoid Locomotion · p. 2
- Prior methods address state prediction with a separate velocity estimator, multi-stage training, or prediction as part of a denoising process.These approaches respond to humanoid robots’ high-dimensional action and observation spaces.Source: A. Learning Humanoid Locomotion · p. 2
- The proposed policy uses perceptive information for state prediction without additional training stages or explicit decoding of predicted states.Source: A. Learning Humanoid Locomotion · p. 3
B. Perceptive Legged Locomotion
Humanoid perceptive locomotion faces a sim-to-real perception gap, and existing approaches can compromise accuracy or struggle with precise tasks such as stair climbing. The proposed system uses a LiDAR-based elevation map for state prediction, alongside proprioceptive and perceptive observations, and optimizes its policy with PPO.
B. Perceptive Legged Locomotion
- Existing perceptive locomotion approaches face a sim-to-real perception gap, and hardware-noise training can compromise accuracy on tasks requiring precise perception, such as stair climbing.Source: B. Perceptive Legged Locomotion · p. 3
- A LiDAR-based, robot-centered elevation map provides hardware perception without placing vision in the historical observation sequence, and experiments cover stairs, gaps, and platforms.Source: B. Perceptive Legged Locomotion · p. 3
A. Framework Overview
- The framework estimates state from proprioceptive and perceptive information and optimizes the policy with PPO.Source: A. Framework Overview · p. 3
B. Observations
- Policy observations include commands, proprioceptive information, the elevation-map percept, and the previous action; the critic can use privileged information such as linear velocity during training.Proprioception includes joint position and velocity, base angular velocity, and gravity direction in the robot frame.Source: B. Observations · p. 3
C. Preliminary: Hybrid Internal Model
HIM estimates next-step linear velocity and latent proprioceptive information from proprioceptive observation history. It trains velocity estimation by regression and next-step prediction through contrastive learning.
C. Preliminary: Hybrid Internal Model
- HIM predicts next-step linear velocity and latent proprioceptive information from proprioceptive observation history.Velocity estimation uses regression to simulator ground truth, while next-step prediction uses contrastive learning with positive and negative trajectory pairs.Source: C. Preliminary: Hybrid Internal Model · p. 3
D. Perceptive Internal Model
PIM combines elevation-map perception with proprioceptive history to estimate the robot’s next state. Its real-world perception uses gravity-aligned, grid-based terrain heights sampled around the robot.
D. Perceptive Internal Model
- Real-world elevation maps use odometry and ground points to create a grid of terrain heights, with the map’s z-axis aligned to gravity.The point cloud is filtered to preserve ground points before map generation.Source: D. Perceptive Internal Model · p. 3
- The robot receives terrain perception from 96 points distributed across a centered 0.8m × 1.2m area, represented by their relative heights to the base link.The sampling frame’s z-axis points opposite gravity.Source: D. Perceptive Internal Model · p. 3
- A LiDAR-based elevation map is the perception model, supporting perception alignment between simulation and real-world environments.The framework integrates this perception into the state predictor.Source: Figure 2 · p. 4
- PIM concatenates current perceptive observations with proprioceptive history to estimate the robot’s next state, using terrain information to improve estimation.Unlike HIM, it adds current perception to the estimator’s inputs.Source: D. Perceptive Internal Model · p. 4
E. Action Curriculum
The action curriculum gradually expands the allowed motion of selected arm and waist joints. Symmetry regularization augments training with reflected observations and actions to promote gait harmony.
E. Action Curriculum
- The action curriculum starts selected arm and waist joints with zero range, then increases their range as training proceeds.These joints are chosen as less important to locomotion.Source: E. Action Curriculum · p. 4
F. Symmetry Regularization
- Symmetry regularization flips proprioceptive observations, perceptive observations, and actions across the x-z plane, and augments training data with the transformed inputs.The technique is introduced to improve gait harmony.Source: F. Symmetry Regularization · p. 4
G. Training Process
Training alternates policy optimization with optimization of the perceptive internal model using collected trajectories. The section also points to reward functions and figures on robot hardware and training metrics.
G. Training Process
- The perceptive internal model estimates next-step linear velocity and latent next-proprioception information from observation history and perception; the policy then uses these estimates with the current observation to produce actions.During policy optimization, the model is frozen and PPO updates the policy and value networks; collected trajectories are later used to optimize the model.Source: G. Training Process · p. 4
H. Reward Functions
- Detailed reward functions are presented in Table I.The supplied table passage identifies it as the rewards table but does not provide its entries.Source: Table I · p. 5
- Figure 4 presents the robot hardware setups in terms of height, weight, and degrees of freedom.Source: Figure 4 · p. 5
G. Training Process
- Figure 5 presents estimation loss and terrain level during training.Source: Figure 5 · p. 5
IV. RESULTS
The experiments assess perception and locomotion on two humanoid robots across challenging terrains. The reported demonstrations include difficult obstacles and cross-platform validation.
IV. RESULTS
- The experiments validate the method on Unitree H1 and Fourier GR-1 humanoids in challenging environments.Source: §IV · p. 5
- The evaluation separately examines perception’s effectiveness for estimating the next-step state.Source: §IV · p. 5
- The proposed locomotion succeeds on extreme terrains including stairs at least 0.15m high, wooden platforms at least 0.4m high, 15° slopes, and gaps.Source: §IV · p. 5
- The terrain-based internal model is reported to predict robot states accurately, enhancing locomotion stability and adaptability.Source: §IV · p. 5
A. PIM vs HIM
PIM achieves higher training efficiency and estimation accuracy than HIM while enabling traversal of more difficult terrain. Hardware tests include 15 cm stairs, consecutive platform and gap jumps, and deployment on GR-1.
A. PIM vs HIM
- PIM achieves higher training efficiency and estimation accuracy than HIM and enables traversal of more difficult terrain.The comparison concerns estimation: PIM uses perception for estimating, whereas HIM does not; both locomotion policies use perception.Source: A. PIM vs HIM · p. 5
B. Stairs
- H1 traverses 15 cm stairs, exceeding the cited 10 cm maximum achieved by existing methods [6], [8].The 30 cm stair width equals the robot’s foot length, requiring precise placement to avoid getting stuck or losing balance.Source: Figure 6 · p. 6, B. Stairs · p. 6
C. High platform & gap
- H1 performs consecutive jumps onto platforms and over gaps, actions that require faster responses to changing perception than stair climbing.Source: C. High platform & gap · p. 6
D. Cross-Platform Validation
The method transfers to Fourier GR-1 despite differences in ankle-joint configuration, and the robot performs coordinated upper-body movements during extreme actions. The paper presents PIM as part of a framework deployed across humanoid platforms with different physical configurations.
D. Cross-Platform Validation
- The same training process enables Fourier GR-1, with two ankle degrees of freedom powered by parallel joints, to traverse varied terrains.The authors report robustness across differences in robot height, mass, joint configuration, and foot design.Source: D. Cross-Platform Validation · p. 6
E. Whole-body Movement
- During jumping, the robot swings its left arm forward as its right leg jumps, coordinating upper-body movement to maintain balance.The authors report simple, human-like movements without prior data or imitation learning, given appropriate reward design and model prediction.Source: E. Whole-body Movement · p. 7
V. CONCLUSION
- The framework combines PIM for next-state prediction with action curriculum, symmetry regularization, and reward-function design.The authors report sim-to-real deployment across robots differing in height, mass, joint configuration, and foot design.Source: §V · p. 7