Source-linked AI summary
RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains
Yuanhang Zhang, Younggyo Seo, Juyue Chen, Yifu Yuan, Koushil Sreenath, Pieter Abbeel, Carmelo Sferrazza, Karen Liu, Rocky Duan, Guanya Shi
TL;DR
Humanoid perceptive locomotion still lacks robust multi-directional performance on complex terrains, especially with payloads and changing views. RPL addresses this with terrain-specific privileged experts distilled into a multi-view depth transformer, plus rendering and distillation techniques for asymmetric observations. Real-world trials demonstrate bidirectional traversal across slopes, stairs, and stepping stones with a 2 kg payload, while the paper identifies sideways locomotion and active viewpoint selection as limitations.
Problem
Existing humanoid perceptive locomotion methods are brittle under multi-directional locomotion and upper-body self-occlusions, while robust terrain traversal with payloads remains underexplored.
Method
RPL trains terrain-specific privileged-height-map experts, distills them into a unified multi-view depth transformer, and adds DFSV, RSM, and scalable realistic multi-depth rendering.
Results
RPL achieves 6/10 whole-course real-world success, including 3/5 trials with a 2 kg payload, across slopes, stairs, and stepping stones.
Takeaways & Limitations
The system demonstrates robust bidirectional locomotion over challenging terrains with payloads, including zero-shot generalization to a curved building staircase.
Takeaways & Limitations
RPL does not demonstrate real-world sideways locomotion and does not explicitly learn active viewpoint selection for highly occluded or ambiguous scenarios.
Abstract
from arXiv · showhide
Humanoid perceptive locomotion has made significant progress and shows great promise, yet achieving robust multi-directional locomotion on complex terrains remains underexplored. To tackle this challenge, we propose RPL, a two-stage training framework that enables multi-directional locomotion on challenging terrains, and remains robust with payloads. RPL first trains terrain-specific expert policies with privileged height map observations to master decoupled locomotion and manipulation skills across different terrains, and then distills them into a transformer policy that leverages multiple depth cameras to cover a wide range of views. During distillation, we introduce two techniques to robustify multi-directional locomotion, depth feature scaling based on velocity commands and random side masking, which are critical for asymmetric depth observations and unseen widths of terrains. For scalable depth distillation, we develop an efficient multi-depth system that ray-casts against both dynamic robot meshes and static terrain meshes in massively parallel environments, achieving a 5-times speedup over the depth rendering pipelines in existing simulators while modeling realistic sensor latency, noise, and dropout. Extensive real-world experiments demonstrate robust multi-directional locomotion with payloads (2kg) across challenging terrains, including 20° slopes, staircases with different step lengths (22 cm, 25 cm, 30 cm), and 25 cm by 25 cm stepping stones separated by 60 cm gaps.
1 Introduction
RPL addresses the brittleness of existing humanoid perceptive locomotion under multi-directional movement, upper-body self-occlusion, and payload-related demands. It combines terrain-specific privileged experts, unified depth-based distillation, efficient rendering, and targeted robustness techniques.
- Motivation: Existing humanoid perceptive locomotion methods often rely on a single forward camera and idealized upper-body conditions, limiting robustness during multi-directional movement and dynamic self-occlusion.Mapping-based methods can support bidirectional locomotion but rely on noisy state estimation.
- RPL framework: RPL trains terrain-specific expert policies with privileged height maps and force perturbations before distilling them into one multi-view depth transformer policy.The experts learn decoupled locomotion and manipulation skills for unified multi-directional control.
- Robustness techniques: RPL introduces DFSV and RSM to address asymmetric multi-view inputs and generalize across unseen terrain widths.DFSV conditions perception features on velocity commands, while RSM masks lateral depth regions during distillation.
- Scalable distillation: 5× speedup over existing simulators is achieved by ray-casting dynamic robot and static terrain meshes in massively parallel environments with realistic sensor effects.The renderer models latency, noise, and dropout for scalable depth distillation.
- Validation: RPL validates long-horizon bidirectional locomotion under whole-body motions and payload disturbances in real-world experiments.The contribution statement identifies payload disturbances as part of the validation scope.
2 Related Works
Prior legged-robot perceptive locomotion methods mainly use mapping or end-to-end depth policies, but multi-camera, multi-directional humanoid locomotion remains insufficiently evaluated. RPL’s framework is presented against these sensing and scalability limitations.
- Mapping-based methods: LiDAR-based mapping compresses terrain into elevation maps but depends on calibration and state estimation that can introduce sim-to-real drift.This is one of the two dominant paradigms described for perceptive locomotion.
- Depth-based methods: End-to-end depth methods map raw depth directly to actions, while most prior work uses a single forward camera and demonstrates only forward locomotion.The related-work comparison contrasts depth policies with mapping-based approaches.
- Multi-camera evaluation: Prior multi-camera work provides wider coverage but evaluates only stairs, not general asymmetric multi-view settings where cameras observe distinct terrain types.This leaves broader multi-directional terrain perception under-evaluated.
- Rendering scalability: Existing depth-rendering pipelines remain expensive, and accelerated Warp-based systems may lack dynamic-mesh support or fail to scale to multiple cameras.These limitations motivate an efficient multi-depth rendering system.
- RPL overview: RPL’s two-stage framework combines privileged terrain-specific experts with a unified multi-view depth transformer policy.The overview places expert training and visual-policy distillation in consecutive stages.
3 RPL: Two Stage – From Multiple Experts to One General Visual Policy
RPL first trains terrain-specialized policies with privileged height maps and then distills them into one unified visual policy using multi-view depth for bidirectional locomotion. The policy separates lower-body locomotion and upper-body manipulation while restricting terrain perception to locomotion control.
- Two-stage framework: Stage 1 trains terrain-specialized experts with privileged height-map observations, whereas Stage 2 distills them into a unified visual policy using front and back depth observations.The two stages convert multiple privileged experts into one perceptive policy for bidirectional locomotion.
- Dual-agent control: The dual-agent formulation factorizes whole-body control into lower-body locomotion and upper-body manipulation policies sharing proprioceptive history.The lower-body policy handles locomotion goals, while the upper-body policy handles manipulation targets.
- Control interface: Both policies use proprioceptive histories and their outputs are tracked by a low-level PD controller.The available formulation identifies proprioceptive histories as shared policy inputs.
- Perceptual inputs: Stage 1 supplies a privileged height map, while Stage 2 supplies multi-view depth observations to the lower-body policy.Terrain perception is used only by the lower-body policy because it primarily governs legged locomotion.
3.1 Stage 1: Training Multiple Terrain Experts via Height Map
Stage 1 trains terrain-specific experts with privileged height maps, PPO, asymmetric actor–critic learning, and symmetry augmentation. Specialized terrain rewards address foot placement, terrain edges, foothold validity, and torso orientation across slopes, stairs, and stepping stones.
- Expert training: One terrain-specialized expert is trained per terrain family using a privileged 1.6 m × 1.0 m height map at 0.1 m resolution.The dual-agent policies are jointly optimized with independent reward functions using PPO.
- Training design: Asymmetric actor–critic training gives critics privileged root velocity and end-effector force information, while symmetry augmentation encourages symmetric gait patterns.These techniques improve training information and gait regularity without changing the stated policy inputs.
- Terrain settings: Experts cover slopes up to 37°, stairs with 0.25–0.30 m step lengths and 0.05–0.27 m heights, and stepping stones with 0.25–0.40 m diameters and 0.05–0.70 m gaps.The terrain families include slopes, stairs up and down, and stepping stones.
- Terrain difficulty: A 0.21 m foot on 0.25 m supports leaves little margin for foothold placement errors on stairs and stepping stones.This geometric constraint motivates explicit foot-placement and edge-related penalties.
- Terrain rewards: The training adds foot-edge, foothold, and torso-orientation terms to promote stable terrain traversal.Foothold penalties are used for stairs because edge dilation is too conservative on narrow treads.
3.2 Stage 2: Distillation into Unified Visuomotor Policy
RPL distills terrain-expert policies into a unified visuomotor controller using multi-camera depth observations. The distillation pipeline combines scalable multi-depth rendering with velocity-conditioned feature scaling and random side masking.
- Distillation into Unified Visuomotor Policy: The unified visuomotor policy is distilled from terrain experts and combines with a blind upper-body policy for whole-body control.The student uses multi-camera depth inputs and noisy proprioception, while the final controller produces low-level PD-tracked actions.
- Distillation into Unified Visuomotor Policy: DAgger-only imitation learning minimizes action regression from expert policies, preserving cross-terrain gradient alignment and improving terrain levels across terrain families.The paper reports that adding reinforcement learning to DAgger was not used because DAgger-only performed better in this comparison.
- Efficient Multi-Depth System Simulation: The multi-depth system ray-casts each camera ray against dynamic robot meshes and a shared static terrain mesh, retaining the closest intersection.Canonical body meshes are queried in local frames through pose-transformed rays, followed by terrain queries in the world frame.
- Efficient Multi-Depth System Simulation: A fused Warp kernel parallelizes environments, cameras, and pixels, with CUDA-graph replay and batched intrinsics supporting low-overhead depth synthesis.The system is summarized as a multi-depth ray-casting kernel and supports per-environment camera intrinsics randomization.
- Depth Feature Scaling Based on Velocity Commands: Depth Feature Scaling emphasizes cameras aligned with the commanded velocity and suppresses cameras facing away before feature concatenation.The attention scales are computed from the planar velocity command and each camera’s normalized viewing direction.
- Random Side Masking for Unseen Terrain Widths: Random Side Masking fills peripheral depth regions with random noise during training, encouraging reliance on central visible geometry for narrower terrain widths.Mask sizes are sampled by terrain type, with larger masks for stairs and slopes and smaller masks for stepping stones.
4 Experiments
Experiments evaluate RPL’s rendering efficiency, camera requirements, distillation design, multi-terrain training objective, and real-world robustness. RPL supports scalable multi-depth rendering, benefits from multi-view inputs and DFSV/RSM, and traverses challenging terrains with payloads, though failures remain near terrain edges and with payload swing.
- Scalable Multi-Depth Rendering: 5× speedup over the fastest baseline is achieved by RPL’s depth renderer while supporting both dynamic robot and static terrain meshes.The comparison uses a single L40S GPU across Ncam = 1, 2, 4 and 240 × 135 resolution; IsaacSim Warp lacks dynamic-mesh support.
- Camera Ablation: At least two cameras are required for reliable multi-directional locomotion because performance degrades sharply on stepping stones with fewer cameras.The camera study compares Ncam = 1, 2, 4 for bidirectional and omnidirectional locomotion; sparse footholds require coverage aligned with walking direction.
- Robustness Ablations: Removing DFSV, RSM, or both reduces whole-course success from 10/10 to as low as 0/10, while removing Stage-1 force perturbation reduces success to 3/10 under a 2.5 kg payload.Without RSM, the policy fails on unseen narrow stairs; without DFSV, irrelevant rear-view cues distract the policy during asymmetric observations.
- Training Architecture: CNN+Transformer achieves the lowest distillation loss and highest deployment success among the compared backbones through attention-based multi-view fusion.U-Net reconstruction is incompatible with random side masking, limiting robustness under partial observations.
- Multi-Terrain Gradient Analysis: DAgger Only preserves cross-terrain gradient alignment and reaches higher terrain levels faster than Hybrid DAgger + RL on every terrain family.Hybrid training drives pairwise gradient cosine toward 0, indicating competing terrain updates and slower convergence with lower final terrain levels.
- Real-World Locomotion: 6/10 real-world whole-course trials succeed, including 3/5 with a 2 kg payload, across slopes, stairs, stepping stones, and bidirectional traversal.Successful trials cover approximately 200 m and 2 minutes; remaining failures concentrate on foot-edge contacts and payload swing.
5 Conclusion
RPL combines terrain-specialized expert training with unified depth-based visuomotor control for multi-directional humanoid locomotion. Simulation and real-world experiments demonstrate long-horizon back-and-forth locomotion across challenging terrains.
- RPL trains terrain-specialized experts before distilling them into a single depth-based transformer policy that fuses multi-view depth inputs.
- 5× speedup over existing simulation renderers is achieved by parallel ray-casting against dynamic robot and static terrain meshes with realistic sensor effects.
- DFSV and RSM improve robustness under asymmetric multi-view observations and unseen terrain widths.
6 Limitations
RPL has two stated limitations: no real-world sideways-locomotion demonstration and no explicit active viewpoint selection for highly occluded or ambiguous scenarios.
- RPL does not demonstrate real-world sideways locomotion on discrete terrains such as stepping stones.
- Achieving high-level sideways performance remains non-trivial for both expert training and distillation.
- DFSV improves robustness with fixed cameras but does not explicitly learn active viewpoint selection for highly occluded or ambiguous loco-manipulation scenarios.
A.1 Notation Summary
This appendix section provides a notation summary for the paper.
- The section is presented as a notation summary.
- The notation summary is organized in Table 5.
- No specific notation definitions are provided in the supplied passage.
A.2 Domain Randomization
This appendix section presents the domain-randomization terms used during distillation.
- The section summarizes domain-randomization terms used during distillation.
- The domain-randomization summary is organized in Table 6.
- No individual randomization terms are specified in the supplied passage.