Source-linked AI summary
Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids
Efe Ongan, Chong Zhang, Boyang Sun, Andrei Cramariuc, Cesar Cadena, Marco Hutter
TL;DR
Humanoids must perceive thin, overhanging structures while executing agile, accurate whole-body motions, a challenge studied through monkey-bar traversal. The paper uses raw solid-state lidar with attention and recurrent memory, phase-scheduled distillation from privileged experts, and hardware-aware transfer modeling. On hardware, the system completed full traversal in 14 of 15 trials and also supported ducking beneath 2 cm obstacles.
Problem
Sparse 3D traversal requires humanoids to perceive thin, overhanging geometry while executing agile, accurate whole-body motions.
Method
A perceptive controller processes raw solid-state lidar with attention and recurrent memory, distills phase-scheduled privileged experts, and models lidar, battery, and actuator effects for hardware transfer.
Results
The policy completed the full jump-up–brachiation–jump-down sequence in 14 of 15 hardware trials and reached 0.5 m s−1 brachiation speed, while a separate policy cleared 2 cm obstacles.
Takeaways & Limitations
Raw lidar perception supported explosive contact-accurate traversal and transferred to ducking under thin overhanging obstacles using the same perception backbone.
Takeaways & Limitations
The system remains limited to a small set of separately trained task policies, and robustness to substantially more diverse geometry remains un demonstrated.
Abstract
from arXiv · showhide
Traversing sparse 3D structures requires humanoid robots to perceive thin, overhanging geometry while executing agile, accurate whole-body motions. We study this problem through monkey-bar traversal, where the robot must jump to the structure, traverse it through sparse bar interactions, and land safely. For this task, we present a reinforcement-learning-based perceptive control system that operates directly on observations from a head-mounted solid-state lidar. To extract task-relevant geometry from the sparse returns, the policy consumes the raw lidar scan through an attention-based encoder with recurrent memory. This policy is obtained by a phase-scheduled teacher- student pipeline that combines privileged experts for jumping up, brachiating, and jumping down. For transfer to hardware, we model lidar noise, battery-voltage sag, and actuator thermal limits, and equip the humanoid with passive hook end-effectors for robust bar interaction. On hardware, the resulting policy completes the full jump-up->brachiation->jump-down sequence in 14 of 15 trials across three bar configurations and reaches brachiation speeds up to 0.5 m/s. Beyond brachiation, the same perception backbone supports a separately trained policy that ducks beneath thin overhead obstacles with 2 cm cross-sections.
I. INTRODUCTION
The paper addresses agile humanoid traversal of sparse 3D structures, where thin bars demand fine-grained perception, precise contact sequences, and whole-body control. It combines raw-lidar perception, phase-scheduled distillation, hardware-aware modeling, and passive hooks to demonstrate traversal and ducking.
- Sparse 3D traversal remains difficult because elevation maps lose thin or overhanging geometry, while voxel grids become costly at higher resolution.
- Monkey-bar brachiation stresses fine-grained bar perception, precise contact exploration, and agile whole-body control under hardware limits.
- The system extracts task-relevant returns from raw head-mounted solid-state lidar using attention and recurrent memory, then distills privileged subtask experts into one perceptive student.
- The contribution includes phase-scheduled multi-teacher distillation with PPO post-refinement for a single deployable policy.
- Hardware transfer models battery-voltage sag, actuator thermal limits, and lidar noise, while passive hooks support robust interaction during explosive maneuvers.
- The same perception backbone also supports ducking beneath thin overhead obstacles without architectural modifications.
II. RELATED WORK
The related work spans intermediate spatial representations, raw-sensor locomotion, privileged distillation, sim-to-real modeling, and dynamic traversal. This paper combines these directions for sparse-structure brachiation with phase-aware teacher scheduling and sensor-specific transfer modeling.
- Perceptive locomotion: Elevation maps discard overhangs and thin structures, whereas voxel grids recover full 3D geometry at resolution-dependent memory and compute cost.
- Perceptive locomotion: Raw exteroceptive inputs have supported parkour, recurrent depth encoding, and humanoid collision avoidance, while sparse-structure traversal remains the target here.
- Privileged distillation: Teacher–student training separates exploration from partial observability, and prior methods have distilled specialists with or without later reinforcement-learning fine-tuning.
- Privileged distillation: This work schedules the active teacher by task phase within one episode and normalizes advantages per phase.
- Sim-to-real modeling: Prior sim-to-real work models actuation and lidar nonidealities; this paper adapts battery-voltage and actuator-thermal proxies to explosive humanoid maneuvers.
- Dynamic traversal and brachiation: Dynamic traversal research includes model-based humanoid ladder climbing, learned quadruped climbing, perceptive humanoid ladder climbing, and specialized-platform brachiation.
III. HARDWARE
The hardware platform uses passive hooks and a head-mounted solid-state lidar to tolerate dynamic bar contacts and perceive thin structures during fast whole-body motion. The system overview connects these hardware choices to the learned control pipeline.
- A. Passive Hook End-Effectors: The end-effector must carry full body weight under swinging loads, tolerate placement error, release without large arm torques, and survive impacts.
- A. Passive Hook End-Effectors: The passive hook replaces the hand with a single water-jet-cut stainless-steel plate.
- A. Passive Hook End-Effectors: A 60 mm circular opening is wide relative to 1–3 cm bar radii, providing tolerance to contact-placement error in the swing plane.
- A. Passive Hook End-Effectors: Wrist-yaw rotation releases the hook without lifting the body, reducing actuator effort and helping maintain thermal limits; symmetric geometry enables bidirectional traversal.
- B. Head Mounted Solid-State Lidar: The RoboSense E1R provides a dense 192 × 144 scan over a 120°× 90° field of view, with an integrated IMU and reduced motion-induced scanning distortion.
- System overview: The system overview comprises privileged subtask teachers, a phase scheduler, a grid-structured attention encoder, GRU memory, action and auxiliary heads, and multi-teacher distillation.
IV. LEARNING APPROACH
The learning approach uses privileged reinforcement-learning teachers to discover contact modes, then trains one onboard-sensing student through phase-scheduled distillation and PPO refinement. Three teachers specialize in brachiation, jumping up, and jumping down.
- Learning approach: Privileged experts with ground-truth bar positions discover contact modes for jumping up, brachiating, and jumping down.
- Learning approach: A single student observes only onboard sensing and reproduces expert behavior from raw lidar through phase-scheduled distillation followed by PPO refinement.
- Learning approach: The three privileged teachers specialize separately in brachiation, jumping-up, and jumping-down subtasks.
- Learning approach: The brachiation teacher uses goal-position commands, whereas the jumping teachers use contact-based rewards to encourage distinct contact modes without human motion priors.
C. Training Terrain and Curriculum
Training uses procedurally randomized ladder terrains and curricula to develop specialized privileged teachers, then distills them into one perceptive student through scheduled imitation and reinforcement learning.
- C. Training Terrain and Curriculum: 53Jumping curricula increase ladder height, while brachiation curricula increase bar spacing and reduce bar width; random objects add spatial disturbances.
- C. Training Terrain and Curriculum: The reward combines task-specific teacher terms with shared limit terms that penalize thermal load, battery voltage, limb torque budgets, and soft joint-limit violations.The limit group encourages physically plausible motion while protecting hardware.
- C. Training Terrain and Curriculum: The active teacher is scheduled by task phase so its action and reward are forwarded to the student.This organizes the transition across jumping-up, brachiation, and jumping-down behaviors within a single episode.
- C. Training Terrain and Curriculum: Distillation proceeds from DAgger behavior cloning, through critic warmup, to regularized PPO with a decaying behavior-cloning anchor.Advantages and value losses are standardized or weighted per phase to prevent differing reward scales from dominating updates.
F. Policy Architecture
The policy combines proprioception, commands, and raw lidar returns with grid-preserving attention and recurrent memory, producing joint-position targets for whole-body control.
- F. Policy Architecture: The student combines proprioception, a planar goal and jump-down trigger, and a 4-frame history of lidar observations.Lidar returns are refreshed at 10 Hz and use different spatial crops for brachiation and ducking.
- F. Policy Architecture: The lidar observation retains the native 2D scan grid, with each node containing hit position relative to the sensor and range.Brachiation crops the upper-half workspace, while ducking retains the full field of view.
- F. Policy Architecture: An attention-based map encoder processes the grid directly, and a GRU integrates perception features with proprioceptive and command embeddings.The GRU hidden state is decoded by an MLP into the action.
- F. Policy Architecture: Hardware measurements show summed mechanical joint power peaking at 2.11 kW during jump-up while battery voltage reaches a minimum of 34.7 V.Eighteen low-torque actuators approach their limits during voltage sags.
G. Auxiliary Guidance Losses
Auxiliary and transfer mechanisms strengthen recurrent perception while modeling battery, thermal, and sensor effects for robust hardware deployment and reuse beyond brachiation.
- G. Auxiliary Guidance Losses: An auxiliary decoder predicts the closest bar’s relative position and orientation from the GRU hidden state.This provides a guidance signal for the recurrent perceptive backbone during distillation.
- G. Auxiliary Guidance Losses: The standalone ducking policy reuses the AME-2 encoder and GRU unchanged while consuming raw lidar observations under the lidar noise model.It is separately trained from a privileged height-map teacher for avoiding overhanging obstacles.
- G. Auxiliary Guidance Losses: Sim-to-real randomization models battery-voltage sag, actuator thermal limits, and lidar noise, targeting physical effects that dominate near platform limits.These parameters are randomized according to Table II.
- G. Auxiliary Guidance Losses: Battery voltage follows a first-order recovery lag driven by summed joint-torque magnitude, while voltage ratio rescales motor stall torque and no-load speed.Parameters are tuned to real jumping logs; voltage penalties prevented brownouts in hardware trials.
- G. Auxiliary Guidance Losses: The thermal model uses a leaky integrator for each low-torque actuator, charging with load ratio and leaking toward zero.A sustained load ratio of 0.2 or higher saturates the integrator.
- G. Auxiliary Guidance Losses: The same perception backbone supports ducking beneath a 2 × 2 cm overhead wooden slat at 1.2 m clearance.This demonstrates reuse of the perception setup for a task beyond brachiation.
B. Lidar Noise Model
The lidar noise model targets sensor-specific ray-cone artifacts and missing returns at depth discontinuities, using both lightweight training simulation and higher-fidelity validation.
- B. Lidar Noise Model: Two E1R noise models are used: a lightweight model for training throughput and a higher-fidelity model for sim-to-sim validation.Their parameters are identified from the sensor rather than taken from generic assumptions.
- B. Lidar Noise Model: Ray-cone divergence is identified as the dominant hardware distortion, producing edge-bleed or mixed-pixel artifacts at depth discontinuities.The simulated models reproduce these artifacts observed on hardware.
- B. Lidar Noise Model: The training-time raycaster dithers each direction within a 0.625° beam cone, applies Gaussian range noise with σ = 2 cm, and injects edge-based dropout and background mixing.These operations mirror missing returns at depth discontinuities.
- B. Lidar Noise Model: The model additionally randomizes mounting calibration, corrupts 1% of returns, freezes frames with probability 0.1, and gates ranges to 0.3–1.5 m.Invalid returns are masked as −1.
C. Sim-to-Sim Validation Model
The sim-to-sim validation model reproduces key solid-state lidar effects, including beam divergence, finite pixel footprints, inverse-square fusion, edge dropout, and spurious returns.
- Sensor model: Per-sample lidar hits are fused with inverse-square weighting so nearer surfaces within the beam dominate the return.
- Sensor model: Each pixel is rendered with 16 rays inside a 0.625° beam-divergence cone at 10 Hz to model finite sensor footprints.The model reproduces ray-divergence edge-bleed artifacts observed on hardware.
- Sensor model: Depth-discontinuity pixels drop to no-return with probability increasing linearly with the neighboring range gap and saturating beyond a threshold.Edge dropout peaks at 0.5 for gaps above 0.3 m.
VI. EXPERIMENTS
Experiments evaluate encoder and auxiliary-supervision choices, sim-to-sim transfer under an unseen sensor model, and robustness across rung heights and overhead-obstacle geometries.
- Experimental questions: The experiments test encoder and auxiliary-supervision choices, transfer to an unseen physics and sensor model, and hardware performance on traversal and ducking.
- Encoder and auxiliary-loss ablation: The AME-2 attention encoder achieves the lowest BC loss in every distillation phase with roughly an order of magnitude fewer parameters than the CNN.It uses two orders fewer parameters than the MLP, while the blind student performs worst.
- Encoder and auxiliary-loss ablation: Adding centerline auxiliary loss lowers BC loss in every phase, indicating that explicit geometric supervision aids the recurrent perception backbone.
- Sim-to-sim validation: 70%–90% success is maintained across rung heights from 1.65 m to 1.90 m, with moderate degradation as the first bar nears the jump limit.
- Sim-to-sim validation: 100% ducking success is achieved for cylindrical bars 0.01–0.05 m in diameter across clearances of 1.1–1.5 m whenever the robot fits underneath.
C. Real-World Results
On hardware, the humanoid autonomously executes the complete jump-up–brachiation–jump-down sequence and separately clears thin overhead obstacles, while the paper notes limits in policy and geometry diversity.
- Full traversal: 14 of 15 hardware trials succeed, yielding a 93% full-sequence success rate across three ladders, including structures that shift under load.The single failure occurred when the hook did not advance to the next bar after a successful jump-up.
- Full traversal: Brachiation speed reaches 0.5 m s−1 during the full traversal sequence.
- Ducking: The separately tested ducking policy clears 2 cm × 2 cm wooden bars in random unseen orientations without contact and recovers to a stable stance.A successful crossing is shown with 1.2 m minimum clearance; ten randomly placed-bar passes are reported.
- Limitations: The framework’s robustness remains limited to a small set of separately trained task policies and substantially more diverse geometry remains un demonstrated.