PaperlayerView exact revision on arXiv

Source-linked AI summary

REACH: An RL-based Hierarchical Humanoid Crawling Framework for Adaptable Confined Space Traversal

Shengzhang Xu, Yuning Guo, Weiyi Hu, Yifeng Xu, Zhongyu Li, He Kong

arXiv:2610.36546v1cs.ROcs.AI

TL;DR

Confined-space traversal remains challenging for approaches centered on bipedal walking, while crawling adds contact-rich dynamics and mode-switching challenges. REACH combines variable-height walking and crawling with history-conditioned adaptation, bidirectional transitions, and LiDAR-based planning. The reported results include improved crawling curriculum progression, reliable simulation-to-hardware transitions, and real-world confined-space demonstrations.

  • Problem

    Existing approaches remain predominantly bipedal, and leg-workspace limits constrain aggressive crouching; crawling also introduces contact-rich dynamics and difficult transitions between modes.

    Source: §I · p. 1

  • Method

    REACH combines variable-height bipedal walking and crawling, history-conditioned latent adaptation, a bidirectional transition policy, and a LiDAR-based planner.

    Source: §I · p. 2

  • Results

    Ablations show that history-conditioned latent adaptation improves curriculum progression and joint command tracking, particularly for crawling, while the transition policy transfers reliably to hardware.

    Source: 1) RQ1: Curriculum Progression · p. 6, §VI · p. 7

  • Takeaways & Limitations

    Real-world demonstrations show complementary walking and crawling under low clearances, with planner-issued velocity and height commands.

    Source: §I · p. 2, §VI · p. 7

Abstract

from arXiv · show

Humanoid locomotion in confined spaces requires continuous height adaptation across multiple locomotion modes, yet most existing approaches remain limited to height-adjustable bipedal walking. We present REACH, an RL-based hierarchical framework that enables a Unitree G1 humanoid to traverse height-constrained environments through variable-height bipedal walking and crawling. The learned policies regulate body height over complementary command ranges of 1.27--0.50m in bipedal locomotion and 0.50--0.20m in crawling, extending traversal beyond the clearance limits of crouched walking. To address contact-rich crawling dynamics, we employ a history-conditioned latent representation that captures terrain and interaction context from proprioceptive observations. A bidirectional transition policy further connects crouched bipedal locomotion and crawling across large kinematic and contact discontinuities. These skills are integrated with a LiDAR-based planner that converts point clouds into collision-free three-dimensional paths and locomotion commands, including proactive height targets. Real-world experiments demonstrate accurate height and velocity tracking, robust terrain traversal, reliable mode transitions and confined-space traversal with the proposed pipeline. Code and videos are available at https://aislab-sustech.github.io/REACH/.

I. INTRODUCTION

Confined-space traversal remains limited by bipedal approaches that rely on fixed height or crouching, while aggressive crouching reduces maneuverability and balance margins. REACH combines variable-height walking and crawling with contact adaptation, bidirectional transitions, and LiDAR-based planning.

A. Humanoid Locomotion under Complex Terrains

  • Bipedal approaches dominate confined-space locomotion, but fixed-height walking and leg-workspace limits on aggressive crouching restrict access to severely constrained spaces.Aggressive crouching also reduces maneuverability and balance margins.Source: §I · p. 1

I. INTRODUCTION

  • REACH combines variable-height bipedal walking, crawling, bidirectional transitions, and a LiDAR-based planner on an unmodified Unitree G1.The planner converts onboard LiDAR observations into collision-free 3D paths and proactive height commands.Source: §I · p. 2
  • REACH uses history-conditioned latent adaptation for contact-rich crawling and a bidirectional policy to connect walking and crawling across kinematic and contact discontinuities.The latent module extracts terrain and interaction context from proprioceptive histories.Source: §I · pp. 1–2

B. Humanoid Postural Adaptation for Confined Spaces

  • Crawling commands span 0.50–0.20 m, complementing variable-height bipedal walking from 1.27–0.50 m and extending traversal beyond crouched-walking clearance limits.Experiments include traversal through clearances as low as 0.40 m.Source: §I · p. 2

C. Motion Tracking and Multi-Contact Crawling

REACH uses a shared deployable actor and context-estimation interface for separate variable-height walking and crawling policies. It also uses generated motion references and one bidirectional tracker for transitions between the modes.

C. Motion Tracking and Multi-Contact Crawling

A. RL-based Variable-Height Locomotion

  • Separate walking and crawling policies share a deployable interface; a five-step proprioceptive-history encoder estimates velocity and latent terrain, contact, and dynamics context.Its decoder reconstructs subsequent observations, encouraging the latent representation to capture information unavailable from a single observation.Source: A. RL-based Variable-Height Locomotion · p. 2
  • The architecture uses privileged information only for critic training, while the actor uses proprioceptive observations and estimated context.The overview also depicts a LiDAR planner producing locomotion and height commands.Source: Figure 2 · p. 3

1) Bipedal Walking:

The walking and crawling policies share a command-conditioned learning architecture but use mode-specific contact objectives, observations, and reward shaping. Their height references and gait timing are tailored to the different postures and contact patterns.

1) Bipedal Walking:

  • Walking’s training critic augments proprioception with terrain and foot-contact information available only in simulation.The crawling critic instead represents four hand-and-knee contacts, while its actor uses latent adaptation to compensate for impacts, uncertain contacts, and modeling errors.Source: 2) Crawling · p. 3, 1) Bipedal Walking · p. 3

3) Reward Design:

  • Command tracking and regularization are shared across modes, but height reference, posture prior, and gait shaping differ.The policies share a command-conditioned architecture while addressing different contact-control problems.Source: 3) Reward Design · p. 3
  • Walking tracks head height as an overhead-clearance proxy, whereas crawling tracks base height for more stable clearance measurement with a nearly horizontal torso.Torso-pitch constraints prevent walking from satisfying its height target through upper-body tilt; in crawling, head height varies with arm placement.Source: 3) Reward Design · p. 3
  • Walking uses a periodic contact schedule, while crawling uses phase-free hand–knee coordination because limb timing varies with height, terrain, and impacts.The crawling objective favors longitudinal and lateral support without prescribing an exact stance sequence; timing terms also suppress contact chatter and asymmetric cycles.Source: 3) Reward Design · pp. 3–4

4) Training Settings:

Training uses terrain-relative height commands within each policy’s specified range and a shared terrain curriculum.

4) Training Settings:

  • Training height commands span 0.50–1.27 m for walking and 0.20–0.50 m for crawling; they target head height in walking and base height in crawling, not physical robot height.Velocity commands are resampled every 5–10 s and height commands every 2–10 s; both policies use a gradually increasing curriculum over varied terrain.Source: 4) Training Settings · p. 4

B. Bidirectional Bipedal–Crawling Transition

REACH connects independently trained bipedal and crawling policies with a generated reference motion and one bidirectional tracking policy. This avoids direct switching through dynamically sensitive intermediate states and supports adding skills without modifying existing low-level policies.

B. Bidirectional Bipedal–Crawling Transition

  • A generated reference motion and single bidirectional tracking policy connect the independently trained bipedal and crawling skills.The two policies operate around different posture and contact distributions.Source: B. Bidirectional Bipedal–Crawling… · p. 4
  • Directly switching between the policies is unreliable because neither is trained to recover from intermediate states between a deep squat and four-limb support.Those states lie between the policies' distinct posture and contact distributions.Source: B. Bidirectional Bipedal–Crawling… · p. 4
  • A new locomotion skill can be connected by generating a transition between compatible endpoint postures and training a tracker, without modifying existing low-level policies.Source: B. Bidirectional Bipedal–Crawling… · p. 4

1) Reference Motion Generation:

The transition reference is generated from crawling to a deep squat so its endpoints match the learned policies. Whole-body keyframes and contact-order constraints guide the motion, while the reverse sequence serves as a tracking reference.

1) Reference Motion Generation:

  • The crawling endpoint matches the stationary crawling policy at its maximum commanded height of 0.5 m, and the squat endpoint matches the stationary bipedal policy at its minimum height of 0.5 m.Kimodo generates the coherent transition, allowing direct control over terminal postures rather than collecting and retargeting motion-capture data.Source: 1) Reference Motion Generation · p. 4
  • The 5 s reference at 30 Hz uses whole-body keyframes to guide motion from initial crawl through compact legs-advanced and single-foot–single-knee support with hands lifted, ending in a stabilized deep squat.The text prompt specifies sequential leg advancement, arm retraction, and successive knee lift.Source: 1) Reference Motion Generation · p. 4
  • The dynamically harder crawl-to-squat motion is generated, while its reverse is used as a tracking reference and the physics-based policy learns feasible closed-loop realizations in both directions.Source: 1) Reference Motion Generation · p. 4

2) Unified Transition Motion Tracking:

A shared transition controller handles direction-dependent contact and balance changes, with phase-specific shaping and a staged policy handoff. The navigation stack combines persistent LiDAR obstacle geometry with path planning and learned locomotion commands.

2) Unified Transition Motion Tracking:

  • A single transition controller tracks the reference in both directions because contact changes and balance constraints make a one-way tracker insufficient.Training alternates reference direction across parallel environments and resamples playback direction every 2–6 s; a three-step observation history helps distinguish rising from descending.Source: 2) Unified Transition Motion Tracking · p. 4
  • The supervisor reaches the shared 0.5 m endpoint, runs the closed-loop tracker, then transfers control; phase-gated shaping targets ambiguous contact changes without constraining the full trajectory.State and dynamics randomization train the tracker to absorb handoff mismatch and disturbances; shaping addresses a kneeling local optimum that can arise near the squat endpoint.Source: 2) Unified Transition Motion Tracking · p. 4

IV. PERCEPTION AND PLANNING

  • The navigation hierarchy turns LiDAR measurements into a collision-free 3D body path, converts it into velocity and height commands, and dispatches them to locomotion or transition policies.Source: §IV · p. 4

A. Spatiotemporal Point-Cloud Representation

  • Real-world demonstrations show variable-height walking across an 80 cm-clearance obstacle, crawling through passages as low as 40 cm, and transitions into and out of low-clearance obstacles.Source: Figure 3 · p. 5
  • Fusing world-frame obstacle points from current scans with the accumulated static map retains overhang geometry that a ground-height map alone cannot represent.The fused cloud is supplied to front-end A* search and the subsequent path optimizer.Source: A. Spatiotemporal Point-Cloud… · pp. 4–5

B. Humanoid-Constrained Spatial Planning

The planner adapts an aerial path optimizer into a static spatial planner for humanoid locomotion, adding height and overhead-obstacle costs. A 3D tracker converts the path into policy commands and coordinates mode changes at the shared height boundary.

B. Humanoid-Constrained Spatial Planning

  • The planner removes dynamic-feasibility cost from the aerial objective because low-level policies handle timing, velocity tracking, and dynamic balance, then adds humanoid-specific height and overhead terms.The resulting spatial objective combines smoothness, collision avoidance, terrain-relative height regularization, and overhead traversal suppression.Source: B. Humanoid-Constrained Spatial Planning · p. 5
  • Terrain-relative height regularization suppresses unnecessary vertical oscillations in open space while allowing path lowering near overhead obstacles when cheaper than a lateral bypass.Referencing local ground elevation makes planned height consistent with the low-level policies' height commands.Source: B. Humanoid-Constrained Spatial Planning · p. 5
  • An overhead penalty rises rapidly for path points above an obstacle footprint, directing paths around solid obstacles while preserving collision-free underpasses beneath elevated structures.Together with collision avoidance, the penalty discourages aerial-planner fly-over solutions beyond humanoid stepping capability.Source: B. Humanoid-Constrained Spatial Planning · p. 5

C. 3D Path-to-Policy Interface

  • The 3D Pure Pursuit tracker converts a forward path target into planar velocity, yaw-rate, and terrain-relative height commands; requested height also selects walking or crawling at the shared 0.5 m boundary.The supervisor completes the transition before handing control to the target policy, and commands stay within each policy's training range.Source: C. 3D Path-to-Policy Interface · pp. 5–6

V. EXPERIMENTS AND DISCUSSIONS

The experiments evaluate curriculum progression, joint command tracking, and bidirectional transitions, with real-world sequences demonstrating composable walking and crawling. For variable-height locomotion, curriculum progression distinguishes how quickly policies acquire command-responsive motion on increasingly complex terrain.

V. EXPERIMENTS AND DISCUSSIONS

  • The evaluation asks how ablations affect curriculum progression, how policies jointly track velocity and height commands, and which components support transitions and sim-to-real transfer.Walking and crawling are tested in simulation; transition ablations are evaluated in simulation and the full policy is deployed on hardware.Source: §V · p. 6

A. Variable-Height Locomotion

  • Four variants per locomotion mode are trained in 8192 parallel environments for 4000 iterations, contrasting the full history-conditioned architecture with changes to idle penalties or inputs.PRIV-SCAN adds privileged terrain heights and is simulation-only.Source: A. Variable-Height Locomotion · p. 6

1) RQ1: Curriculum Progression:

  • FULL reaches walking terrain level 4 first, at approximately 1700 iterations; in crawling it reaches approximately 3.6 by iteration 4000, while PROPRIO and PRIV-SCAN do not progress beyond the initial level.NO-IDLE remains below walking level 3 and reaches approximately crawling level 3.3 only after a pronounced delay.Source: 1) RQ1: Curriculum Progression · p. 6

2) RQ2: Joint Command Tracking:

REACH’s full transition policy succeeds reliably in simulation and on the Unitree G1, while the low-level full policy provides balanced velocity and height tracking. The results support a planner–controller stack combining variable-height walking, crawling, and mode switching for confined-space traversal.

2) RQ2: Joint Command Tracking:

  • FULL provides the best joint velocity-and-height tracking in walking and combines crawling velocity errors comparable to NO-IDLE with lower lateral, yaw-rate, and height errors.PROPRIO and PRIV-SCAN can show smaller velocity or yaw-rate errors while failing to achieve commanded crouching or progress through crawling terrain.Source: 2) RQ2: Joint Command Tracking · p. 6, Figure 4 · p. 7

B. RQ3: Bidirectional Transition Robustness

  • 25/25 squat-to-crawl and 23/25 crawl-to-squat transitions succeed on hardware; FULL (SIM.) succeeds in 25/25 and 24/25 trials, respectively.The hardware bars show deployment of the unchanged full policy.Source: Figure 5 · p. 7, B. RQ3: Bidirectional Transition… · p. 7
  • In MuJoCo, NO SHAPING retains 23/25 squat-to-crawl successes but has 0/25 crawl-to-squat successes; REDUCED ROBUSTNESS succeeds in only 15/25 and 17/25 trials.The results associate phase-gated shaping with crawl-to-squat recovery and broad randomization with recovery from switching-state variation.Source: B. RQ3: Bidirectional Transition… · pp. 6–7

VI. CONCLUSIONS

  • The conclusion identifies history-conditioned latent adaptation as improving curriculum progression and joint command tracking, particularly for contact-rich crawling.It also reports reliable simulation-to-hardware transition transfer and planner-issued velocity and height commands in real-world demonstrations.Source: §VI · p. 7

APPENDIX (A) Reinforcement Learning Reward Summary

The appendix lists reward terms for variable-height bipedal walking and humanoid crawling, plus additional terms for bidirectional motion tracking.

APPENDIX (A) Reinforcement Learning Reward Summary

  • The bipedal-walking reward summary applies signed coefficients to its listed nonnegative rewards or penalties.It identifies standing and crawling reference configurations, projected horizontal gravity, and PD-requested and postsaturation torques.Source: APPENDIX (A) Reinforcement Learning… · p. 7, Table II · p. 7

(B) Transition-Policy Reward Terms

Loading 2610.36546v1…