PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Terrain-Adaptive Humanoid Locomotion on Granular Terrain

Junnosuke Kamohara, Feiyang Wu, Andy Ningan Zong, Daniel I. Goldman, Yashwanth Nakka, Seth Hutchinson, Ye Zhao

arXiv:2609.10286v1cs.RO

TL;DR

Humanoid locomotion on granular terrain is difficult because foot-terrain interaction dynamics are complex and poorly captured by existing approaches. The paper introduces a 3D RFT contact model and terrain-adaptive teacher-student RL, demonstrating multi-gait locomotion across simulated and real-world granular terrains.

  • Problem

    Granular foot-terrain interaction is difficult to model, while rigid and simplified contact models inadequately represent sinkage, slippage, and tangential forces.

  • Method

    The method combines a 3D RFT-based granular contact model with teacher-student RL and a VAE-based terrain encoder for adaptive locomotion.

  • Results

    The method achieves higher success rates and tracking accuracy than rigid-contact baselines in simulation, while enabling successful running and improved velocity tracking in hardware experiments.

  • Takeaways & Limitations

    The framework supports adaptive humanoid walking, running, and jumping across diverse simulated and real-world granular terrains.

Abstract

from arXiv · show

Humanoid locomotion on granular terrain remains a significant challenge due to its complex foot-terrain interaction dynamics that are difficult to model. Existing approaches either ignore granular contact dynamics or incorporate simplified normal force models with heuristic tangential components. In this work, we present a physics-grounded granular contact model based on three-dimensional resistive force theory (3D RFT) and efficiently simulate granular terrain for reinforcement learning (RL) training. Unlike traditional rigid contact models and simplified granular contact models with ad-hoc heuristics, our contact solver produces physically accurate granular intrusion dynamics without resorting to heuristics. It captures realistic penetration and tangential drag during training, enabling the policy to learn behaviors that transfer reliably to real-world granular terrain where rigid contact models fail. To adapt to varying terrain conditions, we train a terrain-adaptive locomotion controller via teacher-student RL, using a variational autoencoder to encode terrain information into a compact latent representation. Simulation studies using material point method (MPM) with NVIDIA Newton demonstrate that our method generalizes to unseen granular terrains, achieves a significantly higher success rate than baselines, and demonstrates zero-shot terrain identification and adaptation. We further validate our approach through extensive hardware experiments across diverse real-world granular terrains including basalt, dry sand, and beach sand. To the best of our knowledge, this is the first demonstration of agile humanoid locomotion on real-world granular terrain. Project page: https://humanoid-gm-locomotion.github.io/HUMANOID-GM/

I. INTRODUCTION

Granular terrain makes humanoid locomotion difficult because foot-terrain forces are complex and difficult to model. The paper combines 3D RFT contact modeling with terrain-adaptive teacher-student RL for multi-gait locomotion across granular terrains.

  • Granular terrain is challenging because its contact forces are highly nonlinear and difficult to model, complicating balance and walking control.
  • 3D RFT is integrated into the simulation pipeline to model granular foot-terrain contact without relying on simplified contact heuristics.
  • The terrain-adaptive policy changes swing-foot clearance and gait frequency as terrain stiffness varies.
  • The framework demonstrates circular walking, running, and jumping over simulated and real-world granular terrain.

II. RELATED WORK

Prior granular-locomotion methods use terradynamics in model-based control or reinforcement learning with domain randomization. Existing simulation choices trade physical fidelity against the computational cost required for large-scale policy training.

  • A. Granular Contact Modeling: Rigid-body simulators miss granular sinkage and slippage, while DEM and MPM provide greater fidelity but are too computationally burdensome for large-scale RL training.
  • Model-based approaches incorporate granular contact models as gait constraints, contact-force constraints, or system dynamics in trajectory optimization.
  • Learning-based approaches often use domain randomization and privileged learning without explicitly modeling intricate terrain dynamics.

C. Terrain-Adaptive Locomotion

The framework combines a 3D RFT granular contact model with terrain-adaptive teacher-student RL, using terrain information to support humanoid locomotion across varying granular conditions.

  • Granular contact model: The terrain model predicts contact force and moment from penetration depth, velocity angle, and orientation.
  • Granular contact model: 3D RFT models granular foot contact by deriving resistive forces from penetration, surface geometry, velocity direction, and material properties.The formulation incorporates normal and tangential components through a friction-cone modification.
  • Granular contact model: The RFT function is re-expressed in an orthogonal basis using characteristic angles that describe surface orientation, velocity direction, and twist.The characteristic-angle functions are fitted to plate-intrusion data generated with MPM simulation.
  • Whole-body integration: Contact forces and moments are accumulated over active foot surface points and applied as a net wrench in whole-body dynamics to reduce simulation cost.The net wrench consists of the summed forces and moments from penetrating grid points.
  • Terrain-adaptive policy: Terrain adaptation uses a privileged teacher with a VAE terrain encoder before distilling its behavior into a deployable student policy.The teacher receives privileged terrain and robot information, while the student is trained for deployment with restricted observations.

IV. METHOD

This section introduces the terrain-adaptive reinforcement-learning framework built on the granular contact model.

  • IV. METHOD: The terrain-adaptive RL framework builds on the granular contact model introduced previously.

A. Terrain-Adaptive Reinforcement Learning

The controller uses teacher-student RL with a terrain encoder to condition locomotion on terrain information, then distills the teacher into a deployable student policy. A VAE shapes the terrain latent through reconstruction, while student training matches teacher actions and representations.

  • Teacher-Student RL: Teacher-student RL trains a privileged teacher and distills it into a deployable student policy conditioned on inferred terrain information.The teacher uses privileged observations and terrain properties, while the student learns from teacher actions and latent embeddings.
  • Terrain Representation: A terrain encoder produces a latent representation that conditions the policy and captures terrain information.The teacher encoder receives privileged observations and proprioceptive history; the student encoder imitates its output.
  • Terrain Representation: The VAE trains the terrain representation with an explicit reconstruction objective rather than relying only on policy-gradient loss.The decoder reconstructs a normalized media-dependent scaling coefficient from the latent variable.
  • Loss Function: The teacher combines PPO and VAE losses, while the student combines PPO, action distillation, and representation-matching losses.Representation matching aligns teacher and student encoders in latent and decoded scaling-coefficient spaces.

B. Environment Design

The environment exposes proprioceptive observations and command information to the policy, with actions defined as whole-body joint-position offsets. Training uses PPO, high-throughput soft-contact simulation, domain randomization, and a curriculum for progressively harder locomotion.

  • Observation and Action Space: The policy observes proprioception, projected gravity, command velocity, and previous actions, while privileged inputs include terrain and contact information.Proprioceptive history supports the deployable student, whereas privileged observations support teacher training.
  • Observation and Action Space: Actions are desired whole-body joint-position offsets from a nominal pose.The action vector specifies joint-position changes relative to q0,j.
  • Implementation Detail: PPO trains the teacher policy, using separate multilayer actor and critic networks and temporal convolution for the student encoder.Soft contact simulation runs at hundreds of thousands of frames per second on one RTX 4090 GPU.
  • Training: Training uses 4096 parallel agents, 20,000 iterations for both teacher and student, and 50 Hz RL with 200 Hz low-level control and simulation.The student is trained with DAgger together with PPO.
  • Training: Domain randomization and a curriculum progressively increase command difficulty while shifting the robot from hard contact toward granular terrain.Command velocity and tracking-reward weight are ramped during the first 15,000 iterations.

V. RESULTS

The proposed control system is evaluated for robust walking and running across multiple granular terrains in simulation and real-world experiments.

  • The control system is evaluated for robust walking and running on different granular terrains in simulation and real-world experiments.

A. Simulation Evaluation

Simulation evaluation compares contact-model fidelity, locomotion policies, terrain adaptation, and jumping under granular-terrain conditions. Across these tests, 3D RFT more closely matches MPM dynamics, supports stronger locomotion policies, and enables jumping where rigid contact fails.

  • Evaluation setup: The evaluation uses NVIDIA Newton’s MPM solver for deformable sand and reports success rate, velocity tracking error, and yaw tracking error.The study also evaluates walking, running, turning, and robustness to unmodeled payload.
  • Contact-model fidelity: 3D RFT achieves the closest horizontal travel distance, sinkage, and x/z GRF profiles to MPM, while producing lateral forces naturally.Heuristic-RFT and Cone-RFT-Multi travel substantially farther, and Heuristic-RFT has no lateral force response.
  • Terrain evaluation: Across basalt, sand, and poppy seed, 3D-RFT policies achieve substantially higher success rates and tracking accuracy than rigid-contact policies, while Ours has the best tracking accuracy overall.Rigid-policy success drops sharply as evaluation terrain becomes softer.
  • Dynamic jumping: PPO-3D-RFT successfully achieves jumping on granular terrain, whereas PPO-Rigid fails to compensate for ground slippage before flight.The jumping controller is trained with imitation RL and an assistive wrench from ZEST.

B. Hardware Experiments

Hardware experiments show that 3D-RFT-based controllers support stable locomotion on granular terrain, while the teacher-student policy improves velocity tracking and ankle compliance.

  • Experimental setup: Hardware evaluation uses a Unitree G1 with joint targets sent at 50 Hz across basalt, beach volleyball sand, and beach terrain.Episodes count as failures when the robot cannot remain upright without human intervention.
  • Hardware performance: At commanded speeds of 1.5–2.5 m/s on dry sand, Rigid fails above 1.5 m/s, while 3D-RFT and Ours remain stable and Ours achieves the best velocity tracking.On basalt, Rigid fails at all commanded velocities, whereas 3D-RFT and Ours remain robust.
  • Compliant ankle control: The proposed policy produces lower peak ankle pitch torque than Rigid and 3D-RFT during 1.5 m/s walking, indicating more compliant ankle control.The authors associate this behavior with terrain-stiffness estimation and reduced stamping motion.

C. Locomotion under Terrain Transition

During transitions across granular layers, the teacher-student policy estimates terrain stiffness and raises swing-foot clearance as the terrain softens, preserving clearance on hardware.

  • Simulation transition: Swing-foot clearance increases from about 9 cm on stiff terrain to 16 cm on softer layers as decoded terrain stiffness drops.The adaptation is evaluated in MPM simulation across basalt, dry sand, and poppy-seed layers.
  • Hardware transition: On hardware, Ours maintains positive foot clearance during transition, whereas PPO repeatedly drops below the ground reference and exhibits reactive torque spikes.The bounded torque profile of Ours accompanies terrain-stiffness estimates that track the changing terrain.

VI. CONCLUSION

The paper combines 3D-RFT granular contact modeling with terrain-adaptive teacher-student reinforcement learning and validates adaptive locomotion in simulation and hardware.

  • Conclusion: The proposed framework combines a 3D RFT-based granular contact model with a terrain-adaptive teacher-student policy for humanoid locomotion.The conclusion identifies explicit granular contact modeling and terrain estimation as central components.
  • Future work: Future work extends the method to rough terrain such as slopes and wavy terrain and studies energy-efficient walking gaits on sand.
Loading 2609.10286v1…