PaperlayerView exact revision on arXiv

Source-linked AI summary

World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain

Yuxi Liu, Lijun Han, Ziming Wang, Ao Zhang, Cong Yang, Wei Sui

arXiv:2609.02542v1cs.RO

TL;DR

Foothold-constrained terrain limits feasible contacts and leaves little recovery room, challenging policies that rely mainly on immediately visible terrain. WM-LOCO jointly trains a recurrent world model with PPO, using proprioception and a single onboard depth image to provide predictive recurrent features without explicit foothold labels. In simulation it succeeds where the baseline fails on gaps and stepping stones, while preserving stair performance and improving gait quality; onboard a Unitree G1, it achieves 93.3% average success across all three terrain classes.

  • Problem

    Foothold-constrained terrain has sparse, discontinuous, or geometrically restricted contacts, making immediately observation-based foot-placement decisions unreliable when a misstep leaves little room for recovery.

  • Method

    WM-LOCO jointly trains a recurrent world model and PPO policy, conditioning predictive recurrent features on proprioception and a single onboard depth image without explicit foothold labels.

  • Results

    WM-LOCO achieves 78.2%–100% success on gaps and stepping stones where PPO reaches 0% at every tier, improves stair gait metrics, and reaches 93.3% average hardware success across three terrain classes.

  • Takeaways & Limitations

    Predictive recurrent context supports visual humanoid locomotion across the evaluated foothold-constrained terrain classes, including onboard traversal of stepping stones, stairs, and a gap.

  • Takeaways & Limitations

    The study evaluates one Unitree G1 and one RSSM backbone; outdoor unstructured terrain, continuous narrow supports, and other morphologies remain outside its demonstrated scope.

Abstract

from arXiv · show

Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps, and on narrow stair treads. On such terrain, a single misstep often leaves little room to recover, so policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure. We ask whether a learned predictive summary of near-future observations and rewards can provide the anticipatory information required in such settings. We present World-Model-Augmented Visual Locomotion (WM-LOCO), which jointly trains a recurrent world model and a PPO policy. Conditioned on proprioception and a single onboard depth image, the world model produces a predictive recurrent feature that guides the policy, without explicit foothold labels. In simulation, WM-LOCO succeeds on gaps and stepping stones where a matched baseline fails completely, and matches the baseline's success rate on stairs while improving stride efficiency and reducing pelvis acceleration. We deploy the same policy onboard a physical Unitree G1 humanoid using onboard proprioception and a single depth stream; it traverses all three terrain classes with an average success rate of 93.3%.

1 Introduction

Foothold-constrained terrain combines sparse or geometrically restricted contacts with little room for recovery after a misstep. WM-LOCO addresses this challenge by jointly training a recurrent world model and PPO policy, outperforming a matched baseline on the hardest sparse-contact settings and transferring to hardware.

  • Motivation: Foothold-constrained terrain has sparse, discontinuous, or geometrically restricted feasible contacts, including gaps, stepping stones, and narrow stair treads.A single misstep can cause a fall, with bipeds especially vulnerable because they have fewer supporting contacts after a missed foothold.
  • Related approaches: Existing approaches address sparse foothold signals with dedicated objectives, staged training, privileged-to-visual distillation, or explicit footstep planning.Examples include separate dense and sparse critics, privileged experts distilled into depth-based students, teacher–distillation loops, and mixed-integer quadratic programming.
  • Proposed approach: WM-LOCO co-trains a recurrent world model with PPO to produce predictive context from proprioception and a single onboard depth image without explicit foothold labels.The resulting predictive recurrent feature guides the locomotion policy.
  • Simulation results: 0%: the PPO baseline's success rate on gaps and stepping stones at every difficulty tier, while WM-LOCO achieves 78.2%–100%.On stairs, WM-LOCO achieves consistently high success rates while the baseline also performs successfully.
  • Hardware validation: 93.3%: WM-LOCO's average success rate traversing stepping stones, stairs, and a gap on a physical Unitree G1 humanoid.The same policy runs onboard using proprioception and a single depth stream.

2 Method

WM-LOCO couples an RSSM world model with PPO so a predictive recurrent feature derived from proprioception and depth can guide locomotion on foothold-constrained terrain. Terrain-specific reward shaping supplies contact, boundary, and penetration signals for stairs, gaps, and stepping stones.

  • World-Model-Augmented Policy: The policy combines proprioceptive history, velocity command, depth image, and predictive recurrent feature through a mixture-of-experts backbone feeding actor and critic heads.
  • World-Model-Augmented Policy: WM-LOCO jointly trains an RSSM world model and PPO, passing only a transformed recurrent memory feature to the policy without imagined rollouts or foothold labels.The model processes proprioception and a single egocentric depth image, while gradients flow through shared encoders during one joint update.
  • Terrain-Specific Rewards: Terrain-specific shaping augments standard locomotion rewards because sparse foothold signals and early irreversible missteps make eventual-fall feedback difficult to assign.These terms are applied identically to WM-LOCO and PPO within each terrain-training setting.
  • Stair Rewards: Stair rewards penalize boundary detours and riser penetration while rewarding sequential contact with ordered stair-tread regions.The riser penalty excludes the upper 4 cm of each riser to avoid misclassifying oblique leading-edge contacts; without it, riser strikes are observed in simulation and hardware testing.
  • Gap Rewards: Gap rewards use sequential region contact, alternating-foot sole-contact credit, and overlap with the other foot, with wrong-foot landings retaining discounted learning signal.The next region activates only after sustained contact on the current region.
  • Stepping-Stone Rewards: Stepping-stone rewards activate all valid stone-top regions independently and add center-plateau shaping plus per-foot lane gating for lane-tagged stones.

3 Experiments

The experiments compare WM-LOCO with a matched PPO baseline in simulation across stairs, gaps, and stepping stones, using held-out terrain instances and success-rate and stair-gait metrics.

  • Study scope: The evaluation examines performance across foothold constraints and validates the trained policy on physical hardware.The comparison focuses on traversal rather than disturbance recovery.
  • Experimental setup: The study evaluates a simulated Unitree G1 in 8192 parallel IsaacLab environments using a single simulated head-mounted depth camera.Stairs, gaps, and stepping stones are procedurally generated, and both methods receive the same training budget.
  • Experimental comparison: WM-LOCO is compared with PPO under matched training conditions, sharing rewards, perception, proprioceptive encoding, motion prior, actor-critic architecture, and iteration budget.The baseline omits only the world-model pathway and auxiliary loss.
  • Evaluation protocol: Each terrain class is tested separately on held-out instances across Easy, Medium, and Hard tiers, with at least 50 episodes per method, terrain, and difficulty setting.Push perturbations and domain randomization are disabled, and success requires crossing the goal line within 45 seconds.
  • Metrics: Success rate is the primary metric, while stair gait quality is assessed using stride length, steps per meter, normalized mechanical energy, pelvis acceleration, and action rate.Success means the robot’s center of mass crosses the goal line within the time limit.

4 Results

WM-LOCO substantially outperforms PPO on gaps and stepping stones, remains comparable on stairs while improving gait quality, and transfers to a physical G1. Reconstruction errors decrease after recurrent-state initialization, although the analysis measures posterior reconstruction rather than open-loop prediction.

  • Success rates: WM-LOCO achieves 78.2%–100% success on gaps and stepping stones, while PPO achieves 0.0% at every difficulty level; on stairs, WM-LOCO reaches 92.0%–95.7% versus PPO’s 87.0%–91.4%.The performance gap is modest on stairs and widens on the more foothold-constrained terrains.
  • Gait quality on stairs: WM-LOCO improves stair gait quality with 15%–35% longer strides, 9%–21% fewer steps per meter, 6%–20% lower normalized mechanical energy consumption, and 24%–33% lower pelvis acceleration.These gains occur while both methods achieve similar stair success rates.
  • Failure modes: On stepping stones, WM-LOCO fails mainly through illegal footholds (23.0%), whereas PPO failures are mostly falls (61.0%) or lack of progress (36.0%).WM-LOCO falls account for 1.9% of failures aggregated over levels.
  • Failure modes: On gaps, PPO remains at 0% across difficulty levels, while WM-LOCO remains at or above 90%.The baseline’s failures often occur near the beginning of episodes.
  • Posterior reconstruction: The reconstruction analysis characterizes encoder-decoder fidelity for the observation stream, not open-loop forward-prediction quality.The observation stream includes depth and proprioception.
  • Sim-to-real evaluation: The same WM-LOCO policy runs fully onboard a Unitree G1 from proprioception and a single head-mounted depth stream, without offboard perception, a terrain map, or an additional state estimator.The deployment uses a Jetson Orin and an Intel RealSense D435 corresponding to the simulation camera model.

5 Limitations

The framework is scoped to foothold-constrained terrain dominated by discrete pads or narrow geometric constraints. Its evaluation covers one Unitree G1 robot and one RSSM world-model backbone, leaving broader generalization for future work.

  • Scope: Continuous narrow supports and stepping stones with varying elevations are outside the study’s scope.These settings combine lateral and vertical constraints and may require separate reward shaping.
  • Scope: Outdoor unstructured terrain is beyond the framework’s stated scope.
  • Platform and architecture: Results are reported on a single Unitree G1 humanoid and a single RSSM world-model backbone.The policy uses a generic predictive recurrent feature that can accommodate alternative world-model architectures.
  • Platform and architecture: Evaluating other humanoid morphologies and additional foothold-constrained tasks remains future work.

6 Conclusion

WM-LOCO combines a recurrent world model with PPO for visual locomotion on foothold-constrained terrain. It succeeds where the PPO baseline records no successful trials, while both methods succeed on stairs and WM-LOCO improves additional locomotion metrics.

  • 6 Conclusion: WM-LOCO supplies a predictive recurrent feature to an actor–critic using onboard proprioception and depth.The feature is produced by a recurrent world model jointly trained with PPO.
  • 6 Conclusion: On stepping-stone and gap classes, WM-LOCO achieves high success rates, whereas the PPO baseline records no successful trials.
  • 6 Conclusion: On stairs, both methods succeed, while WM-LOCO improves stride efficiency and reduces pelvis acceleration.
  • 6 Conclusion: The same policy runs onboard a Jetson Orin and exhibits simulation-like contact patterns across all three terrain classes in hardware experiments.

A Terrain Generation Parameters

The terrain-generation appendix specifies dimensions for simulation difficulty tiers and the real course. Each tier uses a fixed generator-grid row with parameters sampled within that row’s range.

  • Generator configuration: Each difficulty tier corresponds to generator-grid rows 1/4/9, with parameters sampled within the selected row’s range.
  • Generator configuration: Gap tiers set gap width directly, and evaluation uses terrain instances sampled independently of training.
  • Terrain dimensions: Table 2 lists terrain dimensions for each simulation difficulty tier and the real course.

B Reward Terms

The reward appendix identifies the complete configurations for the two terrain-training settings. Gap-and-stairs terms are defined in Section 2.2, while stepping-stone terms are listed separately with their weights.

  • Reward configurations: Table 3 lists the complete reward configurations for the two terrain-training settings.
  • Stair-specific rewards: Gap-and-stairs terms are defined in closed form in Section 2.2, Equations (2) to (5).
  • Stair-specific rewards: Weights wb, wr, and wt appear in the Stair-specific rows of Table 3.
  • Stones-specific rewards: Separate stepping-stone terms and their weights appear in the Stones-specific rows.

C Training and Architecture

Both methods use PPO under matched training conditions, while WM-LOCO adds a jointly optimized recurrent world-model pathway and terrain-adaptive curriculum.

  • Architecture: WM-LOCO additionally includes an RSSM pathway, an auxiliary objective, and a recurrent feature supplied to the policy.The world-model component is optimized jointly with the policy.
  • Training: Both methods are trained with PPO using the same hyperparameters and training budget.
  • Curriculum: The terrain curriculum advances independently for each environment according to exponential velocity-tracking scores.Difficulty changes depend on environment-specific tracking scores.
  • Curriculum: Environments move up one difficulty tile when both rxy exp > 0.5 and move down when rxy exp > 0.75 and ryaw...

D Additional World-Model Reconstruction Results

The world model reconstructs normalized depth and proprioceptive signals with low steady-state error after a one-tick warm-up, though depth error rises around high-relief structures.

  • Depth reconstruction: 2–7 × 10^-4: per-tick depth MSE settles into this band after one warm-up tick.It transiently rises to ∼1.2 × 10^-3 near steps 45–55 before decreasing to ∼1 × 10^-4.
  • Proprioceptive reconstruction: Below 5 × 10^-4: total per-tick proprioceptive reconstruction MSE remains near this level after warm-up.Base angular velocity has the largest channel error at ∼1 × 10^-3.
  • Error localization: Raised obstacles, the depth-window edge, and a near-field foot target concentrate the largest depth residuals, while smooth floors reconstruct with minimal error.

E Stair Gait Metrics by Difficulty

On stairs, the methods achieve similar success rates, while gait, energy, and smoothness metrics assess efficiency and motion quality across difficulty tiers.

  • Metrics: Similar success rates on stairs accompany comparisons of stride length, steps per meter, energy consumption, pelvis acceleration, and action rate.Torque relative to the actuator limit is also reported.
  • Efficiency interpretation: Longer strides and fewer steps per meter, with lower normalized mechanical energy, pelvis acceleration, and action rate, indicate a more efficient gait.
  • Per-difficulty evaluation: Table 4 reports stair gait, energy, and smoothness metrics separately for each difficulty tier.Arrows mark the preferred direction, and bold identifies the better value in each row.
Loading 2609.02542v1…