PaperlayerView exact revision on arXiv

Source-linked AI summary

DWMP: Leveraging Dual World Models for Humanoid Obstacle Traversal

Rongjun Jin, Jianming Ma, Yue Gao

arXiv:2609.12347v1cs.RO

TL;DR

Humanoid obstacle traversal requires policies to use proprioceptive dynamics and visual obstacle geometry despite their different representational challenges. DWMP combines complementary world models for these modalities, and reports higher traversal success than teacher-student baselines alongside successful randomized-layout tests on a Unitree G1.

  • Problem

    Obstacle-traversal policies must jointly use proprioceptive observations of nonlinear robot dynamics and high-dimensional, noisy, redundant egocentric vision, which existing methods often do not represent according to their different characteristics.

    Source: §I · p. 1

  • Method

    DWMP fuses Koopman-based proprioceptive dynamics latents with RSSM-based compact stochastic depth representations for student-policy action generation.

    Source: §I · p. 1, D. Dual World Model for Obstacle… · p. 4

  • Results

    About 10% higher obstacle-traversal success than strong teacher-student baselines, with better visual reconstruction on unseen terrains and successful traversal on randomized real-world layouts.

    Source: §I · p. 1, E. Real-world Experiment · p. 6

  • Takeaways & Limitations

    DWMP was deployed on a Unitree G1 and traversed randomized Ceil, Mceilbar, and Narrow obstacle layouts despite differences from simulation layouts.

    Source: E. Real-world Experiment · p. 6, §V · p. 7

Abstract

from arXiv · show

Humanoid robots must traverse cluttered obstacle fields using onboard proprioceptive and visual observations, yet existing methods usually process multimodal observations without explicitly considering their different characteristics: proprioceptive observations are low-dimensional but governed by highly nonlinear robot dynamics, while egocentric visual observations are high-dimensional, noisy, and redundant. We propose DWMP (Dual World Model Policy), a framework that provides the actor with separate but complementary world-model representations for humanoid obstacle traversal. A Koopman-based dynamics world model lifts proprioceptive observations into a latent space where their temporal evolution is approximately linear, making the dynamics features easier for the actor to learn from. An RSSM-based visual world model compresses egocentric depth observations into compact stochastic states while preserving obstacle-related geometry. The student policy receives the fused latent representation for action generation, combining linearized proprioceptive dynamics with compressed visual perception. Experiments in simulation and on a Unitree G1 humanoid robot show that DWMP improves obstacle traversal performance over baselines and supports real-world deployment under randomized obstacle layouts.

I. INTRODUCTION

DWMP addresses the mismatch between proprioceptive dynamics and visual geometry by giving each modality a complementary predictive representation. It reports improved traversal in simulation and successful traversal on randomized real-world layouts.

I. INTRODUCTION

  • Proprioception is low-dimensional but governed by nonlinear dynamics, whereas egocentric depth is high-dimensional, noisy, and redundant, requiring modality-specific representations.The proposed motivation is to simplify dynamics features while preserving obstacle geometry during visual compression.Source: §I · p. 1
  • DWMP pairs Koopman-based linearization of proprioceptive dynamics with RSSM-based compression and prediction of depth observations, then fuses their latents for action generation.The two representations target predictable body dynamics and compact obstacle-aware perception.Source: §I · p. 1
  • About 10% higher obstacle-traversal success than strong teacher-student baselines, with better visual reconstruction on unseen terrains and successful traversal on randomized Unitree G1 layouts.The real-world tests used Ceil, Mceilbar, and Narrow obstacle layouts.Source: §I · p. 1

A. World Models for Robot Dynamics and Perception

  • Prior world models address dynamics or visual prediction, but the cited approaches do not address visually guided obstacle traversal or often combine modalities in one latent state.DWMP instead assigns Koopman linearization to proprioception and RSSM compression to depth perception.Source: A. World Models for Robot Dynamics and… · pp. 1–2

B. Obstacle Traversal Policies for Humanoid Robots

  • Existing obstacle-traversal approaches show agility but usually do not explicitly model transitions in both robot state and visual scene.The cited approaches include spatial-relation encoding, vision/blind policy switching, vision-based policies, and imitation of human demonstrations.Source: B. Obstacle Traversal Policies for… · p. 2

III. METHOD

The traversal task is formulated as a partially observable decision process with proprioceptive and depth observations. The policy outputs target joint positions, while the objective is to reach a target through obstacles while minimizing collisions.

A. Preliminaries

  • The task is a POMDP in which a policy maps observations to actions to traverse an obstacle field and reach a target while minimizing collisions.The formulation specifies observation space, action space, transition, reward, and discount factor.Source: A. Preliminaries · p. 2
  • At each timestep, observations combine proprioceptive state information with an egocentric depth image that provides obstacle geometry.Proprioception includes joint positions and velocities, gravity direction, base linear and angular velocities, and target position relative to the robot.Source: A. Preliminaries · p. 2
  • Actions specify target joint positions for lower-body PD controllers, which compute torques at a higher control frequency to track them.The controller uses proportional and derivative gain matrices.Source: A. Preliminaries · p. 2
  • The two observation modalities differ: proprioceptive states follow physical dynamics amenable to a lifted linear representation, while visual observations reflect ego-motion and 3D scene geometry.The task starts in a cluttered environment with obstacles of varying sizes and geometries and a sampled target position.Source: A. Preliminaries · p. 2

B. Koopman based Dynamic world model

The dynamics world model uses Koopman theory to represent nonlinear robot dynamics through linear evolution in a lifted observable space. Its training pipeline pre-trains from teacher interaction data before adapting encoders to student observations.

1) Koopman Operator Theory:

B. Koopman based Dynamic world model

  • The pipeline pre-trains the dual world model on teacher interaction trajectories, then encodes student observations while keeping encoders trainable and forward-propagation layers frozen.A sampled trajectory is reconstructed and predicted during pre-training.Source: Figure 1 · p. 3

1) Koopman Operator Theory:

  • The Koopman operator advances observable functions forward in time and remains linear even when the underlying state transition is nonlinear.Source: 1) Koopman Operator Theory · p. 3

2) Auto-Koopman network:

The Auto-Koopman network lifts proprioceptive states into a finite-dimensional latent space with linear transitions, while the RSSM represents visual history with deterministic and stochastic states.

2) Auto-Koopman network:

  • The Auto-Koopman encoder maps states into a finite-dimensional latent space with a linear transition matrix, and a decoder can recover the original state.Repeated application of the transition matrix supports long-horizon prediction of proprioceptive states.Source: 2) Auto-Koopman network · p. 3
  • The observation function transfer layer transforms each input dimension into corresponding observation-function values before encoding.Source: 2) Auto-Koopman network · p. 3

1) Recurrent State-Space Model:

  • The RSSM maintains a deterministic recurrent state summarizing past observations and actions, alongside a stochastic state for the instantaneous latent representation.Source: 1) Recurrent State-Space Model · p. 3
  • During training, the RSSM posterior incorporates the current observation, while imagination uses the prior; training includes observation reconstruction, reward prediction, and posterior-prior KL regularization.Source: 1) Recurrent State-Space Model · p. 3

2) Depth Dreamer network:

DepthDreamer uses an RSSM to encode depth observations into stochastic states, update recurrent state, and reconstruct depth while predicting reward.

2) Depth Dreamer network:

  • A convolutional VAE encoder processes each depth map and outputs the mean and log-variance of the posterior over stochastic states.Source: 2) Depth Dreamer network · p. 4
  • A GRU updates the deterministic recurrent state, and a prior network predicts the stochastic-state prior from the recurrent state.Source: 2) Depth Dreamer network · p. 4
  • Training samples stochastic states from the posterior, while imagination samples from the prior; the concatenated states reconstruct depth and predict reward under the Eq. (13) loss.Source: 2) Depth Dreamer network · p. 4

D. Dual World Model for Obstacle Traversal Policy

DWMP gives the traversal policy structured, predictive inputs by pairing a linearized proprioceptive representation with a compact stochastic representation of depth images.

D. Dual World Model for Obstacle Traversal Policy

  • The dual world model combines a Koopman model that maps proprioception into a linear latent space with DepthDreamer, which compresses depth images into stochastic representations.Together, they provide the traversal policy with structured and predictive inputs.Source: D. Dual World Model for Obstacle… · p. 4

1) Policy Input from Dual-World Model Representations:

  • The policy encodes proprioceptive observations as a temporally predictable latent state and depth images as a compact representation of surrounding obstacle structure.Source: 1) Policy Input from Dual-World Model… · p. 4
  • The fused representation supports anticipating near-future movement through Koopman states and perceiving upcoming obstacles through compressed depth features.The policy network outputs a Gaussian distribution over target joint positions.Source: 1) Policy Input from Dual-World Model… · p. 4

2) Co-Training Loop:

DWMP pre-trains its two world models on teacher exploration data, then co-trains them with the student policy using student interaction data. The student acts from fused world-model representations and learns through reward optimization and teacher behavior cloning.

2) Co-Training Loop:

  • Teacher exploration trajectories jointly pre-train the Koopman and visual world models to reconstruct proprioceptive states and depth images and predict temporal evolution.Teacher training uses privileged environment information, and obstacle collisions or reaching the target trigger resets that produce trajectories of varying lengths and outcomes.Source: 2) Co-Training Loop · p. 4
  • During student training, the policy receives fused world-model representations without privileged information, while the models are updated on student replay data.Student rollouts store transitions in a replay buffer used to update the Koopman model, RSSM, and policy.Source: 2) Co-Training Loop · pp. 4–5
  • Student interaction data updates the world models online, adapting them to the student’s action distribution and states that teacher exploration may not have visited.The student’s early training can visit states absent from the teacher’s exploratory data.Source: 2) Co-Training Loop · p. 5
  • The student policy objective combines cumulative task reward with behavior cloning from the teacher.The two terms are reward maximization and a behavior-cloning loss.Source: 2) Co-Training Loop · p. 5

A. Experiment Setup

The experiments use a Unitree G1 humanoid in MuJoCo, where obstacle-terrain episodes command the robot to reach a target position.

A. Experiment Setup

  • Experiments use a Unitree G1 humanoid in MuJoCo, with generated obstacle terrains and an episode goal of reaching a specified target position.Source: A. Experiment Setup · p. 5

B. Effect of Two-Stage Training on Visual Reconstruction

The visual-representation experiment compares methods on seen and unseen obstacle terrains, while the two-stage-training experiment tests whether teacher-data pretraining improves visual reconstruction. The reported comparisons favor Dreamer-Koopman on unseen terrains and warm-up initialization for final reconstruction loss.

B. Effect of Two-Stage Training on Visual Reconstruction

  • Phase 1 warm-up reaches a distinctly lower final visual reconstruction loss than training without warm-up, although both settings reduce loss during fine-tuning.Each validation value averages compression-reconstruction loss over one episode.Source: B. Effect of Two-Stage Training on… · p. 5, Figure 4 · p. 5

C. Visual Representation Capability Experiment

D. Obstacle Traversal Capability Experiment

In simulation, DWMP achieves higher success rates on complex terrains than teacher-student distillation methods, and its Koopman representation is associated with more stable, smoother locomotion. Unitree G1 tests show successful traversal of randomized obstacle layouts; the conclusion identifies dynamic environments and long-horizon planning as future directions.

D. Obstacle Traversal Capability Experiment

  • DWMP achieves higher success rates on more complex terrains than teacher-student distillation methods, while using a simpler training procedure.The conclusion also reports improved representation quality over teacher-student baselines in simulation.Source: D. Obstacle Traversal Capability… · p. 6, §V · p. 7
  • With the Koopman dynamics representation, locomotion is more stable and smooth; Fig. 7 compares Y-axis trajectories with and without it on Bar terrain.The figure caption identifies the compared trajectories and terrain, but does not state a trajectory outcome.Source: D. Obstacle Traversal Capability… · p. 6, Figure 7 · p. 6

E. Real-world Experiment

  • The Unitree G1 successfully traverses randomly arranged Ceil, Mceilbar, and Narrow obstacles despite layouts differing from simulation, suggesting performance is not tied to memorized placements.Table III reports real-world traversal pass rates and includes a HumanoidPF direct-distillation baseline, but the supplied passages give no values.Source: E. Real-world Experiment · p. 6, Table III · p. 6

V. CONCLUSION

  • The conclusion names more diverse dynamic environments and tighter integration with long-horizon motion planning as future work.It also reports improved traversal success and representation quality in simulation, and deployability on randomized real-world layouts.Source: §V · p. 7
Loading 2609.12347v1…