PaperlayerView exact revision on arXiv

Source-linked AI summary

DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model

Jie Yin, Xingyu Lai

arXiv:2608.22278v1cs.RO

TL;DR

Vision-based humanoid loco-manipulation is difficult under partial observability, contact-rich dynamics, and long horizons. DreamMimic distills privileged teachers using predictive world-model features, auxiliary interaction supervision, and performance-conditioned guidance, improving tracking-based performance over vision-based baselines without online privileged interaction states at deployment.

  • Problem

    Vision-based whole-body humanoid loco-manipulation lacks robust handling of partial observability, contact-rich dynamics, and long-horizon learning from high-dimensional visual inputs.

  • Method

    DreamMimic repurposes a Dreamer-style RSSM for predictive latent dynamics and multi-step supervision, adds interaction-aware auxiliary heads, and adaptively schedules teacher guidance with PCG.

  • Results

    Experiments on OMOMO and BEHAVE show improved tracking-based loco-manipulation performance over vision-based baselines without exposing online privileged interaction states at deployment.

  • Takeaways & Limitations

    World models provide a mechanism for stabilizing visual policy distillation in contact-rich humanoid behaviors.

Abstract

from arXiv · show

Vision-based whole-body loco-manipulation on humanoid robots is challenging due to partial observability, contact-rich dynamics, and the difficulty of learning long-horizon behaviors from high-dimensional visual inputs. We present \href{https://github.com/DreamMimic/DreamMimic}{DreamMimic}, a framework that distills privileged teacher policies into vision-based humanoid controllers via world-model-assisted distillation. Instead of using a Dreamer-style RSSM for planning, we repurpose it to learn predictive latent dynamics that serve as both a representation space and an action-conditioned multi-step supervision signal, while exposing compact predictive features to the student policy to reduce long-term drift. Beyond standard reconstruction objectives for proprioceptive and visual observations, we add auxiliary prediction heads for privileged state, contact, object state, and reward estimation. These heads provide additional supervision related to agent--object interaction and task progress, encouraging the latent representation to retain signals that are useful for contact-rich loco-manipulation. We further introduce Performance-Conditioned Guidance (PCG), a reward-driven adaptive distillation schedule that computes performance scores for both teacher and student to dynamically balance guidance and exploration. PCG prevents both premature teacher annealing and excessive teacher interference in challenging visual settings. Experiments on OMOMO and BEHAVE show improved tracking-based loco-manipulation performance over strong vision-based baselines, without exposing online privileged interaction states to the student at deployment. Qualitative simulations further examine morphology and simulator changes. These results suggest that world models can provide a useful mechanism for stabilizing visual policy distillation in contact-rich humanoid behaviors.

I. INTRODUCTION

DreamMimic addresses the difficulty of vision-based humanoid loco-manipulation by distilling privileged teachers into a visual student with predictive world-model supervision. Its adaptive guidance and interaction-aware objectives improve tracking-based performance across tasks and settings.

  • Vision-based humanoid loco-manipulation remains difficult because of discontinuous contact dynamics, high-dimensional actions, and partial observability.
  • Existing visuomotor methods often lack temporally structured predictive representations, leaving long-horizon contact-rich behavior vulnerable to compounding errors.
  • DreamMimic repurposes a Dreamer-style RSSM for predictive state representations and action-conditioned multi-step supervision instead of planning.
  • Auxiliary reward, contact, and object-dynamics prediction heads shape representations toward interaction-rich behaviors, while PCG adapts teacher involvement using relative performance.
  • Experiments on OMOMO and BEHAVE report consistent performance improvements, with qualitative evidence across Unitree G1 morphology and Isaac Sim.

II. RELATED WORK

Prior work has enabled increasingly complex simulated whole-body behaviors, but deployment remains constrained by privileged observations and unstable direct visual reinforcement learning. DreamMimic combines teacher–student distillation, auxiliary stabilization, PCG, and a reference-buffer curriculum to address these challenges.

  • Physics-based imitation and reinforcement learning have produced high-fidelity motion reproduction and increasingly complex simulated whole-body skills.
  • Transfer to real robots remains challenging because successful methods often rely on privileged contact, task-state, or object-pose observations unavailable during deployment.
  • Direct visuomotor reinforcement learning is unstable with high-dimensional observations and delayed credit assignment, while latent-dynamics methods improve pixel-based learning efficiency.
  • DreamMimic combines DAgger, auxiliary PPO regularization, PCG, and an InterMimic-style reference-buffer curriculum for stable visual whole-body control.
  • PCG adapts teacher-driven rollout allocation from reward-based relative performance while keeping the imitation coefficient fixed.

C. World Models for Robots

DreamMimic targets contact-rich humanoid loco-manipulation as a partially observable control problem in which deployable policies must infer interaction variables from onboard sensing. Privileged teachers provide full simulator-state supervision, while students operate without online privileged interaction states.

  • World models summarize observation histories into latent dynamics and predict future states, supporting control and planning in high-dimensional settings.
  • The task is to reproduce physically consistent interactions from 153-DoF SMPL-X motion and object trajectories without simulator-only privileged information at test time.
  • The problem is formulated as a POMDP because the underlying Markovian simulator state is not directly accessible from onboard sensing.
  • The student uses non-privileged proprioception, a compact goal containing target object pose and robot trajectory cues, and inferred world-model features.
  • Privileged teachers access ground-truth object pose, interaction graphs, and contact signals that support stable optimization of complex contact behaviors.
  • Teacher rewards integrate whole-body tracking, end-effector alignment, object consistency, and contact-aware terms across diverse interaction scenarios.

C. Visuomotor Student Distillation

The student receives compact predictive features rather than raw images or online privileged interaction states. World-model supervision, auxiliary predictions, adaptive teacher allocation, and student-state curriculum updates support stable distillation.

  • The world model encodes onboard depth and segmentation into predictive latent features passed to the policy with proprioception and a compact goal condition.
  • The compact goal combines target object pose with shorthorizon root and key-body trajectory targets while excluding simulator-only interaction labels.
  • The deterministic RSSM state, reward prediction, and auxiliary privileged-state, contact, and object-state predictions provide compact interaction cues without online ground-truth inputs.
  • Student-driven rollouts use PPO regularization as an auxiliary stabilizer alongside supervised action distillation.
  • PCG adjusts the teacher-driven environment fraction from relative teacher–student reward performance while keeping imitation supervision fixed.
  • Failed student rollouts update a replayed reference buffer around difficult interaction windows, while teacher-driven environments are excluded.

D. World Model for Predictive Representation

DreamMimic uses an RSSM world model to learn predictive latent representations from visual-proprioceptive observations for distillation rather than planning. Its latent state combines recurrent deterministic and stochastic components, supports open-loop prediction, and feeds reconstruction and auxiliary prediction objectives.

  • World-model role: The RSSM learns predictive representations for distillation, with planning explicitly excluded from its role.It addresses compounding perceptual errors in contact-rich settings through recurrent predictive state learning.
  • Encoding: The encoder maps the current observation and previous action to an embedding using visual convolutional processing and proprioceptive MLP processing.Temporal context is accumulated through the recurrent RSSM state instead of a long explicitly stacked observation history.
  • RSSM dynamics: The latent state z_t = (h_t,s_t) contains deterministic recurrent state h_t and stochastic state s_t, with posterior inference conditioned on the observation embedding.The deterministic transition uses a GRU, while the posterior infers the stochastic component from the recurrent state and embedding.
  • Prior prediction: A prior over the stochastic state enables open-loop prediction by conditioning only on the deterministic transition.This allows future latent states to be predicted without consuming the current observation.
  • Decoding and auxiliary objectives: The decoder reconstructs visual and proprioceptive observations, while auxiliary heads predict reward, privileged state, contact, and object state.Reconstruction preserves sensory fidelity, and the auxiliary predictors add supervision for interaction-relevant signals.

E. Multi-step Latent Distillation

DreamMimic constrains future behavior by comparing teacher- and student-conditioned latent rollouts from a shared inferred state. The frozen RSSM prior supplies a local, action-conditioned consistency metric over multiple imagined steps.

  • Motivation: Single-step action imitation leaves future state evolution unconstrained, so DreamMimic adds action-conditioned latent consistency supervision.The loss compares predicted consequences of student and teacher actions in the learned dynamics.
  • Shared rollout initialization: Teacher and student imagined rollouts branch from the same posterior latent state inferred from the current student-side observation.The world-model parameters remain frozen during these rollouts.
  • Multi-step rollout: H=3 imagined steps are used in the main experiments without consuming additional observations.The RSSM prior is repeatedly conditioned on the respective teacher or student policy mean action.
  • Latent consistency: Matching deterministic latent components aligns history summaries, while stochastic matching penalizes divergence in the discrete RSSM state used by the policy.The weighting λ_stoch balances deterministic and stochastic matching.
  • Supervision mechanism: The world model provides action-conditioned multi-step latent targets that stabilize policy learning without requiring separate teacher-side visual observations.It functions as a learned consistency metric between teacher and student policy means.

F. Training Objective

Training separates predictive world-model learning from policy distillation. The world model combines KL-regularized representation learning, reconstruction, and auxiliary task supervision, while the policy combines imitation, latent consistency, and PPO regularization.

  • World-model objective: The world-model objective combines KL-regularized representation learning with auxiliary predictions anchoring latent states to task-relevant signals.The model is trained on student experience to capture action-conditioned dynamics in a compact latent space.
  • Representation learning: L_KL balances dynamics and representation learning, while reconstruction covers both visual and proprioceptive observations.A free-bits mechanism is used to prevent posterior collapse.
  • Auxiliary supervision: The remaining auxiliary terms supervise instantaneous reward, teacher-side privileged vectors, target contact, and target object state.These objectives provide interaction- and task-progress-related supervision for the latent representation.
  • Policy objective: The student policy loss combines mean-action matching, multi-step latent supervision, and PPO regularization.The optimization is imitation-dominant: PPO is introduced after warm-up as a lower-weight regularizer for student-driven rollouts.
  • Adaptive guidance: PCG adapts teacher guidance through the teacher-driven environment ratio ρ using reward-based relative performance while keeping the imitation coefficient fixed.World-model learning and latent supervision remain separate because they serve different roles and use different update pathways.

IV. EXPERIMENTS

Experiments evaluate vision-based visuomotor policies on OMOMO and BEHAVE using tracking-based metrics, with shared privileged teachers and student-side inputs across baselines. The comparisons focus on long-horizon whole-body human-object interaction sequences and differing visual representations and distillation strategies.

  • Datasets: OMOMO supplies full-body motions, object trajectories, and contact information across five objects, while BEHAVE contributes long-horizon human-object interactions across three everyday objects.BEHAVE sequences have average durations exceeding 300 steps, emphasizing sustained loco-manipulation.
  • Metrics: The reported tracking metrics are Success Rate, average tracked Duration before early termination, and Robot Tracking Error.These metrics assess successful clip tracking, persistence, and tracking accuracy.
  • OMOMO evaluation: Table I compares visuomotor policies on SMPL-X in the OMOMO dataset.Use it to compare the listed policies across the tracking metrics for OMOMO.
  • BEHAVE evaluation: Table II compares visuomotor policies on SMPL-X in the BEHAVE dataset.Use it to compare the listed policies across the tracking metrics for BEHAVE.
  • Baselines: All vision-based students share the same privileged teacher and student-side inputs, while baselines vary in visual encoder and distillation strategy.Non-world-model variants use ResNet-18, ViT, or lightweight CNN encoders with RL-only, DAgger, or DAgger+RL training.

A. Quantitative Evaluation

DreamMimic consistently improves quantitative loco-manipulation performance through temporal latent supervision, interaction-aware auxiliary prediction, predictive policy features, recurrent conditioning, and adaptive teacher guidance. Depth plus segmentation is the strongest tested visual input, while qualitative comparisons show more stable contact than baselines.

  • Overall performance: 92.2% OMOMO success is achieved by DreamMimic, with lower robot and object tracking errors than direct visual-encoder students.Under increased object mass, gains are more modest but remain visible in success rate and object tracking error.
  • Temporal latent supervision: Removing multi-step latent distillation reduces OMOMO success from 92.2% to 70.6% and increases both robot and object tracking errors.The ablation compares instantaneous observation supervision with multi-step latent evolution.
  • Auxiliary interaction prediction: The full auxiliary model reduces object error to 8.8 cm on OMOMO and 14.8 cm under increased object weight.Object-state prediction alone lowers error from 12.7 to 10.0 cm on OMOMO and from 16.4 to 15.1 cm under increased weight.
  • Predictive features for policy conditioning: Without providing prediction-head outputs to the policy, success drops to 84.3% on OMOMO and 33.3% under increased weight.The object-pose-goal-only variant has competitive tracking errors but remains below the full model in success rate.
  • Recurrent dynamics conditioning: Replacing the recurrent deterministic state with a current-step stochastic feature reduces success to 74.5%, while removing action conditioning reduces it from 92.2% to 90.2%.Removing the recurrent deterministic state reduces success to 88.2%, indicating that recurrent memory and action-conditioned prediction both matter.
  • Distillation schedule: PCG adapts teacher-driven rollouts from relative teacher–student reward and outperforms tuned naive annealing on tracking errors and sustained interaction duration.Naive annealing can match PCG in success rate after tuning but is sensitive to its decay schedule.
  • Visual input modality: Depth plus segmentation reaches 92.2% success and 184.18 average execution steps, exceeding segmentation-only and depth-only success rates of 86.3% and 88.2%.RGB also reaches 88.2% success with a slightly longer average execution time.

C. Qualitative Evaluation

Qualitative simulations show DreamMimic maintaining stable whole-body contact, balance, and object interaction in challenging loco-manipulation behaviors. These behaviors persist across a different humanoid morphology and after transfer between simulators, although the evidence remains simulation-only.

  • Contact-Rich Loco-Manipulation Behaviors: DreamMimic lifts and transports a large table while preserving balance and coordinated motion, whereas the baseline fails to establish stable contact and collapses.The baseline also loses contact after initially grasping a container, leading to a fall.
  • Cross-Embodiment Simulation: On the Unitree G1 humanoid in Isaac Gym, DreamMimic produces stable pushing with consistent balance and controlled object interaction, while Dreamer becomes unstable and falls.The comparison examines behavior under a different morphology and actuation configuration.
  • Cross-Simulator Qualitative Test: After transfer from Isaac Gym to Isaac Lab under matched OMOMO sequences, DreamMimic completes the selected qualitative tasks.The result is qualitative sim-to-sim evidence that learned latent dynamics capture interaction-relevant structure to some extent.

D. Failure Case Analysis

DreamMimic improves vision-based humanoid loco-manipulation through predictive latent dynamics, multi-step consistency, auxiliary interaction supervision, and adaptive teacher guidance. However, failures remain under occlusion and imprecise hand contacts, and evaluation is limited to simulation.

  • Failure cases: Severe occlusion and visually ambiguous contacts still cause policy errors despite multi-step supervision.Weak perceptual evidence remains problematic in these settings.
  • Failure cases: In G1 experiments, imprecise upper-body hand contacts lead policies to push heavy objects rather than lift them.The teacher preserves stable lower-body locomotion, but retargeting may miss fine-grained hand–object interaction dynamics.
  • Scope: Evaluation is confined to simulation with ground-truth depth and segmentation, without reported real-robot experiments.The authors identify robust perception, tactile or local hand–object control, and occlusion recovery as future needs.
  • Method: DreamMimic learns predictive RSSM latent dynamics with action-conditioned multi-step consistency to address long-horizon drift under partial observability.The framework uses predictive dynamics and multi-step supervision to stabilize vision-based policy distillation.
  • Method: Auxiliary supervision for interaction- and task-related quantities is combined with PCG, which adapts teacher involvement using reward feedback.PCG is described as a competence-aware guidance schedule for balancing teacher guidance during optimization.
  • Results: Experiments show improved tracking-based loco-manipulation performance over vision-based baselines, with qualitative evidence across embodiments and simulators.The reported cross-setting evidence is qualitative for morphology and simulator changes.
Loading 2608.22278v1…