DreamMimic: Learning Visuomotor Whole-Body Loco-Manipulation via World Model

• Vision-based humanoid loco-manipulation remains difficult because of discontinuous contact dynamics, high-dimensional actions, and partial observability.Source: §I · p. 1
• Existing visuomotor methods often lack temporally structured predictive representations, leaving long-horizon contact-rich behavior vulnerable to compounding errors.Source: §I · p. 1
• DreamMimic repurposes a Dreamer-style RSSM for predictive state representations and action-conditioned multi-step supervision instead of planning.Source: §I · pp. 1–2
• Auxiliary reward, contact, and object-dynamics prediction heads shape representations toward interaction-rich behaviors, while PCG adapts teacher involvement using relative performance.Source: §I · p. 2
• Experiments on OMOMO and BEHAVE report consistent performance improvements, with qualitative evidence across Unitree G1 morphology and Isaac Sim.Source: §I · p. 2, Figure 2 · p. 2

• Physics-based imitation and reinforcement learning have produced high-fidelity motion reproduction and increasingly complex simulated whole-body skills.Source: A. Physics-based Character Control · p. 2
• Transfer to real robots remains challenging because successful methods often rely on privileged contact, task-state, or object-pose observations unavailable during deployment.Source: A. Physics-based Character Control · p. 2
• Direct visuomotor reinforcement learning is unstable with high-dimensional observations and delayed credit assignment, while latent-dynamics methods improve pixel-based learning efficiency.Source: B. Visual Policy Learning for Robotic… · p. 2
• DreamMimic combines DAgger, auxiliary PPO regularization, PCG, and an InterMimic-style reference-buffer curriculum for stable visual whole-body control.Source: B. Visual Policy Learning for Robotic… · pp. 2–3
• PCG adapts teacher-driven rollout allocation from reward-based relative performance while keeping the imitation coefficient fixed.Source: B. Visual Policy Learning for Robotic… · p. 3

• World models summarize observation histories into latent dynamics and predict future states, supporting control and planning in high-dimensional settings.Source: C. World Models for Robots · p. 3
• The task is to reproduce physically consistent interactions from 153-DoF SMPL-X motion and object trajectories without simulator-only privileged information at test time.Source: §III · p. 3
• The problem is formulated as a POMDP because the underlying Markovian simulator state is not directly accessible from onboard sensing.Source: A. Problem Formulation · p. 3
• The student uses non-privileged proprioception, a compact goal containing target object pose and robot trajectory cues, and inferred world-model features.Source: A. Problem Formulation · p. 3
• Privileged teachers access ground-truth object pose, interaction graphs, and contact signals that support stable optimization of complex contact behaviors.Source: B. Privileged Teacher Policy · p. 3
• Teacher rewards integrate whole-body tracking, end-effector alignment, object consistency, and contact-aware terms across diverse interaction scenarios.Source: B. Privileged Teacher Policy · p. 3

• The world model encodes onboard depth and segmentation into predictive latent features passed to the policy with proprioception and a compact goal condition.Source: C. Visuomotor Student Distillation · p. 3
• The compact goal combines target object pose with shorthorizon root and key-body trajectory targets while excluding simulator-only interaction labels.Source: C. Visuomotor Student Distillation · p. 3
• The deterministic RSSM state, reward prediction, and auxiliary privileged-state, contact, and object-state predictions provide compact interaction cues without online ground-truth inputs.Source: C. Visuomotor Student Distillation · p. 3
• Student-driven rollouts use PPO regularization as an auxiliary stabilizer alongside supervised action distillation.Source: C. Visuomotor Student Distillation · pp. 3–4
• PCG adjusts the teacher-driven environment fraction from relative teacher–student reward performance while keeping imitation supervision fixed.Source: C. Visuomotor Student Distillation · p. 4
• Failed student rollouts update a replayed reference buffer around difficult interaction windows, while teacher-driven environments are excluded.Source: C. Visuomotor Student Distillation · p. 4

World-model role

• The RSSM learns predictive representations for distillation, with planning explicitly excluded from its role.It addresses compounding perceptual errors in contact-rich settings through recurrent predictive state learning.Source: D. World Model for Predictive… · p. 4

Encoding

• The encoder maps the current observation and previous action to an embedding using visual convolutional processing and proprioceptive MLP processing.Temporal context is accumulated through the recurrent RSSM state instead of a long explicitly stacked observation history.Source: D. World Model for Predictive… · p. 4

RSSM dynamics

• The latent state z_t = (h_t,s_t) contains deterministic recurrent state h_t and stochastic state s_t, with posterior inference conditioned on the observation embedding.The deterministic transition uses a GRU, while the posterior infers the stochastic component from the recurrent state and embedding.Source: D. World Model for Predictive… · p. 4

Prior prediction

• A prior over the stochastic state enables open-loop prediction by conditioning only on the deterministic transition.This allows future latent states to be predicted without consuming the current observation.Source: D. World Model for Predictive… · p. 4

Decoding and auxiliary objectives

• The decoder reconstructs visual and proprioceptive observations, while auxiliary heads predict reward, privileged state, contact, and object state.Reconstruction preserves sensory fidelity, and the auxiliary predictors add supervision for interaction-relevant signals.Source: D. World Model for Predictive… · p. 4

Motivation

• Single-step action imitation leaves future state evolution unconstrained, so DreamMimic adds action-conditioned latent consistency supervision.The loss compares predicted consequences of student and teacher actions in the learned dynamics.Source: E. Multi-step Latent Distillation · p. 4

Shared rollout initialization

• Teacher and student imagined rollouts branch from the same posterior latent state inferred from the current student-side observation.The world-model parameters remain frozen during these rollouts.Source: E. Multi-step Latent Distillation · p. 4

Multi-step rollout

• H=3 imagined steps are used in the main experiments without consuming additional observations.The RSSM prior is repeatedly conditioned on the respective teacher or student policy mean action.Source: E. Multi-step Latent Distillation · p. 4

Latent consistency

• Matching deterministic latent components aligns history summaries, while stochastic matching penalizes divergence in the discrete RSSM state used by the policy.The weighting λ_stoch balances deterministic and stochastic matching.Source: E. Multi-step Latent Distillation · p. 4

Supervision mechanism

• The world model provides action-conditioned multi-step latent targets that stabilize policy learning without requiring separate teacher-side visual observations.It functions as a learned consistency metric between teacher and student policy means.Source: Figure 3 · p. 5, E. Multi-step Latent Distillation · p. 5

World-model objective

• The world-model objective combines KL-regularized representation learning with auxiliary predictions anchoring latent states to task-relevant signals.The model is trained on student experience to capture action-conditioned dynamics in a compact latent space.Source: F. Training Objective · p. 5

Representation learning

• L_KL balances dynamics and representation learning, while reconstruction covers both visual and proprioceptive observations.A free-bits mechanism is used to prevent posterior collapse.Source: F. Training Objective · p. 5

Auxiliary supervision

• The remaining auxiliary terms supervise instantaneous reward, teacher-side privileged vectors, target contact, and target object state.These objectives provide interaction- and task-progress-related supervision for the latent representation.Source: F. Training Objective · p. 5

Policy objective

• The student policy loss combines mean-action matching, multi-step latent supervision, and PPO regularization.The optimization is imitation-dominant: PPO is introduced after warm-up as a lower-weight regularizer for student-driven rollouts.Source: F. Training Objective · p. 5

Adaptive guidance

• PCG adapts teacher guidance through the teacher-driven environment ratio ρ using reward-based relative performance while keeping the imitation coefficient fixed.World-model learning and latent supervision remain separate because they serve different roles and use different update pathways.Source: F. Training Objective · p. 5

Datasets

• OMOMO supplies full-body motions, object trajectories, and contact information across five objects, while BEHAVE contributes long-horizon human-object interactions across three everyday objects.BEHAVE sequences have average durations exceeding 300 steps, emphasizing sustained loco-manipulation.Source: §IV · p. 5

Metrics

• The reported tracking metrics are Success Rate, average tracked Duration before early termination, and Robot Tracking Error.These metrics assess successful clip tracking, persistence, and tracking accuracy.Source: §IV · p. 5

OMOMO evaluation

• Table I compares visuomotor policies on SMPL-X in the OMOMO dataset.Use it to compare the listed policies across the tracking metrics for OMOMO.Source: Table I · p. 6

BEHAVE evaluation

• Table II compares visuomotor policies on SMPL-X in the BEHAVE dataset.Use it to compare the listed policies across the tracking metrics for BEHAVE.Source: Table II · p. 6

Baselines

• All vision-based students share the same privileged teacher and student-side inputs, while baselines vary in visual encoder and distillation strategy.Non-world-model variants use ResNet-18, ViT, or lightweight CNN encoders with RL-only, DAgger, or DAgger+RL training.Source: §IV · p. 6

Overall performance

• 92.2% OMOMO success is achieved by DreamMimic, with lower robot and object tracking errors than direct visual-encoder students.Under increased object mass, gains are more modest but remain visible in success rate and object tracking error.Source: A. Quantitative Evaluation · p. 6

Temporal latent supervision

• Removing multi-step latent distillation reduces OMOMO success from 92.2% to 70.6% and increases both robot and object tracking errors.The ablation compares instantaneous observation supervision with multi-step latent evolution.Source: B. Ablation Study · p. 6

Auxiliary interaction prediction

• The full auxiliary model reduces object error to 8.8 cm on OMOMO and 14.8 cm under increased object weight.Object-state prediction alone lowers error from 12.7 to 10.0 cm on OMOMO and from 16.4 to 15.1 cm under increased weight.Source: B. Ablation Study · p. 6

Predictive features for policy conditioning

• Without providing prediction-head outputs to the policy, success drops to 84.3% on OMOMO and 33.3% under increased weight.The object-pose-goal-only variant has competitive tracking errors but remains below the full model in success rate.Source: B. Ablation Study · p. 6

Recurrent dynamics conditioning

• Replacing the recurrent deterministic state with a current-step stochastic feature reduces success to 74.5%, while removing action conditioning reduces it from 92.2% to 90.2%.Removing the recurrent deterministic state reduces success to 88.2%, indicating that recurrent memory and action-conditioned prediction both matter.Source: B. Ablation Study · p. 6

Distillation schedule

• PCG adapts teacher-driven rollouts from relative teacher–student reward and outperforms tuned naive annealing on tracking errors and sustained interaction duration.Naive annealing can match PCG in success rate after tuning but is sensitive to its decay schedule.Source: B. Ablation Study · pp. 6–7

Visual input modality

• Depth plus segmentation reaches 92.2% success and 184.18 average execution steps, exceeding segmentation-only and depth-only success rates of 86.3% and 88.2%.RGB also reaches 88.2% success with a slightly longer average execution time.Source: B. Ablation Study · p. 7

Contact-Rich Loco-Manipulation Behaviors

• DreamMimic lifts and transports a large table while preserving balance and coordinated motion, whereas the baseline fails to establish stable contact and collapses.The baseline also loses contact after initially grasping a container, leading to a fall.Source: C. Qualitative Evaluation · p. 7

Cross-Embodiment Simulation

• On the Unitree G1 humanoid in Isaac Gym, DreamMimic produces stable pushing with consistent balance and controlled object interaction, while Dreamer becomes unstable and falls.The comparison examines behavior under a different morphology and actuation configuration.Source: C. Qualitative Evaluation · p. 7

Cross-Simulator Qualitative Test

• After transfer from Isaac Gym to Isaac Lab under matched OMOMO sequences, DreamMimic completes the selected qualitative tasks.The result is qualitative sim-to-sim evidence that learned latent dynamics capture interaction-relevant structure to some extent.Source: C. Qualitative Evaluation · p. 7

Failure cases

• Severe occlusion and visually ambiguous contacts still cause policy errors despite multi-step supervision.Weak perceptual evidence remains problematic in these settings.Source: D. Failure Case Analysis · p. 7
• In G1 experiments, imprecise upper-body hand contacts lead policies to push heavy objects rather than lift them.The teacher preserves stable lower-body locomotion, but retargeting may miss fine-grained hand–object interaction dynamics.Source: D. Failure Case Analysis · p. 7

Scope

• Evaluation is confined to simulation with ground-truth depth and segmentation, without reported real-robot experiments.The authors identify robust perception, tactile or local hand–object control, and occlusion recovery as future needs.Source: D. Failure Case Analysis · p. 7

Method

• DreamMimic learns predictive RSSM latent dynamics with action-conditioned multi-step consistency to address long-horizon drift under partial observability.The framework uses predictive dynamics and multi-step supervision to stabilize vision-based policy distillation.Source: §V · pp. 7–8
• Auxiliary supervision for interaction- and task-related quantities is combined with PCG, which adapts teacher involvement using reward feedback.PCG is described as a competence-aware guidance schedule for balancing teacher guidance during optimization.Source: §V · p. 8

Results

• Experiments show improved tracking-based loco-manipulation performance over vision-based baselines, with qualitative evidence across embodiments and simulators.The reported cross-setting evidence is qualitative for morphology and simulator changes.Source: §V · p. 8