Source-linked AI summary
ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning
Siheng Zhao, Yanjie Ze, Yue Wang, C. Karen Liu, Pieter Abbeel, Guanya Shi, Rocky Duan
TL;DR
GMT policies reproduce human motion but lack the object awareness and precision needed for whole-body loco-manipulation. ResMimic combines a pretrained GMT base with a task-specific residual policy and training aids for object tracking and contact. The framework reports substantial gains in task success, training efficiency, and robustness, including 92.5% success versus 10% for GMT alone.
Problem
GMT policies reproduce diverse human motions but lack the object awareness and precision required for humanoid loco-manipulation.
Method
ResMimic pretrains a GMT motion prior, then trains a task-specific residual policy with object and contact rewards and a curriculum-based virtual object controller.
Results
92.5% task success for ResMimic versus 10% for GMT alone.
Takeaways & Limitations
The framework supports precise, expressive, and robust humanoid loco-manipulation across simulation and real Unitree G1 deployment.
Abstract
from arXiv · showhide
Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks. While recent advances in general motion tracking (GMT) have enabled humanoids to reproduce diverse human motions, these policies lack the precision and object awareness required for loco-manipulation. To this end, we introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data. First, a GMT policy, trained on large-scale human-only motion, serves as a task-agnostic base for generating human-like whole-body movements. An efficient but precise residual policy is then learned to refine the GMT outputs to improve locomotion and incorporate object interaction. To further facilitate efficient training, we design (i) a point-cloud-based object tracking reward for smoother optimization, (ii) a contact reward that encourages accurate humanoid body-object interactions, and (iii) a curriculum-based virtual object controller to stabilize early training. We evaluate ResMimic in both simulation and on a real Unitree G1 humanoid. Results show substantial gains in task success, training efficiency, and robustness over strong baselines. Videos are available at https://resmimic.github.io/ .
I. INTRODUCTION
Humanoid loco-manipulation requires precise, object-aware whole-body control, but GMT policies lack object awareness and direct retargeting introduces embodiment-gap errors. ResMimic addresses this with a two-stage residual framework that adapts a general motion prior using task-specific corrections.
- Humanoid loco-manipulation combines locomotion and manipulation but demands higher precision, rich whole-body contacts, and difficult-to-obtain data.
- GMT policies reproduce diverse human motions with high fidelity but lack awareness of manipulated objects.
- Existing loco-manipulation methods use task-specific controllers or handcrafted data pipelines, limiting scalability and generality.
- ResMimic trains a GMT policy as a robust whole-body motion prior, then learns a task-specific residual policy conditioned on object trajectories to refine manipulation.
- Evaluations in simulation and on a real Unitree G1 report improvements in motion tracking, object tracking, task success, training efficiency, robustness, and generalization.
II. RELATED WORK
Prior work spans reinforcement learning, human-motion imitation, teleoperation, and residual learning, but existing approaches remain limited in generality, object awareness, or whole-body dynamic interaction. ResMimic formulates the task as goal-conditioned RL with robot and object states, motion and object goals, and PD-executed joint targets.
- RL enables real-time humanoid whole-body control but typically has low data efficiency and requires task-specific reward design.
- Human-motion methods improve motion tracking through kinematic retargeting, yet prior systems described here do not extend to dynamic object interactions.
- Teleoperation and imitation approaches provide promising loco-manipulation results but require operators or remain restricted to tabletop manipulation.
- Residual learning refines predefined, model-predictive, or demonstration-initialized policies for more precise and contact-rich manipulation.
- ResMimic casts loco-manipulation as goal-conditioned RL with robot proprioception, object state, motion goals, and object goals in the MDP state.
- Actions specify target joint angles executed through a PD controller, while learning maximizes expected cumulative discounted reward.
A. Two-Stage Residual Learning
ResMimic separates general human-motion tracking from object interaction: a pretrained GMT controller provides coarse whole-body actions, and a task-specific residual policy refines them using robot and object information.
- The two-stage framework is designed to avoid task-specific reward engineering while transferring human–object interaction data into humanoid control.
- Stage I: General Motion Tracking: Stage I trains a general GMT policy from robot proprioception and reference motion to produce coarse actions for human-motion imitation.
- Stage II: Residual Refinement: Stage II trains a per-task residual policy that uses robot and object information to refine the pretrained GMT action.
- Both stages are trained using PPO.
- Stage I: General Motion Tracking: The GMT policy is intended as a real-world-deployable base and can be replaced by another GMT policy within the general framework.
1) Dataset:
ResMimic decouples motion tracking from object interaction so the general policy can learn from abundant human-motion data without costly manipulation data. The dataset combines large public motion-capture resources with retargeted humanoid references and uses future motion context for smoother tracking.
- Dataset: The general motion-tracking policy relies only on human motion-capture data, avoiding costly and difficult-to-obtain manipulation data.
- Dataset: AMASS and OMOMO provide over 15,000 clips, approximately 42 hours, after filtering motions impractical for the target setting.
- Dataset: Kinematics-based motion retargeting transfers curated human motions into humanoid reference-motion data.
- General Motion Tracking Policy: The GMT policy uses proprioception, recent action history, and reference motion as inputs for control in simulation without privileged information.
- General Motion Tracking Policy: Including future reference motion enables anticipation and planning for upcoming targets, yielding smoother tracking.
- Pipeline: The overall pipeline trains a GMT base policy on large-scale human motion data before residual refinement and deployment.
3) Reward and Domain Randomization:
The residual policy is trained with motion-tracking rewards and domain randomization, using retargeted human and object trajectories as references.
- Motion-tracking reward combines task rewards, penalties, and regularization terms, with domain randomization supporting sim-to-real transfer.
- The residual policy refines coarse actions from the pretrained GMT policy to complete the desired task.
- Human motion is recorded with MoCap and retargeted to humanoid reference trajectories, while object motion directly supplies the object reference.
2) Training Strategy:
Residual learning uses PPO to refine GMT behavior, with near-zero initialization and a virtual-object curriculum that gradually transfers control from assistance to the policy.
- PPO trains a residual policy conditioned on robot state and the reference object trajectory.
- The PPO actor’s final layer is initialized with a small Xavier gain so initial residual outputs remain close to zero.
- Virtual-object assistance addresses retargeting penetration and heavy-object instability that can cause early failures.
- Controller gains decay gradually, providing strong early stabilization before requiring autonomous task completion.
3) Reward and Early Termination:
The method augments motion tracking with object and contact rewards, using point-cloud geometry and contact-link supervision to improve physical interaction and deployment robustness.
- Reward Design: Object-tracking and contact-tracking rewards supplement the motion reward to encourage task completion and guide body–object contact.
- Reward Design: Object tracking uses sampled mesh-surface point clouds, producing a smoother reward that captures translation and rotation without task-specific weight tuning.
- Reward Design: Contact tracking discretizes interactions across meaningful torso, hip, and arm links while excluding feet that primarily contact the ground.
- Early Termination: Episodes terminate when the object deviates beyond a threshold or required body–object contact is lost for more than 10 consecutive frames.
IV. EXPERIMENTS
The experiments combine large-scale simulation with real-world Unitree G1 deployment and test task capability, training efficiency, fine-tuning versus residual learning, and robustness beyond simulation.
- Experiments use large-scale simulation and real-world deployment on a 29-DoF, 1.3 m Unitree G1 humanoid.
- The evaluation asks whether GMT alone solves diverse loco-manipulation tasks and whether GMT initialization improves efficiency and final performance.
- The study compares residual learning with direct fine-tuning and tests precise, expressive, and robust real-world control.
- Figure 5 compares IsaacGym and MuJoCo object-tracking-error curves for Chair and Carry across three training approaches.
A. Experiment Setup
The evaluation uses four challenging whole-body loco-manipulation tasks, multiple tracking and success metrics, and comparisons against representative baselines.
- Tasks: Four tasks—Kneel, Carry, Squat, and Chair—test expressive motion, balance, whole-body contact, and generalization to heavy irregular objects.Human–object interaction references are collected with an OptiTrack motion capture system.
- Evaluation Metrics: Training Iterations measure convergence speed, defined by approximately stopped reward improvement rather than wall-clock time.This avoids hardware-dependent timing discrepancies.
- Evaluation Metrics: Object, motion, and joint tracking errors quantify object alignment, link-position fidelity, and joint-level precision, respectively.Task success additionally requires object error below a predefined threshold while the robot remains balanced.
- Evaluation Metrics: Task Success Rate counts rollouts that satisfy the object-tracking threshold and maintain robot balance.
- Baselines: ResMimic is compared with direct deployment of the base GMT policy, training from scratch, and fine-tuning-based alternatives.The baselines differ in access to object information and adaptation strategy.
B. Sim-to-Sim Evaluation
Sim-to-sim and real-world evaluations compare ResMimic with strong baselines, showing stronger transfer, object-aware manipulation, and broad whole-body capabilities.
- Sim-to-Sim Evaluation: 10% versus 92.5% success rate: the base GMT policy underperforms ResMimic because it lacks object information.GMT retains slightly lower joint tracking error but performs poorly on object tracking and task completion.
- Sim-to-Sim Evaluation: Training from scratch fails in MuJoCo and converges more slowly, while ResMimic maintains strong performance with minimal sim-to-sim degradation.From-scratch policies can show partial IsaacGym success but collapse after transfer.
- Sim-to-Sim Evaluation: Direct fine-tuning neither surpasses the base GMT policy nor approaches ResMimic, especially under randomized object poses and MuJoCo transfer.Its restricted human-motion input prevents explicit object observations and can overwrite GMT generalization.
- Real-World Evaluation: Real-world deployment on Unitree G1 demonstrates blind and non-blind whole-body loco-manipulation, including expressive carrying and interaction beyond manipulation.Non-blind deployment supports random initial poses, consecutive tasks, and reactions to external perturbations.
- Real-World Evaluation: 4.5 kg box carrying exceeds the G1 wrist payload limit of around 2.5 kg through whole-body contact.The robot also lifts and carries 4.5 kg and 5.5 kg chairs, demonstrating instance-level generalization to irregular geometries.
- Real-World Evaluation: In real-world qualitative comparisons, the base policy lacks object awareness, while training from scratch and fine-tuning fail because of the sim-to-real gap.
D. Ablation Studies
Ablations show that the virtual object controller stabilizes difficult early training, while contact rewards encourage whole-body strategies that transfer beyond IsaacGym.
- Virtual Object Controller: Curriculum-based virtual forces guide objects toward reference trajectories during early training, addressing noisy references, penetration, and heavy-object instability.Without this guidance, the policy may knock objects over and retreat into a low-reward local minimum.
- Contact Reward: Without contact reward, the policy relies on wrists and hands, a strategy that may succeed in IsaacGym but fails in MuJoCo and the real world.
- Contact Reward: With contact reward, the humanoid uses coordinated torso and arm contact, improving sim-to-sim and sim-to-real transfer.This strategy aligns with the whole-body contacts shown in human demonstrations.
- Conclusion: ResMimic combines residual refinement with training mechanisms that support precise, expressive, and robust humanoid loco-manipulation.The conclusion reports gains in task success, motion fidelity, and training efficiency alongside Unitree G1 deployment.