Source-linked AI summary
Expressive Whole-Body Control for Humanoid Robots
Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang, Xiaolong Wang
TL;DR
Humanoid robots often lack the rich, diverse expressiveness of human motion, while direct imitation is mismatched to their degrees of freedom and physical capabilities. ExBody combines human motion capture with reinforcement learning, imitating upper-body expression while relaxing leg imitation for robust root-motion tracking. Simulation and real-world experiments show expressive, robust behaviors including varied walking, handshaking, and dancing with a person.
Problem
Whole-body controllers commonly produce singular motions, while direct transfer of human-motion imitation and graphics techniques is limited by humanoid hardware and capability mismatches.
Method
ExBody trains a goal-conditioned controller that imitates expressive upper-body motion while requiring the legs to robustly follow root movement commands.
Results
Simulation and real-world evaluations show expressive humanoid control with diverse walking, waving, handshaking, and dancing behaviors.
Takeaways & Limitations
Relaxing lower-body imitation enables expressive upper-body tracking alongside robust locomotion on a real humanoid robot.
Takeaways & Limitations
Motion retargeting to the lower-DoF H1 loses information and can make behavior deviate from the original motion; falls can also damage the costly robot.
Abstract
from arXiv · showhide
Can we enable humanoid robots to generate rich, diverse, and expressive motions in the real world? We propose to learn a whole-body control policy on a human-sized robot to mimic human motions as realistic as possible. To train such a policy, we leverage the large-scale human motion capture data from the graphics community in a Reinforcement Learning framework. However, directly performing imitation learning with the motion capture dataset would not work on the real humanoid robot, given the large gap in degrees of freedom and physical capabilities. Our method Expressive Whole-Body Control (Exbody) tackles this problem by encouraging the upper humanoid body to imitate a reference motion, while relaxing the imitation constraint on its two legs and only requiring them to follow a given velocity robustly. With training in simulation and Sim2Real transfer, our policy can control a humanoid robot to walk in different styles, shake hands with humans, and even dance with a human in the real world. We conduct extensive studies and comparisons on diverse motions in both simulation and the real world to show the effectiveness of our approach.
I. INTRODUCTION
The paper asks how humanoid robots can produce diverse, expressive whole-body motions rather than singular task-driven patterns. ExBody combines large-scale human motion data with reinforcement learning, relaxing leg imitation to achieve real-robot robustness.
- Motivation: Conventional whole-body control often produces singular motions lacking the grace, personality, diversity, and expressiveness of human or animal movement.Common objectives such as velocity, heading, and gait do not describe the full subspace of expressive motion.
- Motivation: Directly transferring graphics-based character-animation techniques to real humanoids is infeasible because they can require actuator gains around 60kg/m and hardware-unrealistic capabilities.The paper also identifies a large mismatch between human reference motions and robot physical capabilities.
- Approach: ExBody encourages the humanoid’s upper body to imitate diverse references while its legs robustly follow root movement commands instead of matching exact joint angles.The policy receives both a reference motion and a root movement command, and is trained in randomized challenging terrains.
- Evaluation: The controller demonstrates walking in varied styles, waving and shaking hands while walking, and dancing with a person on diverse terrains.The paper describes the robot as sufficiently compliant and robust for hand-holding and human-robot dancing.
- Evaluation: The Unitree H1 policy uses around 780 CMU MoCap reference motions, with evaluation in simulation and the real world showing benefits for expressive motion and robust walking.The study reports comparisons against stronger leg-imitation constraints and finds that relaxing those constraints gives better, more robust results.
II. PROBLEM FORMULATION
The formulation treats humanoid control as a goal-conditioned motor policy mapping behavior goals and observations to joint-position and torque actions. ExBody separates root movement commands from expressive upper-body targets so lower-body imitation does not over-constrain control.
- Policy formulation: Humanoid motion control is formulated as a goal-conditioned policy π: G × S 7→ A, where G specifies behavior, S contains observations, and A contains joint positions and torque.The formulation assumes the Unitree H1 design but states that the approach should generalize to similar body forms with different actuated degrees of freedom.
- Command-conditioned locomotion: Command-conditioned locomotion uses root linear velocity, roll/pitch/yaw body pose, and root body height as the movement goal Gm = ⟨v, rpy, h⟩.The policy is designed for robust control of Unitree H1 hardware and uses local yaw difference rather than privileged absolute deployment quantities.
- Command-conditioned locomotion: Actions are 19-dimensional target joint positions executed by joint-level proportional-derivative controllers.The passage specifies that the PD controllers compute motor torque from target positions and controller gains.
- Expressive control: ExBody extends the movement goal to G = Ge × Gm by adding expressive targets beyond root pose and velocity, including joint angles and 3D body keypoints.The expression target is intended to describe movement features not captured by root commands alone.
- Expressive control: The relaxed formulation excludes lower-body joints and keypoints from Ge because the robot’s body plan differs from humans and those features can over-constrain control, producing brittle policies.The paper therefore defines Ge using upper-body features while assigning the lower body the root-movement objective.
III. EXPRESSIVE WHOLE-BODY CONTROL
ExBody builds a repertoire of robot-compatible motion clips from selected human-motion data and uses them to train and deploy expressive control. The data pipeline supports multiple motion sources and accommodates the H1 robot’s kinematic constraints.
- Data curation: ExBody curates and retargets human motion-capture data into motion clips compatible with humanoid robot hardware, then uses them to train the policy.The method is presented as combining motion-data curation and retargeting with reinforcement-learning training.
- Data curation: Motions involving physical interactions, heavy objects, or rough terrain are excluded because the framework cannot realistically implement substantial environmental interactions.Selection is performed semi-automatically, and the resulting motions are listed in the dataset details.
- Deployment: At deployment, motion can come from static datasets, diffusion models, or video-to-skeleton models after retargeting to the Unitree H1.The H1 has 19 total degrees of freedom, with three perpendicular degrees at each shoulder and hip and one degree at other joints.
- Retargeting: Retargeted data preserves important expressions despite losing some degrees of freedom from hardware constraints.The paper illustrates diverse original and retargeted motions and notes that direct lower-body retargeting can exaggerate movement, motivating ExBody.
- Data curation: The training dataset uses a selected subset of CMU MoCap, while testing covers expressive motions in simulation and the real world.The dataset table is described as containing training data and diverse expressive test motions.
B. Motion Retargeting to Hardware
The method retargets human motion to the 19-DoF Unitree H1 by mapping local joint rotations to its skeleton and converting spherical joints into three revolute-joint angles. One-dimensional joints use rotation angles projected onto their corresponding axes, preserving important expressive motion despite hardware constraints.
- Hardware adaptation: Human motion is adapted to the Unitree H1 by mapping local joint rotations onto the robot’s skeleton despite morphological differences.The H1 has a mass of around 51.5 kg, a height of around 1.8 m, and 19 DoFs.
- Quaternion retargeting: Spherical joint rotations represented by normalized quaternions are remapped to three revolute-joint angles through exponential mapping.The quaternion is first converted to axis-angle form before mapping.
- Quaternion retargeting: For small rotations, the mapped three-DoF angle is computed as m = θa, where a is the rotation axis and θ is the rotation angle.The last axis is used when 1 − q_w^2 is close to zero.
- One-dimensional joints: For one-dimensional elbow, torso, knee, and ankle joints, the method projects the rotation angle onto each joint’s corresponding rotation axis.Retargeted motions lose some DoFs because of hardware constraints but retain important expressive motion.
C. Guiding State Initialization from Human Mocap Data
ExBody trains in massively parallel simulation by initializing each environment from randomly sampled human-motion states and using tracking rewards for expression and root movement. This state-guided setup supports diverse upper-body expressions and root movements while the lower body maintains balance without imitation rewards.
- State initialization: Each simulation environment is initialized or reset to a randomly sampled state g = [g_e, g_m] from the human motion dataset.The policy is trained with massively parallel Isaac Gym simulation.
- Learned motion diversity: Diverse goal-state initialization and tracking rewards produce diverse root movements and arm expressions while the lower body maintains balance without mimicking rewards.The resulting motions include forward and backward walking, sideways walking, yaw turning, root-height variation, and roll or pitch adjustment.
- Reward guidance: The reward combines expression-goal tracking, root-movement-goal tracking, and regularization terms.The expressive reward specification is listed in Table III, while regularization items are referenced separately.
- Reward guidance: Reference signals include upper-body joint positions, upper-body keypoint positions, body velocity, and body roll and pitch.The corresponding references are compared with actual body quantities during reward computation.
IV. RESULTS
ExBody tracks expressive upper-body motions while maintaining robust whole-body root movement, and its design choices improve motion coverage, locomotion, and real-world transfer. Comparisons show that relaxed lower-body imitation and large-data training outperform or avoid several alternative behaviors.
- Tracking performance: ExBody tracks root movement commands beyond the motion dataset, covering most sampled velocity commands while producing a smaller roll-and-pitch state distribution.The authors attribute the narrower roll-and-pitch coverage to robot properties such as a heavy upper body.
- Tracking performance: The learned policy produces nearly identical hand-position distributions to the retargeted motion dataset, indicating effective upper-body expression tracking.The comparison samples 10,000 hand positions relative to the robot.
- Large-data training: The method achieves the best linear velocity tracking performance (MELV) among the reported baselines.The authors link much of this benefit to randomized state initialization (RSI).
- Large-data training: Without RSI, the policy fails to discover positive-reward states, while random command sampling produces kneeling that prioritizes stability over root movement.The random-sample baseline gives up gm and focuses on ge, whereas motion data provide a more advantageous root-command distribution.
- Design comparisons: Full-body tracking reduces performance across all metrics and causes lower-body artifacts, including multiple stabilizing steps for a reference motion designed as a single step.The result motivates relaxing imitation constraints on the legs because of the real robot’s limited torque and degrees of freedom.
- Design comparisons: ExBody + AMP improves MEL and MERP but sacrifices MELV, and its straight-leg gait, low foot clearance, and stumbling complicate sim-to-real transfer.ExBody alone produces more knee flexion and foot-height clearance during real-world walking.
- Real-world results: In real-world demonstrations, the robot replays CMU MoCap trajectories, responds to text-prompted boxing motions, and synchronizes stepping frequency with upper-body poses.The text-to-motion evaluation reports robust, responsive, and precise tracking performance.
V. RELATED WORK
Prior work combines model-based whole-body control, learning-based locomotion, and physics-based character animation, but expressive real-world control remains difficult for high-degree-of-freedom humanoids. ExBody addresses this gap by using a relaxed lower-body objective and a whole-body root movement goal.
- Whole-Body Control with Legged Robots: Whole-body control for high-degree-of-freedom humanoids requires substantial engineering and modeling and remains sensitive to real-world dynamics changes.Earlier approaches relied primarily on dynamics modeling and control for tasks such as dancing and reaching.
- Expressive motion generation: The real-world text-to-motion demonstration replays an offline boxing trajectory generated from the prompt “a man mimics boxing punches.”The robot is reported to show robust, responsive, and precise tracking performance.
- Learning-based control: Recent learning-based advances improve humanoid control, but most studies emphasize locomotion or use relatively small motion datasets.The paper distinguishes its work by targeting expressive whole-body motions on a human-sized robot in the real world.
- Expressive Whole-Body Control: ExBody relaxes lower-body tracking and uses a whole-body root movement goal, while selecting mainly walking, everyday, and expressive motions for one network.The design is tailored to the robot’s capabilities and supports a single policy across the selected motions.
VI. DISCUSSIONS
The paper presents ExBody as a method for expressive upper-body tracking with robust locomotion, supported by extensive simulation and real-world evaluation. Its intended outcome is a reliable and versatile humanoid capable of multiple functions.
- Discussion: The method is designed to track expressive upper-body motions while maintaining robust locomotion capabilities in real-world settings.The framework uses large motion datasets and randomized state initialization, with evaluations in simulation and the real world.
- Discussion: The authors conclude that the approach could support reliable and versatile humanoid robots capable of performing multiple functions effectively.This conclusion is stated within the scope of the presented framework and evaluations.
VII. LIMITATIONS
Direct joint-angle mapping from human motion capture to the lower-DoF H1 robot loses information and can make retargeted behavior diverge from the original motion. Falls are also costly, motivating higher-fidelity retargeting and reliable protection and recovery.
- Directly mapping MoCap joint angles to H1 loses information because the robot has fewer degrees of freedom.This loss can cause retargeted behavior to deviate from the original motion.
- High-fidelity motion retargeting could significantly reduce discrepancies between reference and retargeted motions.
- Because the humanoid is heavy and expensive, falls can break parts and make protective and recovery systems imperative.
Appendix
The appendix documents dataset curation, reward regularization, PPO and AMP training settings, and text-to-motion generation procedures used in the experiments.
- Dataset Curation: The curated training and inference data contain single-person motions on flat terrain selected from CMU MoCap using motion-description keywords.The curation aims to ensure expressive motions are reasonable to track.
- Reward Regularization: Regularization rewards include foot height, airborne duration, new contacts, contact forces, joint acceleration, actions, collisions, joint limits, and projected gravity.These terms define the listed regularization specification in Table VI.
- Reward Regularization: Foot-related rewards are added so feet lift comfortably and contact the ground with reasonable force during placement.
- Training Parameters: PPO hyperparameters and AMP baseline parameters are provided in Tables VII and VIII.
- Text2motion Diffusion Model: The pre-trained transformer-based MDM generates human motions from text prompts, with ten repetitions requested and the most reasonable motion manually selected for retargeting.
C. Additional Real World Results Visualization
Additional real-world visualizations show eight CMU MoCap motions and two motions generated from text prompts, after offline retargeting to the H1 morphology.
- Figure 10 presents eight real-world motions from CMU MoCap and two motions generated by a text-to-motion diffusion model.
- The diffusion-model motions are generated with MDM on the SMPL skeleton and then retargeted offline to the H1 morphology.
- The two prompts request motions for moving an arm out to shake hands and crossing and uncrossing the arms.