Source-linked AI summary
HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu
TL;DR
Humanoid controllers often specialize in a single command space, limiting adaptation across locomotion and manipulation tasks. HOVER uses full-body kinematic motion imitation and policy distillation to consolidate diverse modes into one unified policy, which outperforms specialist controllers across modes and supports seamless transitions. The paper positions this as a scalable way to reuse motor skills without retraining a separate policy for each command configuration.
Problem
Existing humanoid whole-body controllers use task-specific command spaces and interfaces, limiting a unified controller’s ability to adapt across diverse control modes.
Method
HOVER distills motor skills from a full-body kinematic motion imitator into a unified policy using proprioception and mode- and sparsity-based command masking.
Results
HOVER outperforms specialist controllers across control modes, achieves lower tracking error across 32/32 metrics and modes, and outperforms specialists in 11 of 12 real-world metrics.
Takeaways & Limitations
A single HOVER policy shares motor skills across modes and supports seamless multi-mode control in simulation and on a real humanoid.
Abstract
from arXiv · showhide
Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications.
I. INTRODUCTION
HOVER addresses the difficulty of building humanoid controllers that adapt across task-specific command modes by unifying them around shared motion objectives. It distills motor skills from an oracle motion imitator into one generalist policy supporting diverse modes and transitions.
- Motivation: Task-specific whole-body controllers use different command spaces, making development repetitive and limiting seamless switching across locomotion and manipulation.Examples include root velocity tracking for locomotion and joint-angle or end-effector tracking for precise manipulation.
- Motivation: The framework exploits aligned motion objectives—stable, human-like motion—despite differences between control interfaces.This shared objective motivates integrating otherwise distinct command spaces.
- Approach: HOVER unifies diverse humanoid control modes through a common neural controller supporting over 15 modes on a 19-DOF robot.Its command space covers most modes used in prior work.
- Approach: An oracle motion imitator trained on large-scale MoCap data supplies reusable motor skills, which policy distillation transfers into a single generalist policy.The design uses human-motion priors for broad movement and control objectives.
- Results: Experiments report shared motor skills, seamless mode transitions, and superior multi-mode control compared with individually trained policies in simulation and on a real humanoid.These are stated as the paper’s three main contribution-level findings.
II. METHOD
The method formulates humanoid control as goal-conditioned reinforcement learning in which proprioception and a unified target-goal representation guide joint-position actions. Policies are trained with PPO to track commands over time.
- Goal-Conditioned RL: HOVER formulates humanoid control as goal-conditioned reinforcement learning, combining agent proprioception with a target goal state representing the desired motion.The goal state provides a unified representation of the target motion goal.
- Goal-Conditioned RL: The policy outputs 19-dimensional target joint positions that are fed to a PD controller to actuate the robot’s degrees of freedom.The action space is explicitly tied to the robot’s 19 joints.
- Goal-Conditioned RL: PPO maximizes cumulative discounted reward while the humanoid learns to follow target commands at each timestep.This frames training as a command-tracking task.
B. Command Space Design for Humanoid Control
HOVER defines a unified, expressive command space that covers prior control configurations while allowing independent combinations of upper- and lower-body tracking modes. One-hot masking selects the command components active for each task.
- Design Goals: Prior humanoid control modes are tailored to specific task subsets, whereas HOVER seeks a comprehensive framework adaptable across scenarios.Root-only tracking can limit whole-body loco-manipulation capabilities.
- Design Goals: The command space is designed for generality, including prior configurations and interfaces such as joysticks, keyboards, motion capture, exoskeletons, and VR headsets.This supports replacing task-specific controllers without sacrificing versatility.
- Design Goals: Atomic command dimensions allow arbitrary combinations of control options to support different modes.Independent dimensions make selective command activation possible.
- Unified Command Space: HOVER divides control into upper- and lower-body regions and includes kinematic position, local joint-angle, and root tracking modes.Root tracking includes velocity, height, and roll, pitch, and yaw orientation targets.
- Unified Command Space: A one-hot masking vector activates selected command components, allowing prior control designs to appear as subsets of the unified space.The framework supports tracking arbitrary subsets of command elements.
C. Motion Retargeting
The retargeting pipeline converts human motion data into feasible humanoid motions by aligning human-model parameters with humanoid forward-kinematics keypoints.
- The procedure first computes humanoid keypoint positions from joint configurations using forward kinematics.
- It then fits the SMPL model by optimizing its parameters to align with the computed humanoid keypoints.
- Finally, the pipeline converts the large-scale human motion dataset into a dataset containing feasible humanoid motions.
D. Oracle Policy Training from Large-Scale Human Motions
The oracle policy is trained as a motion imitator using humanoid proprioception, reference goal states, and rewards combining penalties, regularization, and task objectives.
- The oracle motion imitator uses proprioception containing rigid-body position, orientation, linear and angular velocity, and the previous action.
- Its goal state encodes reference pose information and one-frame differences between reference and current rigid-body states.
- The oracle reward sums penalty, regularization, and task-reward components.
E. Multi-Mode Versatile Controller via Distillation
HOVER distills an oracle teacher into a student policy that receives masked multi-mode commands and learns to reproduce the teacher’s actions across command configurations.
- The student policy uses a 25-step history of joint state, base motion, gravity, and action terms as proprioceptive input.
- Table III evaluates motion imitation on dataset ˆQ, highlighting mode-tracked metrics and statistically significant results across five random seeds.
- Mode masking independently selects upper- and lower-body command modes, while sparsity masking restricts which body targets are active.
- DAgger rolls out the student, queries the oracle on corresponding states, and supplies teacher actions for distillation.
- The student is trained by minimizing the squared difference between oracle reference actions and student actions.
III. EXPERIMENT
The experiments evaluate HOVER in simulation and on a real humanoid robot against specialist and multi-mode baselines, using motion-tracking and physical-performance metrics.
- Research questions: The study asks whether HOVER outperforms specialist policies, alternative multi-mode training, and transfers to real-world hardware.
- Experiment setup: Evaluation covers simulation with retargeted AMASS motions and real-world tests on 20 standing sequences using a 19-DOF Unitree H1.
- Baselines: Specialist baselines track distinct command configurations, including upper-body joints, root velocity, whole-body joints, or selected body keypoints.
- Metrics: Reported metrics include survival rate, global and root-relative body-position error, joint tracking error, root velocity, and root orientation error.
- Metrics: Table IV reports only the tracking metrics associated with each evaluated control mode.
A. Comparison with Specialists
HOVER is evaluated against specialist controllers across multiple command modes and consistently generalizes better. It outperforms prior-work specialists across the evaluated control configurations.
- A. Comparison with Specialists: HOVER outperforms prior-work specialist controllers in at least 7 of 12 metrics for every evaluated command mode.The comparison includes additional left-hand, right-hand, two-hand, and head modes.
- A. Comparison with Specialists: Distilling from an oracle policy surpasses RL-trained specialists even when evaluating a single control mode.
B. Comparison with Other Generalist Training Methods.
HOVER is compared with a multi-mode RL policy across eight control modes and four tracking metrics. It achieves consistently lower tracking error across all evaluated metric-mode combinations.
- B. Comparison with Other Generalist Training Methods.: The comparison evaluates HOVER and Multi-Mode RL across eight humanoid control modes using four tracking metrics.The metrics are root orientation, upper joint angle, and local and global body positions.
- B. Comparison with Other Generalist Training Methods.: HOVER achieves lower tracking error across 32/32 evaluated metrics and modes than the multi-mode RL baseline.The tracking errors are scaled for visualization so larger radar webs indicate better performance.
- B. Comparison with Other Generalist Training Methods.: The results support distilling from an oracle policy that tracks full-body kinematics for learning a generalist whole-body controller.
C. Real-World Evaluation
Real-world evaluations test HOVER on standing-motion tracking, locomotion with abrupt mode switches, and teleoperation with masked reference positions. The reported experiments show specialist-comparable tracking and successful multi-mode transitions on hardware.
- C. Real-World Evaluation: HOVER outperforms specialist policies in 11 of 12 real-world tracking metrics across 20 standing motions.The evaluation also demonstrates root-pitch tracking and full-body kinematic tracking of highly dynamic running motions.
- C. Real-World Evaluation: HOVER transitions successfully between command modes during forward walking, turning, and backward walking.The tested switches include ExBody to H2O and HumanPlus to OmniH2O.
- C. Real-World Evaluation: In Vision Pro teleoperation, HOVER smoothly tracks across modes when head and hand positions are randomly masked.The demonstration includes tracking only the human head position while ignoring waving hands.
IV. RELATED WORK
Prior humanoid whole-body controllers use distinct abstractions, including kinematic motion, joint-angle, and root-velocity tracking. HOVER consolidates motor skills across such modes through kinematic imitation and policy distillation, while future work targets automated mode switching.
- IV. RELATED WORK: Learning-based humanoid controllers commonly use kinematic motion tracking, local joint-angle tracking, or root-velocity tracking.These abstractions reflect different task and application requirements.
- IV. RELATED WORK: Existing control modes are independently developed and tailored to specific task subsets, limiting a general-purpose control framework.
- IV. RELATED WORK: HOVER uses a kinematic motion imitator and policy distillation to consolidate motor skills across multiple control modes into one policy.The paper reports that this unified policy outperforms specialized controllers.
- IV. RELATED WORK: Future work will develop an automated mode-switching module for real-world applications.