PaperlayerView exact revision on arXiv

Source-linked AI summary

HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots

Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, Yuke Zhu

arXiv:2410.21229v2cs.RO

TL;DR

Humanoid controllers often specialize in a single command space, limiting adaptation across locomotion and manipulation tasks. HOVER uses full-body kinematic motion imitation and policy distillation to consolidate diverse modes into one unified policy, which outperforms specialist controllers across modes and supports seamless transitions. The paper positions this as a scalable way to reuse motor skills without retraining a separate policy for each command configuration.

Abstract

from arXiv · show

Humanoid whole-body control requires adapting to diverse tasks such as navigation, loco-manipulation, and tabletop manipulation, each demanding a different mode of control. For example, navigation relies on root velocity tracking, while tabletop manipulation prioritizes upper-body joint angle tracking. Existing approaches typically train individual policies tailored to a specific command space, limiting their transferability across modes. We present the key insight that full-body kinematic motion imitation can serve as a common abstraction for all these tasks and provide general-purpose motor skills for learning multiple modes of whole-body control. Building on this, we propose HOVER (Humanoid Versatile Controller), a multi-mode policy distillation framework that consolidates diverse control modes into a unified policy. HOVER enables seamless transitions between control modes while preserving the distinct advantages of each, offering a robust and scalable solution for humanoid control across a wide range of modes. By eliminating the need for policy retraining for each control mode, our approach improves efficiency and flexibility for future humanoid applications.

I. INTRODUCTION

HOVER addresses the difficulty of building humanoid controllers that adapt across task-specific command modes by unifying them around shared motion objectives. It distills motor skills from an oracle motion imitator into one generalist policy supporting diverse modes and transitions.

Motivation

  • Task-specific whole-body controllers use different command spaces, making development repetitive and limiting seamless switching across locomotion and manipulation.Examples include root velocity tracking for locomotion and joint-angle or end-effector tracking for precise manipulation.Source: §I · p. 1
  • The framework exploits aligned motion objectives—stable, human-like motion—despite differences between control interfaces.This shared objective motivates integrating otherwise distinct command spaces.Source: §I · p. 1

Approach

  • HOVER unifies diverse humanoid control modes through a common neural controller supporting over 15 modes on a 19-DOF robot.Its command space covers most modes used in prior work.Source: §I · p. 1
  • An oracle motion imitator trained on large-scale MoCap data supplies reusable motor skills, which policy distillation transfers into a single generalist policy.The design uses human-motion priors for broad movement and control objectives.Source: §I · pp. 1–2

Results

  • Experiments report shared motor skills, seamless mode transitions, and superior multi-mode control compared with individually trained policies in simulation and on a real humanoid.These are stated as the paper’s three main contribution-level findings.Source: §I · p. 2

II. METHOD

The method formulates humanoid control as goal-conditioned reinforcement learning in which proprioception and a unified target-goal representation guide joint-position actions. Policies are trained with PPO to track commands over time.

Goal-Conditioned RL

  • HOVER formulates humanoid control as goal-conditioned reinforcement learning, combining agent proprioception with a target goal state representing the desired motion.The goal state provides a unified representation of the target motion goal.Source: A. Goal-Conditioned RL for Humanoid… · p. 2
  • The policy outputs 19-dimensional target joint positions that are fed to a PD controller to actuate the robot’s degrees of freedom.The action space is explicitly tied to the robot’s 19 joints.Source: A. Goal-Conditioned RL for Humanoid… · p. 2
  • PPO maximizes cumulative discounted reward while the humanoid learns to follow target commands at each timestep.This frames training as a command-tracking task.Source: A. Goal-Conditioned RL for Humanoid… · p. 2

B. Command Space Design for Humanoid Control

HOVER defines a unified, expressive command space that covers prior control configurations while allowing independent combinations of upper- and lower-body tracking modes. One-hot masking selects the command components active for each task.

Design Goals

  • Prior humanoid control modes are tailored to specific task subsets, whereas HOVER seeks a comprehensive framework adaptable across scenarios.Root-only tracking can limit whole-body loco-manipulation capabilities.Source: B. Command Space Design for Humanoid… · p. 2
  • The command space is designed for generality, including prior configurations and interfaces such as joysticks, keyboards, motion capture, exoskeletons, and VR headsets.This supports replacing task-specific controllers without sacrificing versatility.Source: B. Command Space Design for Humanoid… · p. 3, Table I · p. 3
  • Atomic command dimensions allow arbitrary combinations of control options to support different modes.Independent dimensions make selective command activation possible.Source: B. Command Space Design for Humanoid… · p. 3

Unified Command Space

  • HOVER divides control into upper- and lower-body regions and includes kinematic position, local joint-angle, and root tracking modes.Root tracking includes velocity, height, and roll, pitch, and yaw orientation targets.Source: B. Command Space Design for Humanoid… · p. 3
  • A one-hot masking vector activates selected command components, allowing prior control designs to appear as subsets of the unified space.The framework supports tracking arbitrary subsets of command elements.Source: B. Command Space Design for Humanoid… · p. 3, Table I · p. 3

C. Motion Retargeting

The retargeting pipeline converts human motion data into feasible humanoid motions by aligning human-model parameters with humanoid forward-kinematics keypoints.

  • The procedure first computes humanoid keypoint positions from joint configurations using forward kinematics.Source: C. Motion Retargeting · p. 3
  • It then fits the SMPL model by optimizing its parameters to align with the computed humanoid keypoints.Source: C. Motion Retargeting · p. 3
  • Finally, the pipeline converts the large-scale human motion dataset into a dataset containing feasible humanoid motions.Source: C. Motion Retargeting · p. 3

D. Oracle Policy Training from Large-Scale Human Motions

The oracle policy is trained as a motion imitator using humanoid proprioception, reference goal states, and rewards combining penalties, regularization, and task objectives.

E. Multi-Mode Versatile Controller via Distillation

HOVER distills an oracle teacher into a student policy that receives masked multi-mode commands and learns to reproduce the teacher’s actions across command configurations.

III. EXPERIMENT

The experiments evaluate HOVER in simulation and on a real humanoid robot against specialist and multi-mode baselines, using motion-tracking and physical-performance metrics.

Research questions

  • The study asks whether HOVER outperforms specialist policies, alternative multi-mode training, and transfers to real-world hardware.Source: §III · p. 4

Experiment setup

  • Evaluation covers simulation with retargeted AMASS motions and real-world tests on 20 standing sequences using a 19-DOF Unitree H1.Source: §III · p. 4

Baselines

  • Specialist baselines track distinct command configurations, including upper-body joints, root velocity, whole-body joints, or selected body keypoints.Source: §III · p. 4

Metrics

  • Reported metrics include survival rate, global and root-relative body-position error, joint tracking error, root velocity, and root orientation error.Source: §III · pp. 4–5
  • Table IV reports only the tracking metrics associated with each evaluated control mode.Source: Table IV · p. 5

A. Comparison with Specialists

HOVER is evaluated against specialist controllers across multiple command modes and consistently generalizes better. It outperforms prior-work specialists across the evaluated control configurations.

A. Comparison with Specialists

  • HOVER outperforms prior-work specialist controllers in at least 7 of 12 metrics for every evaluated command mode.The comparison includes additional left-hand, right-hand, two-hand, and head modes.Source: A. Comparison with Specialists · p. 5
  • Distilling from an oracle policy surpasses RL-trained specialists even when evaluating a single control mode.Source: A. Comparison with Specialists · p. 5

B. Comparison with Other Generalist Training Methods.

HOVER is compared with a multi-mode RL policy across eight control modes and four tracking metrics. It achieves consistently lower tracking error across all evaluated metric-mode combinations.

B. Comparison with Other Generalist Training Methods.

C. Real-World Evaluation

Real-world evaluations test HOVER on standing-motion tracking, locomotion with abrupt mode switches, and teleoperation with masked reference positions. The reported experiments show specialist-comparable tracking and successful multi-mode transitions on hardware.

C. Real-World Evaluation

  • HOVER outperforms specialist policies in 11 of 12 real-world tracking metrics across 20 standing motions.The evaluation also demonstrates root-pitch tracking and full-body kinematic tracking of highly dynamic running motions.Source: C. Real-World Evaluation · p. 5, Table V · p. 6
  • HOVER transitions successfully between command modes during forward walking, turning, and backward walking.The tested switches include ExBody to H2O and HumanPlus to OmniH2O.Source: C. Real-World Evaluation · p. 5, Figure 5 · p. 6
  • In Vision Pro teleoperation, HOVER smoothly tracks across modes when head and hand positions are randomly masked.The demonstration includes tracking only the human head position while ignoring waving hands.Source: C. Real-World Evaluation · p. 6

IV. RELATED WORK

Prior humanoid whole-body controllers use distinct abstractions, including kinematic motion, joint-angle, and root-velocity tracking. HOVER consolidates motor skills across such modes through kinematic imitation and policy distillation, while future work targets automated mode switching.

IV. RELATED WORK

  • Learning-based humanoid controllers commonly use kinematic motion tracking, local joint-angle tracking, or root-velocity tracking.These abstractions reflect different task and application requirements.Source: §IV · p. 6
  • Existing control modes are independently developed and tailored to specific task subsets, limiting a general-purpose control framework.Source: §IV · p. 6
  • HOVER uses a kinematic motion imitator and policy distillation to consolidate motor skills across multiple control modes into one policy.The paper reports that this unified policy outperforms specialized controllers.Source: §V · p. 6
  • Future work will develop an automated mode-switching module for real-world applications.Source: §V · p. 6

Related papers

Loading 2410.21229v2…