PaperlayerView exact revision on arXiv

Source-linked AI summary

KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

Weiji Xie, Jinrui Han, Jiakun Zheng, Huanyu Li, Xinzhe Liu, Jiyuan Shi, Weinan Zhang, Chenjia Bai, Xuelong Li

arXiv:2506.12851v3cs.ROcs.AI

TL;DR

Humanoid whole-body control methods have struggled to track highly dynamic human motions because reference motions can violate robot constraints and difficult motions lack suitable tracking tolerances. PBHC combines physics-based motion processing with adaptive motion tracking and asymmetric actor-critic training. It achieves lower tracking errors than existing approaches and demonstrates stable, expressive behaviors on the Unitree G1, while remaining limited in environment awareness and multi-motion skill generalization.

  • Problem

    Existing humanoid control algorithms mainly track smooth, low-speed motions, while human references may violate humanoid physical constraints and difficult motions lack suitable tolerance mechanisms.

  • Method

    PBHC extracts, filters, corrects, and retargets motions using physics-based processing, then adapts tracking tolerance through bi-level optimization and trains an asymmetric actor-critic policy.

  • Results

    PBHC consistently outperforms OmniH2O and ExBody2 across evaluation metrics and demonstrates stable, expressive highly dynamic behaviors on the Unitree G1.

  • Takeaways & Limitations

    Physics-based filtering and adaptive tracking support robust humanoid behaviors across athletic and artistic domains, including Kungfu and dancing.

  • Takeaways & Limitations

    The method lacks environment awareness and trains each policy for a single motion, restricting unstructured deployment and efficient diverse-skill repertoires.

Abstract

from arXiv · show

Humanoid robots are promising to acquire various skills by imitating human behaviors. However, existing algorithms are only capable of tracking smooth, low-speed human motions, even with delicate reward and curriculum design. This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing through multi-steps motion processing and adaptive motion tracking. For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints to the maximum extent. For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance based on the current tracking error, creating an adaptive curriculum mechanism. We further construct an asymmetric actor-critic framework for policy training. In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions. Our method achieves significantly lower tracking errors than existing approaches and is successfully deployed on the Unitree G1 robot, demonstrating stable and expressive behaviors. The project page is https://kungfubot.github.io.

1 Introduction

Humanoid whole-body control aims to imitate diverse human behaviors, but physical infeasibility and difficult agile motions limit existing methods. PBHC addresses these challenges through physics-based motion processing, adaptive tracking, and asymmetric policy optimization.

  • Motivation: Human motion datasets make whole-body imitation promising, but humanoid control must handle high-dimensional actions and robot physical constraints.Captured motions may violate joint limits and dynamics, complicating direct transfer to humanoid robots.
  • Prior Work: Existing RL-based frameworks track reference kinematic motions, while prior methods remove infeasible motions through privileged policies or language-based filtering.H2O and OmniH2O produce cleaned datasets using a trained privileged imitation policy, whereas ExBody filters motions using language labels.
  • Limitations of Prior Work: Previous methods mainly track smooth, low-speed motions because difficult motions lack suitable tolerance mechanisms during training.Finding an optimal dataset can also require costly initial policy training, limiting these approaches for highly dynamic behaviors.
  • PBHC Framework: PBHC processes motions by extracting, filtering, correcting, and retargeting them with physics-based metrics, contact masks, and differential inverse kinematics.The pipeline estimates physical quantities, corrects contacts, and retargets processed motions to the robot.
  • PBHC Framework: PBHC adapts tracking tolerance through bi-level optimization and trains policies with an asymmetric actor-critic architecture using privileged information for the critic.The actor uses local observations, while the critic uses reward vectorization and privileged information to improve value estimation.
  • Results: PBHC tracks highly dynamic motions with lower errors than existing methods and produces stable, expressive Kungfu and dancing behaviors on Unitree G1.The real-world demonstration includes complex athletic and artistic motions.

2 Preliminaries

The paper formulates motion imitation for a 23-DoF Unitree G1 as goal-conditional reinforcement learning. Policies generate joint-position targets for a PD controller and are optimized with PPO using an actor-critic architecture.

  • Robot and Problem Formulation: The Unitree G1 provides 23 controllable degrees of freedom, excluding 3 wrist degrees of freedom on each hand.
  • Robot and Problem Formulation: Motion imitation is formulated as a goal-conditional Markov Decision Process with robot states, reference-motion states, actions, rewards, and transitions.The transition function depends on robot morphology and physical constraints.
  • Robot and Problem Formulation: At each time step, the policy observes the robot’s proprioceptive state and generates an action intended to follow the corresponding reference state.
  • Robot and Problem Formulation: Each action is a 23-dimensional target joint position supplied to a PD controller that computes motor torques.
  • Policy Optimization: Policy optimization uses PPO with an actor-critic architecture.
  • Motion Representation: Human motions are represented with SMPL parameters for body shape, joint rotations, and global translation, which map to a 3D mesh.The differentiable skinning function maps these parameters to 6,890 mesh vertices.

3 Methods

PBHC combines physics-aware motion processing with adaptive tracking and reinforcement-learning control to imitate highly dynamic humanoid motions. Its pipeline filters, corrects, and retargets reference motions before policy training, while adapting tracking tolerance through an optimization-based feedback mechanism.

  • Motion processing: PBHC processes human videos into robot-ready references through SMPL motion estimation, physics-based filtering, contact-aware correction, and IK-based retargeting.The pipeline is designed to transfer physically plausible motions to the Unitree G1 while respecting joint limits and end-effector alignment.
  • Motion processing: Physics-based filtering evaluates projected center-of-mass and center-of-pressure distance, requiring stable boundary frames and bounded instability gaps.Motions failing these stability conditions are excluded as unable to maintain dynamic stability.
  • Motion processing: Contact-mask correction offsets global translation when either foot contacts the ground, then applies exponential moving averaging to reduce resulting frame-to-frame jitter.The corrected vertical position uses the lowest SMPL-mesh vertex as the reference height.
  • Adaptive motion tracking: PBHC uses exponential tracking rewards in which σ controls error tolerance, avoiding insensitivity caused by either excessively large or excessively small values.The reward combines task-specific tracking terms for joint, rigid-body, and foot-contact states with regularization rewards for stability and smoothness.
  • Adaptive motion tracking: A bi-level optimization selects σ to minimize the converged policy’s accumulated tracking error, while the lower level trains the policy under the current σ.The outer objective is performance-driven and based on absolute external tracking metrics rather than reward maximization alone.
  • Adaptive motion tracking: Under additional technical assumptions, the optimal tracking factor equals the average optimal tracking error, enabling closed-loop σ adjustment during training.PBHC constrains σ to be non-increasing and initializes it relatively large before progressively tightening tolerance as tracking improves.

4 Related Works

Humanoid motion imitation must bridge diverse human behaviors and the physical differences between humans and robots. Existing approaches include task-specific control and data-intensive foundation models, while many datasets omit physical information important for policy learning.

  • Humanoid Motion Imitation: Human motion datasets contain diverse behaviors, but humanoid robots cannot directly learn them because human and robot physical structures differ substantially.Many datasets also lack physical annotations such as foot contacts, which are important for robot policy learning.
  • Humanoid Whole-Body Control: Traditional humanoid control commonly separates locomotion and manipulation, requiring delicate reward design and making human-like behaviors difficult to obtain.RL-based locomotion controllers have been applied to tasks including complex-terrain walking, gait control, standing up, jumping, and parkour.
  • Humanoid Whole-Body Control: Humanoid foundation models require a large number of trajectories, whereas this work requires only a small number of reference motions to learn diverse behaviors.

5 Experiments

The experiments evaluate motion filtering, simulation tracking, adaptive tracking, real-world deployment, and training stability. PBHC filters untrackable motions, outperforms deployable baselines, adapts across motion types, and demonstrates dynamic skills on the Unitree G1.

  • Evaluation Method: The evaluation uses easy, medium, and hard highly-dynamic motions, with three random seeds and 1,000 rollout episodes per setting.Policies are trained in IsaacGym using a dataset constructed by the proposed motion-processing pipeline.
  • Motion Filtering: Four of 10 motion sequences are rejected by physics-based filtering, while six are accepted and evaluated using Episode Length Ratio.ELR is the ratio of average episode length to reference motion length.
  • Motion Filtering: Rejected motions reach a maximum ELR of only 54%, whereas accepted motions consistently achieve high ELRs.The results support excluding inherently untrackable motions and focusing training on viable candidates.
  • Main Result: PBHC consistently outperforms OmniH2O and ExBody2 across all evaluation metrics.MaskedMimic performs well on some metrics but is not deployable for robot control because it does not account for partial observability and action smoothness.
  • Impact of Adaptive Motion Tracking Mechanism: Fixed tracking factors vary in performance across motion types, while the adaptive mechanism achieves near-optimal performance consistently.The ablation compares Coarse, Medium, LowerBound, and UpperBound fixed configurations.
  • Real-World Deployment: The Unitree G1 demonstrates boxing, kicking, spins, deep squats, stretches, dance, and Tai Chi in real-world deployment.The reported repertoire spans athletic and artistic behaviors.
  • Real-World Deployment: Real-world Tai Chi tracking metrics are closely aligned with MuJoCo simulation metrics across 10 trials.The real-world measurements use onboard sensor readings.
  • Learning Curves: Mean episode length and mean reward for Jabs Punch, Tai Chi, and Roundhouse Kick gradually stabilize after approximately 20k training steps.

6 Conclusion & Limitations

PBHC combines physics-based motion processing and adaptive motion tracking to improve highly-dynamic humanoid control, while remaining limited in environmental awareness and motion-repertoire breadth.

  • PBHC uses physics-based motion processing and adaptive motion tracking to achieve highly-dynamic humanoid behaviors and superior tracking accuracy.
  • The motion-filtering metric efficiently removes trajectories that are difficult to track, while adaptive tracking consistently outperforms baselines on tracking error.
  • Real-world deployments demonstrate robust athletic and artistic behaviors, including Kungfu and dancing on the Unitree G1 robot.
  • Limitations: The method lacks terrain perception and obstacle avoidance, restricting deployment in unstructured real-world settings.
  • Limitations: Each policy imitates a single motion, limiting efficiency for applications requiring diverse motion repertoires.

NeurIPS Paper Checklist

The checklist records the paper’s claimed scope, theory status, reproducibility disclosures, experimental-detail disclosures, statistical-reporting justification, and discussion of limitations and societal impacts.

  • The paper states that its contributions and scope are claimed in the abstract and introduction, with key contributions summarized in Section 1.
  • The paper discusses work limitations in Section 6 and societal impacts in Appendix E.
  • Theory assumptions and proofs: The checklist indicates that the paper does not include theoretical results, while the authors state that theory is proposed and proved in Section 3 with further analysis in Appendix A.
  • Experimental result reproducibility: The authors report that simulation and real-world settings are described sufficiently for reproducing the main experimental results.
  • Experimental setting/details: Training and test details are reported in Section 5 and Appendices C and D, while compute-resource information is provided in Appendix D.
  • Experiment statistical significance: The authors state that evaluation metrics are defined in Section 5.1 and baseline statistical comparisons are presented in Section 5.3.

A Derivation of Optimal Tracking Sigma

The paper derives the optimal tracking factor through a bi-level optimization formulation and solves its gradient conditions using implicit differentiation under stated regularity assumptions.

  • The derivation begins by recalling the bi-level optimization problem in Eq. (6).
  • Assuming R(x)=Ax+b, twice continuously differentiable objectives, and a unique lower-level solution x*(σ), the method uses an implicit gradient approach.
  • Differentiating Eq. (11) with respect to σ provides the gradient relation used in the derivation.
  • The resulting derivative is substituted into Eq. (10), followed by computation of first- and second-order gradients in Eq. (14).
  • Element-wise multiplication is used in the gradient expressions, and setting the gradient to zero yields the optimality condition.

B Dataset Description

The dataset combines video-extracted and open-source motions, organizes 13 sequences by difficulty, and defines separate actor and critic observation spaces for policy training.

  • The dataset integrates video-based motions processed through the proposed multi-step pipeline with selected motions from AMASS and LAFAN.
  • 13 distinct motions are categorized into easy, medium, and hard difficulty levels.
  • Linear interpolation at each sequence boundary transitions between the default pose and reference motion to ensure smooth starts and endings.
  • Actor observation space: The actor observes five-step proprioceptive history together with a time-phase variable.
  • Critic observation space: The critic additionally observes base velocity, reference and positional differences, and domain-randomized physical parameters to improve value estimation robustness.

C.2 Reward Design

The reward design combines task tracking with regularization for physically plausible control, while two curricula progressively increase training difficulty and constraint enforcement.

  • Reward Components: Reward functions combine task rewards with regularization rewards, including penalties for exceeding soft joint-position limits.The soft limits are symmetrically scaled from hard limits using α = 0.95; analogous limits apply to velocity and torque.
  • Transferability: Domain randomization is incorporated during training to support robust sim-to-sim and sim-to-real transfer.The specific randomization settings are provided in Table 7.
  • Implementation Details: PPO hyperparameters, PD controller gains, and simulator ankle-link inertia are specified for implementation and numerical stability.The ankle-link inertia is manually fixed at 5 × 10^-3.
  • Curriculum Mechanisms: A termination curriculum gradually decreases the tracking-error threshold, making motion imitation progressively more difficult.The initial threshold is θ = 1.5, bounded by θmin = 0.3 and θmax = 2.0, with decay rate δ = 2.5 × 10^-5.
  • Curriculum Mechanisms: A penalty curriculum progressively increases the regularization weight to strengthen physical constraints during training.The penalty scale is updated by clipping α · (1 + δ) between αmin and αmax; it starts at α = 0.1 and uses growth rate δ = 1.0 × 10^-4.

D.1 Experiment Setup

Experiments evaluate tracking errors across separately trained baselines and multiple tracking-factor configurations, using six motion-error metrics and a Unitree G1 deployment setup.

  • Compute Setup: Each experiment uses a 24-core Intel i7-13700 CPU, 32 GB RAM, and one NVIDIA RTX 4090 GPU, with each model trained for 27 hours.
  • Real-Robot Setup: Policies are deployed on a Unitree G1 through an onboard control board and external PC connected via Ethernet and DDS.The PC maintains observation history, performs policy inference, and sends target joint angles to the control board.
  • Evaluation Metrics: Tracking is measured using global and root-relative body-position, joint-position, joint-velocity, body-velocity, and body-acceleration errors.The metrics use units of mm, 10^-3 rad, 10^-3 rad/frame, mm/frame, and mm/frame² as defined respectively.
  • Baselines: All baseline methods are trained separately for each motion to ensure fair comparison.
  • Baselines: The baselines include OmniH2O, ExBody2, and MaskedMimic, with differing teacher-student or staged-training procedures.MaskedMimic’s first phase directly optimizes pose-level accuracy without explicit physical-plausibility regularization.
  • Tracking-Factor Configurations: Tracking-factor evaluations compare Coarse, Medium, UpperBound, LowerBound, and initial Ours configurations with converged adaptive factors.The converged factors are reported across motions in the ablation study.

E.1 Analysis of Contact Mask Estimation and Motion Correction Method

The contact-mask and motion-correction analyses assess contact estimation, floating-artifact reduction, and contact-aware tracking across dynamic motions and real-world deployments.

  • Contact Mask Estimation: 91.4% accuracy is achieved by the proposed contact-mask estimation method on a manually labeled dataset containing 10 motion samples.
  • Motion Correction: Motion correction aligns the SMPL motion more accurately with the ground plane and reduces floating artifacts.Before correction, the model height is noticeably elevated relative to the ground; after correction, the motion aligns more closely with the ground plane.
  • Ablation Studies: The adaptive motion-tracking ablation evaluates four motion tasks: Jabs Punch, Charleston Dance, Roundhouse Kick, and Bruce Lee’s Pose.
  • Contact-Mask Ablation: The contact-mask ablation evaluates Charleston Dance, Jump Kick, and Roundhouse Kick, which have distinct foot-contact patterns.Mean foot contact mask error is introduced as an evaluation metric.
  • Contact-Mask Ablation: The proposed contact-aware method significantly reduces foot contact-mask errors compared with the baseline without a contact mask.The method also produces noticeable improvements in other tracking metrics.
  • Real-World Results: Additional real-world deployments cover more highly dynamic motions and demonstrate stable, expressive behaviors, including complex martial-arts and dance skills.The broader-impact discussion identifies safety, labor, security, and environmental concerns associated with such capabilities.
Loading 2506.12851v3…