PaperlayerView exact revision on arXiv

Source-linked AI summary

PHUMA: Physically Reliable Humanoid Locomotion Dataset

Kyungmin Lee, Sibeen Kim, Youngdo Lee, Minho Park, Hyunseung Kim, Dongyoon Hwang, Donghu Kim, Hojoon Lee, Jaegul Choo

arXiv:2510.26236v2cs.RO

TL;DR

Humanoid locomotion must be reliable and natural for real-world deployment, but task-reward reinforcement learning can produce effective yet non-humanlike gaits. PHUMA builds a locomotion corpus by filtering infeasible motions and applying physics-constrained, shape-adaptive retargeting. PHUMA-trained policies outperform AMASS and Humanoid-X in success rate across 11 motion types and achieve 16.3% lower real-world tracking error than AMASS on 24 test motions.

  • Problem

    Humanoid locomotion must be reliable and natural for real-world deployment, but task-reward reinforcement learning can produce effective yet non-humanlike gaits.

    Source: §1 · p. 1

  • Method

    PHUMA builds a locomotion corpus by filtering infeasible motions and applying physics-constrained, shape-adaptive retargeting.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    PHUMA-trained policies outperform AMASS and Humanoid-X in success rate across 11 motion types and achieve 16.3% lower real-world tracking error than AMASS on 24 test motions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    PHUMA-trained policies extend their tracking advantage to a real Unitree G1, with lower tracking errors across all reported metrics than the AMASS-trained policy.

    Source: §4.5 · p. 7

    PHUMA focuses on flat-terrain locomotion and does not yet cover scene interaction or uneven terrain.

    Source: §5 · p. 8

Abstract

from arXiv · show

Motion imitation is a promising approach for humanoid locomotion, enabling agents to acquire humanlike behaviors. Existing methods typically rely on high-quality motion capture datasets such as AMASS, but these are scarce and expensive, limiting scalability and diversity. Recent studies attempt to scale data collection by converting large-scale internet videos, exemplified by Humanoid-X. However, they often suffer from physical artifacts such as floating, penetration, and foot skating, which hinder stable imitation. To address this, we introduce PHUMA, a Physically Reliable HUMAnoid locomotion dataset produced by a two-stage pipeline combining physics-aware curation and physics-constrained retargeting, aggregating both motion capture and internet video into a physically reliable, 73-hour corpus. On motion tracking benchmarks, PHUMA-trained policies achieve higher success rates than those trained on AMASS and Humanoid-X, and successfully transfer zero-shot to a real Unitree G1. The code is available at https://davian-robotics.github.io/PHUMA.

1 Introduction

Humanoid motion imitation is constrained by limited motion-data scale and diversity, while video-derived motions can contain physical artifacts. PHUMA combines physics-aware curation with constrained retargeting and is evaluated against existing datasets and on a real Unitree G1.

1 Introduction

  • Task-oriented reinforcement-learning rewards can produce effective but non-humanlike humanoid gaits because they struggle to capture natural whole-body coordination.Motion imitation instead trains policies to replicate human movements through data collection, retargeting, and reinforcement-learning tracking.Source: §1 · pp. 1–2
  • High-quality motion-capture datasets such as LaFAN1 and AMASS are limited in scale and diversity, while video-derived Humanoid-X motions can contain floating, penetration, joint violations, and foot skating.The cited problems are attributed to translation errors in video-to-motion recovery and retargeting that prioritizes joint alignment over physical plausibility.Source: §1 · p. 2
  • PHUMA filters infeasible motions and enforces physical constraints during retargeting, yielding 73.0 hours of physically feasible motion—3.5× AMASS and more than Humanoid-X’s 69.1 hours.The 69.1 hours are drawn from 231.4 hours of Humanoid-X data.Source: §1 · p. 2
  • PHUMA-trained policies outperform AMASS and Humanoid-X in success rate across 11 motion types and achieve 16.3% lower real-world tracking error than AMASS on 24 test motions.The real-robot comparison uses motions sampled from the PHUMA test set.Source: §1 · p. 2, Figure 2 · p. 2

2 Related Work

Humanoid locomotion datasets draw on motion capture and video reconstruction, while retargeting must reconcile differences between human and robot bodies. PHUMA combines physics-aware curation with physically constrained, shape-adaptive retargeting.

2 Related Work

  • PHUMA constructs a large-scale, physically reliable humanoid locomotion dataset by collecting diverse motion data and retargeting it to humanoid robots.Its related-work discussion frames the task around both source data and robot adaptation.Source: §2 · p. 2

2.1 Human Motion Data

  • Motion capture offers accurate kinematics but requires costly instrumentation and studio effort; video reconstruction scales more readily but can contain jitter, foot sliding, and ground penetration.LaFAN1 has only a few hours of motion, while AMASS remains walking-centric; Humanoid-X is cited as an abundant internet-video dataset.Source: §2.1 · p. 3

2.2 Humanoid Motion Retargeting

  • PhySINK augments shape-adaptive inverse kinematics with joint-feasibility, grounding, and anti-skating constraints, using curation-derived contact signals to retarget noisy in-the-wild video.The passage contrasts this with methods that rely on the lowest foot position to infer ground contact.Source: §2.2 · p. 3

3 Method

PHUMA’s pipeline curates motion data and retargets it to humanoids with physical constraints. Its curation removes corrupted clips while preserving usable segments, and PhySINK addresses physical violations during retargeting.

3 Method

  • The PHUMA pipeline curates diverse motion, retargets it with PhySINK, trains a policy to imitate the retargeted motions, and uses that policy to control the humanoid.At inference, the policy can imitate motions from unseen videos processed by a video-to-motion model.Source: §3 · p. 3, Figure 3 · p. 4

3.1 Physics-Aware Motion Curation

  • Curation targets severe jitter, instability from interactions with absent objects, and incorrect foot-ground contact; it filters 4-second clips using jerk, center-of-mass support, and contact criteria.Clip-level filtering preserves valid segments from sequences that are only partly flawed.Source: §3.1 · pp. 3–4
  • The resulting PHUMA corpus contains 73.0 hours of physically reliable motion across 76.0K clips, augmented with LaFAN1, LocoMuJoCo, and in-house video captures.The full dataset composition is reported in Appendix A.2.Source: §3.1 · p. 4

3.2 Physics-Constrained Motion Retargeting

  • Standard IK can distort motion when human and robot skeleton proportions differ, while SINK preserves style through shape adaptation but leaves physical plausibility unaddressed.The described artifacts include joint violations and unrealistic ground interactions.Source: §3.2 · p. 5
  • PhySINK augments SINK with fidelity, joint-feasibility, grounding, and skating losses, jointly optimizing motion similarity and physical reliability.The objective weights the feasibility, grounding, and skating losses alongside the fidelity loss.Source: §3.2 · p. 5
  • Retargeting is evaluated by frame percentages meeting five criteria: motion fidelity, joint feasibility, non-floating, non-penetration, and non-skating.The evaluation compares a standard IK solver, GMR, and SINK on Unitree G1 and H1-2 motions.Source: §3.2 · p. 5
  • PhySINK is the only compared retargeting method that performs consistently well across all five physical-reliability metrics.IK and GMR lose motion fidelity, SINK has physical violations, and GMR’s high non-penetration comes with severe floating elsewhere.Source: §3.2 · p. 5, Table 1 · p. 5

4 Experiments

The experiments test curation, retargeting, dataset quality, and real-robot transfer. PHUMA performs strongly across motion categories and improves real-world tracking errors over AMASS.

4 Experiments

  • The experiments evaluate curation, retargeting, dataset comparisons, and real-robot transfer using MaskedMimic in simulation and BeyondMimic on hardware.Simulation uses Unitree G1 and H1-2; hardware evaluation uses the Unitree G1.Source: §4 · p. 6, §4.1 · p. 6

4.1 Experiment Setup

  • Success rate uses a 0.15 m threshold for RQ1–RQ3; real-world evaluation instead uses local MPJPE, DoF velocity error, and acceleration error.The real-world setup cannot record global translations because mocap equipment is unavailable.Source: §4.1 · p. 6

4.2 Effect of Physics-Aware Curation

  • Applying all curation filters yields a substantially larger tracking gain than any single filter, despite retaining roughly one quarter of the unfiltered motion volume.The controlled comparison uses SINK retargeting to isolate curation effects.Source: §4.2 · p. 6, Table 2 · p. 6

4.3 PhySINK Retargeting Method Effectiveness

  • PhySINK consistently outperforms IK, GMR, and SINK on both evaluation sets, with the largest gains on angular and vertical motions.The comparison retargets identical AMASS motions and trains a separate full-state policy for each method.Source: §4.3 · p. 7, Table 3 · p. 7

4.4 PHUMA Dataset Effectiveness

  • PHUMA achieves the highest success rates across motion categories on both humanoids, while Humanoid-X’s scale and AMASS and LaFAN1’s cleaner data alone are insufficient.PHUMA combines large scale with high-quality motions.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 Real-World Experiment

  • On the Unitree G1, PHUMA-trained policies have lower tracking errors across all metrics and more faithful motion tracking than AMASS-trained policies.The comparison evaluates six motions per category from the PHUMA test split.Source: §4.5 · p. 7, Table 5 · p. 8, Figure 4 · p. 8

5 Limitations

PHUMA is limited to flat-terrain locomotion and does not yet cover scene interaction or uneven terrain; video-derived motions are substantially filtered, and retargeting does not eliminate all physical artifacts.

5 Limitations

  • The pipeline covers flat-terrain locomotion, not scene interaction or uneven terrain, and PHUMA motions still retain some retargeting artifacts.Extending coverage would require jointly modeling humanoid dynamics with environment geometry and contact; video-to-motion recovery also limits how many estimated motions survive curation.Source: §5 · p. 8

6 Conclusion

The conclusion presents PHUMA as a large-scale, physically reliable locomotion dataset. Its appendix describes motion cleaning and a majority-vote procedure for estimating a consistent ground plane.

6 Conclusion

  • PHUMA combines diverse human videos, physics-aware curation, and PhySINK retargeting to produce diverse, physically reliable motions.Policies trained on PHUMA outperform those trained on LaFAN1, AMASS, and Humanoid-X on Unitree G1 and H1-2, with advantages also reported on a real Unitree G1.Source: §6 · p. 8

A.1 Data Preprocessing

  • The preprocessing stage cleans motion data because sensor errors, tracking inaccuracies, and estimation artifacts can negatively affect retargeting.All sequences are resampled to 30 Hz and smoothed with a zero-phase, fourth-order Butterworth filter; cutoffs are 3 Hz for root translation and 6 Hz for global orientation and body pose.Source: §A.1 · p. 16, §A.1.1 · p. 16

A.1.1 Low-Pass Noise Filtering for Motion Data

  • The low-pass filter smooths motion by removing high-frequency components while preserving the overall trajectory.Root translation uses a 3 Hz cutoff; global orientation and body pose use 6 Hz.Source: §A.1.1 · p. 16

A.1.2 Extracting Ground Contact Information

  • Ground height is estimated by majority vote over foot-vertex heights across frames, then the motion is shifted so the selected ground lies at zero.The method uses 22 low vertices from each of four foot regions and selects the height with the most vertices within a 2.5 cm band.Source: §A.1.2 · pp. 16–17, Figure 5 · p. 16, Figure 6 · p. 17

A.1.3 Filtering Motion Data by Physical Information

The section describes physical screening of motion clips and the constraints used in PhySINK retargeting. PHUMA combines diverse sources into a balanced 73-hour dataset.

A.1.3 Filtering Motion Data by Physical Information

  • Motion clips are screened using root jerk, foot contact, pelvis height, and pelvis/spine-1 distance to the base of support; clips failing thresholds are discarded.Source: §A.1.3 · p. 17, Table 7 · p. 18

A.2 Dataset Composition and Statistics

  • 73.0 hours of physically reliable clips give PHUMA more balanced motion coverage than existing datasets, including categories absent or unevenly represented in LaFAN1 and AMASS.PHUMA’s composition spans diverse sources and is not dominated by a single motion type.Source: §A.2 · p. 18, Table 8 · p. 19

B Motion Retargeting

  • PhySINK combines motion-fidelity, joint-feasibility, grounding, and skating losses in its retargeting objective.Source: B Motion Retargeting · p. 18

B.1.1 Motion Fidelity Loss

  • Motion fidelity combines global joint-position matching, local joint-configuration matching, and temporal smoothness penalties.Source: §B.1.1 · pp. 19–20

B.1.2 Joint Feasibility Loss

  • Joint feasibility penalizes joint angles and velocities that exceed 98% of their allowable limits.Source: §B.1.2 · p. 20

B.1.3 Grounding Loss

  • The grounding loss keeps foot regions on the ground plane during detected contact to correct floating or penetration.Source: §B.1.3 · p. 20

B.1.4 Skating Loss

  • The skating loss penalizes horizontal foot-region velocity while the foot is in contact with the ground.Source: §B.1.4 · p. 20

B.2 Comparison of Retargeting Methods

PhySINK adapts human motion to robot proportions while retaining joint angles and applying physical constraints. Its retargeted policies outperform GMR-retargeted policies on both PHUMA evaluation benchmarks.

B.2 Comparison of Retargeting Methods

  • Unlike GMR’s height-ratio and manually tuned limb scaling, PhySINK fits SMPL shape parameters to the robot in a shared T-pose, then preserves original joint angles while adapting limb lengths.Source: §B.2 · p. 21
  • PhySINK-retargeted policies outperform GMR-retargeted policies on both PHUMA Test and Unseen Video benchmarks.Table 9 compares MaskedMimic policies trained under the same setup.Source: Table 9 · p. 21, §B.2 · p. 21

C.1 Observation Space Compositions

  • The observation space has proprioceptive-state and goal-state components.Source: §C.1 · p. 23

C.1.1 MaskedMimic

  • MaskedMimic proprioception includes root height and body positions, rotations, velocities, and angular velocities; its goals encode relative future states and root-relative absolute states.Source: §C.1.1 · p. 23

C.1.2 BeyondMimic

BeyondMimic adds a sampled future reference trajectory to its goal observation, which is associated with higher success rates across the reported settings. The section also details its training, randomization, and unseen-video evaluation setup.

C.1.2 BeyondMimic

  • BeyondMimic’s proprioceptive state includes the previous action, body-frame base angular velocity, and DoF positions and velocities; its instantaneous goal includes reference DoF states and torso orientation.Source: §C.1.2 · p. 23
  • Adding a future-trajectory component to BeyondMimic’s goal state yields substantially higher success rates across both simulators and both PHUMA Test and Unseen Video splits.The added trajectory samples 20 frames over a 1.9-second horizon and is encoded into a 128-D embedding.Source: §C.1.2 · p. 23, Table 12 · p. 24

C.2 Reward Function

C.2.1 MaskedMimic

  • MaskedMimic regularization includes action-rate penalties to discourage large consecutive action changes and promote smooth joint movements.Source: §C.2.1 · p. 24

C.2.2 BeyondMimic

  • BeyondMimic tracks both global reference motion and relative motion re-anchored to the robot’s root, and uses its standard regularization terms.Source: §C.2.2 · p. 25

C.3 PPO Hyperparameters

C.4 Domain Randomization

  • Domain randomization is used during training to improve robustness to model and sensor inaccuracies.Source: §C.4 · p. 25

C.4.1 BeyondMimic

  • BeyondMimic randomizes actuator gains, base inertial and contact properties, external pushes, initial poses, and actor-stream observation noise; randomization is disabled at evaluation.Source: §C.4.1 · p. 27, Table 17 · p. 27

D.2 Success Rate Threshold Analysis

The threshold analysis finds that dataset-trained policies look more similar under a loose success criterion, while stricter evaluation reveals larger differences. PHUMA’s advantage also persists in a zero-shot simulator transfer evaluation.

D.2 Success Rate Threshold Analysis

  • Under the 0.5 m threshold, dataset success rates differ modestly; under 0.15 m, differences become substantially more pronounced, consistent with more precise PHUMA-trained tracking.Source: §D.2 · p. 28, Table 18 · p. 28, Table 19 · p. 28

D.3 Cross-Simulator Generalization Analysis

E.1 Mocap only and Video only data performance

Video-sourced PHUMA data yields stronger imitation than its mocap subset, which the authors attribute to broader coverage and nearly twice the data. The section also reports filtering, retargeting, and pelvis-path results across several evaluations.

E.1 Mocap only and Video only data performance

  • Video-sourced PHUMA consistently outperforms its mocap-sourced subset across motion categories, with broader coverage and nearly twice the data attributed as the main reason.The passage says PhySINK provides similar retargeting quality for both subsets; Table 24 evaluates MaskedMimic on Unitree G1.Source: §E.1 · p. 29, Table 24 · p. 30

E.2.1 Data Distribution Based on Physics-based Filtering

  • 27.1% of the original Humanoid-X dataset remains after sequential jerk, contact, height, and base-of-support filtering.The retained motions satisfy the stated physical-plausibility constraints; Figure 12 shows the filtering distribution.Source: Figure 12 · p. 30, §E.2.1 · p. 30

E.2.2 PhySINK’s Robustness to Noisy Motion Sources

  • PhySINK robustly retargets Humanoid-X sources with varying noise levels, including raw data and versions filtered by individual criteria or all criteria.The analysis excludes pre-retargeted LaFAN1 and LocoMuJoCo data to isolate filtering effects on Humanoid-X.Source: §E.2.2 · pp. 30–31, Table 25 · p. 31

E.3 Impact of Motion Retargeting Quality on Policy Performance

  • SINK-based retargeting methods outperform IK-based methods, while SINK variants show comparable tracking despite differences in penetration and skating artifacts.The authors identify motion fidelity as the most critical factor for tracking; after it reaches a threshold, reducing other artifact types yields diminishing returns.Source: Table 26 · p. 31, §E.3 · p. 31

F Pelvis-Only Path Following Control Performance

G Results on H1-2 Robot

G.1 PhySINK Ablation on H1-2

The appendix reports H1-2 ablations of PhySINK constraints, retargeting-method comparisons, and dataset comparisons. The supplied passages describe the evaluations but do not state their comparative outcomes.

G.1 PhySINK Ablation on H1-2

G.2 Retargeting Method Comparison on H1-2

G.3 Dataset Effectiveness on H1-2

Related papers

Loading 2510.26236v2…