PaperlayerView exact revision on arXiv

Source-linked AI summary

Morphology-Aware Human Motion Retargeting for Wheeled-Humanoid Loco-Manipulation

Chenbo Xia, Chao Ye

arXiv:2609.11357v1cs.RO

TL;DR

The paper addresses the limited support for coupled locomotion and manipulation when retargeting general human motion to wheeled humanoids without leg joints. It introduces a morphology-aware retargeting, planning, and policy pipeline for the Galaxea R1 Pro, and reports qualitative evidence that the resulting system can track screened human-motion references while maintaining wheeled locomotion and manipulation behavior.

  • Problem

    Existing human-motion retargeting is largely developed for legged humanoids, whose assumptions do not hold for wheeled platforms lacking leg joints during coupled locomotion and manipulation.

  • Method

    The method combines canonical SMPL-X preprocessing, planar-base normalization, morphology-aware IK, shoulder-rooted arm scaling, continuous torso substitution, reference-twist wheel planning, and a 21-dimensional BaseDecode PPO policy.

  • Results

    15 of 16 selected references complete without a fall, while the combined planner and contact layer preserves wheel contact and planar tracking during large translations and turns.

  • Takeaways & Limitations

    The system establishes a practical bridge from screened human motion to wheeled-humanoid loco-manipulation rather than a visualization-only retargeter.

  • Takeaways & Limitations

    Current evidence is entirely from Isaac Sim simulation and generated reference motion, with no hardware experiment included.

Abstract

from arXiv · show

Human-to-humanoid retargeting has largely been studied on legged platforms, while comparatively few wheeled-humanoid systems support coupled locomotion and manipulation from general human motion. Building on GMR's configurable general-motion retargeting and BeyondMimic's physically simulated R1 Pro learning framework, we present a reproducible pipeline that converts multi-dataset SMPLX motion into executable loco-manipulation behavior for the Galaxea R1 Pro wheeled humanoid. The robot has a planar three-wheel base, a serial torso, and two arms but no leg joints, so human lower-body motion must be redistributed across base motion and torso posture without sacrificing manipulation-relevant arm geometry. Our pipeline combines canonical body-shape preprocessing, planar-base normalization, morphology-aware differential inverse kinematics, shoulder-rooted hierarchical arm retargeting, and continuous torso substitution for bending and squatting. A reference-twist-driven planning layer then decodes planar base motion into continuous three-wheel steering and rolling commands subject to hysteresis, kinematic continuity, acceleration, and actuator-rate limits. Finally, a 21-dimensional BaseDecode policy is trained in Isaac Lab with directional joint-limit scaling, focused upper-body tracking, and a staged wheel-contact reward. The resulting system provides a complete bridge from human motion data to physically trackable wheeled-humanoid loco-manipulation rather than a visualization-only retargeter; quantitative policy comparisons remain scheduled for a later revision.

1 Introduction

The paper develops a reproducible pipeline for converting human motion into physically trackable loco-manipulation on the legless, wheeled R1 Pro. It redistributes lower-body motion across the planar base and torso while preserving manipulation-relevant arm geometry and adds planning and policy layers for execution.

  • Motivation: The R1 Pro cannot map human knee and ankle motion directly, so locomotion, torso posture, and upper-body geometry must be coordinated differently from legged-humanoid retargeting.Its morphology couples planar base displacement, torso folding, and arm posture during tasks such as squatting and manipulation.
  • Pipeline: The pipeline standardizes SMPL-X inputs, normalizes the planar base, performs morphology-aware IK, and converts outputs into versioned motion references with explicit data contracts.Canonical body-shape preprocessing separates human body-shape variation from robot morphology, while planar normalization removes the unobservable absolute-height component of the wheeled base.
  • Retargeting: Shoulder-rooted hierarchical arm scaling and task-partitioned differential IK address the invalid arm positions that can result from global pelvis-rooted scaling.The approach preserves direct shoulder and distal-arm correspondences because manipulation depends on relative arm geometry.
  • Retargeting: Continuous torso substitution maps human lower-body folding to the R1 Pro through a knee-height feature, sigmoid gating, temporal smoothing, and arm re-IK.The four-joint torso is treated as a limited resource: upright motion prioritizes the spine and upper body, whereas squat motion reallocates joints to the missing lower-body fold.
  • Execution and evaluation: A reference-twist-driven planner and 21-dimensional BaseDecode policy produce bounded three-wheel commands and physically trackable behavior using focused upper-body rewards and staged wheel-contact regularization.The evaluation protocol covers geometric retargeting, smoothness, feasibility, and qualitative policy tracking, with fixed manifests and configurations.

3 Problem Formulation

The formulation separates human-motion retargeting from wheel-command generation by representing the robot with planar base, body, and wheel states. Explicit interfaces connect canonical motion inputs, IK-generated references, BaseDecode actions, and physical rollout evaluation.

  • 3 Problem Formulation: The target state combines planar base pose, torso and arm joints, and wheel variables, while retargeting outputs the state without wheel variables.The wheel decoder generates steering and rolling variables from the base trajectory.
  • 3 Problem Formulation: Each configured body correspondence compares a robot link pose with a transformed human target through a FrameTask.The per-frame objective is solved under joint and velocity limits using Jacobians, pose errors, and damping.
  • 3 Problem Formulation: The IK formulation uses Jacobian-weighted pose errors with damping to produce velocity updates that are integrated into the robot state.Robot joint and velocity limits constrain the optimization.
  • 4 System Overview: The implementation defines five interfaces spanning fixed-rate canonical SMPL-X frames, raw R1 Pro states, 24-joint references, BaseDecode actions, and physical rollout metrics.These interfaces make the differing representations between reference generation and physical tracking explicit.
  • 4 System Overview: The BaseDecode policy action contains three virtual base dimensions and eighteen upper-body dimensions before physical rollout.The final interface records configuration, input identity, and evaluation metrics.
  • 4 System Overview: Figure 1 summarizes the pipeline from canonical SMPL-X preprocessing and morphology-aware IK to filtered planar references, wheel decoding, and PPO/PD tracking.The filtered reference excludes wheel variables until the planner and three-wheel decoder stage.

5 Canonical Human Motion and Planar-Base Preprocessing

Preprocessing canonicalizes body shape and centers motion so heterogeneous SMPL-X sequences provide a stable planar reference for R1 Pro retargeting. The original shape remains available for audit, while the canonical sequence is used for IK.

  • 5 Canonical Human Motion and Planar-Base Preprocessing: A single canonical SMPL-X shape is used to recompute body-model forward kinematics across sequences with varying original shape parameters.The fixed shape table addresses body-shape variation before accounting for robot morphology.
  • 5 Canonical Human Motion and Planar-Base Preprocessing: The original body shape is retained for visualization and audit, whereas the canonical sequence supplies the R1 Pro IK input.The authors identify canonicalization as a conditioning step requiring comparison with a noncanonical baseline.
  • 5 Canonical Human Motion and Planar-Base Preprocessing: All sequences are resampled to 30 Hz, and R1 Pro positions are centered by subtracting the first-frame pelvis because the base cannot observe absolute pelvis height.This makes the planar base start at a stable origin.
  • 5 Canonical Human Motion and Planar-Base Preprocessing: Planar centering preserves relative body geometry while keeping the pelvis near the base, avoiding an ankle-ground assumption that would impose legged-robot morphology.Removing centering raises the reference root and makes nominal base height inconsistent with the R1 Pro.

6 Morphology-Aware Retargeting

Morphology-aware retargeting adapts the IK task structure to the R1 Pro’s topology by assigning absolute position primarily to the base and using shoulder-rooted local arm scaling. A yaw safeguard further reduces redundant torso rotation.

  • 6 Morphology-Aware Retargeting: The task tables establish and refine base, torso, shoulder, elbow, and wrist correspondences with asymmetric weights tailored to reachable robot geometry.The planar base carries absolute position, while torso and arm links primarily carry orientation tasks where absolute positions are unreachable.
  • 6 Morphology-Aware Retargeting: This task partition avoids forcing unreachable human positions onto topologically different robot links and permits independent configuration versioning.The solver iteratively applies the task tables with MuJoCo Jacobians and damping.
  • 6 Morphology-Aware Retargeting: The default pelvis-rooted global transform is poorly conditioned because R1 Pro arms sit atop a tall torso while human arms may hang below the pelvis frame.The mismatch contaminates arm targets through the pelvis-to-shoulder displacement.
  • 6 Morphology-Aware Retargeting: Shoulder-rooted hierarchical scaling keeps the shoulder connected to the torso/pelvis transform while scaling elbow and wrist segments locally.An independent three-axis shoulder scale and separate elbow and wrist scales replace global arm scaling.
  • 6 Morphology-Aware Retargeting: A yaw decomposition safeguard assigns global yaw to the planar base whenever possible instead of leaking redundant rotation into the fourth torso joint.The safeguard is evaluated as a regression condition rather than a separate learned component.

7 Continuous Torso Substitution for Bending and Squatting

The torso substitutes for missing lower-body joints through mode-dependent, continuously blended posture control. Upright motion preserves direct upper-body correspondences, while squat motion uses a filtered pelvis–knee height signal and arm correction.

  • 7 Continuous Torso Substitution for Bending and Squatting: Table 1 distinguishes upright and squat correspondence contracts, with synthesized entries generated from a filtered pelvis–knee height signal rather than one-to-one knee IK targets.The table’s model names refer to R1 Pro URDF links and SMPL-X joints, not interchangeable actuator and link labels.
  • 7 Continuous Torso Substitution for Bending and Squatting: Upright mode uses the pelvis as the mobile-base root and preserves direct shoulder, elbow, and wrist correspondences for upper-body tracking.Squat mode instead reallocates limited torso joints toward the missing lower-body fold.
  • 7 Continuous Torso Substitution for Bending and Squatting: Squat mode removes direct spine constraints, retains the pelvis root and manipulation-critical arm correspondences, and uses three torso joints as a surrogate for the missing hip–knee–ankle chain.The reallocation prioritizes body-height reduction while preserving arm behavior.
  • 7 Continuous Torso Substitution for Bending and Squatting: The lower-body folding feature signals when the human body is folded but is not intended to estimate physical knee angle.This signal supports morphology-specific posture substitution because the robot cannot directly reproduce knee and ankle rotations.
  • 7 Continuous Torso Substitution for Bending and Squatting: A sigmoid-filtered activation and guard-and-ramp mechanism blend the torso override continuously instead of switching abruptly between walking and squat modes.The algorithm updates the height gate, solves constrained IK, blends toward squat targets above the guard, and optionally applies EMA smoothing.
  • 7 Continuous Torso Substitution for Bending and Squatting: After torso folding, a constrained arm-only IK pass re-solves shoulder, elbow, and wrist tasks so the arms follow the updated shoulder position.This ordering prevents the spine task from pulling the torso upright and maintains continuous joint and end-effector transitions.

8 Feasibility Filtering and Motion Contracts

The pipeline filters retargeted sequences through stability, collision, penetration, and optional dynamics checks before training, while preserving reproducible failure and configuration records. It also separates planar geometric references from executable wheel commands for the R1 Pro.

  • Feasibility Filtering: Sequences are screened for COM stability, ZMP stability, self-collision, ground penetration, and optionally full MuJoCo dynamics before entering the reference library.Each layer reports passed and failed frames, failure indices, and a human-readable reason.
  • Feasibility Filtering: Because the free-base R1 Pro cannot reliably self-right after falling, unstable references are rejected before PPO training.The screened pool excludes motions with large transient torques or support-region violations.
  • Motion Contracts: Each retargeted frame contains a planar base pose and 18 torso/arm joint positions, plus frame-rate, sequence, and configuration metadata.Wheel steering and rolling values are intentionally absent at this interface.
  • Motion Contracts: Wheel commands are generated later from the planar base trajectory, keeping geometric references distinct from executable commands and separating planar yaw from quaternion roots.This contract avoids ambiguity between reference geometry and wheel-level actuation.

9 Planning Layer and Three-Wheel Base Decoder

A planning-and-decoding layer converts planar base references into continuous three-wheel commands using feedforward twists, bounded feedback, hysteresis, continuity rules, filtering, and explicit dynamic limits. The resulting wheel targets are integrated with a 21-action whole-body policy without teleporting the simulated base.

  • Planning Layer: Reference planar poses are yaw-unwrapped and differentiated into twists, then combined with bounded pose feedback, saturation, rate limits, and low-pass filtering.The layer changes executable wheel commands without reparameterizing the geometric x–y–yaw path.
  • Three-Wheel Base Decoder: For each wheel, planar twist and wheel location determine contact-point velocity, steering angle, and rolling velocity.The decoder therefore maps base motion into three-wheel actuation targets.
  • Three-Wheel Base Decoder: Equivalent steering representations reverse wheel rolling when closer to the previous target, preserving physical velocity while avoiding artificial π-scale steering jumps.Wrapped-distance selection and deadband holding maintain continuity near low-speed conditions.
  • Three-Wheel Base Decoder: The online decoder explicitly bounds wheel acceleration, steering rate, base linear and angular acceleration, and steering limits while applying hysteresis and filtering.These operations are part of the active planner used by the reported policy.
  • Planning Layer: The policy emits three virtual base targets and 18 torso/arm actions, with base targets decoded into pose feedback, reference-twist feedforward, and wheel targets.The free base moves through wheel-ground contact rather than simulator root teleportation.

10 Physics-Based Tracking with PPO

The BaseDecode task trains a 21-action PPO policy in Isaac Lab with staged wheel-contact rewards, directional joint-limit scaling, and focused upper-body tracking. The reward curriculum changes training emphasis while retaining the standard PPO environment and objective.

  • Policy and Action Contract: BaseDecode PPO uses a 21-dimensional action with three virtual base targets and 18 torso/arm commands passed through the planner.The base dimensions are not written directly into the simulator root pose.
  • Joint Limits: Directional joint-limit scaling maps upper-body actions from the default pose into simulator limits, reducing systematic excursions beyond usable ranges.The action contract remains unchanged while the usable joint range is enforced directionally.
  • Tracking Reward: Focused tracking weights emphasize manipulation-critical torso and arm joints, with wb = 2.0, wj = 2.0, and wf = 6.0.The focus-set weight is higher than the base and general joint weights in the evaluated configuration.
  • Contact Curriculum: The wheel-contact reward penalizes wheels whose measured contact-force norm falls below a threshold and is introduced through a two-stage curriculum with K = 1500 iterations.The curriculum targets physical wheel-ground contact during training.
  • Training Configuration: Table 2 identifies the verified BaseDecode PPO configuration used for the physics-based tracker.The supplied material provides the table identity but not its individual parameter values.
  • Contact Curriculum: The staged wheel-contact term is a reward curriculum within the standard PPO objective, leaving the environment, actor/critic, optimizer, sensors, and action dimensions unchanged.It is not presented as a new PPO algorithm.

11 Experiments

Experiments evaluate the retargeting front end, feasibility filtering, and physical tracking across fixed motion sets and dataset-scale screening. The results show qualitative preservation of locomotion and manipulation, stable planner-based execution, and persistent constraints from wheeled morphology and direct wheel control.

  • 11.1–11.2 Evaluation protocol: The evaluation uses fixed manifests and macro-averaged per-motion errors, with representative kinematic snapshots corresponding directly to the 21 Table 3 motions.Figure 2 is a pose-correspondence visualization without gravity or rigid-body dynamics, not a physical-stability evaluation.
  • 11.2 GMR baseline on the fixed A–U set: The 21 fixed A–U motions retain their principal action characteristics, including arm raising, planar translation, turning, bending, and squatting.Squats and forward bends are approximated through continuous torso substitution, while walking cannot reproduce human leg swing.
  • 11.4 Dataset-scale feasibility screening: 6,636 of 16,268 motions (40.8%) pass feasibility screening, with ZMP the dominant first-failure category at 7,904 motions (48.6%).Pass rates vary from 70.1% for GRAB to 3.7% for HDM05, so feasibility depends strongly on motion distribution.
  • 11.6 Qualitative PPO rollouts: 15 of 16 selected PPO rollouts complete without a fall while preserving intended arm, base, and torso behavior.Failures arise from large instantaneous torques and mismatch introduced by downstream planar-target decoding into wheel commands.
  • 11.7 Planner/contact stability: Planner and contact measures maintain stable three-wheel contact during large-range translation and turning, whereas removing them causes inconsistent contact, base instability, and target-motion deviation.The complete interface supports planar (x, y, ψ) learning with downstream wheel decoding rather than direct six-variable wheel control.
  • 11.8 Direct wheel–steer action ablation: Direct wheel control can require TOPP slowdowns exceeding a factor of 40, materially distorting references that violate base-planner constraints.This data-side limitation supports retaining planar actions and planner-based decoding.

12 Limitations and Reproducibility

The pipeline is bounded by morphology, contact, recovery, and validation constraints that require feasibility screening and limit current evidence to simulation-generated references.

  • The torso substitution approximates lower-body folding at the task level rather than reproducing biomechanically equivalent leg behavior.
  • The three-wheel decoder assumes planar geometry and contact but does not guarantee no slip under arbitrary dynamics.
  • Large transient torques can move the center of mass outside the support region, while failed motions cannot be rescued by stand-up behavior.
  • Current evidence comes entirely from Isaac Sim simulation and generated reference motion, with no hardware experiment included.

13 Conclusion

The paper presents a reproducible GMR modification for R1 Pro loco-manipulation, combining morphology-aware retargeting, bounded wheel decoding, and simulated physical execution. Qualitative studies report stable tracking and 15 of 16 PPO rollouts completed without a fall, while hardware validation remains future work.

  • The pipeline replaces direct leg mapping with canonical SMPL-X preprocessing, planar centering, shoulder-rooted arm IK, and continuous torso substitution.
  • The planner and three-wheel decoder convert planar references into bounded commands, while PPO and PD tracking provide physical execution in Isaac Sim.
  • 15 of 16 selected references were completed without a fall in PPO rollouts, supporting the bridge from screened human motion to wheeled-humanoid loco-manipulation.
  • Future work will expand screened motion pools, support additional formats including BVH, and validate the system on hardware.

A IK Weights and Arm Scaling Configuration

The appendix documents the two-stage differential-IK weights, morphology-aware arm scales, fixed evaluation motions, and canonical-shape ablation motions used for reproducibility.

  • Table 5 records two-stage IK position and orientation weights together with morphology-aware arm scales for R1 Pro retargeting.
  • The appendix specifies dataset, subject, action label, and dataset-relative path for the fixed evaluation motions without implying performance ranking.
  • Tables 6 and 7 identify the fixed A–U baseline motions and the B1–B5 canonical-shape ablation motions, respectively.
Loading 2609.11357v1…