PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Kinematic Tracking: Preserving Expressive Characteristics in Humanoid Whole-Body Control

Lingfan Bao, Hanshang Zhu, Yan Pan, Jincheng Wang, Jingyang Yu, Tianhu Peng, Yiyang Jia, Dimitrios Kanoulas, Chengxu Zhou

arXiv:2610.19393v1cs.RO

TL;DR

Whole-body trackers balance motion tracking and physical stability, but low positional error does not guarantee preservation of expressive movement characteristics. The paper adds motion-aware residual compensation targeting dynamic and postural cues, and reports improved characteristic preservation across simulation and physical experiments on the Unitree G1.

  • Problem

    Whole-body trackers may achieve low aggregate tracking error while diminishing expressive postural and dynamic cues, leaving open how to characterize and recover those cues during stable execution.

    Source: §I · p. 1

  • Method

    A residual policy augments a frozen whole-body tracker using compensation targets for amplitude, energy, body expansion, body lean, and pelvis height, with correction ranges allocated by body-part activity.

    Source: §I · p. 1

  • Results

    Residual compensation achieves lower aggregate errors in all five characteristics than tracking-oriented residuals in the reported simulation comparison, with higher positional error; physical Unitree G1 experiments further demonstrate recovery of dynamic and postural characteristics.

    Source: §VI · p. 8

  • Takeaways & Limitations

    Lower mean positional tracking error alone is insufficient to ensure preservation of the measured dynamic and postural characteristics.

    Source: §VI · p. 8

    The five descriptors do not cover every expressive cue, and recovery depends on the underlying tracker; improving one characteristic may worsen another.

    Source: §V · p. 8

Abstract

from arXiv · show

Whole-body control plays a crucial role in enabling humanoid robots to execute expressive motions. Faithful execution requires preserving both the intended action and its expressive characteristics. Existing whole-body trackers typically balance joint and keypoint tracking with physical stability across diverse motions, but these objectives do not necessarily prioritize the expressive characteristics of each action. Our analysis shows that lower average positional tracking error does not guarantee better preservation of dynamic and postural motion characteristics. To address this gap, we propose a residual learning framework with compensation objectives targeting amplitude, energy, body expansion, body lean, and pelvis height. To concentrate compensation while limiting interference with the underlying tracking behavior, motion-aware gating allocates correction capacity according to body-part activity, preserving larger correction ranges in more active parts and restricting them elsewhere. For each frozen tracker, we train motion-specific residual policies and distill them into a unified policy shared across motion categories. Simulation and physical experiments on the Unitree G1 across eight motion categories demonstrate superior preservation of dynamic and postural motion characteristics compared with the corresponding baselines.

I. INTRODUCTION

Whole-body tracking can reproduce reference motions while balancing stability, yet low aggregate tracking error does not ensure preservation of expressive posture and dynamics. The paper proposes motion-aware residual compensation and evaluates it across eight motion categories in simulation and on the Unitree G1.

I. INTRODUCTION

  • Low overall tracking error can coexist with diminished expressive cues, such as hand amplitude or forward lean, motivating questions about how execution changes expression and how to recover it stably.The paper notes that balance and physical constraints may require deviations from reference motion.Source: §I · p. 1
  • The framework targets amplitude, energy, body expansion, body lean, and pelvis height with a unified residual policy for each frozen tracker.Motion-aware gating gives more correction range to active body parts while retaining capacity in less active regions; tracking constraints and bounded corrections balance preservation with stable execution.Source: §I · pp. 1–2, Figure 2 · p. 2
  • Evaluations across eight motion categories in simulation and on the Unitree G1 report better preservation of dynamic and postural characteristics than corresponding baselines.The analysis also finds that lower mean positional error does not necessarily correspond to better preservation of expressive characteristics.Source: §I · p. 2

II. RELATED WORK

Prior work studies expressive motion cues and whole-body tracking or residual adaptation, while this paper focuses on compensating posture and dynamics deviations. Its formulation uses a residual policy to adapt a frozen tracker while retaining tracking constraints and regularization.

A. Expressive Motion Analysis

  • Expressive-motion research links movement dynamics and posture to expressive qualities, while robot applications must account for task requirements and physical embodiment.Prior approaches include human-feedback objectives and physics-aware generation paired with style transfer.Source: A. Expressive Motion Analysis · p. 2

B. Whole-Body Motion Tracking and Residual Adaptation

  • Whole-body trackers reproduce diverse motions while maintaining balance, and residual or adapter methods extend pretrained controllers to challenging motions, interaction, or deployment changes.Unlike those adaptation objectives, this work explicitly compensates for expressive posture and movement-dynamics deviations.Source: B. Whole-Body Motion Tracking and… · p. 2

III. METHOD

  • The framework proceeds from motion curation through residual-policy learning and distillation to deployment.Figure 3 identifies amplitude, energy, body expansion, body lean, and pelvis height as kinematic cues.Source: §III · p. 2, Figure 3 · p. 3

A. Problem Formulation

  • The residual policy observes proprioception, control history, local target states, and keypoint errors, then corrects the frozen tracker's nominal joint-position targets.Its gated residual is bounded by a 0.2-rad scale and an activity-aware gate; PPO optimizes a reward combining characteristic preservation, tracking constraints, and residual regularization.Source: A. Problem Formulation · p. 3

B. Data Curation

The study curates eight motion categories into units suited to complete movements or sustained poses, then computes five dynamic and postural descriptors over those units. The descriptors provide explicit compensation targets for the residual policy.

B. Data Curation

  • Eight AMASS motion categories annotated with BABEL are retargeted to the G1, resampled at 50 Hz, and PHC-preprocessed into robot reference trajectories.The categories are walk, kick, punch, crouch, wave, clap, cross arms, and bow; unsupported interactions and mixed actions are excluded.Source: B. Data Curation · p. 3
  • References are segmented into one-shot movements, periodic cycles, and sustained holds, with boundaries guided by limb extrema, direction reversals, foot contacts, and posture changes.The same unit intervals are used to compute characteristics and compare methods.Source: B. Data Curation · p. 3

C. Expressive Motion Descriptors

  • Five descriptors capture complementary expressive cues: amplitude and energy for dynamics, plus body expansion, body lean, and pelvis height for posture.They are computed over motion units from joint angles and forward-kinematic keypoints; the listed amplitude components track the hands, ankles, and waist.Source: C. Expressive Motion Descriptors · p. 3, Table I · p. 3
  • Motion energy uses squared joint-angle increments as a kinematic-intensity proxy, while body expansion measures upper-body openness and body lean measures torso pitch.Pelvis height is measured relative to the lowest sole point; positive torso pitch denotes forward bending.Source: C. Expressive Motion Descriptors · p. 3

D. Motion-Aware Residual Control

Motion-aware gating varies residual correction capacity by body-part activity, concentrating larger corrections in active regions while retaining capacity elsewhere. Activity is computed from reference motion across five body groups, with minimum and maximum correction bounds.

D. Motion-Aware Residual Control

  • The motion-aware gate assigns larger correction bounds to more active body parts while retaining correction capacity in less active regions.The gate uses reference frames available to the actor.Source: D. Motion-Aware Residual Control · pp. 3–4
  • Group activity is RMS reference joint velocity over up to ten future intervals, computed for five body groups.The groups are the left and right arms, left and right legs, and waist.Source: D. Motion-Aware Residual Control · p. 4
  • For nonzero reference activity, the most active group receives a ±0.2 rad correction bound, while every group retains at least ±0.04 rad.The gate uses only reference frames available to the actor.Source: D. Motion-Aware Residual Control · p. 4

E. Residual Policy Training

For each frozen tracker, the method trains motion-specific residual policies and distills them into a unified policy. Training combines characteristic-preservation rewards with tracking constraints and correction regularization.

E. Residual Policy Training

  • For each tracker, eight motion-specific residual policies are distilled into a unified policy shared across motion categories.The unified policy is trained on an accumulated dataset with equal sampling across categories and deployed with the same observations and motion-aware gating.Source: E. Residual Policy Training · pp. 4–5
  • The reward combines characteristic preservation, tracking constraints, and residual smoothness, with characteristic terms for amplitude, energy, expansion, lean, and pelvis height.Postural rewards compare body configurations at corresponding frames.Source: E. Residual Policy Training · p. 4
  • Dynamic rewards cover amplitude and energy, with energy computed from executed and reference sums of squared joint-angle increments within each body group.Amplitude and postural errors are mapped to bounded rewards after normalization.Source: E. Residual Policy Training · p. 4
  • The tracking constraint allows a 25% increase in positional error before penalty, and residual regularization penalizes squared correction changes at every control step.The tracker-only reference error and mean across units define the constraint comparison; reward scales are shared across categories and trackers.Source: E. Residual Policy Training · p. 4
  • Figure 4 illustrates compensation on representative development motions across amplitude/energy, upper-body geometry, body lean, and pelvis height.It compares reference ghosts with executions with and without residual compensation.Source: Figure 4 · p. 5

IV. EXPERIMENT

The experiment evaluates whether positional tracking error predicts preservation of dynamic and postural characteristics, and whether residual compensation recovers those characteristics. It also examines effects on tracking accuracy, motion completion, and residual intervention from motion-aware gating.

IV. EXPERIMENT

  • The first evaluation question asks whether lower positional tracking errors imply better preservation of dynamic and postural characteristics.Source: §IV · p. 5
  • The second question assesses characteristic recovery from residual compensation alongside its effects on tracking accuracy and motion completion.Source: §IV · p. 5
  • The third question examines how motion-aware gating affects residual intervention, tracking accuracy, and characteristic recovery.Source: §IV · p. 5

A. Experimental Setup

The framework is evaluated in simulation and on the Unitree G1 across eight motion categories, against three trackers and a tracking-focused residual baseline. The protocol measures characteristic errors and tracking accuracy, with main tables reporting equally weighted action averages.

A. Experimental Setup

  • Evaluation covers eight motion categories in simulation and on the physical Unitree G1, comparing SONIC, GMT, and TWIST2 under shared motions and evaluation starts.All trackers operate at 50 Hz.Source: A. Experimental Setup · p. 5
  • Track-Res trains motion-specific residual policies to minimize mean pelvis-relative joint-position error, while retaining gating, regularization, and constraints but omitting characteristic rewards.Its policy uses two 256-unit hidden layers, as in the unified policy.Source: A. Experimental Setup · p. 5
  • The protocol measures errors in amplitude, energy, body expansion, body lean, and pelvis height on completed motion units.Characteristic errors are evaluated on units completed by all three methods in Table III or both conditions in Fig. 5.Source: A. Experimental Setup · p. 5
  • Main tracking accuracy is mean per-keypoint position error, while main tables average per-action P70 values and weight the eight actions equally.Physical-experiment main tables use reference-level mean errors; lean is reported as a degree-equivalent sine discrepancy, not angular tracking error.Source: A. Experimental Setup · p. 5
  • Trained-gate comparisons average correction means across five equal activity bins, while low-activity analyses use a fixed reference-velocity threshold.The low-activity errors average eligible wrists and ankles, then rollouts; MPJPE uses common completed starts and equal motion and tracker weights.Source: A. Experimental Setup · p. 5

B. Expressive Preservation and Recovery

Expressive compensation yields the lowest action-averaged errors across five characteristics for each backbone, while physical and qualitative comparisons show that recovery can remain incomplete. Positional accuracy alone does not capture preservation of every motion characteristic.

B. Expressive Preservation and Recovery

C. Ablation

Motion-aware gating allocates smaller corrections at low activity and comparable or larger corrections at the highest activity, with lower low-activity endpoint errors in several motions. At matched correction bounds, it lowers several errors across trackers, though the comparison does not establish superiority over uniform-gate training.

C. Ablation

  • Compared with open gates, motion-aware gates apply smaller corrections at low activity and comparable or larger corrections at the highest activity.Low-activity endpoint errors are lower for walking, kicking, crossing arms, and bowing across trackers, while effects on other motions vary.Source: C. Ablation · p. 7
  • At matched correction bounds, motion-aware gating lowers amplitude, group-energy, MPJPE, and low-limb errors for every tracker, although walking energy error rises 3.3%.Equal-tracker reductions are 15.8%, 4.9%, 8.8%, and 18.1%, respectively; applied correction RMS increases from 4.35° to 4.72°.Source: C. Ablation · p. 7, Table IV · p. 7
  • The matched-bound comparison evaluates fixed motion-aware-trained policies at inference and does not establish superiority over uniform-gate training.The comparison keeps policies, observations, references, initial states, and evaluation seed fixed across all eight categories.Source: C. Ablation · p. 7, Table IV · p. 7

D. Real-World Robot Experiments

On the physical Unitree G1, residual compensation lowers all five characteristic errors on the selected references for each tracker. The evaluation uses screened references run once per condition.

D. Real-World Robot Experiments

  • The physical evaluation comprises 48 executions per tracker and 144 in total.References vary across trackers but match between compensated and uncompensated conditions.Source: D. Real-World Robot Experiments · p. 7
  • On each tracker’s selected physical references, compensation lowers errors in all five characteristics, indicating improved dynamic and postural preservation in these motions.The study uses three references per action and tracker, screened in simulation for executability and expressive recovery; each reference is run once per condition.Source: D. Real-World Robot Experiments · pp. 7–8, Table V · p. 8

V. LIMITATIONS

The conclusion reports improved recovery of five expressive characteristics through residual compensation, while the limitations note incomplete descriptor coverage and possible trade-offs among targets.

V. LIMITATIONS

  • Five descriptors capture complementary expressive cues but do not cover every dynamic or postural characteristic.Agreement on these measures does not imply preservation of every expressive cue.Source: §V · p. 8
  • Unit-mean energy and lean errors can mask temporal discrepancies, while total energy can hide redistribution across body groups.Recovery depends on the tracker; with limited waist excursion, adjusting lean may involve hip and leg motion and alter other targets, so improving one characteristic may worsen another.Source: §V · p. 8

VI. CONCLUSION

  • Residual compensation achieves lower aggregate errors in all five expressive characteristics than tracking-oriented residuals, with higher positional error.The reported simulation comparison covers amplitude, energy, body expansion, body lean, and pelvis height.Source: §VI · p. 8
Loading 2610.19393v1…