PaperlayerView exact revision on arXiv

Source-linked AI summary

Compliant GMT: Making General Motion Tracking Compliant via Residual Reference Adaptation

Venki Parsuram, Michelle Wang, Jin-Hyuk Hong, Pulkit Agrawal

arXiv:2610.22119v1cs.RO

TL;DR

General motion trackers can imitate diverse motions but are trained to track stiffly, while prior compliant humanoid methods trade off generality and compliance. C-GMT adds compliance through a proprioceptive residual adapter on a pretrained tracker. It demonstrates competitive tracking accuracy and compliance adherence, with improved safety, robustness, and error tolerance in simulation and real-world G1 experiments.

  • Problem

    General motion trackers are trained to track stiffly, while prior compliant humanoid-control approaches trade off generality and compliance.

    Source: §I · p. 1

  • Method

    C-GMT uses a proprioceptive residual adapter, distilled from a force-privileged teacher, to adapt references for a pretrained general motion tracker.

    Source: §I · p. 2

  • Results

    C-GMT achieves competitive motion-tracking accuracy and compliance adherence relative to state-of-the-art general motion trackers and single-motion compliant policies.

    Source: §V · p. 8

  • Takeaways & Limitations

    Simulation and real-world G1 experiments demonstrate improved safety, robustness to disturbances, and error tolerance to high-level controller errors.

    Source: §V · p. 8

Abstract

from arXiv · show

We propose Compliant GMT (C-GMT), a modular framework for full-body compliant responses in general motion tracking. As humanoid robots move into real world deployment, they not only need to perform any motion but also respond safely and robustly to interaction forces. Existing general motion tracking policies can imitate diverse motions but are trained to be stiff, leading to unexpected and unsafe behaviors under disturbance. Prior work in compliant humanoid control trades off compliance and generality. There exist examples of full-body compliant responses, but these are heavily curated to single-reference policies. On the other hand, general motion trackers (GMT) are trained to rigidly imitate massive motion capture datasets. Our key insight is to leverage the generality of pretrained GMTs for motion tracking while introducing compliance from a residual adapter that conditions on a history of proprioception. This enables our approach, C-GMT, to achieve both general motion tracking and whole-body compliant responses. Compared to state-of-the-art general motion trackers and single-motion compliant policies, C-GMT demonstrates competitive performance in both motion tracking accuracy and compliance adherence. We further demonstrate improved safety, robustness, and error tolerance in simulation and real-world G1 experiments.

I. INTRODUCTION

General motion trackers imitate diverse motions but are stiff, so unexpected contact can drive robots forcefully toward their references. C-GMT separates tracking from compliance by adapting references with a proprioceptive residual adapter, achieving competitive tracking and compliance in simulation and real-world G1 experiments.

I. INTRODUCTION

  • Stiff tracking can forcefully drive a robot toward its reference during unexpected contact, potentially damaging the robot or environment.The paper frames this as a barrier to humanoid deployment alongside people.Source: §I · p. 1
  • C-GMT adds adjustable whole-body compliance to a pretrained general motion tracker through a proprioceptive residual adapter distilled from a force-privileged teacher.The tracker executes motion and maintains balance under disturbances; the adapter modifies the reference to realize the desired compliant response.Source: §I · p. 2
  • C-GMT matches single-motion compliant policies in compliance adherence while maintaining tracking performance comparable to state-of-the-art general motion trackers.Simulation and real-world Unitree G1 experiments also demonstrate compliance during dynamic whole-body motions and coordinated whole-body responses.Source: §I · p. 2

A. General Motion Tracking

  • Compliant control can reduce the precision and reactivity required of higher-level systems that generate motion references.The paper motivates this for teleoperation and vision-language-action outputs, which may have delayed or imperfect feedback.Source: A. General Motion Tracking · p. 2

B. Learned Compliant Humanoid Control

C-GMT targets a gap between general motion tracking and whole-body compliance, using residual adaptation rather than retraining a single policy for both objectives. Its force-robust tracker is evaluated on held-out motions under external loads.

B. Learned Compliant Humanoid Control

III. METHODS

  • C-GMT separates a force-robust tracker, which executes reference motions under external loads, from a residual adapter that modifies references for compliant responses.This lets desired compliant behavior be specified independently of the motion-tracking objective.Source: §III · p. 2
  • A privileged teacher uses a spring law and inverse kinematics to design and verify feasible compliant responses, which are then distilled into an adapter operating from proprioception histories.At deployment, the adapter modifies the motion reference using proprioceptive observations and commanded stiffness.Source: §III · p. 3, Figure 2 · p. 3

A. Force-Robust General Motion Tracker

  • SONIC’s learned latent motion tokens provide a compact reference space for adaptation, avoiding direct residual changes to kinematic quantities that can induce large motion changes.The authors observed unintended walking under small variations in root velocity.Source: A. Force-Robust General Motion Tracker · p. 3
  • The force-robust tracker is produced by finetuning the SONIC decoder for force robustness while keeping its motion encoders frozen and adding load perturbations.During training, hand loads combine carried payloads and sustained forces, with the force budget constrained and increased over time.Source: A. Force-Robust General Motion Tracker · p. 3
  • Per-motion load scales reduce perturbations for clips with high failure rates and increase them as robustness improves, limiting infeasible motion-load combinations while preserving feasible disturbances.The scale is updated from each motion clip’s cumulative failure rate.Source: A. Force-Robust General Motion Tracker · pp. 3–4
  • A force-robust tracker improves local tracking under external loads on 128 held-out clips, with a 4.8 cm increase in global tracking error, equal to 2.6% of mean motion displacement of 1.82 m.The comparison is against the original SONIC policy; Table I reports MPJPE in mm.Source: A. Force-Robust General Motion Tracker · p. 4, Table I · p. 4

B. Compliant Teacher and Feasibility Tuning

A privileged teacher converts an applied force and commanded stiffness into a feasible whole-body compliant reference. An IK solver prioritizes the forced link while preserving balance and posture, and simulation sweeps identify stable force ranges for adapter training.

B. Compliant Teacher and Feasibility Tuning

C. Residual Adapter Architecture

The adapter makes residual corrections to SONIC’s motion token using proprioception and stiffness, reproducing a privileged teacher’s reference modifications. Its training combines teacher imitation with staged student-driven rollouts to improve robustness to prediction-induced states.

C. Residual Adapter Architecture

D. Residual Adapter Training

  • Training imitates teacher actions and modified tokens, supervises force prediction, regularizes residuals, and encourages zero residuals without external force; staged student-driven rollouts expose the adapter to its own prediction errors.Source: D. Residual Adapter Training · pp. 4–5

1) Anti-Oscillation Loss:

Two complementary losses discourage residual actions from reinforcing ongoing joint motion. One targets larger dynamic responses; the normalized five-step loss is designed to catch low-amplitude and slower oscillations.

1) Anti-Oscillation Loss:

  • Early adapter versions oscillated during compliant responses and jittered near rest; increasing action-rate and jerk losses did not remove this without degrading compliance.Source: 1) Anti-Oscillation Loss · p. 5
  • The unnormalized loss penalizes alignment between changes in residual action and joint velocity across 14 arm joints, but weakens when motion is small.Source: 1) Anti-Oscillation Loss · p. 5
  • The normalized loss uses five-step differences to reduce motion-magnitude dependence and detect small persistent or slower oscillations; it complements the unnormalized loss’s focus on larger dynamic responses.Source: 1) Anti-Oscillation Loss · p. 5

E. Experimental Setup

C-GMT evaluates compliance across commanded stiffness levels and general tracking on walking and dynamic motions, alongside a shared tracker and adapter across motion interfaces. It tracks close to SONIC while substantially outperforming TWIST2 and GentleHumanoid in success rate, especially on dynamic motions.

E. Experimental Setup

  • Direct playback, whole-body teleoperation, and VLA references all use the same residual adapter and force-robust GMT.Source: E. Experimental Setup · p. 5

A. C-GMT accurately tracks compliance levels

B. C-GMT is an accurate general motion tracker

  • C-GMT stays close to SONIC in tracking, with only a modest increase in global position error, and substantially exceeds TWIST2 and GentleHumanoid in success rate, especially on dynamic motions.Source: B. C-GMT is an accurate general motion… · p. 6

C. C-GMT enables safe interaction during dynamic motions

C-GMT reduces contact forces during dynamic motion while continuing to track, and coordinated whole-body responses help the robot accommodate sustained forces.

C. C-GMT enables safe interaction during dynamic motions

  • C-GMT produces the lowest wall-contact forces during a dynamic dance, yielding to contact while closely tracking the motion.SONIC produces larger forces and recovers after moving away from the wall; TWIST2 produces the largest impacts and is launched far away.Source: C. C-GMT enables safe interaction… · p. 6, Figure 4 · p. 6

1) Whole-body compliant responses improve robustness:

  • Under increasing downward hand force, C-GMT and SoftMimic squat while maintaining balance, whereas GentleHumanoid stays upright and eventually falls.C-GMT and SoftMimic distribute the response across arms, torso, and legs, allowing the robot to redistribute sustained disturbance through the body.Source: 1) Whole-body compliant responses… · pp. 6–7, Figure 6 · p. 7
  • C-GMT supports contact-rich teleoperation tasks, including opening a door, wiping a whiteboard, dragging a chair while walking, and holding a rubber ball.The described tasks involve unknown contact dynamics that are imperceivable to the operator.Source: Figure 7 · p. 7

2) Compliance provides robustness during high-level controller execution:

C-GMT's low-level compliance helps higher-level policies withstand disturbances, including in real-world teleoperation and VLA task execution, without sacrificing nominal task performance.

2) Compliance provides robustness during high-level controller execution:

  • Real-world C-GMT teleoperation enables contact-rich tasks when object and interaction dynamics are unknown and imperceivable to the operator.Examples include opening a door, wiping a whiteboard, dragging a chair while walking, and holding a rubber ball.Source: 2) Compliance provides robustness… · p. 7
  • Under external disturbances, the C-GMT-based VLA retains considerably higher task success than the SONIC-based VLA, while nominal success is nearly identical.Each system is evaluated over 50 trials from the same initial states.Source: 2) Compliance provides robustness… · p. 7, Table III · p. 7
  • Low-level motion-tracking compliance improves higher-level policy robustness without sacrificing nominal task performance.In a representative trial, C-GMT yields to a 60 N force applied for 1.5 seconds while walking, then recovers and successfully grasps and lifts the box.Source: 2) Compliance provides robustness… · pp. 7–8, Figure 8 · p. 7

V. CONCLUSION

  • The conclusion reports competitive tracking accuracy and compliance adherence, with improved safety and robustness to disturbances and high-level controller errors.The authors identify extending compliant responses to any body link and context-dependent stiffness selection as future work.Source: §V · p. 8

APPENDIX

The appendix describes force-robust tracker training and a compliant teacher that modifies reference motions through whole-body inverse kinematics.

A. Force-Robust Tracker Training

  • The force-robust tracker is initialized from SONIC, then its decoder and critic are fine-tuned with PPO while the motion encoders and FSQ quantizer remain frozen.Training uses randomized hand payloads and sustained external forces under a motion-dependent force budget that rises from 10 to 30 N.Source: A. Force-Robust Tracker Training · p. 9

B. Compliant Teacher Implementation

  • The compliant teacher converts applied force and commanded stiffness into desired hand displacement, then uses whole-body IK to generate a modified reference motion.The weighted, box-constrained IK problem tracks the displaced hand while maintaining foot placement and balance through center-of-mass tracking, with regularization toward the original motion.Source: B. Compliant Teacher Implementation · p. 9, Table IV · p. 9
  • Teacher-feasible force ranges are identified across 4,096 representative motions, eight force directions, and multiple stiffness values, then used to limit adapter training.The retained ranges are those for which the teacher remains stable without falling.Source: B. Compliant Teacher Implementation · p. 9

C. Residual Adapter Architecture

The residual adapter uses proprioceptive history and motion information to adjust motion tokens, and is trained against teacher-generated compliant behavior while the force-robust tracker stays frozen.

C. Residual Adapter Architecture

  • The adapter takes a 10-step proprioceptive history, a motion representation, and commanded stiffness as inputs to a two-layer MLP with three output heads.The heads predict motion-token residuals, gates, and wrist forces; the residual is zero-initialized and gated corrections are applied after FSQ with a 0.005 deadband.Source: C. Residual Adapter Architecture · p. 9

D. Residual Adapter Training

  • Adapter training keeps the force-robust tracker frozen while randomizing applied hand forces, commanded stiffness, robot properties, initial states, and proprioceptive observations.Stiffness is sampled log-uniformly from 40 to 600 N/m, and force magnitudes are bounded by teacher feasibility limits and capped at 140 N.Source: D. Residual Adapter Training · p. 9
  • The training objective combines action and motion-token imitation, force prediction, residual regularization, and temporal smoothness, with force-free penalties encouraging preservation of tracker behavior.Additional rate and jerk penalties of 0.5 apply to force-free samples.Source: D. Residual Adapter Training · p. 9, Table VI · p. 10
  • Training starts with 500 teacher-driven updates, transitions linearly to student-driven rollouts over 1,000 updates, and then continues on student-driven trajectories.The adapter is optimized with Adam at a learning rate of 2 × 10^-5 and gradient-norm clipping at 1.0.Source: D. Residual Adapter Training · p. 10, Table V · p. 10

E. Deployment

Deployment pauses motion playback when predicted force exceeds a stiffness-dependent threshold. Feasibility-based force sampling reduces training falls and speeds give-way learning.

E. Deployment

  • On Unitree G1, the residual adapter and force-robust GMT run at 50 Hz; playback pauses above a stiffness-dependent EMA force threshold and resumes below it.The system uses an NVIDIA RTX 3080 GPU.Source: E. Deployment · p. 10

F. Effect of Feasibility Tuning

  • Approximately 13× fewer falls during training (0.4% vs. 5.4% of training steps), with over 95% give-way adherence within 6k updates versus around 74% for uniform sampling.Feasibility-based sampling also reduces the training time needed to achieve the desired compliance.Source: Figure 9 · p. 10, F. Effect of Feasibility Tuning · p. 10

Related papers

Loading 2610.22119v1…