PaperlayerView exact revision on arXiv

Source-linked AI summary

Track Any Motions under Any Disturbances

Zhikai Zhang, Jun Guo, Chao Chen, Jilong Wang, Chenghuai Lin, Yunrui Lian, Han Xue, Zhenrong Wang, Maoqi Liu, Jiangran Lyu, Huaping Liu, He Wang, Li Yi

arXiv:2509.13833v3cs.RO

TL;DR

Humanoid motion trackers need to reproduce diverse, highly dynamic, contact-rich motions while remaining stable under real-world dynamics disturbances. Any2Track addresses this with a two-stage RL framework separating general tracking from history-informed adaptation, achieving zero-shot sim2real transfer and strong disturbed-motion tracking on Unitree G1.

  • Problem

    Humanoid motion trackers need to combine diverse, highly dynamic, contact-rich motion tracking with stability under terrains, external forces, and physical property changes.

  • Method

    Any2Track uses two-stage RL: AnyTracker learns general motion tracking, while AnyAdapter adds history-informed online dynamics adaptability.

  • Results

    Any2Track achieves zero-shot sim2real transfer on Unitree G1 and tracks diverse motions under multiple real-world disturbances.

  • Takeaways & Limitations

    Any2Track provides a foundational humanoid motion tracker intended for general practical use and downstream tasks.

Abstract

from arXiv · show

A foundational humanoid motion tracker is expected to be able to track diverse, highly dynamic, and contact-rich motions. More importantly, it needs to operate stably in real-world scenarios against various dynamics disturbances, including terrains, external forces, and physical property changes for general practical use. To achieve this goal, we propose Any2Track (Track Any motions under Any disturbances), a two-stage RL framework to track various motions under multiple disturbances in the real world. Any2Track reformulates dynamics adaptability as an additional capability on top of basic action execution and consists of two key components: AnyTracker and AnyAdapter. AnyTracker is a general motion tracker with a series of careful designs to track various motions within a single policy. AnyAdapter is a history-informed adaptation module that endows the tracker with online dynamics adaptability to overcome the sim2real gap and multiple real-world disturbances. We deploy Any2Track on Unitree G1 hardware and achieve a successful sim2real transfer in a zero-shot manner. Any2Track performs exceptionally well in tracking various motions under multiple real-world disturbances.

I. INTRODUCTION

Any2Track addresses the need for humanoid tracking that combines diverse, dynamic motion reproduction with stability under real-world disturbances. Its two-stage design separates general motion tracking from online dynamics adaptation and transfers zero-shot to Unitree G1 hardware.

  • Existing humanoid trackers do not simultaneously provide general motion tracking and dynamics adaptability under terrains, external forces, and physical property changes.
  • Any2Track is a two-stage RL framework that adds dynamics adaptability to basic action execution through AnyTracker and AnyAdapter.
  • AnyTracker: AnyTracker uses canonicalized action spaces and a specialist-to-generalist strategy to track diverse, highly dynamic, and contact-rich motions.
  • AnyAdapter: AnyAdapter uses history-informed dynamics-aware world-model prediction and an adapter architecture to adjust the base policy for varied disturbances.
  • Any2Track achieves zero-shot sim2real transfer on Unitree G1 and tracks diverse motions under terrains, external forces, and physical property changes.

II. RELATED WORKS

Humanoid motion tracking has progressed from character animation to real hardware, but real-world systems remain constrained in motion diversity and disturbance adaptability. Any2Track is presented as advancing both capabilities within a unified policy.

  • Humanoid robots reproduce human motions for more expressive behavior, but cross-embodiment, actuator, and sim-to-real challenges limit real-world tracking.
  • Existing systems include quasi-static loco-manipulation approaches and highly dynamic tracking that overfits to limited motion clips.
  • Any2Track demonstrates unprecedented motion diversity, high dynamism, and contact complexity within a unified tracking policy.
  • Unlike domain-specific prior approaches, Any2Track adapts online to multiple real-world disturbances for general practical use.

B. Online Dynamics Adaptation for Legged Robots

Online dynamics adaptation is needed because disturbance-unaware policies can behave conservatively under large dynamic variation. Any2Track decouples motion tracking from adaptation and learns dynamics representations from interaction history.

  • Naive domain randomization lacks environment-dynamics awareness, so policies cannot adapt actions and may behave conservatively under large dynamic variance.
  • Prior online adaptation methods estimate real-world dynamics from robot-environment interaction history and adjust behavior accordingly.
  • Any2Track uses dynamics-aware world-model prediction as a proxy task to learn more informative dynamics representations from the history buffer.
  • The framework first builds AnyTracker as an RL policy mapping humanoid proprioception and tracking goals to low-level robot actions.
  • Any2Track uses AMASS and LAFAN1 motion data while retaining highly dynamic and contact-rich motions that GMT removed.

1) Canonicalized Action Spaces

AnyTracker simplifies general motion-tracking control through canonicalized action spaces and specialist-to-generalist training, while AnyAdapter adds disturbance variation and history-informed adaptation.

  • Canonicalized Action Spaces: Canonicalized action spaces compact multi-joint action distributions to reduce the difficulty caused by diverse joint action ranges.The policy rescales joint-specific action scales and predicts residual PD offsets relative to reference motion.
  • Canonicalized Action Spaces: Specialist training clusters motions by category so each policy handles similar action distributions before contributing to a general tracker.This reduces training difficulty and improves final tracking results.
  • Canonicalized Action Spaces: AnyAdapter introduces environment dynamics variance and uses a history-informed module to address terrains, external forces, and physical property changes.The disturbance types are explicitly randomized in the training setup.
  • Canonicalized Action Spaces: Dynamics-aware world-model prediction serves as a proxy task for extracting informative dynamics features from interaction history.These features support online adaptation across different disturbances.

1) Learn Informative Dynamics Embeddings

AnyAdapter learns dynamics embeddings from interaction history by training a history encoder with an autoregressive world model to predict future robot states.

  • Learn Informative Dynamics Embeddings: The history encoder extracts dynamics features because raw robot–environment interaction histories contain irrelevant information and noise.A proxy task is introduced to make the learned representation more informative.
  • Learn Informative Dynamics Embeddings: The world model uses a history window of H = 79 state-action pairs and predicts the next robot state autoregressively.The history comprises earlier states and actions, while the model predicts forward dynamics.
  • Learn Informative Dynamics Embeddings: The history encoder must identify environment configurations so the world model can predict next-frame states under different dynamics.The autoregressive prediction window is N = 20 states.
  • Learn Informative Dynamics Embeddings: Training samples H + 1 + N = 100 state-action pairs, initializes an embedding from the first H pairs, and predicts the following N states.The history encoder and world model are trained jointly through the world-model loss.

2) Dynamics Adaptability Injection

AnyAdapter injects dynamics adaptability on top of a frozen motion tracker, separating basic action execution from disturbance adaptation to preserve expressive tracking.

  • Dynamics Adaptability Injection: Coupling action execution and dynamics adaptability can make learning harder and motion tracking over-conservative under large dynamics variation.The paper identifies this trade-off as harmful to expressive, anthropomorphic motion tracking.
  • Dynamics Adaptability Injection: Any2Track freezes AnyTracker and fine-tunes a zero-initialized adapter that injects dynamics adaptability through layer-wise feature fusion.The adapter initially leaves the base tracker output unchanged and progressively adds adaptive behavior.
  • Dynamics Adaptability Injection: The adapter and history encoder are alternately updated using the same rewards within environments containing dynamics variance.This training procedure builds adaptation on the previously trained tracker.
  • Dynamics Adaptability Injection: Experiments evaluate general tracking, online adaptation to disturbances, and real-world performance on the 29-DoF Unitree G1.The questions cover comparisons for AnyTracker, AnyAdapter, and deployment behavior.

A. Results of Tracking Any Motions

The general-tracking evaluation compares AnyTracker with baselines on AMASS and LAFAN1 without dynamics variance, using success and motion-error metrics.

  • Results of Tracking Any Motions: AnyTracker is compared with baseline methods on curated-motion simulations using the AMASS test set and LAFAN1.This experiment isolates motion-tracking quality by excluding dynamics variance.
  • Results of Tracking Any Motions: Success Rate records the percentage of successful tracking trials.A trial is unsuccessful when averaged joint-position error or root-height error exceeds 0.2 m.
  • Results of Tracking Any Motions: Mean Per Joint Position Error measures the average position error across all links.The metric is reported in millimeters.
  • Results of Tracking Any Motions: Mean Per Joint Velocity Error measures the average velocity error across all links.The metric is reported in millimeters per frame.

3) Baselines

The evaluation compares reproduced open-source general motion trackers with AnyTracker and ablates its canonicalized action spaces and specialist-to-generalist distillation designs.

  • 3) Baselines: Existing general motion trackers with open-source training code are reproduced in MuJoCo using curated motion datasets as baselines.The paper re-implements OmniH2O and ExBody because direct comparison across differing simulators, datasets, and robots is unfair.
  • 3) Baselines: AnyTracker, Ours w/o CAS, and Ours w/o distillation are evaluated to test the effects of canonicalized action spaces and specialist-to-generalist distillation.The ablations remove each major AnyTracker design separately.
  • 3) Baselines: AnyTracker outperforms baseline methods in motion tracking quality, while both major designs improve the final quantitative results.The reported explanation is that the designs alleviate optimization difficulty from complex action spaces.
  • 3) Baselines: Bold numbers in Table IV indicate the best performance across compared motion tracking methods and the authors’ major designs.The table uses curated motion datasets for evaluation.

B. Results of Adapting to Any Disturbances

The online adaptation experiments compare Any2Track with history-based, privileged-information, and PPO baselines under terrain, external-force, and physical-property disturbances. Any2Track achieves the strongest reported simulation performance, while ablations support both world-model learning and decoupled adaptation.

  • Baselines and setting: The experiments compare Any2Track with Vanilla PPO, Dual History, RMA, and DWL for online adaptation under disturbances.All algorithms are trained with asymmetric PPO, and the study also ablates the adapter and world model.
  • Baselines and setting: The tested disturbances include Perlin-noise terrains, randomly directed torso forces, and changes to torso mass, center of mass, and joint friction.An undisturbed environment is also included to evaluate basic motion tracking capability.
  • 4) Experiment Results: Table V reports simulation performance for the compared online dynamics-adaptation methods, with bold numbers marking the best performance.The table summarizes results across the disturbance settings.
  • 4) Experiment Results: Any2Track surpasses all baseline methods under all disturbances and has the highest success rate and lowest tracking error without disturbance.It also shows a smaller motion-tracking performance drop when dynamic disturbances are introduced.
  • 4) Experiment Results: The world model provides informative dynamics embeddings, while removing the adapter causes a clear performance drop.Without the world model, tracking can perform worse than vanilla PPO under external pushes or physical-property changes.

C. Real-World Evaluations

Any2Track is evaluated on a real Unitree G1 under complex terrains, an external rope constraint, and a 5 kg payload. It outperforms vanilla PPO with domain randomization in every tested environment.

  • Experiment setting: The real-world evaluation deploys Any2Track on a Unitree G1 robot under multiple disturbance conditions.The study explicitly addresses performance in various real-world scenarios.
  • Experiment setting: The test conditions include wooden boards, cardboard, foam, fabric, a fixed-length hoist rope, and a 5 kg payload.These represent terrain variation, external constraint, and changed robot mass, respectively.
  • Experiment metrics: Performance is measured with MPJPE and MPJVE computed from motor-sensor joint positions and velocities through forward kinematics.All results are averaged over five different motion trajectories.
  • Experiment results: Any2Track beats PPO with domain randomization in all tested real-world environments, with its advantage increasing as disturbances are introduced.The comparison uses identical real-world evaluation settings.

V. CONCLUSIONS

Any2Track is presented as a two-stage humanoid motion-tracking framework for diverse motions under multiple real-world disturbances. The authors position it as a foundation for open-world humanoid tracking and future downstream applications.

  • V. CONCLUSIONS: Any2Track tracks diverse, highly dynamic, and contact-rich motions under terrain, external-force, and physical-property disturbances.The conclusion frames this capability as applying to humanoid robots in real-world settings.
  • V. CONCLUSIONS: The framework is proposed as a foundational motion tracker for open-world humanoid applications.The authors identify whole-body tele-operation, humanoid skill learning, and humanoid VLA models as future downstream uses.
Loading 2509.13833v3…