PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control

Yunshen Wang, Shaohang Zhu, Peiyuan Zhi, Yuhan Li, Jiaxin Li, Yong-Lu Li, Yuchen Xiao, Xingxing Wang, Baoxiong Jia, Siyuan Huang

arXiv:2602.23843v1cs.RO

TL;DR

High-fidelity humanoid tracking loses scalability as motion libraries become more diverse and physically demanding, especially when simulation policies meet real actuators. OmniXtreme addresses this with flow-based unified pretraining followed by actuation-aware residual refinement, and reports robust tracking and execution across diverse extreme motions, including on a real robot.

  • Problem

    High-fidelity tracking remains difficult to scale across large, heterogeneous motion libraries and to transfer reliably to physical humanoid robots.

  • Method

    OmniXtreme combines specialist-to-unified flow-based pretraining with actuation-aware residual reinforcement learning post-training.

  • Results

    The unified policy preserves tracking fidelity across increasing motion diversity and executes diverse extreme behaviors on real hardware.

  • Takeaways & Limitations

    A single unified policy can support scalable, high-fidelity humanoid control across high-dynamic motions while mitigating the fidelity–scalability trade-off.

Abstract

from arXiv · show

High-fidelity motion tracking serves as the ultimate litmus test for generalizable, human-level motor skills. However, current policies often hit a "generality barrier": as motion libraries scale in diversity, tracking fidelity inevitably collapses - especially for real-world deployment of high-dynamic motions. We identify this failure as the result of two compounding factors: the learning bottleneck in scaling multi-motion optimization and the physical executability constraints that arise in real-world actuation. To overcome these challenges, we introduce OmniXtreme, a scalable framework that decouples general motor skill learning from sim-to-real physical skill refinement. Our approach uses a flow-matching policy with high-capacity architectures to scale representation capacity without interference-intensive multi-motion RL optimization, followed by an actuation-aware refinement phase that ensures robust performance on physical hardware. Extensive experiments demonstrate that OmniXtreme maintains high-fidelity tracking across diverse, high-difficulty datasets. On real robots, the unified policy successfully executes multiple extreme motions, effectively breaking the long-standing fidelity-scalability trade-off in high-dynamic humanoid control.

I. INTRODUCTION

OMNIXTREME targets the scalability barrier in high-fidelity humanoid tracking by addressing both multi-motion learning limits and real-world actuation constraints. It combines flow-based unified pretraining with actuation-aware residual refinement to support diverse extreme behaviors.

  • High-fidelity tracking tests whole-body coordination and contact timing underlying locomotion, manipulation, and expressive interaction.
  • Scaling heterogeneous motion libraries remains difficult because representation and optimization limitations degrade multi-motion tracking.
  • Real-robot deployment adds actuator nonlinearities and power-related effects that can undermine simulation-level tracking fidelity.
  • OMNIXTREME uses specialist-to-unified flow-based pretraining and actuation-aware residual reinforcement learning to address the two bottlenecks.
  • The framework evaluates extreme motions involving high speed, frequent contact transitions, and tight timing constraints, including flips, acrobatics, and breakdancing on a Unitree G1.
  • A single unified policy is reported to execute diverse extreme motions while addressing the conventional fidelity–scalability trade-off.

A. Scalable Flow-based Policy Pretraining

The scalable pretraining stage distills motion-specific experts into a unified flow-matching policy using DAgger-based supervision. It learns from diverse retargeted motion data while using conservative noise and randomization to preserve stability.

  • Reference motions from LAFAN1, AMASS, MimicKit, and Reallusion are retargeted to the Unitree G1 and used to train motion-specific PPO experts.
  • DAgger rolls out the current flow-based policy, queries the corresponding expert at visited states, and aggregates expert-labeled trajectories.
  • Flow matching trains a velocity field to recover expert actions from actions interpolated with Gaussian noise across flow timesteps.
  • The unified policy generates actions by integrating the learned velocity field from noise toward the reconstructed action, with denoising-step count controlling approximation accuracy.

B. Actuation-Aware Post-training Phase

The post-training stage freezes the flow-matching base policy and learns residual corrections under realistic actuation constraints. It combines aggressive randomization, power-aware regularization, and actuator modeling to improve deployment robustness.

  • 1) Residual Policy Modeling: A lightweight residual policy adds corrective actions to the frozen flow-matching policy rather than relearning motion tracking.
  • 2) Actuation-aware Physical Constraint Modeling: The residual policy is trained with actuation-aware physical constraints and domain randomization for sim-to-real refinement.
  • 1) Residual Policy Modeling: Post-training uses randomized environments and relaxed termination thresholds to expose the residual policy to large but recoverable deviations.
  • 2) Actuation-aware Physical Constraint Modeling: An explicit penalty on excessive negative joint mechanical power targets braking loads that can trigger overcurrent protection or thermal stress.
  • 2) Actuation-aware Physical Constraint Modeling: Torque-speed envelopes and nonlinear friction modeling prevent simulation from relying on actuator behavior unavailable on physical hardware.
  • 2) Actuation-aware Physical Constraint Modeling: The structured refinement stage yields controllers described as safer, more disturbance-robust, and more aligned with real-world actuator dynamics.

C. Real World Deployment

The deployment pipeline integrates onboard state estimation, flow-based action generation, and residual correction into a real-time system on the Unitree G1. TensorRT optimization keeps the full pipeline responsive for high-quality control.

  • The pipeline uses the pelvis IMU for orientation estimation and Forward Kinematics to compute torso rotation.
  • TensorRT executes state estimation, the base flow-matching policy, and the residual policy onboard.
  • About 10ms end-to-end inference latency is achieved on the Unitree G1’s onboard Orin NX.

IV. EXPERIMENTS

Experiments evaluate whether OMNIXTREME preserves high-fidelity tracking as motion diversity, difficulty, and model capacity scale, using simulation and real-robot tests. Across these settings, OMNIXTREME maintains stronger tracking and robustness than conventional multi-motion baselines.

  • Motion libraries: The motion libraries combine standard LAFAN1 benchmarks with about 60 highly challenging XtremeMotion motions featuring intense dynamics, frequent contact transitions, and tight timing constraints.XtremeMotion draws from LAFAN1, AMASS, MimicKit, and Reallusion.
  • Experimental setup: The experiments compare OMNIXTREME with specialist-to-unified MLP distillation and from-scratch multi-motion reinforcement learning under matched capacity and training data.Evaluation uses the combined LAFAN1 and XtremeMotion library, with full-library, high-dynamic, and unseen-motion test sets.
  • Scalable high-fidelity tracking: OMNIXTREME consistently outperforms both simulation baselines, with larger gaps on XtremeMotion and unseen motions as difficulty increases.Metrics include success rate, root-relative mean per-joint position error, joint-space velocity discrepancy, and acceleration discrepancy.
  • Real-world evaluation: Across 157 real-world trials spanning 24 high-dynamic motions, OMNIXTREME achieves consistently high success rates across flips, acrobatics, breakdancing, and martial-arts-style behaviors.Trials use a Unitree G1, and success requires execution without manual intervention or safety-triggered termination.
  • Capacity scaling: Tracking fidelity and robustness improve more strongly with model capacity for OMNIXTREME, while conventional MLP controllers saturate earlier.Fig. 4 reports both quantities as functions of model capacity.
  • Fidelity–scalability trade-off: As motion diversity increases, from-scratch multi-motion RL degrades earlier and more sharply, whereas OMNIXTREME maintains higher tracking robustness over a broader scaling range.From-scratch RL shows steadily worsening precision followed by a sharp robustness loss beyond a critical scale.

E. Capacity scaling (Q3)

OMNIXTREME scales multi-motion tracking with flow-matching policies whose larger capacity yields stronger fidelity gains than MLP controllers, then uses execution-oriented refinement for physical deployment. The framework supports reliable unified-policy execution across diverse, extreme whole-body behaviors.

  • Capacity scaling: Increasing model capacity improves tracking quality more directly for flow-matching policies than for MLP-based controllers.Models were trained under the same data and training recipe; MLP policies showed weaker gains.
  • Execution robustness: OMNIXTREME combines scalable tracking with post-training mechanisms that address complementary real-world failure modes.The mechanisms target actuator constraints, contact perturbations, and unsafe energy absorption during high-impact contacts.
  • Execution robustness: Motor constraints stabilize impulsive motions, while aggressive domain randomization benefits contact-rich skills and power-safety regularization supports high-impact landings.The mechanisms are associated with distinct motion classes and failure modes.
  • Qualitative results: Real-world rollouts span flips, acrobatics, breakdance, martial arts, rapid contact switching, high angular velocities, and diverse whole-body styles.These behaviors are executed with a single unified policy on physical hardware.
  • Overall outcome: The framework preserves tracking fidelity deeper into motion diversity than baselines and reliably executes diverse extreme behaviors on a real robot.The conclusion characterizes this as breaking the conventional fidelity–scalability trade-off.

APPENDIX

The appendix defines motion-complexity metrics from kinematics, center-of-mass motion, airborne behavior, and contact switching, then normalizes them into a six-dimensional difficulty vector for dataset comparison.

  • Metric construction: Motion complexity is summarized from joint and base kinematics, center-of-mass velocity, airborne ratio, and contact-switch frequency.The metrics capture velocity, acceleration, angular motion, dynamic liveliness, airborne behavior, and contact transitions.
  • Metric construction: Joint and base kinematics provide maximum linear velocity, linear acceleration, and angular velocity through finite differencing.The same three scalar quantities are defined for base torso kinematics.
  • Metric construction: Airborne ratio measures the fraction of frames in which all feet remain above a small ground-height threshold.Foot body indices define the set used for this calculation.
  • Metric construction: Contact-switch frequency counts end-effector contact-state flips per second and is normalized as s_sw = min(f_switch/10, 1) in radar plots.Contact states are binary vectors over K end-effectors.
  • Difficulty scores: The scalar metrics are clamped and linearly scaled into [0, 1] difficulty scores, forming a six-dimensional vector for radar-plot comparison.The vector comprises s_ang, s_v, s_a, s_com, s_air, and s_sw.

C. Teacher Training

Teacher training uses BeyondMimic reinforcement-learning motion tracking in IsaacLab with full-mesh collision geometry and robot-consistent system parameters. The teacher observes reference motion, torso errors, proprioception, and noisy state inputs, while training uses early termination for large tracking errors.

  • Teacher setup: BeyondMimic trains the teacher policy with reinforcement learning in IsaacLab for motion tracking.The setup uses full-mesh collision geometry and system parameters consistent with the BeyondMimic framework.
  • Teacher architecture: The teacher actor and critic use MLPs with hidden dimensions [512, 256, 128].This architecture is specified for both actor and critic networks.
  • Teacher observations: Teacher observations combine reference joint positions and velocities with torso position and orientation differences.The torso orientation difference is represented using the 6D orientation from the reference torso rotation.
  • Teacher observations: The actor additionally receives proprioception, relative joint positions, joint velocities, and previous actions subject to noise, while the critic receives ground-truth inputs and privileged state information.The asymmetric observations support robust policy inputs and stable value estimation during training.
  • Termination criteria: Early termination is triggered when torso or end-effector vertical position error exceeds 0.25m or torso gravity-vector discrepancy exceeds 0.8.The training protocol terminates episodes when tracking errors exceed safety-related thresholds.

D. Flow matching policy training

The flow-matching policy processes proprioception, motion commands, and history with a Transformer-based high-dimensional representation, while residual reinforcement learning refines the base action under noisy proprioceptive observations and privileged simulation state.

  • Flow matching policy: Robot proprioception and motion commands become single tokens, while 15 history steps become individual tokens for a 17-token Transformer input.Dedicated two-layer MLP preprocessors map these inputs before Transformer encoding and 1024-dimensional pooling.
  • Flow matching policy: The policy combines state, action, and sinusoidally encoded time embeddings to predict the velocity v_t through a deep MLP.The embeddings are concatenated and processed by hidden layers of [2048, 2048, 2048].
  • Residual refinement: Residual learning adds an asymmetric actor-critic stage whose actor uses proprioception, motion commands, previous total action, and the flow-matching base action.Uniform observation noise is aligned with the flow-matching policy noise to improve sim-to-real robustness.
  • Residual refinement: The residual critic uses privileged simulation information, including full ground-truth state and precise global body, velocity, and torso positions.These variables are hidden from the residual actor.
  • Training specification: Residual policy training uses specified reward terms and power-safety regularization, together with actuator modeling parameters.The reward and actuator details are documented in Tables VIII and IX.

G. PD control

The control and evaluation protocol combines actuator-aware PD control with fixed-length motion clips, randomized simulation conditions, unseen-motion testing, physics-aware metrics, and timeout-based success measurement.

  • PD control: Action outputs determine PD setpoints relative to default joint positions, and torques are then computed from current joint positions and velocities using PD control.The control law uses q and q̇ together with the policy action and default position q0.
  • Evaluation protocol: The evaluation protocol segments reference motions into 10-second clips corresponding to 500 control steps, while shorter motions remain unsegmented.Simulation metrics use the same clip-level organization for reporting.
  • Evaluation protocol: Simulation evaluation retains training sensor noise and base domain randomization, evaluates each clip over 10 independent rollouts, and averages the metrics.These settings target basic sim-to-real robustness rather than idealized noise-free tracking.
  • Generalization evaluation: The unseen-motion set contains 1000 ten-second clips excluded from training and extreme-motion selections.It primarily tests locomotion, turning, and simple dance-like behaviors beyond the training distribution.
  • Evaluation metrics: Success rate is the fraction of episodes that reach timeout instead of early termination, using the same safety thresholds as training.Metrics are averaged across control steps within episodes and then across episodes and motion clips.

I. Skill-level Grouping for Real-world Evaluation

Real-world motions are grouped into interpretable extreme-skill categories based on semantic intent and dynamic structure, solely for reporting and analysis.

  • Skill grouping: Each real-world skill category groups multiple motion instances sharing semantic intent and high-level dynamic structure.The grouping does not affect training or policy execution.
  • Skill grouping: Grouping considers coordination patterns, contact configurations, and dynamic features such as aerial phases, rapid contact switching, and impulsive landing.The categories are based on human semantic understanding rather than automated clustering or learned representations.
  • Hardware evaluation: Table XI maps skill categories to the motion instances used for hardware evaluation.Motion IDs refer to retargeted motions evaluated on hardware.

J. Motion Subsets Used in Fidelity–Scalability Analysis

The fidelity–scalability analysis fixes a common evaluation set while expanding training libraries from 10 to 20 and all extreme motions. The section also records deployment latency and hardware-related failure boundaries, alongside a residual-policy limitation.

  • Motion subsets: The controlled subsets provide a benchmark for analyzing how tracking fidelity changes as motion-library size and diversity increase.The analysis explicitly specifies the motion subsets used at each scale.
  • Motion subsets: All quantitative Q2 evaluations use the same first 10 extreme motions, isolating training-scale growth from evaluation-set changes.The set includes fast flips, contact-rich acrobatics, and extreme balance behaviors.
  • Motion subsets: Training libraries expand from 10 to 20 and all extreme motions, adding diversity in contact patterns and dynamic regimes while preserving difficulty.The larger sets extend the same core 10-motion evaluation set with additional extreme motions.
  • Deployment: 10 ms per control step enables closed-loop execution of high-dynamic whole-body motions entirely on the onboard Unitree G1 compute.The deployment uses ONNX Runtime with TensorRT acceleration on an NVIDIA Jetson Orin NX.
  • Failure boundaries: Real-world failures predominantly arise during impulsive landings, where transient braking loads trigger motor overcurrent, power-limit, or battery-undervoltage protection.The motions remain reliable in simulation and sim-to-sim transfer, pointing to discrepancies between simulated actuation models and hardware limits.
  • Failure boundaries: The lightweight residual post-training policy refines a frozen flow-based controller but may limit adaptation of the base model to hardware-specific constraints.The paper suggests directly adapting the full base policy under actuation-aware objectives as future work.
Loading 2602.23843v1…