PaperlayerView exact revision on arXiv

Source-linked AI summary

OmniXtreme: Breaking the Generality Barrier in High-Dynamic Humanoid Control

Yunshen Wang, Shaohang Zhu, Peiyuan Zhi, Yuhan Li, Jiaxin Li, Yong-Lu Li, Yuchen Xiao, Xingxing Wang, Baoxiong Jia, Siyuan Huang

arXiv:2602.23843v1cs.RO

TL;DR

High-fidelity humanoid tracking loses scalability as motion libraries become more diverse and physically demanding, especially when simulation policies meet real actuators. OmniXtreme addresses this with flow-based unified pretraining followed by actuation-aware residual refinement, and reports robust tracking and execution across diverse extreme motions, including on a real robot.

Abstract

from arXiv · show

High-fidelity motion tracking serves as the ultimate litmus test for generalizable, human-level motor skills. However, current policies often hit a "generality barrier": as motion libraries scale in diversity, tracking fidelity inevitably collapses - especially for real-world deployment of high-dynamic motions. We identify this failure as the result of two compounding factors: the learning bottleneck in scaling multi-motion optimization and the physical executability constraints that arise in real-world actuation. To overcome these challenges, we introduce OmniXtreme, a scalable framework that decouples general motor skill learning from sim-to-real physical skill refinement. Our approach uses a flow-matching policy with high-capacity architectures to scale representation capacity without interference-intensive multi-motion RL optimization, followed by an actuation-aware refinement phase that ensures robust performance on physical hardware. Extensive experiments demonstrate that OmniXtreme maintains high-fidelity tracking across diverse, high-difficulty datasets. On real robots, the unified policy successfully executes multiple extreme motions, effectively breaking the long-standing fidelity-scalability trade-off in high-dynamic humanoid control.

I. INTRODUCTION

OMNIXTREME targets the scalability barrier in high-fidelity humanoid tracking by addressing both multi-motion learning limits and real-world actuation constraints. It combines flow-based unified pretraining with actuation-aware residual refinement to support diverse extreme behaviors.

  • High-fidelity tracking tests whole-body coordination and contact timing underlying locomotion, manipulation, and expressive interaction.Source: §I · p. 2
  • Scaling heterogeneous motion libraries remains difficult because representation and optimization limitations degrade multi-motion tracking.Source: §I · p. 2
  • Real-robot deployment adds actuator nonlinearities and power-related effects that can undermine simulation-level tracking fidelity.Source: §I · p. 2
  • OMNIXTREME uses specialist-to-unified flow-based pretraining and actuation-aware residual reinforcement learning to address the two bottlenecks.Source: §I · pp. 2–3
  • The framework evaluates extreme motions involving high speed, frequent contact transitions, and tight timing constraints, including flips, acrobatics, and breakdancing on a Unitree G1.Source: §I · p. 3
  • A single unified policy is reported to execute diverse extreme motions while addressing the conventional fidelity–scalability trade-off.Source: §I · p. 3

A. Scalable Flow-based Policy Pretraining

The scalable pretraining stage distills motion-specific experts into a unified flow-matching policy using DAgger-based supervision. It learns from diverse retargeted motion data while using conservative noise and randomization to preserve stability.

B. Actuation-Aware Post-training Phase

The post-training stage freezes the flow-matching base policy and learns residual corrections under realistic actuation constraints. It combines aggressive randomization, power-aware regularization, and actuator modeling to improve deployment robustness.

1) Residual Policy Modeling

2) Actuation-aware Physical Constraint Modeling

1) Residual Policy Modeling

2) Actuation-aware Physical Constraint Modeling

C. Real World Deployment

The deployment pipeline integrates onboard state estimation, flow-based action generation, and residual correction into a real-time system on the Unitree G1. TensorRT optimization keeps the full pipeline responsive for high-quality control.

IV. EXPERIMENTS

Experiments evaluate whether OMNIXTREME preserves high-fidelity tracking as motion diversity, difficulty, and model capacity scale, using simulation and real-robot tests. Across these settings, OMNIXTREME maintains stronger tracking and robustness than conventional multi-motion baselines.

Motion libraries

  • The motion libraries combine standard LAFAN1 benchmarks with about 60 highly challenging XtremeMotion motions featuring intense dynamics, frequent contact transitions, and tight timing constraints.XtremeMotion draws from LAFAN1, AMASS, MimicKit, and Reallusion.Source: 1) Motion Libraries · p. 6

Experimental setup

  • The experiments compare OMNIXTREME with specialist-to-unified MLP distillation and from-scratch multi-motion reinforcement learning under matched capacity and training data.Evaluation uses the combined LAFAN1 and XtremeMotion library, with full-library, high-dynamic, and unseen-motion test sets.Source: C. Scalable high-fidelity tracking (Q1) · p. 7

Scalable high-fidelity tracking

Real-world evaluation

  • Across 157 real-world trials spanning 24 high-dynamic motions, OMNIXTREME achieves consistently high success rates across flips, acrobatics, breakdancing, and martial-arts-style behaviors.Trials use a Unitree G1, and success requires execution without manual intervention or safety-triggered termination.Source: C. Scalable high-fidelity tracking (Q1) · p. 7, B. Evaluation Metrics · p. 7

Capacity scaling

  • Tracking fidelity and robustness improve more strongly with model capacity for OMNIXTREME, while conventional MLP controllers saturate earlier.Fig. 4 reports both quantities as functions of model capacity.Source: Figure 4 · p. 7

Fidelity–scalability trade-off

  • As motion diversity increases, from-scratch multi-motion RL degrades earlier and more sharply, whereas OMNIXTREME maintains higher tracking robustness over a broader scaling range.From-scratch RL shows steadily worsening precision followed by a sharp robustness loss beyond a critical scale.Source: D. Fidelity–scalability trade-off (Q2) · p. 7, Figure 3 · p. 7

E. Capacity scaling (Q3)

OMNIXTREME scales multi-motion tracking with flow-matching policies whose larger capacity yields stronger fidelity gains than MLP controllers, then uses execution-oriented refinement for physical deployment. The framework supports reliable unified-policy execution across diverse, extreme whole-body behaviors.

Capacity scaling

  • Increasing model capacity improves tracking quality more directly for flow-matching policies than for MLP-based controllers.Models were trained under the same data and training recipe; MLP policies showed weaker gains.Source: E. Capacity scaling (Q3) · p. 7

Execution robustness

  • OMNIXTREME combines scalable tracking with post-training mechanisms that address complementary real-world failure modes.The mechanisms target actuator constraints, contact perturbations, and unsafe energy absorption during high-impact contacts.Source: F. Real-world executability and… · p. 8, §V · p. 8
  • Motor constraints stabilize impulsive motions, while aggressive domain randomization benefits contact-rich skills and power-safety regularization supports high-impact landings.The mechanisms are associated with distinct motion classes and failure modes.Source: F. Real-world executability and… · p. 8

Qualitative results

Overall outcome

  • The framework preserves tracking fidelity deeper into motion diversity than baselines and reliably executes diverse extreme behaviors on a real robot.The conclusion characterizes this as breaking the conventional fidelity–scalability trade-off.Source: §V · p. 8

APPENDIX

The appendix defines motion-complexity metrics from kinematics, center-of-mass motion, airborne behavior, and contact switching, then normalizes them into a six-dimensional difficulty vector for dataset comparison.

Metric construction

  • Motion complexity is summarized from joint and base kinematics, center-of-mass velocity, airborne ratio, and contact-switch frequency.The metrics capture velocity, acceleration, angular motion, dynamic liveliness, airborne behavior, and contact transitions.Source: B. Motion Complexity Metrics · p. 12
  • Joint and base kinematics provide maximum linear velocity, linear acceleration, and angular velocity through finite differencing.The same three scalar quantities are defined for base torso kinematics.Source: B. Motion Complexity Metrics · p. 12
  • Airborne ratio measures the fraction of frames in which all feet remain above a small ground-height threshold.Foot body indices define the set used for this calculation.Source: B. Motion Complexity Metrics · p. 12
  • Contact-switch frequency counts end-effector contact-state flips per second and is normalized as s_sw = min(f_switch/10, 1) in radar plots.Contact states are binary vectors over K end-effectors.Source: B. Motion Complexity Metrics · p. 12

Difficulty scores

  • The scalar metrics are clamped and linearly scaled into [0, 1] difficulty scores, forming a six-dimensional vector for radar-plot comparison.The vector comprises s_ang, s_v, s_a, s_com, s_air, and s_sw.Source: B. Motion Complexity Metrics · p. 12

C. Teacher Training

Teacher training uses BeyondMimic reinforcement-learning motion tracking in IsaacLab with full-mesh collision geometry and robot-consistent system parameters. The teacher observes reference motion, torso errors, proprioception, and noisy state inputs, while training uses early termination for large tracking errors.

Teacher setup

  • BeyondMimic trains the teacher policy with reinforcement learning in IsaacLab for motion tracking.The setup uses full-mesh collision geometry and system parameters consistent with the BeyondMimic framework.Source: C. Teacher Training · p. 12

Teacher architecture

  • The teacher actor and critic use MLPs with hidden dimensions [512, 256, 128].This architecture is specified for both actor and critic networks.Source: C. Teacher Training · p. 12

Teacher observations

  • Teacher observations combine reference joint positions and velocities with torso position and orientation differences.The torso orientation difference is represented using the 6D orientation from the reference torso rotation.Source: C. Teacher Training · p. 12
  • The actor additionally receives proprioception, relative joint positions, joint velocities, and previous actions subject to noise, while the critic receives ground-truth inputs and privileged state information.The asymmetric observations support robust policy inputs and stable value estimation during training.Source: C. Teacher Training · p. 12

Termination criteria

  • Early termination is triggered when torso or end-effector vertical position error exceeds 0.25m or torso gravity-vector discrepancy exceeds 0.8.The training protocol terminates episodes when tracking errors exceed safety-related thresholds.Source: C. Teacher Training · pp. 12, 14

D. Flow matching policy training

The flow-matching policy processes proprioception, motion commands, and history with a Transformer-based high-dimensional representation, while residual reinforcement learning refines the base action under noisy proprioceptive observations and privileged simulation state.

Flow matching policy

  • Robot proprioception and motion commands become single tokens, while 15 history steps become individual tokens for a 17-token Transformer input.Dedicated two-layer MLP preprocessors map these inputs before Transformer encoding and 1024-dimensional pooling.Source: D. Flow matching policy training · p. 14
  • The policy combines state, action, and sinusoidally encoded time embeddings to predict the velocity v_t through a deep MLP.The embeddings are concatenated and processed by hidden layers of [2048, 2048, 2048].Source: D. Flow matching policy training · p. 14

Residual refinement

  • Residual learning adds an asymmetric actor-critic stage whose actor uses proprioception, motion commands, previous total action, and the flow-matching base action.Uniform observation noise is aligned with the flow-matching policy noise to improve sim-to-real robustness.Source: E. Residual policy training · p. 14
  • The residual critic uses privileged simulation information, including full ground-truth state and precise global body, velocity, and torso positions.These variables are hidden from the residual actor.Source: E. Residual policy training · p. 14

Training specification

G. PD control

The control and evaluation protocol combines actuator-aware PD control with fixed-length motion clips, randomized simulation conditions, unseen-motion testing, physics-aware metrics, and timeout-based success measurement.

PD control

  • Action outputs determine PD setpoints relative to default joint positions, and torques are then computed from current joint positions and velocities using PD control.The control law uses q and q̇ together with the policy action and default position q0.Source: G. PD control · p. 15

Evaluation protocol

  • The evaluation protocol segments reference motions into 10-second clips corresponding to 500 control steps, while shorter motions remain unsegmented.Simulation metrics use the same clip-level organization for reporting.Source: H. Evaluation Protocol and Metrics · p. 15
  • Simulation evaluation retains training sensor noise and base domain randomization, evaluates each clip over 10 independent rollouts, and averages the metrics.These settings target basic sim-to-real robustness rather than idealized noise-free tracking.Source: H. Evaluation Protocol and Metrics · p. 15

Generalization evaluation

  • The unseen-motion set contains 1000 ten-second clips excluded from training and extreme-motion selections.It primarily tests locomotion, turning, and simple dance-like behaviors beyond the training distribution.Source: H. Evaluation Protocol and Metrics · p. 15

Evaluation metrics

  • Success rate is the fraction of episodes that reach timeout instead of early termination, using the same safety thresholds as training.Metrics are averaged across control steps within episodes and then across episodes and motion clips.Source: H. Evaluation Protocol and Metrics · pp. 15–16

I. Skill-level Grouping for Real-world Evaluation

Real-world motions are grouped into interpretable extreme-skill categories based on semantic intent and dynamic structure, solely for reporting and analysis.

Skill grouping

  • Each real-world skill category groups multiple motion instances sharing semantic intent and high-level dynamic structure.The grouping does not affect training or policy execution.Source: §I · p. 16
  • Grouping considers coordination patterns, contact configurations, and dynamic features such as aerial phases, rapid contact switching, and impulsive landing.The categories are based on human semantic understanding rather than automated clustering or learned representations.Source: §I · p. 16

Hardware evaluation

  • Table XI maps skill categories to the motion instances used for hardware evaluation.Motion IDs refer to retargeted motions evaluated on hardware.Source: §I · p. 16

J. Motion Subsets Used in Fidelity–Scalability Analysis

The fidelity–scalability analysis fixes a common evaluation set while expanding training libraries from 10 to 20 and all extreme motions. The section also records deployment latency and hardware-related failure boundaries, alongside a residual-policy limitation.

Motion subsets

  • The controlled subsets provide a benchmark for analyzing how tracking fidelity changes as motion-library size and diversity increase.The analysis explicitly specifies the motion subsets used at each scale.Source: J. Motion Subsets Used in… · p. 16
  • All quantitative Q2 evaluations use the same first 10 extreme motions, isolating training-scale growth from evaluation-set changes.The set includes fast flips, contact-rich acrobatics, and extreme balance behaviors.Source: J. Motion Subsets Used in… · p. 16
  • Training libraries expand from 10 to 20 and all extreme motions, adding diversity in contact patterns and dynamic regimes while preserving difficulty.The larger sets extend the same core 10-motion evaluation set with additional extreme motions.Source: J. Motion Subsets Used in… · p. 16, Table XII · p. 16

Deployment

  • 10 ms per control step enables closed-loop execution of high-dynamic whole-body motions entirely on the onboard Unitree G1 compute.The deployment uses ONNX Runtime with TensorRT acceleration on an NVIDIA Jetson Orin NX.Source: L. TensorRT-Accelerated Deployment · pp. 16–17

Failure boundaries

  • Real-world failures predominantly arise during impulsive landings, where transient braking loads trigger motor overcurrent, power-limit, or battery-undervoltage protection.The motions remain reliable in simulation and sim-to-sim transfer, pointing to discrepancies between simulated actuation models and hardware limits.Source: M. Failure Cases and Discussion · p. 17
  • The lightweight residual post-training policy refines a frozen flow-based controller but may limit adaptation of the base model to hardware-specific constraints.The paper suggests directly adapting the full base policy under actuation-aware objectives as future work.Source: M. Failure Cases and Discussion · p. 17

Related papers

Loading 2602.23843v1…