PaperlayerView exact revision on arXiv

Source-linked AI summary

ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation

Xialin He, Sirui Xu, Xinyao Li, Runpei Dong, Liuyu Bian, Yu-Xiong Wang, Liang-Yan Gui

arXiv:2603.03279v1cs.ROcs.CV

TL;DR

ULTRA addresses the limited physical fidelity, scalability, and autonomy of humanoid control systems that rely on predefined motion references. It combines physics-driven MoCap retargeting with a distilled multimodal controller and reinforcement-learning fine-tuning, then evaluates the resulting system in simulation and on a Unitree G1. The framework supports dense tracking and sparse goal-conditioned behavior from realistic sensing, including egocentric perception, while outperforming tracking-only baselines in the reported evaluations.

  • Problem

    Existing humanoid-control methods face physically inconsistent or limited retargeted data and often rely on predefined references instead of perception and high-level task specifications.

  • Method

    ULTRA uses physics-driven RL retargeting, teacher-student distillation, multimodal availability masking, a latent skill bottleneck, and RL fine-tuning.

  • Results

    ULTRA improves interaction fidelity, matches tracking performance, remains robust under distribution shift, and boosts success on out-of-distribution goals while transferring to Unitree G1.

  • Takeaways & Limitations

    A single controller can support dense tracking and sparse goal following across sensing modalities, including real-world deployment without test-time reference motions.

  • Takeaways & Limitations

    The method can fail under severe point-cloud occlusion that removes critical geometric cues.

Abstract

from arXiv · show

Achieving autonomous and versatile whole-body loco-manipulation remains a central barrier to making humanoids practically useful. Yet existing approaches are fundamentally constrained: retargeted data are often scarce or low-quality; methods struggle to scale to large skill repertoires; and, most importantly, they rely on tracking predefined motion references rather than generating behavior from perception and high-level task specifications. To address these limitations, we propose ULTRA, a unified framework with two key components. First, we introduce a physics-driven neural retargeting algorithm that translates large-scale motion capture to humanoid embodiments while preserving physical plausibility for contact-rich interactions. Second, we learn a unified multimodal controller that supports both dense references and sparse task specifications, under sensing ranging from accurate motion-capture state to noisy egocentric visual inputs. We distill a universal tracking policy into this controller, compress motor skills into a compact latent space, and apply reinforcement learning finetuning to expand coverage and improve robustness under out-of-distribution scenarios. This enables coordinated whole-body behavior from sparse intent without test-time reference motions. We evaluate ULTRA in simulation and on a real Unitree G1 humanoid. Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception, consistently outperforming tracking-only baselines with limited skills.

I. INTRODUCTION

ULTRA addresses the gap between precise reference tracking and autonomous goal-conditioned control by combining physics-aware retargeting with a unified multimodal controller. It targets whole-body loco-manipulation across changing sensing and supervision conditions.

  • Motivation: Existing controllers often separate dense reference tracking from sparse goal conditioning, creating a precision-flexibility trade-off.Dense policies can fail when references are missing or infeasible, while goal-conditioned policies may lack fine-grained coordination.
  • Motivation: Kinematic retargeting can produce physically inconsistent demonstrations, while fixed conditioning structures struggle with partial or diverse supervision.These limitations are especially problematic for contact-rich tasks and shifting deployment observability.
  • Contributions: ULTRA uses physics-driven reinforcement-learning retargeting to scale motion capture to physically feasible humanoid trajectories and support augmentation without per-motion retraining.The retargeting enforces kinematic, dynamic, and contact constraints during simulation-constrained optimization.
  • Contributions: A unified multimodal controller is distilled from a privileged tracker to support dense references, sparse goals, and egocentric perception.Availability masking allows one policy to operate when references or modalities are missing.
  • Contributions: The system combines scaled motion data, multimodal control, and reinforcement-learning fine-tuning to expand interaction-state coverage and improve robustness to out-of-distribution goals and executions.The paper evaluates this approach in simulation and on a Unitree G1 humanoid.

A. Task Interface

ULTRA formulates whole-body loco-manipulation as goal-conditioned control under partial observation, with inputs ranging from dense references to sparse task goals. Its training pipeline converts human-object demonstrations into physically feasible G1 rollouts before teacher-student learning and deployment.

  • Task Interface: The policy receives an observation and task conditioning, then outputs joint-position targets executed by a PD controller.Success is defined by satisfying the terminal task objective within a tolerance.
  • Task Interface: Partial onboard sensing motivates a goal-conditioned partially observable Markov decision process formulation.The policy acts from observations produced by an observation function rather than the full underlying state.
  • Training Pipeline: ULTRA trains through four stages: neural retargeting, privileged tracking, multimodal distillation, and deployment under real sensing.The stages connect physically feasible demonstrations to a student policy operating with sparse goals and available modalities.
  • Neural Retargeting: Retargeting uses reference-aware observations containing simulator state, reference quantities, and heading-aligned simulation-reference differences.The observation includes proprioception, contact signals, correspondence-defined references, and object state.
  • Neural Retargeting: Episodes begin from a default standing pose, transition from initial-frame stabilization to full tracking, and terminate after falls, excessive deviation, or sustained contact mismatch.The retargeting policy uses an idealized simulation controller and no domain randomization during this stage.
  • Neural Retargeting: A single retargeting policy supports anisotropic trajectory scaling and object scaling while preserving physical feasibility without retraining.Interaction and contact rewards correct augmentation imperfections while simulator dynamics enforce feasibility.

B. Dense Motion Tracking for Teacher Policy

The dense-tracking teacher uses privileged state and retargeted references, while the student learns to reproduce its behavior from heterogeneous, selectively available modalities. Distillation and reinforcement-learning fine-tuning extend the same controller from tracking to sparse-goal and vision-based control.

  • Teacher Policy: The privileged teacher tracks retargeted rollouts with full state and dense reference residuals, while randomized physics and perturbations teach recovery.This teacher provides supervision through the deployment control interface and actuation limits.
  • Multimodal Student: The student uses proprioception, goals, object state, egocentric point clouds, and an availability mask to handle missing modalities.A transformer encoder projects modalities into shared tokens, while the mask gates token processing and crossmodal attention.
  • Distillation: The training loop gradually shifts rollouts from teacher to student and queries the teacher on visited states in a DAgger-style procedure.At deployment, the student samples its latent from the prior using only available inputs.
  • Distillation: Distillation matches student actions to teacher actions while aligning a privileged posterior with a prior inferred from masked student observations.Auxiliary reconstruction heads encourage the latent representation to retain task-relevant information.
  • Retargeting Comparison: The retargeting comparison shows more stable contacts for ULTRA than OmniRetarget in the contact frame.The figure also contrasts final-frame standing foot placement between the methods.
  • RL Fine-tuning: RL fine-tuning combines reference-imitation environments with goal-reaching environments that apply state and goal perturbations.A residual shortcut from the full-body goal to the decoder preserves low-level reference information during decoding.
  • Control Modes: Modality masking supports high-fidelity tracking, sparse goal-conditioned control, and vision-based manipulation with the same parameters.Local references are unmasked for tracking, long-horizon goals for goal control, and point clouds for vision-based manipulation.

V. EXPERIMENTAL RESULTS

ULTRA is evaluated end-to-end for retargeting, dense tracking, sparse-goal control, robustness, and sim-to-real transfer. The experiments test whether one controller can preserve interaction quality while operating under partial sensing and distribution shift.

  • Evaluation Goals: The evaluation asks whether MoCap can be converted into physically consistent rollouts, whether dense tracking matches privileged supervision, and whether sparse-goal control improves with fine-tuning.It also tests latent skill organization and transfer to a physical Unitree G1.

A. Experimental Setup

ULTRA is evaluated through simulation-based motion tracking and physical-interaction experiments, using augmented data and comparisons against retargeting and control baselines. Results indicate improved tracking and contact quality, including under out-of-distribution conditions.

  • Evaluation platforms: Experiments use IsaacGym for motion-tracking evaluation, with key results validated in MuJoCo and real trials on a Unitree G1.The setup combines simulation training/evaluation with physical-hardware trials.
  • Data augmentation: Zero-shot augmentation scales trajectories and manipulated objects while preserving plausible motion, expanding the corpus by approximately 6× without retraining the retargeter.The augmented dataset uses anisotropic trajectory scaling and novel object scales for out-of-distribution evaluation.
  • Baselines and metrics: The evaluation compares ULTRA with PHC, GMR, OmniRetarget, HDMI, and ULTRA ablations across tracking and physical-interaction metrics.Metrics include success, tracking errors, foot skating, contact floating, and penetration.
  • Retargeting quality: ULTRA outperforms baselines across nearly all physical-interaction metrics, with lower foot skating, reduced penetration, and near-zero contact floating on Largebox and Suitcase.The reported gains are attributed to physics-aware retargeting that preserves stance and hand-object contact.
  • Motion tracking: ULTRA strongly outperforms tracking baselines on humanoid-object tracking, especially for out-of-distribution motions and object scales.Distillation from a privileged teacher improves success and object errors under partial observation compared with direct reinforcement learning.

D. Goal-Conditioned Following

ULTRA evaluates sparse goal-conditioned following across simulated perception settings and real Unitree G1 deployment. RL finetuning improves robustness to out-of-distribution goals, while learned latents organize control modes and motion semantics.

  • The success metric requires no fall and a terminal state within 0.3 m of the goal.
  • Real-world evaluation uses onboard proprioception, optional OptiTrack object pose, dense OMOMO tracking tasks, and reference-free keyboard-specified object goals.
  • RL finetuning produces modest in-distribution gains but large out-of-distribution gains under random goal offsets, nearly doubling with point clouds and tripling with position-only input.
  • The latent space captures control modes and motion semantics while keeping tracking distinct from sparse goal-following representations within a shared manifold.
  • On hardware, ULTRA reliably grasps and transports objects and achieves reasonable sparse-goal success under out-of-distribution operator commands, including composed motions.
  • ULTRA combines physics-driven retargeting with a distilled multimodal controller and reports improved success on out-of-distribution goals with sim-to-real validation on Unitree G1.

APPENDIX

The appendix organizes supplementary material into demonstrations, retargeting and teacher-policy details, student-policy training, and experimental setups.

  • Section A describes the supplementary demonstration video.
  • Section B provides retargeting and teacher-policy details, including observation and reward design.
  • Section C covers student-policy distillation and reinforcement-learning finetuning.
  • Section D provides experimental details and setups for simulation and real-world experiments.

A. Demo Videos

The supplementary webpage highlights ULTRA’s retargeting, tracking, sparse-goal, and egocentric-perception capabilities across simulation and long-horizon object-centric tasks.

  • The retargeting pipeline learns one policy from all data and performs zero-shot retargeting to unseen rescaled trajectories and objects.
  • The teacher policy transfers reliably across simulators.
  • The all-in-one ULTRA model faithfully tracks reference motions for object interactions.
  • ULTRA supports sparse object-goal commands and fine-grained keyboard-based control, demonstrating strong generalization.
  • ULTRA completes long-horizon object-centric goals using only egocentric perception.

B. Additional Details on Retargeting and Teacher Policy

The teacher-policy pipeline combines tracking and regularization rewards with observations spanning simulated state, reference targets, residuals, and interaction-graph features.

  • Rewards: The teacher reward combines tracking terms with smoothness and regularization penalties to support sim-to-real transfer.The total reward includes tracking components and smoothness terms.
  • Retargeting: Retargeting additionally requires a predefined key-joint mapping between Unitree G1 links and SMPL-X body indices.The correspondence table specifies the mapped links and body parts.
  • Observation: Teacher observations concatenate simulated state, reference targets, simulation–reference residuals, and interaction-graph features.The interaction-graph features encode body–object relationships.
  • Architecture: The teacher uses a three-layer MLP with hidden dimensions 1024, 1024, and 512 and ReLU activations.It uses a separate actor–critic design.
  • Rewards: The tracking reward covers body joints, object pose and velocity, palm-to-surface interaction, and contact matching.The listed terms include body tracking, object tracking, palm-to-surface distance, and contact matching.

C. Additional Details on Student Policy

The student policy fuses multimodal observations into a latent-variable controller, then combines distillation with PPO finetuning and masked goal rewards for robust deployment.

  • Observation: The student observation combines global state, commands, local state, proprioception, task information, and modality masks.The observation space also includes object position, rotation, and point-cloud features.
  • Student Policy Architecture: ULTRA uses a 64-dimensional latent z and a modality-fusion transformer to represent task-relevant information under noisy or missing modalities.Each modality is encoded into a shared token space before fusion.
  • Teacher Inputs: The teacher observation is structured around simulated quantities, reference targets, residuals, and interaction-graph features.These blocks provide the privileged information used during teacher learning.
  • Distillation: Distillation uses supervised and latent-variable losses, with a 64-dimensional latent space and scheduled optimization.The student rollout uses 4096 parallel environments and horizon length 8.
  • RL Finetuning: PPO finetuning is applied to the deployable prior-only student while preserving part of the distillation updates.The finetuning reward combines dense goal-reaching, progress, and auxiliary regularization terms.
  • RL Finetuning: Goal-related rewards and success checks are masked when targets are unobserved.The reward table separately identifies dense proximity, clipped progress, and terminal success terms for visible targets.
  • Perception Robustness: Point-cloud domain randomization includes Gaussian noise, dropout, outliers, occlusion, and camera perturbations.The listed settings include 0.02 m Gaussian noise and 0.15 point-dropout probability.

D. Additional Experimental Details.

The method remains vulnerable to severe point-cloud occlusion, marker occlusion in MoCap control, and overly aggressive or infeasible operator goals.

  • Limitations: Severe point-cloud occlusion can remove critical geometric cues and cause failures under out-of-distribution conditions.The limitation concerns deployment with egocentric perception.
  • Limitations: MoCap-driven control is sensitive to marker occlusions, which can create jitter and drift in estimated object pose and unstable tracking.This limitation applies to real-world experiments.
  • Limitations: Overly aggressive, discontinuous, or physically infeasible goals can degrade performance.The stated examples include large target jumps and goals inconsistent with current contacts.
Loading 2603.03279v1…