Source-linked AI summary
Contact-UAN: Unsupervised Actuator Networks as a Residual World Model for Humanoid Control
Nolan Fey, Khai Nguyen, Yash Prabhu, Gabriel B. Margolis, Pulkit Agrawal
TL;DR
Humanoid policies may transfer qualitative motion without reproducing force interactions needed for torque efficiency, load regulation, and impact handling. Contact-UAN uses proprioception to initialize contact-consistent replay and learn actuator corrections while preserving simulator physics. With 6.5 minutes of walking data, it improves transfer across running, force tracking, and jumping, including lower torque usage and force-tracking error.
Problem
Sim-to-real humanoid policies can transfer qualitative motion while failing to transfer force interactions, which matter for torque efficiency, load regulation, and impact absorption.
Source: §1 · p. 2Method
Contact-UAN preserves simulator physics while learning actuator-input residuals from hardware replay initialized with proprioceptively inferred contact-consistent states.
Source: §1 · p. 2, §2.2 · p. 4Results
Takeaways & Limitations
A single walking-data UAN improves transfer across behaviors that benefit from accurate force and impulse regulation, including running, force and impedance control, and jumping.
Source: §1 · p. 3, §7 · p. 9The current Contact-UAN does not capture coupled actuator effects or motor-temperature-dependent effects.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Humanoid policies trained in simulation can transfer qualitatively to hardware while failing to transfer the force interactions learned during training, compromising torque efficiency, load regulation, and robustness to impacts. This failure can arise from actuator sim-to-real gaps because forces depend on actuator torques. We introduce Contact-UAN, a residual world model that preserves a simulator's physics priors while correcting actuator dynamics to align simulated rollouts with recorded hardware rollouts. For humanoid motion with contact, this requires contact-consistent replay: if hardware playback in simulation begins with misaligned contact wrenches, the residual model learns to correct reset artifacts rather than actuator discrepancies. Using only onboard proprioception, Contact-UAN infers the simulator-side stance-foot pose needed to initialize replay with consistent contact forces. We show that a UAN trained on 6.5 minutes of real-world walking data generalizes to tasks that benefit from accurate force and impulse regulation and produces policies that transfer seamlessly to the real world, yielding 20% lower running torque, 43% lower force-tracking error, extreme compliance, accurate impedance control, and dynamic, robust jumps.
1 Introduction
Humanoid policies can transfer their motions without transferring force interactions, motivating actuator modeling for torque-efficient, force-sensitive control. Contact-UAN uses proprioceptive contact-consistent replay initialization to learn actuator residuals and improve transfer across several behaviors.
1 Introduction
- Qualitative motion transfer alone does not establish that a humanoid reproduces the force profiles needed for efficient movement, load regulation, and impact absorption.The paper focuses on actuator modeling because forces are mediated by joint torques.Source: §1 · p. 2
- Millimeter-scale foot-height and degree-scale orientation errors can substantially change simulated contact wrenches, making contact-consistent initialization necessary for replay-based learning.Figure 3 highlights sensitivity of force to height error and moment to pitch error.Source: §1 · p. 2, Figure 3 · p. 3
- Contact-UAN infers simulator-side stance-foot pose and base velocity from proprioception, enabling replay initialized with contact wrenches consistent with hardware.This initialization lets the residual learn actuator-model errors rather than reset artifacts.Source: §1 · p. 2, Figure 2 · p. 3
- A single UAN trained on 6.5 minutes of real-world walking data improved transfer across running, force and impedance control, and jumping.These tasks were selected for their distinct force- and impulse-sensitive demands.Source: §1 · p. 3
- 20% lower hardware torque usage and 43% lower force-tracking error were achieved relative to the no-UAN baseline; UAN task-reward fine-tuning also transferred for higher jumps.Both reference-tracking jump policies transferred, but task-reward fine-tuning for higher jumps transferred only with UAN.Source: §1 · p. 3
2 Method
Contact-UAN relabels hardware trajectories for contact-consistent replay, trains a residual actuator model, then trains downstream policies in the augmented simulator. Replay-based learning depends on initializing contacts to reproduce hardware contact wrenches.
2.1 Contact-Consistent Replay Initialization
- Replay requires simulator contacts that reproduce hardware wrenches, so the method regresses the simulator-side penetrating foot pose from load-dependent proprioception rather than estimating force alone.Onboard sensing provides neither contact wrench nor absolute base height, and simulator soft-contact behavior depends on foot penetration and tilt.Source: §2.1 · p. 3
- Proprioceptive history is used to infer contact status, stance-foot pose, and base velocity; confident stance estimates are fused into a root pose for replay initialization.The estimator omits absolute base height and conditions pose regression on stance so swing variation does not dominate the contact-sensitive objective.Source: §2.1 · pp. 3–4
2.2 UAN as a Residual World Model
- The UAN learns a residual correction to simulator inputs, preserving the simulator’s rigid-body dynamics, actuator implementation, contact solver, and numerical integration.It is trained by replaying labeled hardware windows and penalizing joint-position and velocity error alongside residual-rate magnitude.Source: §2.2 · pp. 4–5
- Hardware training data come from 6.5 minutes of varied walking, recorded with onboard proprioception and relabeled with base velocity and pose.Gait frequency and swing height are varied to diversify stance durations, swing speeds, and impacts.Source: §2.2 · p. 5
2.3 Policy Training and Deployment
- After UAN training, its frozen correction is used to train task policies without robot-specific domain randomization; only the learned task policy runs on hardware.Terrain mesh, ground friction, and external pushes are still randomized during policy training.Source: §2 · p. 3, Figure 4 · p. 4, §2.3 · p. 5
3 Experimental Setup
Experiments use a Unitree G1 and compare downstream policies trained with and without UAN. The evaluation includes actuator-sensitive tasks and controlled simulator perturbations.
3 Experimental Setup
- The evaluation uses a 29-DoF Unitree G1 with BeyondMimic G1 parameters, including PD gains and manufacturer-provided actuator inertias.No additional robot-specific system identification is performed before Contact-UAN training.Source: §3 · pp. 5–6
- A WTW-style policy is deployed for 6.5 minutes to collect varied walking data, then running, force-control, and jumping policies are compared across UAN-augmented and nominal simulators.No-UAN policies use additional robot-specific randomization; UAN policies do not.Source: §3 · p. 6
- Controlled sim-to-sim evaluation uses easy, medium, and hard target simulators formed by scaling perturbations to damping, friction loss, action delay, encoder bias, and armature.Held-out running torque replay compares traces under identical relabeled initial states and aggregate commanded-torque usage error.Source: §3 · p. 6, Figure 6 · p. 6
4 Results
Contact-UAN improves force-sensitive sim-to-real transfer while preserving running tracking, and contact-consistent initialization supports accurate replay and stronger downstream transfer. Its benefits extend to larger controlled actuator gaps, reset variants, and alternative residual architectures.
4.1 Residual World Model Accuracy
- Contact-conditioned pose estimation reduces contact-height RMSE by an order of magnitude versus direct regression over all gait phases.The contact-conditioned estimator first identifies contact and estimates pose only for confident contacts, avoiding errors driven by swing-foot variation.Source: §4.1 · p. 6
- Contact-UAN reduces all-joint aggregate commanded-torque mismatch by 69.2% during held-out running replay with contacts.The comparison uses identical commands and initial states; the replay also reports lower 24-step joint-position MSE with UAN than direct initialization.Source: §4.1 · p. 6
- With ground-truth initialization, the residual lowers joint-position RMSE from 0.0720 to 0.0184 rad.This supports improved actuator modeling rather than compensation for initialization error.Source: §4.1 · p. 7
4.2 Sim-to-Real Task Transfer
- 42.8 ± 12.1% lower average force-tracking error accompanies retained running tracking and improved efficiency with UAN policies.The UAN force policy also supports compliant interaction and impedance control; see the separately reported jumping results.Source: §4.2 · p. 7, Figure 7 · p. 7
- Task-reward fine-tuning transfers to higher jumps with stable landings under pushes with UAN, while the no-UAN policy fails during landing.Both reference-tracking policies transfer, but the UAN policy tracks better and jumps higher; the UAN uses walking training data.Source: §4.2 · p. 7
4.3 Sim-to-Sim Evaluation
- Across easy, medium, and hard actuator-gap simulators, UAN policies achieve expert-level tracking error and torque usage, with little added benefit from running data beyond walking data.The baseline degrades sharply as the gap grows; the reported WTW UAN training uses five minutes of walking data.Source: §4.3 · p. 8, Figure 8 · p. 8
- Contact-consistent resets raise 20-s survival from 49.8% with direct-estimator resets to 96.7%, near the 96.6–98.4% oracle-reset range.The comparison retrains UANs and policies with the different reset variants.Source: §4.3 · p. 8, Figure 8 · p. 8
- Position residuals, torque residuals, and supervised torque models achieve similar downstream performance in a matched simulation comparison.Position residuals avoid torque supervision and retain the deployed position-control interface.Source: §4.3 · p. 8
5 Limitations
Current Contact-UAN does not capture coupled actuator effects or motor-temperature-dependent effects.
5 Limitations
- Current Contact-UAN does not capture coupled actuator effects or motor-temperature-dependent effects.The authors note that temperature effects may require heat modeling or additional sensors and are difficult to identify from short datasets.Source: §5 · p. 8
6 Related Work
Related approaches estimate dynamics, randomize simulator parameters, or learn world models, each with distinct constraints. Contact-UAN instead preserves analytical simulation structure while learning actuator-level corrections from proprioceptive data.
6 Related Work
- System identification methods estimate inertial and dynamic parameters, while domain randomization trains over parameter distributions but can yield conservative or inefficient policies when randomization is wide.The paper also notes that identification methods remain constrained by the structure of their chosen parametric model.Source: §6 · p. 9
- Fully learned world models replace hand-designed simulators and can reduce parametric structural bias, but require large datasets and safeguards against exploitation outside the data distribution.Contact-UAN retains rigid-body dynamics, contact, and integration while restricting learning to the actuator channel.Source: §6 · p. 9
- Supervised actuator networks require torque sensing, whereas UAN learns residual actions from recorded hardware transitions; Contact-UAN extends this approach to a floating-base 29-DoF humanoid without external sensing.The method uses proprioceptive state estimators for base velocity and contact pose to enable contact-aligned residual learning.Source: §6 · p. 9
7 Conclusion
Contact-UAN preserves an analytical simulator while learning actuator corrections from offline proprioception and estimating contact-consistent replay states. On a humanoid, the approach transfers across force- and impulse-sensitive tasks and remains effective under larger controlled actuator gaps.
7 Conclusion
- Contact-UAN learns actuator-level corrections from offline proprioceptive data while preserving the analytical simulator’s structure.Replay initialization estimates base velocity, contact status, and simulator-side contact pose to align contact forces with hardware trajectories.Source: §7 · p. 9
- Using 6.5 minutes of real-world walking data, the UAN improves transfer across tasks, including lower running torque use, lower force-tracking error, compliant interaction, impedance control, and higher jumps with stable landings.The conclusion reports 20% lower running torque usage and 43% lower force-tracking error.Source: §7 · p. 9
- Reset ablations support contact-consistent initialization, and sim-to-sim experiments show the residual remains effective under larger controlled actuator gaps.The reported results suggest residual world models can retain physics-simulation generalization while improving fidelity where the sim-to-real gap matters.Source: §7 · p. 9
A.1 Data Collection
The appendix describes estimator data and replay initialization: contact-weighted stance-foot estimates produce a root pose for replay, while the training details specify sampled replay windows and residual limits.
A.1 Data Collection
- The data-collection dataset uses 20 simulation seconds across 4096 environments, with a random 10% of environments held out for validation.Collection uses the WTW-style policy in its training environment on flat terrain with the UAN-training contact parameters.Source: §A.1 · p. 14
A.2 Architectures
- Foot-height and foot-projected-gravity estimators are trained separately for left-stance and right-stance samples.The shared estimator settings are ELU activations, Adam, learning rate 3 × 10−4, batch size 4096, history length 3, and 200 epochs.Source: §A.2 · p. 14, Table 1 · p. 15
A.3 Observation Noise
- Estimator-data collection uses the observation-noise model listed in Table 2, which is also used for data-collection and downstream policies except jumping, which uses smaller joint-velocity noise.The appendix identifies the noise model as uniform additive noise.Source: §A.3 · p. 14, Table 2 · p. 15
- A proprioceptive estimator paired with a precise force-control policy tracks a circular end-effector trajectory reasonably well without external sensing.Figure 10 illustrates impedance position control.Source: Figure 10 · p. 15
A.4 Contact-Weighted Root-Pose Fusion
- Contact-weighted pose estimates from confident feet are fused to initialize replay with support forces consistent with hardware; windows without confident contact are excluded.The resulting reset combines fused root height and orientation with separately estimated base velocity.Source: §A.4 · pp. 15–16
B UAN Training Details
- Replay windows use three recorded steps to prime UAN history, then roll 24 recorded joint-position targets forward; windows require a confident contact and residual corrections are clipped to ±1.0 rad.The contact-probability threshold is ≥0.8.Source: B UAN Training Details · p. 16
C Data Collection Policy Training Details
The data-collection policy is a domain-randomized WTW-style velocity tracker whose varied gait commands produce diverse stance durations, swing speeds, and impacts for UAN training.
C Data Collection Policy Training Details
- The WTW-style velocity-tracking policy uses behavior-shaping commands and is trained without a UAN.Its command set includes base velocity commands.Source: C Data Collection Policy Training… · p. 16
- Varying gait-frequency and swing-height commands produces diverse stance durations, swing speeds, and impacts in the UAN training data.The commands are resampled every 5–10 seconds; gait frequency ranges from 1.0 to 2.5 Hz and swing height from 0.05 to 0.1 m.Source: C Data Collection Policy Training… · p. 16
- Domain randomization samples foot friction, joint target offsets, and torso center-of-mass offsets at startup, with each environment’s parameters fixed during training.The sampled ranges are 0.3–1.2 for foot friction, ±0.015 rad for joint target offsets, and ±2.5 cm horizontally and ±3 cm vertically for torso offsets.Source: C Data Collection Policy Training… · p. 16
D Downstream Policy Training Details
Downstream policies use a shared simulation and training setup, with task-specific designs for running, force control, impedance control, and jumping. The jumping policy first tracks a retargeted motion and is then fine-tuned to increase ballistic rise while retaining its motion structure.
D Downstream Policy Training Details
- Downstream policies act at 50 Hz in 200 Hz simulation, with joint-position-offset actions applied through per-joint PD controllers; the frozen UAN adds a residual in simulation but is not exported at deployment.No-UAN policies additionally use robot-specific randomization for running and force control; UAN-augmented policies remove that randomization.Source: D Downstream Policy Training Details · p. 17
- Running and jumping policies use a torque-speed actuator envelope, with commanded targets clipped to asymmetric bounds before the UAN residual is added in simulation.Deployment applies the same clipping.Source: D Downstream Policy Training Details · p. 17
D.1 Running Policy
- The running policy removes base linear velocity observations, extends observation history to five steps, and adds rough-terrain randomization and torque-speed clipping.Other changes include slow-walking commands, a low-height termination, and a stronger joint-position-limit penalty.Source: §D.1 · p. 18
D.2 Force Control Policy
- The force-control policy tracks commanded forces applied through a virtual wrist spring–damper, with commands sampled per axis and clipped to a 50 N applied-force limit.The force is ramped up, held, and ramped down; the policy observes commanded force with a 20-step history.Source: §D.2 · p. 18
- An outer impedance loop converts end-effector position error into a force command for the frozen force-control policy, enabling compliant position control without an arm-specific controller.The described example tracks circular trajectories with radii up to 0.3 m.Source: §D.2 · pp. 18–19
D.3 Jumping Policy
- The jumping policy first tracks a retargeted standing-jump motion, then adds a ballistic center-of-mass rise reward while retaining motion-tracking rewards and replacing reference-error terminations with fall conditions.The reference is a retargeted CMU MoCap jump clip padded with standing at the beginning and end.Source: §D.3 · pp. 19–21
E Sim-to-Sim Experiment Details
The sim-to-sim experiment varies actuator mismatch across three deterministic target simulators and compares downstream policies trained with nominal or UAN-augmented dynamics.
E Sim-to-Sim Experiment Details
- Easy, medium, and hard target simulators apply scaled deterministic perturbations to joint damping, friction loss, action delay, encoder bias, and armature.The easy target creates a measurable but recoverable tracking gap for the nominal baseline.Source: E Sim-to-Sim Experiment Details · p. 21, Table 12 · p. 21
- For each target simulator, downstream velocity-tracking policies are trained in nominal or UAN-augmented simulation, using a walking-only UAN or one trained with walking and running data.The WTW UAN uses five minutes of walking data; WTW+Vel adds five minutes of running data.Source: E Sim-to-Sim Experiment Details · p. 21
F World Model Details
In a controlled hard-target comparison, UAN reached expert-level normalized episode length with far less target data than the fully learned world-model variants. Reset, sensor-corruption, and actuator-model tests further examine the conditions and robustness of replay-based learning.
F World Model Details
- With 5 minutes of target data, UAN reaches 100% expert normalized episode length; WM FT reaches 38.2% versus 32.6% for the nominal-simulator baseline after using about 400× more data.This is a controlled comparison on one hard target environment, not a definitive benchmark of RWM-U.Source: Figure 11 · p. 22, F World Model Details · p. 22
G.1 Reset and Estimator Protocols
- Estimated reset raises 20-s survival from 49.8% to 96.7%; the residual also lowers replay RMSE with estimated and ground-truth initialization, supporting actuator correction beyond reset estimation.Joint-position RMSE falls from 0.0750 to 0.0239 rad with estimated initialization and from 0.0720 to 0.0184 rad with ground-truth initialization.Source: §G.1 · pp. 23–24
G.2 Retraining with Structured Sensor Corruption
- Under fixed IMU pitch biases or causal velocity filtering, 20-s survival remains 97.1–100%, while replay RMSE and command-tracking errors stay near clean-data levels.Joint-position and joint-velocity replay RMSEs remain within 8% and 11% of clean, respectively; command-tracking errors remain within approximately 15%.Source: §G.2 · pp. 24–25, Figure 13 · p. 25
G.3 Matched Actuator-Model Comparison
- In matched 200-Hz simulation, position-residual, torque-residual, and torque-supervised models have similar downstream performance, though the position residual fits target data tightest.Torque supervision is available in this simulation comparison; it does not imply reliable output-torque sensing on G1 hardware.Source: Table 15 · p. 25, §G.3 · p. 25
G.4 Additional Replay Visualizations
- Additional replay visuals compare held-out hardware knee trajectories with nominal and UAN simulator replay, and show recorded-action rollouts using alternative estimators and UAN.The rollout sequence contrasts an estimator trained on stance and swing data, a stance-only estimator, and stance-only estimation with UAN.Source: §G.4 · p. 25, Figure 14 · p. 26, Figure 15 · p. 26