BRACE: Adapting Whole-Body References for Force and Terrain Aware Humanoid Motion Tracking

I. INTRODUCTION

• Whole-body tracking connects human motion to humanoid control, and its trackers also serve as teleoperation interfaces for collecting demonstrations,,.Motion-capture and in-the-wild video data provide large quantities of reference trajectories,.Source: §I · p. 1
• Flat-ground, unloaded motion references omit the exchanged wrench and terrain, leaving teleoperated robots without specifications for tasks such as moving payloads or pushing carts uphill.A slope changes both the force needed and the lean used to provide it.Source: §I · p. 1
• Prior force-capable controllers accept end-effector force or impedance but prescribe no dense whole-body pose –, while terrain-adaptive controllers treat loads as disturbances [3],,.Thus, existing approaches do not combine commanded wrenches with whole-body pose tracking on terrain.Source: §I · pp. 1–2
• A commanded hand force requires end-effector displacement and coordinated body adjustment to preserve balance, while reference footholds and root height may mismatch the robot’s terrain.The authors note that no dataset pairs retargeted humanoid motion with its execution terrain, preventing direct learning of the correction.Source: §I · p. 2
• BRACE reports 40-percentage-point lower average global drift and an 18.25-percentage-point average success-rate improvement, and demonstrates the policy on a Unitree-G1.The reported tasks include inserting a syringe, turning a winch, lifting a barbell, and pushing a cart up a slope.Source: §I · p. 2
• BRACE analytically adapts flat-ground references to terrain and commanded wrenches, then distills separate wrench-exertion and reference-compensation teachers into one proprioception-and-command-driven student.The system targets slopes, rough surface inconsistencies, and small platforms that can be handled with proprioception.Source: §I · p. 2

A. Whole-Body Tracking on Terrain and Under Load

• Prior trackers handle terrain, payloads, or pushes separately, but do not expose a commanded interaction wrench while jointly tracking whole-body motion across terrain and loads.Any2Track handles dynamics mismatch and terrain, while HEFT treats payloads as quasi-static gravitational loads; neither provides the operator a commanded wrench interface.Source: A. Whole-Body Tracking on Terrain and… · p. 2
• BRACE accepts a flat-ground reference and adapts it internally to terrain and interaction demands, separating the operator’s intended motion and wrench from local conditions.This separation supports remote teleoperation even when the operator and robot do not share terrain or physical load.Source: A. Whole-Body Tracking on Terrain and… · p. 2

B. Force Control in Locomotion

• Prior force-exertion methods command end-effector force or impedance, while compensation frameworks hold a pose against external forces using decoupled lower- and upper-body agents.Examples include UNIFP and FACET for exertion, and FALCON, HAFO [12], and KinAware [13] for compensation.Source: B. Force Control in Locomotion · p. 2
• Unlike interfaces that leave whole-body pose degrees of freedom unresolved, BRACE tracks a full-body reference against a commanded wrench and switches between exertion and compensation.Exertion produces the commanded wrench; compensation preserves motion against an external load, including in similar body configurations.Source: B. Force Control in Locomotion · p. 2

C. Behavior Models and Multi-Teacher Distillation

• Generalist controllers combine skills or command interfaces, including HOVER and BFM-Zero [26], while OmniXtreme distills motion-specific experts into a flow-matching policy through DAgger.Other examples include PULSE, MaskedMimic, and BeyondMimic.Source: C. Behavior Models and Multi-Teacher… · p. 2
• BRACE distills separate privileged exertion and compensation teachers through DAgger into one student conditioned on a motion goal, wrench command, and interaction mode.The formulation extends multi-teacher distillation to complementary interaction behaviors while retaining whole-body tracking across terrain and loading conditions.Source: C. Behavior Models and Multi-Teacher… · p. 2

III. METHOD

• BRACE tracks flat-ground human-motion references while exerting or compensating commanded hand forces on uneven terrain.The operator supplies a retargeted kinematic trajectory and commanded hand wrenches specifying force and moment.Source: §III · p. 3
• Analytically adapting references to terrain and wrenches avoids the conflict between tracking rewards and the pose deviations needed for bracing and contact displacement.Unadapted references assume a flat floor and omit the whole-body adjustments needed to exert or resist wrenches.Source: §III · p. 3
• Separate exertion and compensation teachers track adapted targets, and DAgger distills them into one flow-based student operating from onboard observations alone.Reference synthesis adapts the input for terrain and commanded wrench; privileged teacher training and student distillation form the other framework stages.Source: §III · p. 3

A. Terrain- and Wrench-Adapted Reference Synthesis

• Terrain adjustment preserves foothold horizontal placement, lifts foot elevation for local ground and swing clearance, and adjusts root elevation and foot orientation to the surface.Root orientation remains aligned with locomotion direction; whole-body inverse kinematics reconciles foot and root changes with the upper-body pose.Source: A. Terrain- and Wrench-Adapted… · pp. 3–4
• Wrench adjustment caps commands using joint-effort and support constraints, jointly limiting force and moment according to each teacher’s effort budget.Teacher-E uses actuator limits at the terrain-adjusted reference; Teacher-C uses a floored residual-effort proxy at the current configuration. Support-based scaling also bounds hand excursions and center-of-pressure travel and is smoothed across gait transitions.Source: A. Terrain- and Wrench-Adapted… · p. 4
• Hand leads displace targets along commanded force and moment directions using spring–damper compliance matched to the arm’s joint gains.Commands below 1 N or 0.3 N·m are zeroed, making the corresponding lead vanish.Source: A. Terrain- and Wrench-Adapted… · p. 4
• The COM brace counters reaction-wrench overturning moments with a horizontal shift; support-polygon center-of-pressure travel absorbs part of the shift, and the bounded remainder informs hand targets.Whole-body inverse kinematics reconciles the displaced hands, braced COM, and terrain-adjusted feet into a valid adapted target.Source: A. Terrain- and Wrench-Adapted… · p. 4

a) Shared Architecture

• Both teachers use goal-space redirection and proprioception with a privileged latent, plus a shared-trunk 29-joint action head and target-reconstruction head, trained by PPO with asymmetric critics.Only the action head is retained after training; the arm-effort constraint bounds Teacher-E’s exertion and Teacher-C’s resistance.Source: B. Privileged Teacher Policies · p. 4

b) Teacher-E: Force Exertion

• Teacher-E converts hand displacement into exertion by applying a virtual wrench-field reaction at the terrain-adjusted reference hand, crediting part of the planned brace to spring force.Rewards evaluate tracking and wrench exertion against feasible targets, judging performance by what the terrain and joint budget permit.Source: B. Privileged Teacher Policies · pp. 4–5

Teacher-C: Force Compensation

• Teacher-C learns compensation by tracking the reference under bounded external wrenches, with randomized force-and-moment pulses, unloaded intervals, and adaptive load scaling.Loads are constrained by residual joint effort; randomized hand-frame offsets couple force and moment, while wrench history lets the teacher brace before loads peak.Source: B. Privileged Teacher Policies · p. 5

Distillation

• DAgger distills both privileged-signal teachers into one deployable student, with training rollouts covering mid-motion mode switches and ramped load transitions.The E/C mode flag selects the active teacher during rollouts; the student can use the flat-ground motion supplied by a teleoperator at inference.Source: C. Flow-Based Student Distillation · p. 5

Student Architecture

• Flow matching models multimodal teacher actions from 25 frames of proprioceptive history, while a task encoder and decoder combine the base action with task context into joint commands.The task context includes flat-ground goals, mode flag, and commanded wrench; training combines behavior cloning, flow-matching, exertion-weighted, and estimation losses.Source: C. Flow-Based Student Distillation · p. 5

Terrain and Load Estimation

• At deployment, the student receives proprioception, goals, mode flag, and wrench command, but no terrain geometry or measured external wrench.An auxiliary training head regresses privileged simulator states from the task latent, including hand positions, root velocity, external wrench, surface normal, and root height; it is discarded at export.Source: C. Flow-Based Student Distillation · p. 5

Implementation

• The 29-DoF G1 policies command PD targets at 50 Hz; teachers use 8-frame histories and the student 25-frame histories.The flow base uses four Euler steps per action; its height scan is a 16×16 grid covering 2.0 m×2.0 m, and wrench history spans 3 frames.Source: D. Dataset, Implementation and real… · p. 5
• Teacher effort limits are γ = 0.7, with compensation trained up to 20N, exertion up to 9N, and torque up to 4Nm for both.These settings are intended to stay within the G1 arms’ physical limits,; the exertion setup also uses a 0.15 m hand-excursion budget and 0.02 m support-polygon erosion.Source: D. Dataset, Implementation and real… · p. 5

Dataset, Implementation and real robot deployment

• Training uses about 65k selected Bones-Seed trajectories in IsaacLab/IsaacSim with Protomotions, while real-robot flow-policy deployment uses RoboJudo.Teachers and student train on 8 NVIDIA RTX Pro 6000 GPUs each, with 2048 environments and batch size 8192 per GPU.Source: D. Dataset, Implementation and real… · p. 5

• The evaluation covers force compensation and exertion on both flat ground and sloped terrain, alongside whole-body tracking.Source: §IV · p. 5, Figure 3 · p. 6
• Fig. 3 measures success, global and local MPJPE, and velocity error; force error is reported for exertion only.Compensation tests sustained loads of 0–20 N per hand, while exertion tests commanded forces of 3–9 N per hand.Source: Figure 3 · p. 6
• Each plotted point is the equal-weight mean over the PHUMA and Bones-seed test splits, with margins comparing BRACE against the better of SONIC and HEFT.The figure marks success margins in percentage points and error margins as relative changes at the heaviest command.Source: Figure 3 · p. 6

Baselines

• Parallel Isaac Lab simulation on the 29-DoF G1 compares BRACE with SONIC, TWIST2, Any2Track [3], and HEFT [22], using each baseline’s native settings.Force-commanding controllers, are excluded because they do not accept whole-body reference trajectories; BRACE’s deployed student observes neither the simulator height map nor the applied compensation wrench.Source: A. Evaluation Setup · p. 6

Datasets

• The shared test split combines PHUMA motions with held-out Bones-seed motions absent from training, and is identical for every method.Source: A. Evaluation Setup · p. 6

Protocols

• Compensation applies a constant 0–20 N hand load in a uniformly sampled, episode-fixed direction; exertion commands 3–9 N per hand in the torso-heading frame.Exertion tracks the force-compensated reference available at real-robot deployment; a moving-reference spring-damper applies the simulated reaction, and pose-feasibility limits can reduce the effective target.Source: A. Evaluation Setup · p. 6
• Terrain tests use nominal slopes of 4.6–17.7°, while policies receive only flat-ground references and must handle slopes blindly from proprioception.Realized grade varies along the trajectory, and motions are scored against terrain-conforming references.Source: A. Evaluation Setup · p. 6

Metrics and success criteria

• Tracking reports world-coordinate MPJPE-G and root-translation-removed MPJPE-L; success requires height and torso-orientation limits plus a setting-specific position check.Errors average recorded frames until the first height/orientation violation. Torso, wrist, and ankle heights must remain within 0.25 m of reference and torso orientation within 1 rad; passing motions must also meet a 0.5 m average torso horizontal-distance limit on flat ground or a 0.5 m terrain MPJPE-G limit. Success is the share of motions passing both checks, with the same thresholds for all baselines; completed exertion motions also report average measured-versus-target force error after feasibility limits.Source: A. Evaluation Setup · pp. 6–7

B. Force Compensation — 1) Flat Ground Evaluations:

• At 20 N per hand on flat ground, BRACE leads the best baseline by 10.7 percentage points in success and ends with 40% lower global error.Local error stays within 10% of SONIC’s; the passage attributes resistance to drift to the wrench-compensation teacher distilled into the student, which does not observe the load.Source: 1) Flat Ground Evaluations · p. 7

B. Force Compensation — 2) Terrain Evaluations:

• On slopes, BRACE leads at every load, with a 22.9-percentage-point success gap and 43% lower global error than the best baseline at 20 N.All methods use the same flat-ground reference without terrain information; BRACE’s terrain-conformed training helps it hold position while resisting load, with local error indistinguishable from SONIC’s.Source: 2) Terrain Evaluations · p. 7

C. Force Exertion — 1) Flat Ground Evaluations:

• In the force-exertion experiment, trackers receive the flat-terrain force-compensated reference, while a virtual spring-mass-damper simulates interaction forces in the simulator.At 3 N, SONIC and HEFT succeed on most motions and their force errors are within about 1 N of BRACE’s; as command rises, SONIC’s success drops by more than 20 points and global error more than doubles.Source: 1) Flat Ground Evaluations · p. 7
• At 9 N on flat ground, BRACE exceeds HEFT’s success rate by 5.7 percentage points, achieves 35% lower global error, and reduces force error by 34% versus the closest baseline.BRACE’s success and global error stay nearly flat from 3–9 N, and it has the lowest force error at every command.Source: Figure 4 · p. 7, 1) Flat Ground Evaluations · p. 7

C. Force Exertion

• At 9 N on an unobserved slope, BRACE leads the closest baseline by 33.7 percentage points in success, with 42% lower global error and 47% lower force error.Its success remains flat across the command range and matches unloaded terrain tracking; the comparison against force-commanding controllers remains untested.Source: 1) Flat Ground Evaluations · p. 7

D. Torque Analysis

• At commanded torques of 0.5–2.5 N·m, BRACE has the lowest moment error at every command, 33% below the closest baseline on flat ground and 32% on slopes.These results demonstrate accurate commanded-wrench exertion.Source: D. Torque Analysis · p. 7

E. Applications

• Using 362 teleoperated G1 demonstrations, the authors fine-tuned GR00T N1.7 3B, [39] to predict whole-body and Dex3 hand commands; the policy executes autonomously on hardware.The demonstrations involved transferring three bananas from a table into a container using vision, proprioception, and a fixed instruction.Source: E. Applications · p. 8
• BRACE adapted operator references during cross-site teleoperation across sites approximately 1,000 miles apart and tracked VR-retargeted references under varying terrain and loads.VR controller poses streamed through XRoboToolkit [40] and were retargeted to the G1 using GMR [41].Source: E. Applications · p. 8
• On hardware, BRACE tracked whole-body trajectories generated by MotionBricks [42] from gamepad root-velocity commands.The applications section also reports cross-site teleoperation and hardware demonstrations for VLA learning.Source: E. Applications · p. 8, §V · p. 8

2) Terrain Evaluations:

• At each setting’s highest tested load or command, BRACE reduces global tracking error by 40% and improves success by 18.25 percentage points on average.The conclusion summarizes these results across the evaluations in Sec. IV.Source: §V · p. 8

V. CONCLUSION

• A proprioceptive student tracks flat-ground references on slopes while exerting or compensating hand forces, without height maps or wrench measurements.BRACE adapts references to terrain and six-dimensional wrenches.Source: §V · p. 8