Source-linked AI summary
FALCON: Learning Force-Adaptive Humanoid Loco-Manipulation
Yuanhang Zhang, Yifu Yuan, Prajwal Gurunath, Ishita Gupta, Shayegan Omidshafiei, Ali-akbar Agha-mohammadi, Marcell Vazquez-Chanlatte, Liam Pedersen, Tairan He, Guanya Shi
TL;DR
Forceful humanoid loco-manipulation requires precise whole-body control under multidirectional end-effector forces, whereas prior approaches are often limited to lightweight tasks. FALCON jointly trains specialized lower- and upper-body agents with a torque-aware 3D force curriculum, achieving more accurate tracking, robust locomotion, faster convergence, and deployment across humanoid platforms. Its scope is limited to end-effector forces and does not cover other-body-part contacts or external torques.
Problem
Forceful humanoid loco-manipulation remains challenging because precise manipulation under 3D end-effector forces must coexist with stable locomotion.
Method
FALCON jointly trains lower- and upper-body RL agents using shared proprioception and commands with a progressive 3D force curriculum respecting joint torque limits.
Results
FALCON outperforms Lower-RL-Upper-IK and Monolithic-Whole-body-RL with faster convergence, reduced tracking errors, stable force-regime performance, and cross-platform transfer.
Takeaways & Limitations
The same training setup produces policies deployed on Unitree G1 and Booster T1 for payload transport, cart-pulling, and door-opening tasks.
Takeaways & Limitations
FALCON considers end-effector forces but not other-body-part contacts, multi-contact interactions, or external torques.
Abstract
from arXiv · showhide
Humanoid loco-manipulation holds transformative potential for daily service and industrial tasks, yet achieving precise, robust whole-body control with 3D end-effector force interaction remains a major challenge. Prior approaches are often limited to lightweight tasks or quadrupedal/wheeled platforms. To overcome these limitations, we propose FALCON, a dual-agent reinforcement-learning-based framework for robust force-adaptive humanoid loco-manipulation. FALCON decomposes whole-body control into two specialized agents: (1) a lower-body agent ensuring stable locomotion under external force disturbances, and (2) an upper-body agent precisely tracking end-effector positions with implicit adaptive force compensation. These two agents are jointly trained in simulation with a force curriculum that progressively escalates the magnitude of external force exerted on the end effector while respecting torque limits. Experiments demonstrate that, compared to the baselines, FALCON achieves 2x more accurate upper-body joint tracking, while maintaining robust locomotion under force disturbances and achieving faster training convergence. Moreover, FALCON enables policy training without embodiment-specific reward or curriculum tuning. Using the same training setup, we obtain policies that are deployed across multiple humanoids, enabling forceful loco-manipulation tasks such as transporting payloads (0-20N force), cart-pulling (0-100N), and door-opening (0-40N) in the real world.
1 Introduction
FALCON addresses forceful humanoid loco-manipulation by jointly learning specialized lower- and upper-body control with a torque-aware 3D force curriculum. It targets robust locomotion, accurate end-effector tracking, and cross-platform deployment under substantial unknown forces.
- Motivation: Forceful humanoid loco-manipulation requires precise manipulation under multidirectional forces while maintaining lower-body stability.The target systems must adapt to varying payloads and contact forces without compromising precision or robustness.
- Motivation: Existing RL approaches mainly succeed on lightweight tasks and often lack whole-body coordination under significant interaction forces.Lower-RL-Upper-IK lacks whole-body dynamics modeling, while monolithic whole-body RL couples distinct objectives.
- FALCON: FALCON separates lower-body locomotion and upper-body tracking into jointly trained agents sharing whole-body proprioception and commands.The decomposition uses tailored rewards while preserving coordination between the two agents.
- FALCON: A 3D force curriculum progressively increases end-effector forces while enforcing joint torque feasibility through inverse dynamics.This curriculum is designed to improve force adaptation without violating torque limits.
- Results: 100% higher upper-body joint tracking accuracy is reported over prior methods while robust locomotion is maintained under 0–100N unknown end-effector forces.The force range reaches up to 30% of body weight.
- Results: FALCON is validated on Unitree G1 and Booster T1, with cross-platform generalization achieved using minimal tuning overhead.The evaluated tasks include payload transport, cart-pulling, and door-opening.
2 Related Works
Prior humanoid loco-manipulation methods use model-based control or two RL paradigms, but few address significant unknown end-effector forces. FALCON jointly trains task-specialized upper- and lower-body agents to coordinate force responses.
- Existing approaches: Traditional model-based methods provide real-time planning but rely on manual design, limiting flexibility and generalizability.Learning-based sim-to-real RL has instead shown promising results for versatile loco-manipulation.
- Existing approaches: Lower-RL-Upper-IK combines lower-body RL with kinematic upper-body control, while monolithic whole-body RL learns all degrees of freedom together.These represent the two primary paradigms identified for humanoid loco-manipulation.
- Limitations: Few RL methods address significant unknown end-effector forces, and Lower-RL-Upper-IK can delay upper-body force compensation.Monolithic approaches can suffer sample inefficiency, overfitting, and behavioral dominance by one body subsystem.
- Related forceful interaction: Quadrupedal forceful interaction has advanced through MPC, force planning, impedance control, and aggressive force-adaptation RL.The passage contrasts this established quadrupedal literature with the more limited humanoid setting.
- FALCON: FALCON implicitly adapts to unknown end-effector forces using a 3D curriculum that accounts for humanoid joint torque limits.The design aims to maximize force adaptability while supporting safe real-world deployment.
3 FALCON: Force-Adaptive Humanoid Loco-Manipulation
FALCON formulates humanoid loco-manipulation as coordinated dual-agent control, combining lower-body locomotion and upper-body manipulation under shared proprioception. Its torque-limit-aware force curriculum and force randomization support adaptive control under external end-effector disturbances.
- 3.1 Dual-Agent Learning Framework: Unlike Lower-RL-Upper-IK and Monolithic-Whole-body-RL, FALCON uses decomposition learning to improve training efficiency and coordination under force disturbances.The decomposition addresses delayed upper-body force compensation and exploration inefficiency from large, coupled action spaces.
- 3.1 Dual-Agent Learning Framework: FALCON partitions humanoid control into lower-body locomotion and upper-body manipulation agents that share whole-body proprioceptive input.The agents optimize independent goal-conditioned objectives and are jointly trained with PPO.
- 3.1 Dual-Agent Learning Framework: FALCON combines the two agents' actions through a joint-level PD controller and uses asymmetric actor-critic training with privileged force information available only to critics.Upper-body target joint angles are sampled during training and computed through IK during deployment.
- 3.2 Torque-Limit-Aware 3D Force Curriculum: FALCON computes upper-body force bounds from end-effector Jacobians and joint torque limits before sampling feasible 3D applied forces.The force bounds are estimated along Cartesian axes using worst-case joint torque induced by unit forces.
- 3.2 Torque-Limit-Aware 3D Force Curriculum: Force adaptability is trained progressively by scaling estimated end-effector forces with a global factor α_g that increases over training.During walking, planar forces are projected opposite to velocity and low-pass filtered to reduce jitter.
- 3.2 Torque-Limit-Aware 3D Force Curriculum: Force application is randomized along the end-effector link to improve robustness to varying contact points and their torque mappings.The randomization spans from the wrist yaw to the distal segment, while applied forces may differ between left and right end-effectors.
4 Simulation and Real-World Experiments
The experiments compare FALCON with decoupled and monolithic baselines in simulation and real-world humanoid tasks, evaluating tracking, locomotion, training efficiency, and force adaptation. FALCON maintains low tracking errors under disturbances and transfers across platforms and tasks.
- Simulation Evaluation: Experiments evaluate FALCON and baselines on upper-body joint tracking and root-velocity tracking under dynamically unknown 3D end-effector forces.The evaluation uses 252 ACCAD motion targets and no-force, middle-force, and large-force conditions.
- Simulation Results: 0.37 upper-body error under L-Force, compared with 0.60 for PID-Force-Curr. and 0.73 for M-WB-RL.Root error remains 0.45, indicating stable locomotion under the same condition.
- Curriculum Study: Torque-limit-aware force curriculum balances upper-body joint tracking and root-velocity tracking under significant external disturbances.A wide random-force baseline does not enforce torque feasibility, whereas the proposed curriculum evaluates forces within estimated admissible limits.
- Curriculum Study: Increasing the force range has minimal impact on loco-manipulation performance, demonstrating robustness of the torque-limit-aware curriculum.Table 1 uses narrower clipping ranges than Table 2.
- Real-World Evaluation: In real-world evaluation, FALCON carries 1.2kg in each hand while walking at (0.5, 0.0)m/s and achieves the lowest tracking errors among the compared methods.The reported motion is stable and natural during heavy-duty loco-manipulation.
- Cross-Platform Deployment: Policies trained without reward or force-curriculum modifications transfer to Unitree G1 and Booster T1 across payload transport, cart-pulling, and door-opening.The tasks involve 0-20N vertical payload forces, up to 100N longitudinal cart-pulling forces, and up to 40N 3D door-opening forces.
5 Conclusion
FALCON uses dual-agent reinforcement learning with shared proprioceptive coordination to address force-adaptive humanoid loco-manipulation. Across evaluations, it reports lower tracking errors, faster convergence, stable forceful control, and cross-platform sim-to-real transfer.
- Conclusion: FALCON decouples upper- and lower-body learning while maintaining coordination through shared proprioceptive feedback.The framework is designed for adaptation to 3D end-effector forces during complex tasks.
- Conclusion: FALCON outperforms Lower-RL-Upper-IK and Monolithic-Whole-body-RL baselines in tracking errors, training convergence, and stability across force regimes.The conclusion also reports successful transfer to physical humanoids for payload transport, cart-pulling, and door-opening.
6 Limitations
FALCON is limited to end-effector force disturbances and does not currently model multi-contact interactions or external torques. These boundaries constrain applications involving whole-body support or rotational disturbances.
- Scope Boundaries: FALCON does not account for contact forces on other body parts or support multi-contact interactions.This restricts applicability to leaning, bracing, and collaborative lifting scenarios.
- Scope Boundaries: The current curriculum ignores external torques, so rotationally disturbed tasks such as operating eccentric handles or tools remain a limitation.The paper identifies torque-adaptive policies as future work.
A.1 Reward Terms
FALCON augments related reward terms with penalties for locomotion stability and tracking rewards for squat and waist twist. These additional components support sim-to-real loco-manipulation behavior.
- Reward Terms: Additional penalties target undesired behaviors for sim-to-real transfer, while task rewards promote squat and waist-twist loco-manipulation capabilities.The additional reward terms and weights are summarized in Table 5.
A.2 Domain Randomization
FALCON applies domain randomization during training to support sim-to-real transfer, including dynamics randomization and external perturbations.
- Domain randomization is applied during training to support successful sim-to-real transfer.
- The randomization setup includes dynamics randomization and external perturbation terms.
A.3 Lower-RL-Upper-IK with Force Estimator
The Lower-RL-Upper-IK force-estimator variant predicts end-effector forces from proprioception and supplies them to the lower-body policy, while upper-body torques use force compensation. Estimated and applied forces align closely, but deployment still requires sensing the force application point and compensation assumes quasi-static motion.
- A jointly trained 3D force estimator predicts end-effector forces from robot proprioception.The estimator uses proprioceptive history as input.
- Predicted end-effector forces are concatenated with full-body proprioception and fed into the lower-body reinforcement-learning policy.
- Upper-body joint torques include force compensation alongside proportional-derivative tracking control.
- Estimated and applied forces show close alignment in the force-estimator evaluation.The comparison is shown in Figure 7(b).
- Deployment still requires a force sensor to localize force application because contact-point changes alter the effective Jacobian.The compensation also assumes quasi-static conditions, causing additional error during upper-body motion.
A.4 Force Measurement
FALCON is evaluated through force-gauge measurements and an autonomous tote-logistics pipeline combining localization, pose estimation, grasp planning, and state-machine control. Real-world payload transport is bounded by actuator thermal endurance, whereas brief high-torque cart-pulling remains feasible.
- Force Measurement: Peak forces are measured for cart-pulling, door-opening, and stance-pulling tasks using a handheld force gauge.The measurements cover Booster T1 and Unitree platforms for the listed tasks.
- Force Measurement: Stance-pulling measures the maximum longitudinal force resisted in a static stance without losing balance.Forces are applied along the X-Y plane.
- Tote Logistics: The tote-logistics pipeline uses MoCap localization, FoundationPose 6-DoF tote estimation, inverse kinematics, and four state-machine actions.The actions are walking without the tote, pickup, walking with the tote, and drop-off.
- Hardware Limits: Real-world payload transport exceeding 2kg per arm is limited by sustained high joint torques and rapid wrist-motor overheating.In MuJoCo without thermal modeling, the same policy transports over 3kg per end effector.
- Hardware Limits: Brief high-torque cart-pulling is feasible because it does not require sustained high torque output.