Source-linked AI summary
Hold My Beer: Learning Gentle Humanoid Locomotion and End-Effector Stabilization Control
Yitang Li, Yuanhang Zhang, Wenli Xiao, Chaoyi Pan, Haoyang Weng, Guanqi He, Tairan He, Guanya Shi
TL;DR
Fine-grained end-effector stabilization during humanoid locomotion remains a challenge because it requires different control characteristics from robust gait. SoFTA separates upper- and lower-body control into agents with different frequencies and reward structures. It reduces EE acceleration over baselines and is demonstrated carrying liquids and recording stable video.
Problem
Fine-grained EE stabilization during locomotion remains underexplored, despite its importance for safe, precise object interaction.
Source: §1 · p. 2Method
SoFTA decouples upper- and lower-body control into separate agents that share observations but use different action spaces, frequencies, and reward structures.
Source: §1 · p. 2, §3.2 · p. 4Results
Takeaways & Limitations
SoFTA is demonstrated on Unitree G1 and Booster T1 humanoids for liquid carrying and stable first-person video recording.
Source: §1 · p. 2The fixed boundary between locomotion and EE control can be suboptimal for tasks requiring close coordination, such as dynamic reaching or complex interactions.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Can your humanoid walk up and hand you a full cup of beer, without spilling a drop? While humanoids are increasingly featured in flashy demos like dancing, delivering packages, traversing rough terrain, fine-grained control during locomotion remains a significant challenge. In particular, stabilizing a filled end-effector (EE) while walking is far from solved, due to a fundamental mismatch in task dynamics: locomotion demands slow-timescale, robust control, whereas EE stabilization requires rapid, high-precision corrections. To address this, we propose SoFTA, a Slow-Fast Two-Agent framework that decouples upper-body and lower-body control into separate agents operating at different frequencies and with distinct rewards. This temporal and objective separation mitigates policy interference and enables coordinated whole-body behavior. SoFTA executes upper-body actions at 100 Hz for precise EE control and lower-body actions at 50 Hz for robust gait. It reduces EE acceleration by 2-5x relative to baselines and performs much closer to human-level stability, enabling delicate tasks such as carrying nearly full cups, capturing steady video during locomotion, and disturbance rejection with EE stability.
1 Introduction
Fine-grained end-effector stabilization during locomotion remains underexplored, despite its importance for precise interaction. SoFTA separates upper- and lower-body control to address their different objectives and dynamics, reducing end-effector acceleration and supporting real-world tasks.
1 Introduction
- Fine-grained EE stabilization during locomotion remains underexplored, although it is essential for safe, precise object interaction and stable video capture.The default Unitree G1 controller produces around 5 m/s^2 average EE acceleration while tapping in place, over 10× human levels.Source: §1 · p. 2
- Locomotion needs robust control over slower contact-driven dynamics, whereas EE stabilization needs a minimally moving base and rapid, precise corrections.The paper attributes this gap to differences in both task objectives and dynamics.Source: §1 · p. 2
- 50–80% reduction in EE acceleration over baselines; SoFTA achieves less than 2 m/s^2 during diverse locomotion and supports real-world liquid-carrying and stable-video tasks.The reported deployments include Unitree G1 and Booster T1 humanoids.Source: §1 · p. 2
- SoFTA decouples upper- and lower-body action and value spaces, using high-frequency precise EE control and slower robust locomotion control with task-specific rewards.The authors report experiments in simulation and the real world, including analysis across control frequencies.Source: §1 · p. 2
2 Related Work
Prior learning-based work has advanced humanoid locomotion, manipulation, and integrated behaviors, but precise EE stability during humanoid locomotion remains relatively neglected. Existing mobile-manipulator and policy-architecture approaches face limitations that motivate SoFTA’s frequency-aware upper/lower-body decoupling.
2 Related Work
- Learning-based humanoid control spans robust locomotion [23] [25] [1] [2] [3] [4] [5] [6] [7] [8] [9] [26], manipulation [12] [15], and integrated loco-manipulation [27] [28] [29] [30] [31] [32] [17], while precise EE control during locomotion receives relatively little attention.Domain randomization and system identification have supported sim-to-real transfer [18] [19] [20] [21] [22].Source: §2 · p. 2
- Mobile-manipulator approaches rely on accurate models and predefined contact schedules [36] [37] [38] [41] [42] [43], freeze the base in some hybrid methods [44] [45], or mainly evaluate joint learning on non-humanoids [47] [48].The paper presents its method as the first to achieve fine-grained EE stabilization during dynamic humanoid locomotion.Source: §2 · p. 3
- Whole-body policies [21] [53] face high-dimensional state-action spaces, while prior policy splits [50] do not leverage humanoid dynamics through both frequency and upper/lower-body decoupling.Other approaches include multi-critic methods for reward conflicts [55] and decentralized control [51].Source: §2 · p. 3
3 SoFTA for Learning Stable End-effector Control and Robust Locomotion
The problem formulation combines locomotion commands with end-effector stabilization goals and rewards, reflecting their different control objectives and timescales. SoFTA assigns separate, shared-observation agents to upper- and lower-body control, with distinct frequencies and reward groups.
3.1 Problem Statement
- The task is goal-conditioned RL for stabilizing an end-effector at target positions while following locomotion commands; observations include proprioception and goal-state inputs, and actions are target joint positions.The EE command encodes activation, target position, height, and tracking tolerance; locomotion commands include target velocity and gait information.Source: §3.1 · p. 3
- The EE reward formulation penalizes linear and angular acceleration and gravity tilt while encouraging near-zero acceleration.The passage defines the acceleration, angular acceleration, rotation matrix, gravity vector, and XY-plane projection used by these terms.Source: §3.1 · p. 3
- EE control favors a static base and sharp, continuous rewards, whereas locomotion accommodates gait and momentum changes with long-horizon robustness rewards and slower contact-driven dynamics.Because higher frequencies can increase sensitivity and exacerbate sim-to-real gaps [58] [59] [60] [19], the paper motivates modulating control rates by task.Source: §3.1 · pp. 3–4
3.2 SoFTA: Slow-Fast Two-Agent Framework.
- SoFTA uses separate agents for disjoint robot degrees of freedom: the high-frequency upper-body agent controls 14 arm joints for EE stability, while the slower lower-body agent manages legs and waist for locomotion and balance.Both agents use the full-body observation to support coordinated behavior.Source: §3.2 · p. 4
- SoFTA separates heterogeneous upper- and lower-body reward groups and uses separate actor and critic networks without shared parameters, while including termination reward in both streams.The authors describe this decomposition as providing targeted feedback and improving credit assignment [61] [62].Source: §3.2 · p. 4
4 Experimental Results
Across simulation and real-world evaluations, SoFTA outperforms baselines in end-effector stability, with a particular real-world advantage in Acc-Z. Further analyses examine its frequency design, task capabilities, training commands, and transfer to another humanoid embodiment.
4.1 Simulation Results
- SoFTA achieves the best overall simulation performance, reducing end-effector accelerations especially vertically, and outperforms baselines in most simulation metrics.Whole-body RL improves over Lower-body RL + IK but struggles in Push; Table 1 evaluates stability across simulation tasks.Source: §4.1 · p. 5, Table 1 · p. 5
4.2 Real-World Results
- In real-world locomotion, SoFTA maintains consistent, robust performance and outperforms baselines, especially in Acc-Z.The default controller has the highest acceleration across nearly all metrics, while Whole-body RL struggles during large movements such as TrajTrack.Source: §4.2 · pp. 6–7, Table 2 · p. 6
- SoFTA suppresses liquid motion while carrying an almost full cup, compensates during forceful pushes, and records smooth video during continuous turning.The real-world demonstrations cover bottle carrying without spillage, disturbance rejection, and camera stabilization that avoids visible jitter.Source: §4.2 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7
- Random end-effector position commands promote more transferable compensation patterns than fixing the end-effector at one position during training.Fixed-position training often overfits to specific poses and simulator dynamics; Table 3 concerns end-effector tracking and transferability.Source: §4.2 · p. 8, Table 3 · p. 8
4.3 In-Depth Analysis on Frequency Design
- The 50 Hz lower-body and 100 Hz upper-body configuration consistently achieves lower peak end-effector acceleration across simulation and real-world tasks.A 100 Hz lower-body policy caused more oscillations and degraded end-effector performance in real-world Stop trials, while 100 Hz upper-body control consistently enhanced performance.Source: §4.3 · p. 8, Figure 7 · p. 8
4.4 Cross Embodiment Validation
- On Booster T1, the same framework yields better stability and coordination than its Default Controller, particularly for sudden stops and precise end-effector control.The evaluation applies the training procedure to a robot with different joint configuration and body proportions, without embodiment-specific tuning.Source: §4.4 · p. 8, Figure 8 · p. 9
5 Conclusion
SoFTA combines frequency separation and task-specific rewards to support robust locomotion and stable end-effector control. The reported acceleration reductions and real-world demonstrations show its performance and application scope.
5 Conclusion
- SoFTA uses frequency separation and task-specific reward design to support robust locomotion and precise, stable end-effector control.Source: §5 · p. 9
- 50–80% reduction in end-effector acceleration brings SoFTA closer to human-level stability, according to the conclusion.Source: §5 · p. 9
- SoFTA was deployed on the Unitree G1 for tasks including carrying liquids while walking and recording stable video.Source: §5 · p. 9
6 Limitation
SoFTA’s stability remains below human-level performance, and its fixed separation of locomotion and end-effector control can be suboptimal for tasks requiring close coordination. Its performance may also vary with task complexity and robot configuration.
6 Limitation
- SoFTA’s achieved stability still falls short of human-level performance, including in the task of carrying a cup while walking.Source: §6 · p. 9
- The fixed boundary between locomotion and end-effector control can be suboptimal for dynamic reaching or complex interactions requiring close coordination.Source: §6 · p. 9
- Performance may vary with task complexity, robot morphology, and coordination needs; future work could improve adaptability across tasks and configurations.Source: §6 · p. 9
A.1 Training Details
The training setup combines deployable actor observations with privileged critic information, task commands for locomotion and end-effector stabilization, and domain randomization. It also specifies reward shaping and PPO training configuration.
A.1 Training Details
- The actor uses onboard proprioception, command signals, and recent actions, stacked over five timesteps, without global position data.Source: §A.1 · p. 15
- The critic receives privileged base and end-effector information, while selected observations are noised; the setup is intended to support value estimation and sim-to-real transfer.Source: §A.1 · p. 15, Table 4 · p. 15
- The five-value end-effector command toggles stabilization and specifies local target position, target height offset, and tracking tolerance; higher tolerance favors smoother, lower-acceleration motion.Source: §A.1 · p. 15
- Locomotion commands specify base-frame linear and angular velocities plus gait information, including a double-stance flag or dynamic-gait period.Source: §A.1 · p. 15, Table 5 · p. 16
- Sinusoidal gait phase signals derive target foot-contact timings, and a phase-based reward guides the agent toward that contact sequence.Source: §A.1 · p. 16
- Domain randomization is applied before push disturbances, which are introduced after a stable policy is obtained.Source: §A.1 · p. 16, Table 6 · p. 16
- Reward design groups task rewards and penalties by body group, while PPO training configuration includes general settings, module-specific action standard deviations, and shared network architecture.Source: §A.1 · pp. 16–17, Table 7 · p. 17, Table 8 · p. 17
- The reward scaling factor starts at 0.5 and is adjusted according to episode length, with an upper bound of 1.Source: §A.1 · p. 16
A.2 More Analysis on Frequency Ablation
The ablation separates gait-rate robustness from end-effector response speed: 50 Hz lower-body control remains stable across upper-body frequencies, while higher upper-body frequencies improve disturbance recovery.
A.2 More Analysis on Frequency Ablation
- 50 Hz lower-body control consistently achieves stable locomotion across upper-body frequencies, while other lower-body frequencies may degrade under certain upper-body frequencies.Source: §A.2 · p. 17
- At 100 Hz, upper-body policies react faster to base-motion changes and recover end-effector velocity faster than lower-frequency policies.Source: §A.2 · p. 18, Figure 9 · p. 18
- Increasing upper-body control frequency reduces recovery time and peak acceleration and velocity errors under disturbance.Source: Table 9 · p. 17, §A.2 · p. 18