PaperlayerView exact revision on arXiv

Source-linked AI summary

A Hierarchical Framework for Humanoid Locomotion with Supernumerary Limbs

Bowen Zhi

arXiv:2512.00077v1cs.RO

TL;DR

Heavy, articulated supernumerary limbs introduce dynamic perturbations that challenge humanoid walking stability. The paper separates learned locomotion from model-based balancing that actively uses the limbs. In simulation, active balancing better preserves the baseline gait pattern than a static payload and improves within-cycle re-stabilization.

  • Problem

    Heavy, articulated SLs introduce dynamic perturbations to humanoid locomotion, while jointly learning walking and SL balancing presents a complex control problem.

    Source: Introduction · p. 4, §1.1.1 · p. 5, §1.1.2 · p. 5

  • Method

    A hierarchical design pairs a learned low-level walking policy with a high-level model-based controller that uses SLs for active balance.

    Source: Conclusion · p. 23

  • Results

    The active balancing controller yields a lower DTW distance between the robot’s CoM trajectory and the unperturbed baseline than the static payload condition, and improves re-stabilization within gait cycles.

    Source: Conclusion · p. 23

  • Takeaways & Limitations

    The results support a decoupled hierarchical strategy as a viable way to manage humanoid-SL complexity, with measurable improvements in gait-pattern preservation and intra-cycle stability.

    Source: Conclusion · p. 23

    The evaluation is simulation-only without physical motor torque limits, leaving hardware feasibility uncertain; the SLs serve only balancing, not manipulation tasks.

    Source: §4.2 · p. 21

Abstract

from arXiv · show

The integration of Supernumerary Limbs (SLs) on humanoid robots poses a significant stability challenge due to the dynamic perturbations they introduce. This thesis addresses this issue by designing a novel hierarchical control architecture to improve humanoid locomotion stability with SLs. The core of this framework is a decoupled strategy that combines learning-based locomotion with model-based balancing. The low-level component consists of a walking gait for a Unitree H1 humanoid through imitation learning and curriculum learning. The high-level component actively utilizes the SLs for dynamic balancing. The effectiveness of the system is evaluated in a physics-based simulation under three conditions: baseline gait for an unladen humanoid (baseline walking), walking with a static SL payload (static payload), and walking with the active dynamic balancing controller (dynamic balancing). Our evaluation shows that the dynamic balancing controller improves stability. Compared to the static payload condition, the balancing strategy yields a gait pattern closer to the baseline and decreases the Dynamic Time Warping (DTW) distance of the CoM trajectory by 47\%. The balancing controller also improves the re-stabilization within gait cycles and achieves a more coordinated anti-phase pattern of Ground Reaction Forces (GRF). The results demonstrate that a decoupled, hierarchical design can effectively mitigate the internal dynamic disturbances arising from the mass and movement of the SLs, enabling stable locomotion for humanoids equipped with functional limbs. Code and videos are available here: https://github.com/heyzbw/HuSLs.

Introduction

The introduction frames stable bipedal walking with heavy, articulated supernumerary limbs (SLs) as a challenge and positions decoupled learning-based locomotion with model-based balancing as its response.

Introduction

  • Heavy, articulated SLs can impose continuous dynamic perturbations that compromise humanoid locomotion stability.The introduction notes that such limbs may carry payloads up to 30 kg.Source: Introduction · p. 4

1.1.1 Control of Humanoid Locomotion

  • Locomotion research spans model-based ZMP control and DRL, but a monolithic policy for walking and SL balancing faces state-action growth and conflicting objectives.ZMP methods can struggle with uneven terrain and unforeseen disturbances; the cited hierarchical approaches suggest decomposition, but seamless integration remains challenging.Source: §1.1.1 · pp. 4–5

1.1.2 Balance Augmentation with Supernumerary Limbs

  • Static bracing and specialized tails or extra legs address particular settings, while dynamic walking under continuous, high-magnitude disturbances from multi-purpose SL arms remains open.The cited challenge includes large, time-varying torques when repositioning heavy payloads.Source: §1.1.2 · p. 5

1.1.3 Positioning this Research

  • The research positions anthropomorphic, general-purpose arms as dynamic balance aids, pairing DRL locomotion with model-based Center of Mass regulation.The same limbs could also serve manipulation tasks, while the modular design separates the control responsibilities.Source: §1.1.3 · pp. 5–6

1.2 Contribution and Objectives

The project aims to maintain stable bipedal locomotion under disturbances from heavy, articulated SLs by separating gait learning from active balance control.

1.2 Contribution and Objectives

  • The proposed modular strategy separates a low-level locomotion policy from a high-level balancing controller to address humanoid-SL control complexity.The work targets stable walking while managing self-induced dynamic effects from the limbs.Source: §1.2 · p. 6
  • The objectives pair imitation-guided DRL gait learning and progressive SL mass and pose curricula with a model-based controller using real-time CoM and CoS feedback.The controller is intended to modulate arm configuration to counteract instability.Source: §1.2 · p. 6
  • The evaluation compares unladen baseline walking, a fixed-pose SL payload, and the full framework with active balance modulation.The scenarios are framed as progressively more complex tests in physics simulation.Source: §1.2 · p. 6

Methods

  • The methods chapter is organized around DRL locomotion, model-based dynamic balancing, and quantitative evaluation.These are the three primary components of the methodology.Source: Methods · p. 7

2.1 System Architecture

The system combines a simulated H1 humanoid with backpack-mounted robotic arms and separates learned walking from model-based SL balancing.

2.1.1 Robot Model and Simulation Environment

  • The study uses MuJoCo physics simulation and does not saturate actuator torques at the H1’s physical limits.The torque simplification is identified as relevant to the sim-to-real challenge.Source: §2.1.1 · pp. 7–8
  • The simulated platform combines a 12-DoF Unitree H1 base with two 7-DoF Kinova Gen3 arms mounted on a backpack.Each arm has a 2F-85 gripper, and the composite robot has 26 actuated DoF.Source: Figure 2.1 · p. 7, §2.1.1 · p. 8

2.1.2 Hierarchical Control Framework

  • The hierarchical architecture assigns gait generation to a DRL policy and active SL pose adjustment to a separate model-based balancing controller.The separation lets the learning policy focus on locomotion while the controller handles SL-induced dynamics; the framework also includes an outer curriculum scheduler and an inner imitation-learning loop.Source: §2.1.2 · pp. 8–9, Figure 2.2 · p. 8

2.2 Low-Level Control: DRL for Locomotion

The low-level walking policy uses imitation learning and PPO, with rewards balancing survival, stability, and motion matching and a curriculum that progressively increases SL challenges.

2.2 Low-Level Control: DRL for Locomotion

2.2.1 Proximal Policy Optimization (PPO)

  • PPO trains the policy with a clipped surrogate objective designed to prevent excessively large policy updates.The network has three hidden layers of [1024, 512, 256] neurons with Tanh activation.Source: §2.2.1 · pp. 9–10

2.2.2 Imitation Learning and Reward Function

  • The reward combines survival, stability, and imitation, with weights favoring survival and stability so the policy can deviate from the reference to avoid falling.Imitation components reward similarity in joint positions, velocities, and key body-site orientations.Source: §2.2.2 · p. 10, Table 2.1 · p. 10

2.2.3 Curriculum Learning

  • Across 500 million training timesteps, the curriculum raises payload mass from negligible levels to 19–30 kg and moves the arms from a neutral pose toward a forward-reaching posture.The arm-pose progression has four phases, exposing the policy to shifting center of mass and increasing inertial and gravitational effects.Source: §2.2.3 · pp. 10–11

2.3 High-Level Control: Model-Based Dynamic Balancing

The hierarchical controller pairs DRL leg locomotion with independent, model-based balancing through the supernumerary limbs (SLs). It estimates balance from the CoM and CoS, then adjusts SL arm targets and fuses their torques with the gait policy’s actions.

2.3 High-Level Control: Model-Based Dynamic Balancing

  • The Active Balance Controller uses robot state to estimate CoM and CoS, compute balance error, and generate compensatory SL-arm torques alongside the DRL gait policy’s leg actions.The two control signals are combined and sent to the simulation environment.Source: Figure 2.4 · p. 11, §2.3 · p. 11

2.3.1 State Estimation for Balancing

  • CoM is the weighted average of body-link positions, while CoS follows the support phase: stance-foot contact center in single support and force-weighted foot CoPs in double support.Per-foot CoP is calculated from measured ground-reaction forces and moments, providing a stability reference across gait transitions.Source: §2.3.1 · pp. 11–12
  • The controller treats CoM–CoS displacement as a dynamic stability indicator, not a static error to eliminate, because controlled misalignment supports forward propulsion.Source: §2.3.1 · p. 12

2.3.2 Balancing Controller and Control Fusion

  • The reactive controller modulates a fixed, slightly forward-and-down home arm pose using the stability indicator, seeking compensatory momentum that shifts CoM toward a stable region.Its linear heuristic was chosen for computational simplicity and real-time feasibility rather than nonlinear whole-body optimization.Source: §2.3.2 · p. 12
  • A PD controller drives SL joints toward their dynamic targets, with empirically tuned gains; arm torques override the DRL output for SL actuators while leg and torso actions remain from the policy.The fixed forward-reaching pose in the Static Payload condition differs from the dynamic controller’s neutral base pose.Source: §2.3.2 · p. 13

2.4 Experimental Evaluation

The evaluation compares baseline walking, static SL payload, and dynamic balancing, using measures of CoM trajectory similarity, within-cycle recovery, and bipedal coordination. The results chapter is organized around training, CoM trajectories, balance modulation, and exploratory gait coordination.

2.4 Experimental Evaluation

  • Three scenarios are evaluated: unladen baseline walking, walking with SLs fixed in a forward-reaching pose without active balancing, and walking with the fully active hierarchical controller.Source: §2.4 · p. 13
  • Stability is assessed through CoM trajectory similarity, recovery based on CoM–CoS distance, and bipedal coordination from Ground Reaction Force analysis.The GRF phase-plane analysis fits an ellipse with PCA and measures orientation error relative to the ideal 135-degree anti-phase axis.Source: §2.4 · pp. 13–14

Results

  • The results chapter covers DRL training performance, CoM trajectory analysis, dynamic balance modulation, and exploratory gait coordination.Source: Results · p. 15

3.1 DRL Training Performance

The PPO agent was trained for 500 million environment steps with curriculum increases in payload mass and SL arm-pose difficulty every 100 million steps. Episode return and length trended upward, with temporary dips at curriculum changes followed by recovery and continued improvement.

3.1 DRL Training Performance

  • 500 million environment steps of curriculum training produced upward trends in mean episode return and length, with temporary dips at difficulty increases and subsequent recovery.Payload mass and SL arm poses were adjusted every 100 million steps; the authors interpret the recovery and continued improvement as adaptation to SL dynamics.Source: §3.1 · p. 15, Figure 3.1 · p. 15

3.2 Center of Mass Trajectory Analysis

Dynamic balancing brings the CoM trajectory closer to baseline walking than a static SL payload does, while also improving re-stabilization within gait cycles. The analysis uses DTW for temporal trajectory similarity and GCSM to quantify recovery minima.

3.2 Center of Mass Trajectory Analysis

  • 65.54 DTW distance from baseline for Dynamic Balancing, approximately 47% below the Static Payload distance of 123.71.Figure 3.2 compares the three CoM trajectories, with distances measured relative to Baseline Walking.Source: §3.2 · p. 16, Figure 3.2 · p. 16
  • Dynamic Balancing better preserves the unladen gait’s fundamental dynamic pattern and rhythm, while Static Payload produces greater CoM pattern and final-position deviation.Source: §3.2 · p. 17

3.3 Analysis of Dynamic Balance Modulation

  • CoM–CoS distance oscillates through gait, increasing during swing and decreasing after foot-strike; the DRL policy varies double-support timing as it adapts gait timing for stability.The baseline oscillation-amplitude drift is described as an artifact of this adaptive control, not an intended circular path.Source: §3.3 · p. 17, Figure 3.3 · p. 17
  • GCSM measures the minimum CoM–CoS distance during the recovery phase between successive distance peaks; cycles are time-normalized for phase-aligned comparison.Source: §3.3 · p. 18
  • Dynamic Balancing has the lowest median GCSM and a lower trough in the averaged recovery cycle, indicating more effective re-stabilization after each step.Lower GCSM means a smaller minimum CoM–CoS distance during recovery and a more complete recovery at the cycle’s critical point.Source: §3.3 · pp. 18–19, Figure 3.4 · p. 18

3.4 Exploratory Analysis of Gait Coordination

The phase-plane analysis uses vertical GRF orientation relative to the ideal anti-phase pattern as a gait-coordination indicator. Orientation Error is lowest with dynamic balancing, and the Discussion interprets the findings as support for decoupled locomotion and balancing.

3.4 Exploratory Analysis of Gait Coordination

  • The vertical GRF phase-plane analysis treats deviation from the ideal 135° anti-phase orientation as an indicator of gait coordination.Figure 3.5 identifies Orientation Error as the deviation of the ellipse’s major axis from that ideal orientation.Source: §3.4 · p. 19, Figure 3.5 · p. 19
  • The learned baseline is not optimal, and statistical analysis would be needed to confirm that the observed coordination differences are significant.Its benchmark Orientation Error is 8.37°; the analysis therefore focuses on relative changes among conditions.Source: §3.4 · p. 19
  • Orientation Error is 8.37° for baseline walking, 5.59° with static payload, and 1.95° with dynamic balancing.The analysis presents these as relative differences across conditions, not as proof of statistical significance.Source: §3.4 · p. 19, Figure 3.5 · p. 19

Discussion

  • The Discussion concludes that decoupling locomotion and dynamic balancing can support stable walking under dynamic SL perturbations.It interprets the preceding results as evidence for the hierarchical framework’s effectiveness.Source: Discussion · p. 20

4.1 Interpretation of Key Findings

The interpretation reports benefits of curriculum-trained locomotion and active SL balancing, while identifying simulation, task-scope, and analysis-depth limits. The reported results do not establish hardware feasibility or explain the controller’s joint-level corrective actions.

4.1 Interpretation of Key Findings

  • Curriculum-trained imitation learning produced a locomotion policy able to handle the SL load during evaluation.Training progressively increased payload mass and introduced more destabilizing arm poses.Source: §4.1 · p. 20
  • Active balancing yields a significantly lower CoM-trajectory DTW distance than static payload and better preserves the learned gait’s dynamics and rhythm.The trajectory need not perfectly match the unladen baseline; the reported result indicates partial mitigation of SL-related gait-pattern changes.Source: §4.1 · p. 20
  • Dynamic balancing achieves a lower Gait Cycle Stability Minimum, with the SLs used for active re-stabilization at a critical point within each step.The authors describe this as a functional benefit of the actively controlled limbs.Source: §4.1 · pp. 20–21
  • The trend toward lower Orientation Error is exploratory and requires further statistical validation.The authors suggest it may reflect better adherence to the learned coordination pattern when the high-level controller handles primary balancing.Source: §4.1 · p. 21

4.2 Limitations of the Study

  • Simulation omitted Unitree H1 motor torque limits, leaving it uncertain whether hardware can support the SL payload or produce the required compensatory torques.Hardware transfer would require domain randomization and actuator-torque validation against the robot’s physical capabilities.Source: §4.2 · p. 21
  • The framework uses SLs only for balancing; adding manipulation objectives was outside scope because balance and task execution can compete.The authors identify multi-objective arbitration as a challenge for integrating those tasks.Source: §4.2 · p. 21
  • The analysis does not trace SL motion at the joint level, limiting interpretation of the controller’s specific corrective actions.The study did not perform a moment-to-moment analysis of emergent SL motion strategies.Source: §4.2 · p. 21

4.3 Future Work

The conclusion reports a decoupled controller that preserves gait and improves intra-cycle stability with heavy actuated SLs, while identifying sim-to-real transfer and controller integration as next steps.

4.3 Future Work

  • Physical deployment is the immediate next step, contingent on validating actuator torque requirements against the hardware’s physical limits.Source: §4.3 · pp. 21–22
  • Future work proposes gait-locked SL joint-motion analysis, then an optimization-based whole-body controller combining manipulation goals and balance constraints.Correlating arm angles with CoM-CoS distance oscillations could reveal the stabilization strategies and limitations of the current heuristic controller.Source: §4.3 · p. 22

Conclusion

  • Imitation learning with a curriculum strategy produced a locomotion policy capable of adapting to progressively increasing dynamic perturbations from the SLs.Source: Conclusion · p. 23
  • The active balancing controller preserves the underlying gait pattern better than a static payload, with a significantly lower CoM-trajectory DTW distance to the unperturbed baseline.The thesis concludes that the decoupled hierarchy measurably improves gait-pattern preservation and intra-cycle stability.Source: Conclusion · p. 23
  • Active SL use improves re-stabilization within each gait cycle, helping the robot reach a more stable state at the critical recovery point.Source: Conclusion · p. 23
  • Exploratory analysis suggests that offloading primary balancing lets the locomotion policy execute a more coordinated bipedal pattern, but this requires further statistical validation.Source: Conclusion · p. 23

Related papers

Loading 2512.00077v1…