PaperlayerView exact revision on arXiv

Source-linked AI summary

Reinforcement Learning Based Multi-Terrain Adaptive Whole-Body Control for a Quadruped Manipulator

Yanzhou Jin, Fei Suo, Chang Liu, Yasuhisa Hasegawa, Liming Chen, Yusuke Iwasawa, Yutaka Matsuo, Yaonan Zhu

arXiv:2610.23548v1cs.RO

TL;DR

Small-scale quadruped loco-manipulation must maintain task-consistent end-effector behavior despite terrain disturbances and coupling between locomotion and manipulation. The paper combines hybrid task representation with task-prioritized whole-body learning and validates the approach in simulation and real-world terrain experiments. The conclusion reports terrain-adaptive locomotion and task-consistent end-effector behavior, while noting that validation covers one platform and a limited set of conditions.

  • Problem

    Terrain disturbances and strong coupling make task-consistent end-effector behavior during locomotion challenging for small-scale quadruped platforms.

    Source: §I · pp. 1–2

  • Method

    The framework combines task-prioritized whole-body learning with a hybrid representation that expresses end-effector position in the body frame and orientation relative to gravity.

    Source: §I · p. 2, B. Hybrid Manipulation Task… · p. 3

  • Results

    Simulation and real-world experiments on slopes, stairs, and deformable terrain demonstrate coordinated traversal and task-consistent end-effector behavior.

    Source: §V · p. 9

  • Takeaways & Limitations

    The experiments support the feasibility of terrain-adaptive loco-manipulation with task-consistent end-effector behavior on a small-scale quadruped manipulator.

    Source: §V · p. 9

    Validation is limited to a single robot platform and a representative but limited set of terrain and manipulation conditions.

    Source: §V · p. 9

Abstract

from arXiv · show

Legged robots equipped with manipulators offer promising capabilities for loco-manipulation tasks in unstructured environments. However, achieving task-consistent end-effector behavior during locomotion remains challenging due to terrain-induced disturbances and the strong coupling between locomotion and manipulation, particularly for small-scale quadruped platforms. A reinforcement-learning based whole-body control framework is proposed for quadruped loco-manipulation on complex terrain. The proposed framework combines a hybrid task representation with task-prioritized whole-body learning to coordinate locomotion and manipulation while preserving task-consistent end-effector behavior. A lightweight quadruped manipulator platform is further developed to validate the proposed framework on a small-scale quadruped system. Real-world experiments demonstrate successful deployment on complex terrains such as slopes up to 30 degrees, stairs, and deformable terrain. These results demonstrate the feasibility of deploying learned whole-body control policy for quadruped loco-manipulation in real-world environments and highlight the importance of task-consistent whole-body coordination for terrain-adaptive loco-manipulation.

I. INTRODUCTION

Small quadruped loco-manipulation must balance terrain traversal with task-consistent end-effector behavior despite tight payload constraints and strong locomotion–manipulation coupling. The paper proposes a lightweight platform and a reinforcement-learning whole-body controller that combines hybrid task representation with task-prioritized learning.

I. INTRODUCTION

  • Small quadruped manipulators face coupled challenges: arm loading and terrain-induced body motion can undermine traversal and end-effector tracking.Posture changes that improve reachability may conflict with terrain-regulating body motion.Source: §I · p. 1
  • Explicit state-dependent coordination among body regulation, locomotion, and manipulation remains limited, while unified learned policies for these dependencies remain an open challenge.The paper also identifies task representation as a challenge because end-effector position and orientation play different roles.Source: §I · pp. 1–2
  • The proposed framework combines task-prioritized whole-body learning with a hybrid task-space representation and a lightweight quadruped manipulator platform.The custom 1.5 kg Lite Arm is integrated with a Unitree Go1; the policy coordinates the arm and base under remaining coupling.Source: §I · p. 2
  • Simulation and real-world experiments demonstrate end-effector tracking on challenging terrain, including slopes, stairs, and deformable surfaces.The controller is also integrated into an autonomous loco-manipulation pipeline for real-world tasks.Source: §I · p. 2

II. METHOD

The method uses a lightweight quadruped manipulator and formulates rough-terrain loco-manipulation as whole-body control. Its hybrid end-effector command separates body-frame position from gravity-referenced orientation to balance local reachability and task consistency.

A. System Configuration and Problem Formulation

  • The custom 1.5 kg, 6-DoF manipulator reaches over 60 cm, and rough-terrain loco-manipulation is formulated around body regulation, locomotion tracking, and end-effector regulation.Its PLA-CF structure and embedded carbon-fiber tubes provide stiffness without excessive additional mass.Source: A. System Configuration and Problem… · p. 2

B. Hybrid Manipulation Task Representation

  • A world-frame task can preserve orientation relative to the environment but may compromise feasibility for a limited-workspace arm as body motion changes its workspace.The passage notes that full world-frame task specification also requires reliable global pose estimation.Source: B. Hybrid Manipulation Task… · p. 3
  • A body-frame task improves kinematic feasibility, but its orientation tilts with the robot body on terrain such as a 30° slope.This can conflict with practical tasks that require a consistent end-effector orientation.Source: B. Hybrid Manipulation Task… · p. 3
  • The hybrid command specifies end-effector position in the body frame and orientation relative to gravity, supporting local reachability without letting body tilt carry the end effector’s orientation.This representation permits large-scale body locomotion while the manipulator performs local task execution, without requiring global position estimation.Source: B. Hybrid Manipulation Task… · p. 3

C. Task-Prioritized Whole-Body Learning

The framework orders body regulation, locomotion, and manipulation according to their dependencies. Performance-conditioned activation factors attenuate lower-priority objectives when higher-priority performance is insufficient.

C. Task-Prioritized Whole-Body Learning

  • The objective hierarchy prioritizes body regulation, then locomotion, then manipulation because locomotion feasibility depends on body regulation and manipulation depends on sufficient locomotion.The hierarchy addresses conflicts that remain after specifying the desired end-effector behavior.Source: C. Task-Prioritized Whole-Body Learning · p. 3
  • At each control step, an objective’s current performance scales the reward contribution of lower-priority objectives, which become fully active once the required level is reached.Insufficient higher-priority performance attenuates the lower-priority objective.Source: C. Task-Prioritized Whole-Body Learning · p. 3

1) Priority I: Body Regulation:

  • Body regulation has highest priority because excessive roll or pitch can compromise terrain traversability and the two lower-priority objectives.The upright activation factor regulates both locomotion and manipulation, reducing their contributions as the body deviates from the desired upright configuration.Source: 1) Priority I: Body Regulation · pp. 3–4

2) Priority II: Locomotion Regulation:

The second priority regulates locomotion performance, and the third regulates end-effector behavior. Locomotion performance gates manipulation, while the manipulation objective combines coarse reach-to-target behavior with fine local position regulation.

2) Priority II: Locomotion Regulation:

  • The locomotion activation factor reduces manipulation’s influence when locomotion performance is insufficient and retains its full contribution once the required capability is maintained.The factor saturates at one when the nonnegative locomotion-performance objective reaches its threshold.Source: 2) Priority II: Locomotion Regulation · p. 4

3) Priority III: End-Effector Behavior Regulation:

  • The manipulation objective regulates the hybrid end-effector behavior through position and orientation components.Its coarse position objective uses the distance between commanded and measured end-effector positions; fine position regulation concentrates its effect near the target.Source: 3) Priority III: End-Effector Behavior… · p. 4

4) Prioritized Objective Integration:

The framework combines body regulation, locomotion, and manipulation through a state-dependent priority hierarchy. That hierarchy shapes the reward signal used to train one shared policy with PPO.

4) Prioritized Objective Integration:

  • The stepwise prioritized objective uses a state-dependent priority vector whose elements correspond to body regulation, locomotion, and manipulation.The priority vector is updated from body condition and locomotion performance, scheduling the objective groups’ relative contributions throughout each rollout.Source: 4) Prioritized Objective Integration · p. 4
  • The sequence of prioritized rewards supplies returns and advantages for PPO updates to a single shared policy.Maximizing expected return encourages the policy to coordinate the three objective groups under the prescribed hierarchy.Source: 4) Prioritized Objective Integration · p. 4
  • Priority is soft rather than a strict non-interference constraint: when higher-priority performance falls below its required level, its activation factor decreases the influence of dependent lower-priority objectives.The integration of hybrid task representation and objective prioritization is summarized in Fig. 2(e).Source: 4) Prioritized Objective Integration · p. 4

III. TRAINING AND IMPLEMENTATION

The controller uses asymmetric actor–critic observations and 18 joint-position targets. Its objective components cover body regulation, locomotion tracking, and end-effector position and orientation.

A. Observation and Action Space

  • The actor receives a 70-dimensional observation of joint positions and velocities, previous action, WBC command, base angular velocity, and projected gravity; the training critic additionally receives privileged state and terrain information.The critic-only observations include base linear velocity, end-effector accelerations, orientation deviation, pose distance, and a local terrain height map, used to stabilize value estimation during training.Source: A. Observation and Action Space · p. 4
  • The 18-dimensional action specifies normalized target positions for the 12 leg joints and 6 arm joints.Source: A. Observation and Action Space · p. 4

B. Objective Components

  • Body objectives regularize contact, foot clearance and force, body pose, action smoothness, joint limits, torque use, and joint acceleration; locomotion objectives track base linear and angular velocity.The locomotion tracking reward follows the Isaac Lab formulation.Source: B. Objective Components · p. 4
  • Manipulation objectives combine coarse and fine end-effector position tracking with orientation regulation through yaw alignment and upright-orientation terms.The upright term encourages gravity-aligned end-effector orientation; yaw error uses the signed angle between normalized horizontal projections of the end-effector and commanded forward directions.Source: B. Objective Components · pp. 4–5

C. Training Protocol

Training uses PPO in parallel simulation, with commands and end-effector targets sampled to cover both stationary manipulation and locomotion-compatible tasks. A terrain curriculum and randomized dynamics, payloads, delays, and disturbances broaden training conditions.

C. Training Protocol

  • PPO training uses an asymmetric actor–critic in Isaac Lab across 4,096 environments, with 1,000-step episodes at 50 Hz and command resampling at each episode midpoint.Base-velocity and end-effector-pose commands remain fixed during the first half; end-effector target sampling is conditioned on the base-velocity command.Source: C. Training Protocol · p. 5
  • Targets span the manipulator’s reachable workspace under stationary commands and a locomotion-oriented workspace region during movement, retaining task diversity while favoring feasible targets.Training terrains include flat ground, slopes, stairs, boxes, and random obstacles, with difficulty raised from 0.0 to 1.0 through a curriculum.Source: C. Training Protocol · p. 5
  • Domain randomization varies dynamics, arm and end-effector payload, action delay, and external disturbances to facilitate sim-to-real transfer.Randomizations include 0–3 control steps of delay, 0–0.5 kg of added end-effector mass, and trunk-wrench and horizontal-velocity perturbations.Source: C. Training Protocol · p. 5

1) Ablation Study on Prioritized Whole-Body Objectives:

The full policy balances locomotion and end-effector tracking across ablations, while disabling priority components exposes different trade-offs. Across terrain and arm-motion tests, WBC performs better than hierarchical tracking on the reported metrics and is less sensitive to speed and payload.

1) Ablation Study on Prioritized Whole-Body Objectives:

  • The full policy has the best overall balance among ablated variants, lying closest to the lower-left corner of the composite Base Motion Error versus EE Tracking Error plot.The composite metrics summarize locomotion and manipulation performance for compact visualization.Source: 1) Ablation Study on Prioritized… · p. 5
  • Disabling Priority II reduces composite EE Tracking Error but substantially increases Base Motion Error, indicating that the policy sacrifices mobility to maintain a stable end-effector pose.The resulting policy lacks the locomotion capability required for complex-terrain traversal.Source: 1) Ablation Study on Prioritized… · p. 5
  • Disabling Priority III significantly increases end-effector tracking error with little effect on locomotion; position error rises sharply while orientation error remains relatively low.Without this component, the manipulator does not fully exploit its workspace for goal-directed behavior.Source: 1) Ablation Study on Prioritized… · p. 5
  • Disabling Priority I degrades both locomotion and manipulation performance, supporting body regulation and contact-related objectives as the foundation for higher-level behavior.Source: 1) Ablation Study on Prioritized… · p. 5

2) Performance under Terrain and Manipulator Variations:

  • Across evaluated terrain difficulty levels, WBC consistently achieves lower Base Motion Error, EE Tracking Error, and joint overload exposure than Hier; exposure measures the percentage of near-limit actuator torque (|τ| ≥95%τmax).All methods exhibit increasing errors and actuator load as terrain difficulty increases. Manipulation-enabled configurations experience more frequent near-limit actuator operation than the locomotion-only baseline, with the gap widening on more challenging terrains.Source: 2) Performance under Terrain and… · p. 6, Figure 3 · p. 6
  • WBC had two failures over 4,096 lateral-motion rollouts and EE RMSE of 0.057–0.069 m, while Hier’s failure rate rose from 8.50% to 13.77% and RMSE from 0.070 to 0.108 m under slow/light versus fast/heavy conditions.The tests used arm-motion speeds of 0.4 and 0.8 m/s and payloads of 0.2 and 0.5 kg.Source: 2) Performance under Terrain and… · p. 6

B. Real-World Experiments

Real-world demonstrations show the controller coordinating locomotion and manipulation on a 30° slope and stairs while keeping the end-effector approximately upright through substantial body motion. The behavior persists across arm configurations and contrasts with larger disturbances under the built-in MPC baseline.

1) Representative Terrain Traversal Demonstrations:

  • The controller maintained an approximately upright end-effector during slope ascent, descent, and stair traversal despite substantial body pitch and roll variations.The demonstrations included a 30° slope and 10 cm stairs while carrying a container with two 36 mm elastic balls.Source: 1) Representative Terrain Traversal… · p. 6
  • Across different arm configurations, including forward-reaching and centered postures, the robot showed similar end-effector behavior despite changes in mass distribution and body dynamics.The carried container served as a visual indicator of end-effector disturbance during locomotion.Source: 1) Representative Terrain Traversal… · pp. 6–7
  • The built-in MPC baseline with a fixed arm posture showed larger body oscillations and lateral drift, which produced more pronounced disturbances to the carried object.Fig. 4 compares the controller and MPC during stair and slope traversal; blue lines indicate the end-effector’s horizontal direction.Source: 1) Representative Terrain Traversal… · p. 7, Figure 4 · p. 7

2) Quantitative Tracking and Whole-Body Compensation:

On stairs and during manual pitching disturbances, the robot coordinated leg, body, and arm motion to preserve end-effector tracking. Autonomous grasping trials on deformable terrain and lateral stair descents achieved an 80% overall success rate, with failures spanning traversal, localization, and grasp execution.

2) Quantitative Tracking and Whole-Body Compensation:

  • On stairs at a mean foot-plane tilt of 0.421 rad, end-effector position and orientation errors were 0.016 ± 0.004 m and 0.045 ± 0.017 rad, respectively.Errors were estimated using measured joint states and forward kinematics, with body and foot-plane tilts estimated from projected gravity and fitted foot positions.Source: 2) Quantitative Tracking and… · p. 7
  • During five manual disturbances, average peak foot-plane tilt reached 0.752 ± 0.031 rad, while leg compensation limited peak body tilt to 0.499 ± 0.037 rad.Mean absolute x- and y-axis errors stayed below 0.009 m, z-axis error was 0.034 ± 0.029 m, and end-effector orientation error remained 0.039 ± 0.024 rad.Source: 2) Quantitative Tracking and… · pp. 7–8

3) Grasping on Deformable and Uneven Terrains:

  • The semantic-guided pipeline turns a natural-language target description into an object location and then commands the whole-body controller to approach and grasp.The planner outputs base velocity and an end-effector target pose, switching to grasp execution after reachability is confirmed over consecutive observations; the policy receives no visual input.Source: 3) Grasping on Deformable and Uneven… · p. 8
  • Replacing the upstream perception module need not require retraining the controller if its target representation stays within the policy’s training range and arrives at a comparable update rate.This modularity concerns the controller interface and does not establish equivalent end-to-end performance across perception systems.Source: 3) Grasping on Deformable and Uneven… · p. 8
  • 80% (32/40) of autonomous grasping trials succeeded across deformable terrain and lateral stair descent.The deformable terrain used sandbags and sponges with compliant, uncertain contacts absent from simulation training; 17/20 trials succeeded there, while 15/20 succeeded on lateral stairs with footholds at different heights.Source: 3) Grasping on Deformable and Uneven… · p. 8
  • The eight grasping failures spanned the task: two falls during traversal, four target-localization errors, one table-edge collision, and one failed object retention during lifting.Three failures involved motion control, while five stemmed from localization or grasp execution.Source: 3) Grasping on Deformable and Uneven… · p. 8

C. Positioning Among Loco-Manipulation Systems

The paper positions its system as a compact quadruped platform for task-consistent loco-manipulation on steep and complex terrain, combining body-frame position tracking with gravity-aligned orientation regulation. The conclusion reports simulation and real-world validation, while noting limits in measurement coverage and validation scope.

C. Positioning Among Loco-Manipulation Systems

  • The system demonstrates task-consistent loco-manipulation on slopes up to 30° and complex terrain, where few existing quadruped manipulation systems report task execution.Table I compares systems by terrain demonstrations, task-space formulation, system scale, and manipulation during terrain traversal.Source: C. Positioning Among Loco-Manipulation… · p. 9
  • The hybrid task-space formulation tracks position in body coordinates and regulates orientation aligned with the world frame, balancing kinematic feasibility and task consistency under terrain disturbances.The end-effector z-axis is kept parallel to the world z-axis.Source: C. Positioning Among Loco-Manipulation… · p. 9

V. CONCLUSION

  • The proposed system maintains task-consistent end-effector behavior on challenging terrain with a compact 13.5 kg platform.The conclusion describes simulation and real-world experiments on slopes, stairs, and deformable terrain.Source: C. Positioning Among Loco-Manipulation… · p. 9, §V · p. 9
  • Quantitative real-robot evaluation relied on onboard inertial and joint measurements with forward kinematics because external motion-capture coverage was unavailable in the uneven-terrain test area.Validation was also limited to one robot platform and a representative but limited set of terrain and manipulation conditions.Source: §V · p. 9
Loading 2610.23548v1…