PaperlayerView exact revision on arXiv

Source-linked AI summary

HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation

Carmelo Sferrazza, Dun-Ming Huang, Xingyu Lin, Youngwoon Lee, Pieter Abbeel

arXiv:2403.10506v2cs.ROcs.AIcs.LG

TL;DR

Humanoid robot research is constrained by costly, unsafe hardware experiments and limited benchmarks for diverse whole-body control. HumanoidBench provides a high-dimensional simulated benchmark spanning locomotion and manipulation, and its evaluations show that standard RL struggles on most tasks while hierarchical learning benefits from robust low-level skills. The benchmark is intended to expose these challenges and support faster algorithm verification.

  • Problem

    Costly, unsafe hardware setups and limited existing benchmarks constrain research on diverse whole-body humanoid control.

    Source: §I · p. 1

  • Method

    HumanoidBench provides a simulated humanoid with dexterous hands, diverse locomotion and manipulation tasks, standardized evaluation, and a hierarchical RL approach.

    Source: §I · p. 1

  • Results

    State-of-the-art RL algorithms struggle with most tasks, while hierarchical RL substantially outperforms flat baselines on push when supported by pretrained low-level policies.

    Source: §I · p. 2, B. Results · p. 8, D. Flat vs. Hierarchical Reinforcement… · p. 9

  • Takeaways & Limitations

    HumanoidBench provides a platform for identifying challenges in diverse humanoid tasks and rapidly verifying robot-learning algorithms and ideas.

    Source: §I · p. 1, §VI · p. 10

Abstract

from arXiv · show

Humanoid robots hold great promise in assisting humans in diverse environments and tasks, due to their flexibility and adaptability leveraging human-like morphology. However, research in humanoid robots is often bottlenecked by the costly and fragile hardware setups. To accelerate algorithmic research in humanoid robots, we present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands and a variety of challenging whole-body manipulation and locomotion tasks. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies, such as walking or reaching. With HumanoidBench, we provide the robotics community with a platform to identify the challenges arising when solving diverse tasks with humanoid robots, facilitating prompt verification of algorithms and ideas. The open-source code is available at https://humanoid-bench.github.io.

I. INTRODUCTION

HumanoidBench addresses the cost, safety, and limited generality of existing humanoid-control research with a diverse simulated benchmark for whole-body learning. It spans locomotion and manipulation while exposing challenges from high-dimensional dynamics, coordination, and long-horizon tasks.

I. INTRODUCTION

  • Hand-designed humanoid controllers require substantial engineering for each task and environment and often provide limited whole-body control.Source: §I · p. 1
  • Costly and unsafe real-world setups have delayed scaling robot-learning algorithms to humanoid robots.Source: §I · p. 1
  • HumanoidBench provides a fast, safe, inexpensive simulation environment, diverse locomotion and manipulation tasks, and a standardized evaluation platform for high-dimensional humanoid learning.Source: §I · p. 1
  • Its 15 whole-body manipulation and 12 locomotion tasks range from practical interactions to simpler primitive skills, while benchmark results show state-of-the-art RL struggles with complex dynamics and challenging tasks.Source: §I · p. 2

II. RELATED WORK

  • Prior benchmarks commonly use single-arm or floating-hand setups, focus on locomotion or specific tasks, and therefore offer less diversity in coordinated whole-body control.Source: §II · p. 2
  • Compared with prior simulations, HumanoidBench combines high-dimensional actions and degrees of freedom with diverse long-horizon locomotion and manipulation tasks.Source: §II · p. 2

III. SIMULATED HUMANOID ROBOT ENVIRONMENT

HumanoidBench simulates a humanoid with dexterous hands, multimodal sensing, and up to 61 actuators across locomotion and whole-body manipulation tasks. Its task suite combines simpler locomotion scenarios with challenging coordinated interactions.

III. SIMULATED HUMANOID ROBOT ENVIRONMENT

  • HumanoidBench uses a Unitree H1 humanoid with two dexterous Shadow Hands as its primary simulated agent.Source: §III · p. 3
  • The benchmark supports multiple humanoid bodies and end effectors, while the primary H1 configuration is selected partly for its simpler mechanical design and faster observed learning.Source: §III · p. 3
  • The environment includes proprioceptive and task-relevant state observations, head-mounted egocentric cameras, and whole-body tactile sensing with 448 taxels.Source: §III · p. 3
  • The benchmark initially focuses on state-based inputs with consistent robot and object observations across tasks, leaving multimodal perception benchmarking for future work.Source: §III · pp. 3–4

IV. HUMANOIDBENCH

  • Its action space reaches 61 actuators, enabling evaluation of complex whole-body coordination from toy scenarios to applications such as truck unloading and shelf rearrangement.Source: §IV · p. 5
  • HumanoidBench covers 27 tasks: 12 locomotion tasks that bypass dexterous-hand control and 15 whole-body manipulation tasks requiring coordination across the robot body.Source: Figure 3 · p. 4, §IV · p. 5, Figure 4 · p. 5

A. Locomotion Tasks •

HumanoidBench spans 12 locomotion and 15 whole-body manipulation tasks, ranging from basic movement and balance to dexterous, long-horizon interactions. The suite includes tasks such as walking, reaching, pushing, kitchen operation, package transport, and catching and throwing a basketball.

A. Locomotion Tasks •

  • The cited benchmark includes walking forward at approximately 1 m/s without falling, running at 5 m/s, reaching a random 3D point, balancing on an unstable board, and maze navigation.Source: A. Locomotion Tasks • · p. 5
  • Locomotion tasks test varied whole-body abilities, including stair and slide traversal, pole avoidance, hurdle crossing, tunnel crawling, and unstable-board balancing.Source: A. Locomotion Tasks • · pp. 5–6

B. Whole-Body Manipulation Tasks •

  • The suite includes 15 whole-body manipulation tasks involving object transport, tool use, insertion, doors and cabinets, and high-bar interaction.Representative tasks include pushing boxes, using a spoon in a pot, inserting a peg, opening doors, unloading a truck, and swinging on a high bar.Source: B. Whole-Body Manipulation Tasks • · pp. 6–7
  • Kitchen requires a sequence of discrete actions—opening a microwave, moving a kettle, and turning burner and light switches—with a purely discrete sparse reward capped at 4.Source: B. Whole-Body Manipulation Tasks • · p. 6, Figure 6 · p. 7
  • Additional manipulation tasks require package placement, barbell lifting, room organization, in-hand cube reorientation, ordered bookshelf placement, basketball catching and shooting, and window wiping.Source: B. Whole-Body Manipulation Tasks • · pp. 6–7

V. BENCHMARKING RESULTS

The benchmark evaluates four reinforcement-learning methods across HumanoidBench to study learning challenges in high-dimensional humanoid control. It also notes that the environments support controller types beyond the model-free and model-based RL baselines evaluated here.

A. Baselines

  • HumanoidBench evaluates DreamerV3, TD-MPC2, SAC, and PPO across all benchmark tasks.DreamerV3 and TD-MPC2 are model-based methods, while SAC and PPO are model-free baselines.Source: A. Baselines · pp. 7–8
  • The environments support actions from any controller type, but classical model-based control approaches are not benchmarked in this work.Source: A. Baselines · p. 7

B. Results

Baseline RL methods remain below success thresholds on most tasks, especially manipulation and long-horizon whole-body control, while dexterous hands and large action spaces hinder learning. Hierarchical policies improve push performance when supported by pretrained reaching skills, but package lifting remains difficult.

B. Results

  • All baseline algorithms fall below success thresholds on most tasks, with particularly poor performance on long-horizon tasks requiring whole-body coordination in high-dimensional action spaces.Even the walk task requires many environment steps despite being a commonly studied locomotion problem.Source: B. Results · p. 8
  • Manipulation is harder than locomotion: policies barely learn stabilization from dense rewards and fail to acquire complex manipulation skills.Source: B. Results · p. 8

C. With Hands vs. Alternative Configurations

  • The dexterous hands reduce walk performance substantially, while fixing their 42 action dimensions lowers the action space from 61D to 19D and makes learning significantly faster.The hand observations and masses remain present, so the observation space stays 151D; the results attribute most of the drop to increased action dimensionality.Source: C. With Hands vs. Alternative… · p. 8, Figure 7 · p. 8
  • Similar difficulty trends appear on push, whose task dynamics differ substantially depending on whether the robot uses hands.Source: C. With Hands vs. Alternative… · p. 8

D. Flat vs. Hierarchical Reinforcement Learning

E. Common Failures

HumanoidBench exposes recurring failures in long-horizon planning and whole-body coordination, while concluding with a broad benchmark intended to accelerate humanoid control research.

E. Common Failures

  • Short-horizon planning limits performance on long-horizon tasks: highbar learns to maintain contact but struggles with whole-body rotation despite dense rewards.Source: E. Common Failures · pp. 9–10
  • Door opening requires coordinated arm pulling and whole-body backward motion, highlighting the challenge of integrating manipulation with locomotion.Source: E. Common Failures · p. 10
  • Hurdle training favors forward running or conservative collision-stabilizing poses instead of exploring the higher-reward jumping behavior.Source: E. Common Failures · p. 10

VI. CONCLUSION

  • HumanoidBench is presented as a comprehensive, high-dimensional benchmark spanning locomotion and manipulation, with complex tasks intended to stimulate whole-body algorithm development.Source: §VI · p. 10
  • Future work includes studying egocentric vision and tactile sensing, adding more realistic environments, and exploring dexterous tasks such as screwing and furniture assembly.Source: §VI · p. 10
  • The study focuses on reinforcement learning because collecting physical humanoid demonstrations is particularly challenging, while human-video learning is suggested as an alternative.Source: §VI · p. 10

Appendix

The appendix specifies HumanoidBench’s supported robot configurations, high-dimensional controls, simulation performance, tactile sensing, reward primitives, and introductory locomotion and sitting tasks.

APPENDIX A ADDITIONAL COMPONENTS

  • HumanoidBench supports Digit and Unitree G1 variants, plus H1 configurations with either a 2-DoF gripper or 13-DoF dexterous hand.Source: Appendix A · p. 14

A. Observation Space

B. Action Space

  • The action space is normalized to [−1, 1]^61, with 19 body actions and 21 actions per hand, using position control for benchmarks.Source: B. Action Space · p. 14

C. Simulation Performance

E. Task Specification

  • The task specifications define reusable tolerance, posture, distance, stability, velocity, and control-effort terms for reward construction.Source: E. Task Specification · pp. 14–15

1) walk:

5) balance:

These environments test balance, terrain traversal, collision avoidance, reaching, and fast locomotion under randomized initial conditions and task-specific termination rules.

5) balance:

  • Balance requires maintaining the robot on an unstable board, with hard variants allowing the spherical pivot beneath the board to move.Source: 5) balance · pp. 16–17

6) stair:

5) balance:

8) pole:

  • The pole task combines forward progress with collision avoidance, using a collision penalty alongside stability and velocity rewards.Source: 8) pole · p. 17

9) reach:

  • Reach requires the left hand to reach a randomly initialized 3D point, observing robot joints, hand position, and target position.Source: 9) reach · p. 18

11) crawl:

The crawl, push, and cabinet tasks extend the suite to constrained locomotion and sequential object manipulation with randomized goals and multi-stage rewards.

11) crawl:

  • Crawl requires moving at 1 m/s through a tunnel while combining tunnel containment, low body height, crawling orientation, and speed.Source: 11) crawl · p. 18
  • The maze task directs the robot through multiple checkpoints using target velocities, proximity rewards, collision handling, and sparse completion bonuses.Source: 12) maze · pp. 18–19

13) push:

  • Push moves a randomly initialized box to a randomly initialized table target, using observations of the robot, hand, box, destination, and box velocity.Source: 13) push · p. 19

14) cabinets:

  • Cabinet tasks require opening several door types and completing object manipulations inside the cabinet, with rewards changing by subtask.Source: 14) cabinets · p. 19
  • Cabinet completion provides sparse rewards for finished subtasks and an additional 1000 reward when all subtasks are complete.Source: 14) cabinets · p. 19

15) highbar:

The final tasks cover acrobatic, door, package, bimanual cube, and bookshelf manipulation, emphasizing sequential objectives, whole-body coordination, and randomized configurations.

15) highbar:

  • Highbar requires swinging while attached to a horizontal bar until reaching a vertical upside-down position.Source: 15) highbar · p. 20

16) door:

  • Door traversal combines pulling a door open by its doorknob, passing through it, and maintaining the door open within current hinge and latch ranges.Source: 16) door · p. 20

17) truck:

  • Truck unloading rewards moving packages from the truck to a table, with an additional sparse reward of 1000 when all packages are picked up and delivery completes.Source: 17) truck · p. 20

18) cube:

  • The cube task requires both hand-held cubes to match a randomly initialized target orientation, using orientation and hand-proximity terms.Source: 18) cube · pp. 20–21

19) bookshelf:

  • Bookshelf tasks relocate five objects through ordered subtasks, with fixed order in the simple variant and randomized order, objects, and destinations in the hard variant.Source: 19) bookshelf · p. 21

20) basketball:

This section defines basketball catching-and-throwing, window wiping, and spoon-following tasks with task-specific observations, initialization, rewards, and termination conditions.

20) basketball:

  • The basketball task requires catching a randomly arriving ball and throwing it into a basket, using separate catch and throw reward stages.Success occurs when the ball reaches within 0.05 of the basket, triggering a sparse reward of 1000.Source: 20) basketball · pp. 21–22

21) window:

  • The window task requires holding a wiping tool parallel to the window while following a prescribed vertical speed and achieving contact coverage.Five sites on the wipe detect contact coverage across the tool surface.Source: 21) window · p. 22

22) spoon:

  • The spoon task requires grasping a spoon and tracing a circular pattern inside a pot while observing the current target position.The reward combines destination and trajectory terms, and the episode lasts up to 1000 steps unless the pelvis falls below the threshold.Source: 22) spoon · p. 22

23) kitchen:

This section specifies kitchen, powerlifting, room organization, and block-insertion tasks, then describes baseline implementations, reaching-policy training, and benchmark evaluation procedures.

23) kitchen:

  • The kitchen task sequences opening a microwave, moving a kettle, and switching burners and lights, with sparse reward based on completed subtasks.A subtask is complete when an object is within a specified distance of its goal.Source: 23) kitchen · pp. 22–23, 24) package · p. 23

25) powerlift:

  • The powerlift task asks the humanoid to lift a designated-mass barbell, rewarding stability and barbell height.The reward is R(s, a) = 0.2 · stable + 0.8 · heightbarbell.Source: 25) powerlift · p. 23

26) room:

  • The room task organizes randomly scattered objects in a 5 m by 5 m area by minimizing the variance of their x- and y-coordinates.The cleanness term uses the maximum of the coordinate variances.Source: 26) room · p. 23

27) insert:

  • The insert task requires placing both ends of a rectangular block into two pegs, with separate versions differing in object size.The reward uses proximity between block ends and pegs together with proximity between the pegs and the robot’s hands.Source: 27) insert · pp. 23–24

A. Baseline Implementation Details

  • SAC, PPO, DreamerV3, and TD-MPC2 use specified implementations, with DreamerV3 and TD-MPC2 configured through their medium and 5M settings respectively.PPO uses Stable-Baselines3 with four parallel environments, while the other algorithms use stated official or JaxRL implementations.Source: A. Baseline Implementation Details · p. 24

B. Reaching Policy Implementation Details

  • Low-level reaching policies are trained with PPO using GPU-parallelized MuJoCo MJX and PureJaxRL, with modified environment count, rollout length, and entropy coefficient.The entropy coefficient is 0.001.Source: B. Reaching Policy Implementation… · p. 24

C. Benchmarking Results

  • Benchmark results report mean and standard deviation of maximum episode returns over three seeds, with DreamerV3 and SAC trained for 10M steps and TD-MPC2 for 2M.The reported training budgets are intended to correspond roughly to 48 hours for TD-MPC2.Source: C. Benchmarking Results · p. 24, Table V · p. 24, Table VI · p. 24

Related papers

Loading 2403.10506v2…