PaperlayerView exact revision on arXiv

Source-linked AI summary

Berkeley Humanoid: A Research Platform for Learning-based Control

Qiayuan Liao, Bike Zhang, Xuanyu Huang, Xiaoyu Huang, Zhongyu Li, Koushil Sreenath

arXiv:2407.21781v1cs.RO

TL;DR

Mid-scale humanoids are disturbance-sensitive and require precise control, while learning-based algorithms face substantial sim-to-real gaps in rapid, dynamic motion. The authors custom-build a mid-scale humanoid tailored for learning-based control, using simple actuator-joint transmissions and EtherCAT to simplify simulation. The robot traversed 364 m on campus in 10 minutes, climbed rough terrain for more than 5 minutes covering 96 m with 10.5 m elevation gain, and performed omnidirectional and one-leg hops.

  • Problem

    Mid-scale humanoids are disturbance-sensitive and require precise control, while learning-based algorithms face substantial sim-to-real gaps in rapid, dynamic motion.

    Source: §1 · p. 2

  • Method

    The authors custom-build a mid-scale humanoid tailored for learning-based control, using simple actuator-joint transmissions and EtherCAT to simplify simulation.

    Source: §1 · p. 2, §3.2 · p. 5

  • Results

    The robot traversed 364 m on campus in 10 minutes, climbed rough terrain for more than 5 minutes covering 96 m with 10.5 m elevation gain, and performed omnidirectional and one-leg hops.

    Source: §5.1 · pp. 10–11, §5.2 · p. 12

  • Takeaways & Limitations

    The platform demonstrates robust outdoor locomotion across varied terrain with a minimally designed reinforcement-learning algorithm.

    Source: §7 · p. 12

    The platform omits arms, and further work is needed on range of motion, backlash, weight, mechanical strength, torque-current modeling near saturation, and simulated motor-work and heat limits.

    Source: §6 · p. 12

Abstract

from arXiv · show

We introduce Berkeley Humanoid, a reliable and low-cost mid-scale humanoid research platform for learning-based control. Our lightweight, in-house-built robot is designed specifically for learning algorithms with low simulation complexity, anthropomorphic motion, and high reliability against falls. The robot's narrow sim-to-real gap enables agile and robust locomotion across various terrains in outdoor environments, achieved with a simple reinforcement learning controller using light domain randomization. Furthermore, we demonstrate the robot traversing for hundreds of meters, walking on a steep unpaved trail, and hopping with single and double legs as a testimony to its high performance in dynamical walking. Capable of omnidirectional locomotion and withstanding large perturbations with a compact setup, our system aims for scalable, sim-to-real deployment of learning-based humanoid systems. Please check http://berkeley-humanoid.com for more details.

1 Introduction

The paper argues for a mid-scale humanoid that combines dynamic motion with rapid learning-policy deployment, fall robustness, and low cost. It custom-builds a platform around learning-based control, addressing both compact-hardware design and the control challenges of short-legged robots.

1 Introduction

  • Mid-scale robots are attractive for accessible, forgiving experiments, but compact packaging makes high-performance actuators and reliable hardware difficult to achieve affordably.Their lower mass eases handling and falls, while limited space and demanding dynamic tasks raise actuator and impact-reliability requirements.Source: §1 · pp. 1–2
  • Low center of gravity, short stride, and rapid leg movements make mid-scale robots especially disturbance-sensitive and difficult to control across the sim-to-real gap.Small forces can cause large motions, and short time constants demand precise, robust policies.Source: §1 · p. 2
  • The authors propose a custom-built mid-scale humanoid tailored to learning-based control, with hardware design intended to narrow the sim-to-real gap.The platform uses custom modular actuators, integrated transmissions, hollow shafts, and EtherCAT; learning-based control also permits cheaper, noisier sensors.Source: §1 · p. 2

2 Related Work

Related work spans full-scale, mid-scale, and miniature platforms, with different actuator choices and trade-offs in dynamic performance, learning-based control, and research access. Prior control work demonstrates varied locomotion skills, while simple algorithms and architectures remain a challenge for dynamic motions.

Humanoid Design

  • Full-scale platforms often use high-reduction Harmonic Drive actuators, while lighter QDD designs have enabled more dynamic tasks; mid-scale and miniature platforms commonly use QDD but often lack fully articulated legs.The paper groups platforms as full-scale, mid-scale, or miniature and notes that some company robots use other actuator arrangements.Source: §2 · p. 2
  • Company-built robots are described as well-designed and well-tested, but many are unavailable to research labs or do not permit low-level system modification.The passage names Boston Dynamics’ hydraulic and electric Atlas as examples of company platforms.Source: §2 · p. 3

Humanoid Control

  • Prior humanoid control spans heuristic and model-based methods as well as learning-based locomotion and manipulation, with demonstrations including rough-terrain walking, disturbance resistance, running, and parkour.Some prior learning approaches use complex networks or state-action histories, while wide domain randomization can hinder learning or yield conservative policies [42].Source: §2 · p. 3
  • For smaller humanoids, large flat feet and servo motors have supported soccer but limited dynamic motion; the paper contrasts this with smaller feet and more powerful actuators.The authors characterize dynamic motions with simple algorithms as an outstanding challenge.Source: §2 · p. 3

3 Design for Learning-based Control

The Berkeley Humanoid is designed around learning-based control through simple-to-simulate actuation, durable low-cost construction, a compact experiment setup, and anthropomorphic leg mechanics. The design combines custom hardware and system integration to support simulation efficiency, repeated experiments, and human-like motion.

3.1 System Overview

  • The 16 kg robot has a torso and two 6-DoF legs, with a 0.85 m standing height and 220 mm thighs and 180 mm calves.Most actuators attach directly to their links; two designed 4-DoF arms were omitted to focus on locomotion.Source: §3.1 · p. 4

3.2 Simulation-Friendly

  • Simple actuator-joint transmissions and EtherCAT communication let the robot be simulated without actuator dynamics, achieving more than 90,000 simulation steps per second on an NVIDIA A4500 GPU.The design avoids flexible components and closed kinematic chains; latency is 0.5–2 ms and torque control bandwidth is 1 kHz.Source: §3.2 · p. 5

3.3 Reliable and Low-Cost

  • Aluminum main components, steel gearbox and linkage parts, and hollow-shaft cable routing are designed to help the robot withstand impacts and reduce cable tearing.Custom QDD actuators also support joint-torque estimation without strain gauges, enabling foot-contact wrench estimates without dedicated contact or force/torque sensors.Source: §3.3 · p. 6
  • Custom hardware and learning-based control reduce cost: the cell-phone-level ICM42688 IMU costs less than one dollar, and the arm-less robot costs USD 10,000.The authors note that most component costs are expected to decrease with scaled-up production.Source: §3.3 · p. 6

3.4 Experiment-Friendly

  • The 16 kg design allows one operator to conduct indoor experiments and outdoor experiments with an optional cameraman, including commanding, data collection, filming, and some resets.All reported experiments use this setup.Source: §3.4 · p. 6

3.5 Anthropomorphic

  • Six-DoF anthropomorphic legs, including ankle roll actuation, support human-like joint ranges and improve stability in challenging static poses.The design also aims to protect hardware while retaining ranges for imitating human motions.Source: §3.5 · p. 7

4 A Minimally Composed Learning-based Controller

The platform pairs a minimally composed reinforcement-learning controller with hardware design and targeted domain randomization to narrow the sim-to-real gap. The resulting controller zero-shot transfers robust, agile locomotion to the robot.

4.1 Reinforcement Learning Formulation

  • The controller uses immediate state feedback and desired joint positions, with PPO-trained MLP networks and no history, teacher-student training, phase signals, or reference motion.The policy runs at 50 Hz, the state estimator at 1 kHz, and the PD controller at 25 kHz.Source: §4.1 · p. 7

4.2 Closing the Sim-to-Real Gap

  • Hardware design reduces simulation mismatch by simplifying actuator-joint modeling and improving torque execution rate, accuracy, and latency.The design uses direct or linear joint-actuator mappings and high-bandwidth torque control with transparent QDD dynamics.Source: §4.2 · pp. 7–8
  • Figure 4 depicts forward, backward, and turning walks in the lab, and forward and sideways walking outdoors.The controller follows sagittal and lateral velocity commands and yaw angular-velocity commands.Source: Figure 4 · p. 8
  • For robot-physics uncertainty, CAD parameters and actuator friction experiments support small, accurate randomization ranges.The authors note that obtaining similarly detailed hardware parameters for commercial robots would be difficult.Source: §4.2 · p. 8
  • Contact uncertainty is covered by randomizing ground friction, restitution, and perturbation forces, while unidentifiable motor-strength ratios and PD gains are excluded.The paper says heuristic PD approximations can require unnecessarily large randomization ranges.Source: §4.2 · p. 8
  • Design-enabled domain randomization supports robust, agile locomotion after zero-shot transfer to the robot, even with a minimally composed RL controller.The approach targets a narrow sim-to-real gap through hardware design and accurate randomization.Source: §4 · p. 7, §4.2 · p. 8

5 Experimental Validation

The experiments assess learning-based locomotion, sim-to-real transfer, and hardware reliability. Berkeley Humanoid demonstrates long-distance outdoor walking, traversal of varied and challenging terrain, recovery from perturbations, dynamic hopping, and resilience to repeated falls.

5 Experimental Validation

  • The validation evaluates whether a minimally composed RL controller can learn locomotion, how small its sim-to-real gap is, and whether the hardware is reliable.The authors emphasize robust and agile locomotion from a basic RL controller, contrasting their minimal design with advanced architectures.Source: §5 · p. 8, §5.1 · p. 9

5.1 Learning Control Performance

  • The robot walked across eight terrain types, including grass, sidewalks, unpaved trails, roads, bridges, tracks, tiled surfaces, stairs, and inclines.On an average 20-degree unpaved trail, it walked stably, turned, and recovered after stepping on loose rocks; on uneven pathways it navigated forward and backward and recovered from slipping.Source: §5.1 · pp. 9–10, Figure 5 · p. 9
  • The robot traversed and turned on rocky stairs with 4 cm steps, and recovered within a few steps after kicks while stepping in place, including outdoors on uneven grass.The stairs’ steps were 10% of full leg length.Source: Figure 6 · p. 9, §5.1 · p. 10
  • The robot walked 364 m across UC Berkeley campus in 10 minutes, including uphills and downhills, and traversed rough terrain non-stop for more than five minutes, covering 96 m.On the rough terrain, elevation gain was 10.5 m.Source: §5.1 · pp. 10–11

5.2 Evaluation of Sim-to-Real Transfer

  • Over 60 seconds of random velocity commands, tracking errors were 0.051 m/s in simulation and 0.058 m/s on hardware sagittally, and 0.086 m/s and 0.1156 m/s laterally.The controller had no access to history for online system identification or adaptation during training or deployment.Source: §5.2 · p. 11, Figure 8 · p. 11
  • With the same hopping-controller settings except for rewards, the robot performed omnidirectional hops and single-leg hops while maintaining balance, with a mostly slack safety rope and minor assistance.The authors present the dynamic motions as evidence of a small sim-to-real gap and of hardware support for agile motion with simple algorithmic design.Source: §5.2 · pp. 11–12, Figure 9 · p. 11

5.3 Hardware Reliability

  • Across 38 falls on varied terrain, the robot sustained no hardware damage except for two failures caused by loose screws and glue, and could usually resume its policy within 3–5 seconds.The authors identify rapid resetting as important for scalable real-world deployment.Source: §5.3 · p. 12

6 Limitations

The work focuses on locomotion and leaves arms off the platform. The authors also identify hardware refinements and additional torque modeling as future work.

6 Limitations

  • Arms were omitted for simplicity because the work focuses on mid-scale humanoid locomotion.The authors describe future plans for two 4- or 6-DoF arms.Source: §6 · p. 12
  • The authors plan further hardware iterations to improve range of motion, backlash, weight, and mechanical strength.Source: §6 · p. 12
  • Further reducing the sim-to-real gap for more dynamic motion requires detailed identification of torque-current nonlinearities near saturated torque, plus training-time simulation of motor operating regions and heat protection.Source: §6 · p. 12

7 Conclusion

Berkeley Humanoid is a reliable, low-cost platform for learning-based bipedal locomotion, designed for a narrow sim-to-real gap. Its minimally designed RL policy supports robust omnidirectional locomotion and dynamic outdoor demonstrations.

7 Conclusion

  • The Berkeley Humanoid is a reliable, low-cost platform for learning-based bipedal locomotion with a narrow sim-to-real gap.Its design emphasizes low simulation complexity, anthropomorphic ranges of motion, and reliability against falls and impacts.Source: §7 · p. 12
  • A minimally designed RL policy without history or phase-signal inputs withstands large random perturbations and supports omnidirectional locomotion over challenging terrain.The paper describes outdoor experiments across varied terrains and ground conditions.Source: §7 · p. 12
  • The robot demonstrates long-distance campus walking, steady ascent on steep, narrow unpaved trails, and single-leg hopping.Source: §7 · p. 12

Appendix

The appendix details walking rewards for tracking, smoothing, regularization, and gait quality, then describes the adjustments used for hopping.

A.1 Walking

  • Walking rewards track desired versus actual sagittal and lateral linear velocities and yaw angular velocity using L2 differences.Source: §A.1 · p. 18
  • Walking smoothing penalties cover vertical linear velocity, roll and pitch angular velocities, joint torques, and action rates.These terms are intended to improve policy smoothness.Source: §A.1 · p. 18
  • Walking regularization penalizes deviation from nominal hip and knee positions, non-upright body orientation, and actions beyond soft actuator limits.The authors say these terms help prevent aggressive and dangerous motions.Source: §A.1 · p. 18
  • Walking gait-quality terms encourage longer foot airtime, prevent ground slipping, and keep contact forces below a hardware-protective threshold.Source: §A.1 · p. 18

A.2 Hopping

  • Hopping rewards encourage positive vertical velocity with ReLU, remove knee and hip-pitch limits, and penalize ground contact by the in-air leg during single-leg hopping.Other reward terms remain the same as in walking.Source: §A.2 · p. 18

B Outdoor Failure Counts Throughout the Project

The project recorded hardware failure counts across different terrains, but these counts cover testing and debugging rather than the reported experiments.

B Outdoor Failure Counts Throughout the Project

C Joint Ranges of the Hardware

The section describes the robot's anthropomorphic joint ranges; the cited table compares the right leg's ranges with human joints.

C Joint Ranges of the Hardware

D Dynamics Randomization Details

Dynamics randomization was designed to fit the actual hardware and minimize the sim-to-real gap. After system identification, the authors use small ranges for six dynamics parameters and eight noise terms.

D Dynamics Randomization Details

  • The authors carefully design dynamics randomization to best fit the actual hardware.The passage refers to Section 4.2 for this design.Source: D Dynamics Randomization Details · p. 19
  • Six dynamics parameters and eight noise terms are randomized over small ranges after system identification on the in-house hardware.The ranges are summarized in Table 6.Source: Table 6 · p. 19
  • The listed randomization ranges are intended to minimize the sim-to-real gap.Table 6 covers both dynamics parameters and noise terms.Source: Table 6 · p. 19
Loading 2407.21781v1…