Source-linked AI summary
ROBEL: Robotics Benchmarks for Learning with Low-Cost Robots
Michael Ahn, Henry Zhu, Kristian Hartikainen, Hugo Ponte, Abhishek Gupta, Sergey Levine, Vikash Kumar
TL;DR
Real-world reinforcement learning needs affordable, resilient hardware and benchmarks that account for physical constraints. ROBEL provides two modular robots, reusable benchmark infrastructure, and safety-aware evaluation, with results reproduced across independently built systems.
Problem
Simulation and existing benchmarks inadequately expose real-world domain shift, hardware risks, and the practical constraints of scalable robot learning.
Method
ROBEL combines modular D’Claw and D’Kitty robots with real-world and simulated benchmark tasks, reusable software, and dense, sparse, success, and hardware-safety evaluation signals.
Results
ROBEL supports reproducible benchmarking across manipulation and locomotion robots, with platforms sustaining over 14000 hours of real-world training and results reproduced at a remote site.
Takeaways & Limitations
ROBEL provides a reproducible, extensible foundation for comparing learning-based methods directly on physical robot hardware.
Abstract
from arXiv · showhide
ROBEL is an open-source platform of cost-effective robots designed for reinforcement learning in the real world. ROBEL introduces two robots, each aimed to accelerate reinforcement learning research in different task domains: D'Claw is a three-fingered hand robot that facilitates learning dexterous manipulation tasks, and D'Kitty is a four-legged robot that facilitates learning agile legged locomotion tasks. These low-cost, modular robots are easy to maintain and are robust enough to sustain on-hardware reinforcement learning from scratch with over 14000 training hours registered on them to date. To leverage this platform, we propose an extensible set of continuous control benchmark tasks for each robot. These tasks feature dense and sparse task objectives, and additionally introduce score metrics as hardware-safety. We provide benchmark scores on an initial set of tasks using a variety of learning-based methods. Furthermore, we show that these results can be replicated across copies of the robots located in different institutions. Code, documentation, design files, detailed assembly instructions, final policies, baseline details, task videos, and all supplementary materials required to reproduce the results are available at www.roboticsbenchmarks.org.
1 Introduction
Real-world reinforcement learning needs affordable, resilient robots because simulation-to-reality gaps and unsafe exploration limit direct deployment. ROBEL addresses these requirements with two cost-effective, modular robots, benchmark tasks, and reproducible evaluations.
- Simulation-to-reality gaps, including inaccurate physical models and domain shift, motivate developing robotic control solutions directly on physical hardware.
- Real-robot learning must withstand aggressive exploration, which can cause collisions, wear, and permanent hardware damage.
- Learning-based robotics requires resilient, low-cost, maintainable hardware and reliable mechanisms for long-duration trial-and-error and large-scale data collection.
- ROBEL introduces D’Claw for nine-DOF manipulation and D’Kitty for twelve-DOF locomotion, alongside real-world and simulated benchmark tasks and cross-copy policy reproducibility.
2 Related Work
Existing robotics learning benchmarks largely rely on simulation or costly, task-specific real-world setups that omit physical constraints. ROBEL extends benchmarking to high-DOF manipulation and locomotion while explicitly measuring hardware safety.
- Simulation offers fast prototyping and standardized comparisons, but privileges such as precise resets and unrestricted safety do not transfer readily to physical robots.
- Real-world reinforcement learning either transfers from simulation, which scales poorly with task complexity, or requires substantial task-specific time and cost.
- Prior low-cost robots often sacrifice degrees of freedom or replicability, whereas ROBEL uses modularity to retain high DOF, low cost, and easy assembly.
- ROBEL broadens real-world benchmarks with high-DOF dexterous manipulation and quadruped locomotion tasks.
- ROBEL adds dense-reward, sparse-score, and hardware-safety signals because average episodic return can overlook unsafe exploration and hardware risks.
3 ROBEL
ROBEL combines modular, replicable hardware with low-cost construction and benchmark-oriented platform designs. Its two robots target dexterous manipulation and agile locomotion while supporting scalable experimentation.
- Hardware Platforms: Modularity enables reasonably high-DOF robots to remain low-cost and maintainable using off-the-shelf components and common fabrication tools.
- Hardware Platforms: ROBEL platforms have the most economical price point, with costs scaling linearly with degrees of freedom through modular design.
- Hardware Platforms: Detailed CAD models, bills of materials, and assembly instructions support independent replication and reliability testing at geographically remote locations.
- Hardware Platforms: ROBEL’s reproducibility and scalability support standardized benchmarks for sharing and collaborative comparison of robotics results.
- Hardware Platforms: D’Claw provides nine actuated DOFs for dexterous manipulation, while D’Kitty provides twelve DOFs for quadruped locomotion and can walk upside down.
4 Benchmark Tasks
ROBEL defines reusable real-world benchmark tasks for its D’Claw manipulation and D’Kitty locomotion robots, with standardized objectives, evaluation, and hardware-safety measures.
- ROBEL provides benchmark tasks for continuous-control challenges spanning D’Claw manipulation and D’Kitty locomotion.The platform includes task formulations, reusable software components, and both real-hardware and simulated back ends.
- ROBEL encourages simulation for rapid prototyping while treating real-world hardware performance as the benchmark target.
- The task interface uses an MDP with dense rewards, sparse task scores, success evaluation, and hardware-safety counters.Safety counters measure joint-limit, velocity-limit, and current-limit violations over a specified horizon.
- D’Claw manipulation benchmarks: D’Claw benchmarks target fundamental manipulation behaviors including pose matching, turning, and continuously rotating an object toward a moving target.Pose tasks match randomly selected joint-angle targets, while Screw tasks use moving joint-angle targets.
- D’Kitty locomotion benchmarks: D’Kitty benchmarks target quadruped locomotion behaviors including standing and changing torso orientation while maintaining balance.The task variants include fixed, random, and random-dynamics settings with increasing distributional and dynamics variability.
5 Experiments
Experiments evaluate ROBEL with multiple learning algorithms and examine reproducibility, hardware transfer, reliability, and safety. The results support geographically independent replication and extended real-world use, while exposing safety violations in policies that solve tasks.
- Baseline methods: ROBEL benchmarks cover on-policy, off-policy, demonstration-augmented, and supervised baselines, including NPG, SAC, DAPG, and behavior cloning.Figure 7 reports success percentages for these agents across D’Claw and D’Kitty tasks.
- Reproducibility: Independent groups reproduced ROBEL platforms and results at geographically separated sites using the documentation, and policies were evaluated across different hardware.The sites were 60 miles apart, with no in-person visits; Figure 8 summarizes reproducibility and cross-hardware effectiveness.
- Reliability: ROBEL supported multiple-day real-world experimentation, with loose screws as the main maintenance issue and occasional replaceable motor failures.The reliability assessment includes development-period data, while the mature system is reported as more reliable.
- Safety: Policies successfully solve tasks but exhibit significant safety violations, showing that task success alone does not ensure hardware-safe behavior.ROBEL measures violations over trajectories, including joint-limit, velocity-limit, and torque-related safety objectives.
6 Conclusion
ROBEL is an open-source, low-cost platform for on-device reinforcement-learning research, combining D’Claw and D’Kitty with benchmark tasks, baseline methods, and reproducibility materials.
- ROBEL combines 9-DOF D’Claw and 12-DOF D’Kitty platforms with benchmark tasks for real-world reinforcement-learning experimentation.
- The platform has sustained over 14,000 hours of real-world training and provides baselines spanning on-policy, off-policy, demonstration-accelerated, and supervised methods.
- ROBEL includes materials intended to support reproducibility, including documentation, design files, assembly instructions, policies, baseline details, and task videos.
A.1 D’Claw tasks
D’Claw tasks evaluate dexterous manipulation through pose, turn, and screw objectives, with fixed, randomized, and randomized-dynamics variants. Success is measured against task-specific tracking or terminal-error thresholds.
- A.1 D’Claw tasks: D’Claw provides pose, turn, and screw manipulation tasks spanning static targets, moving targets, and object rotation.The task families target joint-angle matching, object turning, and continuous rotation at a desired velocity.
- A.1 D’Claw tasks: Three pose variants use fixed goals, oscillating goals, or randomized joint damping and friction loss.The dynamic variant oscillates between two sampled goal positions and randomizes joint damping and friction loss.
- A.1 D’Claw tasks: Three turn variants progress from fixed initial and goal angles to randomized goals and randomized object and joint dynamics.Random-dynamics episodes vary the hand’s relative position, object size, joint damping, and friction loss.
- A.1 D’Claw tasks: Success thresholds use terminal turn error or mean absolute tracking error, with β = 0.1 for turn and pose evaluation.The turn evaluator checks the final-step error, while pose uses mean absolute tracking error within β = 10◦.
- A.1 D’Claw tasks: Three screw variants vary the initial angle, desired velocity, and physical dynamics while tracking a goal angle updated every timestep.The moving target follows θt,goal = θt−1,goal + ˙θdesired ∗dt.
A.2 D’Kitty tasks
D’Kitty tasks formulate legged control around standing, orienting, and walking while sharing proprioceptive observations and uprightness-based rewards. Random-dynamics variants perturb robot parameters and terrain, and success requires task completion while remaining sufficiently upright.
- A.2 D’Kitty tasks: Uprightness is a shared reward component: falling below the cosine threshold terminates the episode and incurs αfalling.When sufficiently upright, the robot receives αupright reward; when uprightness falls below β, the episode ends.
- A.2 D’Kitty tasks: D’Kitty benchmarks cover standing, orientation matching, and walking toward a Cartesian goal while maintaining balance.The tasks coordinate 12 joints and use shared torso, joint, action, and uprightness observations.
- A.2 D’Kitty tasks: Stand variants range from fixed initial poses to randomized poses and randomized joint, friction, mass, and terrain properties.Randomized-dynamics episodes include height fields with heights up to 0.05m.
- A.2 D’Kitty tasks: Orient variants match fixed or randomized target headings, with random-dynamics episodes also varying robot properties and terrain.The orientation reward penalizes facing error and horizontal displacement while adding uprightness and bonus terms.
- A.2 D’Kitty tasks: Walk variants target fixed or randomized distances and directions, while rewarding both goal proximity and heading alignment.Random-dynamics walking varies robot parameters and generates height fields up to 0.05m.
A.3 Safety metrics
ROBEL defines shared hardware-safety metrics that count joint position, velocity, and current violations across tasks. These metrics quantify proximity to operating bounds and exceedance of speed or force limits.
- A.3 Safety metrics: ROBEL shares three hardware-safety scores across tasks: position violations, velocity violations, and current violations.The scores quantify distinct classes of joint-level safety events.
- A.3 Safety metrics: Position violations count joints whose positions approach their lower or upper operating bounds within threshold ϵ.The metric is defined over all N joints using the respective lower and upper position limits.
- A.3 Safety metrics: Velocity violations count joints whose speeds exceed the per-joint safety limit αi.The score sums violations across the robot’s N joints.
- A.3 Safety metrics: Current violations measure joints exerting forces above their per-joint current limits γi.The current score is likewise defined over the N joints.
B Locomotion benchmark performance on D’Kitty
Figure 11 reports D’Kitty task success percentages for SAC, NPG, DAPG, and BC in simulation. It evaluates each method across all D’Kitty tasks using three seeds and 20 trajectories.
- B Locomotion benchmark performance on D’Kitty: Figure 11 compares SAC, NPG, DAPG, and BC on all simulated D’Kitty tasks using success percentage.The evaluation uses three seeds and 20 trajectories per method.
C ROBEL reproducibility
SAC training performance is compared across two real D’Claw robots at different laboratory locations and across two valve objects.
- SAC training performance is evaluated on two real D’Claw robots located in different laboratories.
- The benchmark uses score as a measure of closeness to the goal during training.
- Each task is trained with both a 3-prong valve and a 4-prong valve, with each timestep representing 0.1 simulated seconds.