Source-linked AI summary
Robot Parkour Learning
Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christopher Atkeson, Soeren Schwertfeger, Chelsea Finn, Hang Zhao
TL;DR
Autonomous parkour requires diverse, vision-based skills that existing approaches do not jointly provide. The paper learns specialized skills with two-stage reinforcement learning, distills them into one depth-based policy, and demonstrates autonomous traversal on two low-cost robots.
Problem
Existing methods provide either diverse but blind skills or vision-based specialized skills, leaving a need for generalizable parkour that is both vision-based and diverse.
Method
A two-stage reinforcement-learning method learns five parkour skills with simple rewards and dynamics constraints, then DAgger distills them into a single vision-based policy.
Results
The system enables two low-cost robots, A1 and Go1, to autonomously select and execute skills including climbing 0.40m obstacles, leaping 0.60m gaps, crawling, squeezing, and running.
Takeaways & Limitations
The same training pipeline can support autonomous selection and execution of diverse parkour skills across different low-cost quadrupedal robots.
Takeaways & Limitations
The current system requires manually constructed simulation environments, so new skills require adding environments with different obstacles and appearances.
Abstract
from arXiv · showhide
Parkour is a grand challenge for legged locomotion that requires robots to overcome various obstacles rapidly in complex environments. Existing methods can generate either diverse but blind locomotion skills or vision-based but specialized skills by using reference animal data or complex rewards. However, autonomous parkour requires robots to learn generalizable skills that are both vision-based and diverse to perceive and react to various scenarios. In this work, we propose a system for learning a single end-to-end vision-based parkour policy of diverse parkour skills using a simple reward without any reference motion data. We develop a reinforcement learning method inspired by direct collocation to generate parkour skills, including climbing over high obstacles, leaping over large gaps, crawling beneath low barriers, squeezing through thin slits, and running. We distill these skills into a single vision-based parkour policy and transfer it to a quadrupedal robot using its egocentric depth camera. We demonstrate that our system can empower two different low-cost robots to autonomously select and execute appropriate parkour skills to traverse challenging real-world environments.
1 Introduction
The paper frames autonomous parkour as a difficult legged-locomotion challenge requiring rapid obstacle negotiation, then presents a system spanning diverse skills and robot platforms.
- Autonomous parkour requires robots to move swiftly through complex environments while running, climbing, and jumping over obstacles.
- Existing approaches demonstrate locomotion capabilities, but autonomous parkour remains difficult because predictive-control systems demand substantial engineering effort and costly hardware.
- The system combines a two-stage reinforcement-learning method with soft and hard dynamics constraints to address difficult parkour exploration.
- It supports climbing 0.40m obstacles, leaping 0.60m gaps, crawling beneath 0.2m barriers, squeezing through 0.28m slits, and running.
- Experiments show that low-cost quadrupedal robots can autonomously select and execute appropriate skills using onboard computation, visual sensing, and power, with the same pipeline demonstrated on A1 and Go1.
2 Related Work
Related work includes model-based control demonstrations of agile locomotion across diverse robots and learning-based methods for locomotion and manipulation.
- Model-based control has enabled agile behaviors including obstacle jumping, vertical jumping, tree climbing, self-righting, door opening, and stepping-stone traversal.
- Boston Dynamics Atlas has demonstrated parkour skills within this model-based-control lineage.
- Learning-based methods have also demonstrated capabilities in agile locomotion and related legged-robot tasks.
3 Robot Parkour Learning Systems
The system learns diverse parkour skills with two-stage reinforcement learning, then distills them into one vision-based policy that uses onboard sensing to select among skills.
- System Overview: The system targets five skills—climbing, leaping, crawling, squeezing through slits, and running—under one end-to-end parkour framework.The policy is intended to use raw depth sensing and proprioception to control every joint of a low-cost robot.
- Two-Stage RL: The two-stage RL procedure first permits obstacle penetration with soft dynamics constraints, then fine-tunes behaviors under realistic hard constraints.The penetration reward measures violation volume and depth, while an automatic curriculum adapts obstacle difficulty.
- Specialized Skills: Specialized policies use privileged visual and physics information in simulation, then output target joint positions through recurrent networks.The five policies are trained separately on corresponding terrains using the same reward structure.
- Reward Design: The skill reward encourages forward motion while conserving mechanical energy, with target speed set around 1 m/s and squared motor power reducing average and variance.The reward uses forward, lateral, yaw, penetration, and motor-power terms with tunable hyperparameters.
- Policy Distillation: DAgger distills the five specialized policies into a single GRU-based parkour policy using proprioception, previous actions, and a CNN embedding of depth images.Each state is supervised by the specialized policy corresponding to the known obstacle type, while the distillation objective uses binary cross-entropy divergence.
- Sim-to-Real Transfer: Simulation and real depth images receive different preprocessing to reduce the visual sim-to-real gap, including clipping, noise or artifacts, hole-filling, and spatial-temporal filtering.Rendered images receive clipping, Gaussian noise, and random artifacts; real images receive clipping, hole-filling, and spatial and temporal filters.
4 Experimental Results
Experiments evaluate the parkour system in simulation and real-world settings against blind, MPC, RND, MLP, RMA, and ablation baselines. Results show benefits from vision, soft-dynamics pre-training, recurrent memory, distillation, and emergent retrying.
- Experimental Setup: The evaluation uses Unitree A1 and Go1 robots with onboard Jetson NX computation and Intel RealSense D435 visual sensing.The simulator is IsaacGym, and experiments include both simulation and real-world comparisons.
- Real-World Experiments: Figure 6 compares the vision-based policy with blind and built-in MPC controllers across skills and obstacle difficulties, with the parkour policy achieving the best performance.The real-world indoor trials use success rates averaged over 10 trials for each skill and difficulty.
- Simulation Experiments: Without depth sensing, the blind policy completes no climbing, leaping, or tilting trials and reaches only 13% crawling success.The comparison uses proprioception alone for the blind policy.
- Simulation Experiments: Soft-dynamics pre-training achieves faster learning and approximately 95% final success across exploration-intensive skills.The comparison averages success over 100 trials across climbing, leaping, crawling, and tilting, using three random seeds per method.
- Simulation Experiments: The GRU-based policy outperforms the MLP baseline on memory-dependent skills because climbing and leaping require past visual perceptions.During climbing, past obstacle dimensions remain relevant after the front legs contact the obstacle.
- Real-World Experiments: In real-world climbing, the policy can retry after failure by backing away to create run-up space, although this behavior was not explicitly programmed.The retrying behavior appears in both real-world experiments and simulation.
5 Conclusion, Limitations and Future Directions
The system demonstrates robust parkour performance for low-cost robots, but expanding its skill set currently depends on manually constructed simulation environments. Future work targets automatic environment construction and RGB-based training.
- Conclusion: The system achieves robust performance for challenging parkour skills in indoor and outdoor environments using low-cost robots.The conclusion describes the system as a parkour learning system for low-cost robots.
- Limitations: New skills currently require manually constructed simulation environments containing different obstacles and appearances.This constraint reduces how automatically new skills can be learned.
- Future Directions: Future work will explore constructing diverse simulation environments automatically from large-scale real-world data using 3D vision and graphics.The authors also plan to investigate training agile locomotion directly from semantic RGB images instead of depth images.
A Experiment Videos
The authors report thorough real-world analysis, with indoor and outdoor experiment videos available on the project website.
- Experiment Videos: Indoor and outdoor experiment videos document the system’s real-world analysis and are available on the project website.The cited website is https://robot-parkour.github.io.
B Details of Training in Simulation
Simulation training uses specialized skill policies and a vision-based parkour policy, with domain randomization, PPO settings, distillation, and comparisons of training variants documented in the supplementary material.
- Specialized Skills: Each specialized skill policy uses a GRU followed by an MLP to output target joint positions from proprioceptive, visual, recurrent, action, and physics information.The GRU has 256 hidden units, and the MLP dimensions are [512, 256, 128].
- Specialized Skills: Training randomizes robot mass, center of mass, motor strength, terrain friction, depth latency, camera parameters, and proprioception delay.These parameters are listed as environment randomizations for simulation training.
- Parkour Policy: The parkour policy combines a CNN visual encoder, GRU, and MLP, feeding visual embeddings alongside proprioception, previous action, and recurrent state.Its simulation training environment is illustrated in Figure 8.
- Ablations: The supplementary experiments compare the method with Oracles without soft dynamics and RND, with fine-tuning beginning late in training for the proposed method.PPO hyperparameters and the parkour-policy structure are also tabulated.
- Distillation: Distillation uses binary cross-entropy between specialized-skill actions and parkour-policy actions, whose outputs range from −1 to 1.The action relationship is specified through aspecialized and aparkour.
C Details of Simulation Setup
Simulation uses IsaacGym Preview 4 with a large, procedurally organized terrain map and curriculum-based track difficulty.
- IsaacGym Preview 4 provides the simulation environment for training specialized policies.
- Each training map contains 800 tracks arranged in a 20 by 40 grid.
- Track difficulty increases linearly by row, while same-row tracks vary in non-essential configurations.
- Tracks within each column connect end to end, allowing the robot to continue forward after completing a track.
D Details of Robot Setup
Real-world experiments use a Unitree A1 with onboard computing, a depth camera, and an asynchronous vision-processing pipeline.
- The Unitree A1 has 12 joints, each with a 33.5Nm instant maximum motor torque, and carries an Nvidia Jetson NX.
- An Intel RealSense D435 camera provides front-facing depth images using inferred and stereo sensing.
- Depth images are filtered, cropped, and down-sampled from 240 X 424 to 48 X 64 resolution.
- A CNN processes visual observations asynchronously, sending visual embeddings to the main network through ROS messages at 10Hz.
E Detailed Comparison Studies on RL Pre-Training with Soft Dynamics Constraints
Soft dynamics constraints are central to exploration: the proposed method succeeds at climbing and leaping, whereas the comparison methods do not learn those skills.
- The method trained with soft dynamics constraints is the only compared approach that completes climbing and leaping skills.
- RND fails to learn successful climbing, leaping, and tilting maneuvers, except for crawling.
- Oracles w/o Soft Dyn learns crawling and tilting but fails at climbing and leaping, the paper’s most difficult skills.