PaperlayerView exact revision on arXiv

Source-linked AI summary

ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots

David Hoeller, Nikita Rudin, Dhionis Sako, Marco Hutter

arXiv:2306.14874v1cs.RO

TL;DR

Agile quadrupedal navigation must handle obstacle-dependent motions, partial perception, and minimal latency. This paper presents a fully learned hierarchical pipeline that transfers from simulation to hardware, enabling navigation across challenging obstacles at speeds up to 2 m/s.

  • Problem

    Agile quadrupedal navigation requires obstacle-dependent motions, partial scene perception, and minimal processing latency.

  • Method

    A fully learned hierarchical pipeline trains obstacle-specific locomotion skills, a high-level selector, and a perception module in simulation for onboard navigation without pre-mapping or offline planning.

  • Results

    The robot reaches targets across randomized obstacle arrangements, selects suitable skills, recovers from falls and slips, and reaches speeds of up to 2 m/s.

  • Takeaways & Limitations

    The approach demonstrates real-world quadrupedal parkour with onboard, real-time computation through complex scenes requiring climbing, jumping, and non-trivial paths.

  • Takeaways & Limitations

    Scalability to more diverse scenarios remains untested, and generalization to completely new environments is unresolved.

Abstract

from arXiv · show

Performing agile navigation with four-legged robots is a challenging task due to the highly dynamic motions, contacts with various parts of the robot, and the limited field of view of the perception sensors. In this paper, we propose a fully-learned approach to train such robots and conquer scenarios that are reminiscent of parkour challenges. The method involves training advanced locomotion skills for several types of obstacles, such as walking, jumping, climbing, and crouching, and then using a high-level policy to select and control those skills across the terrain. Thanks to our hierarchical formulation, the navigation policy is aware of the capabilities of each skill, and it will adapt its behavior depending on the scenario at hand. Additionally, a perception module is trained to reconstruct obstacles from highly occluded and noisy sensory data and endows the pipeline with scene understanding. Compared to previous attempts, our method can plan a path for challenging scenarios without expert demonstration, offline computation, a priori knowledge of the environment, or taking contacts explicitly into account. While these modules are trained from simulated data only, our real-world experiments demonstrate successful transfer on hardware, where the robot navigates and crosses consecutive challenging obstacles with speeds of up to two meters per second. The supplementary video can be found on the project website: https://sites.google.com/leggedrobotics.com/agile-navigation

I. INTRODUCTION · A. Method overview

The paper frames agile quadrupedal navigation as parkour-like traversal requiring coordinated skills, rapid perception and planning, and dynamic control under partial observability. It proposes a three-module pipeline that learns terrain perception, specialized locomotion skills, and navigation in simulation before real-world deployment on ANYmal D.

  • I. INTRODUCTION: Parkour-like navigation requires robots to coordinate walking, running, climbing, and jumping through precisely timed sequences to reach difficult locations efficiently.
  • I. INTRODUCTION: The robot must interpret rapidly changing scenes, select feasible skill sequences, and execute dynamic maneuvers in real time despite limited onboard computing and partial sensing.
  • I. INTRODUCTION: Agile traversal challenges conventional locomotion because obstacles require different motions and limb contacts rather than a stable periodic gait.
  • I. INTRODUCTION: Perception and planning are difficult because self-occlusions and limited sensor fields of view provide only partial scenes, while feasible trajectories must respect the robot’s kinematic, dynamic, and controller limitations.
  • A. Method overview: The proposed system divides agile navigation into perception, locomotion, and navigation modules for reaching target locations quickly in parkour-like environments.
  • A. Method overview: Perception reconstructs surrounding terrain and a compact scene-belief vector from onboard camera and LiDAR point clouds, while locomotion provides five policies specialized for terrains such as irregular ground and gaps.
  • A. Method overview: Training uses randomized fields of stairs, inclined surfaces, boxes, gaps, and tables, with locomotion policies trained on single obstacles and perception and navigation trained on obstacle arrangements.
  • A. Method overview: After simulation training, the complete pipeline is deployed on the ANYmal D quadruped for highly dynamic maneuvers and necessary limb contacts.

B. Contributions · C. Related work

The paper presents a learned hierarchical system that combines terrain reconstruction, agile locomotion skills, and navigation to autonomously cross difficult terrains at up to 2 m/s. It addresses limitations of model-based, planned, and manually designed approaches by enabling skill selection and path planning without a priori maps or planning.

  • B. Contributions: The system autonomously crosses difficult terrains at speeds up to 2 m/s while selecting navigation decisions and executing precise agile movements on narrow and high obstacles.Its locomotion controllers exploit the robot’s full range of motion and can traverse boxes barely matching the robot’s footprint.
  • B. Contributions: The navigation approach plans through intricate scenes from terrain-reconstruction beliefs while selecting among locomotion skills, with millisecond-scale inference.The method modifies PPO [1] with a hybrid actor producing Gaussian low-level commands and categorical skill outputs.
  • B. Contributions: The locomotion policies use new terrains, heading commands, and symmetry augmentation, while terrain reconstruction combines precise local and coarse larger-scale maps.The reconstruction network is modified for efficient large-batch inference during reinforcement-learning training.
  • C. Related work: Model-predictive-control methods support precise foot placement – but depend on an underlying model and can fail under slippage or imperfect terrain perception.Existing approaches also make strong assumptions about foot-contact schedules, limiting allowable motions.
  • C. Related work: Prior agile quadrupedal work includes running [11], jumping,, jumping and climbing, and cat-like landing motions,, alongside agile bipedal demonstrations [20],.These examples reflect growing interest in agile locomotion enabled by commercially available quadrupedal platforms and deep reinforcement-learning frameworks.
  • C. Related work: Conventional legged navigation plans collision-free paths before control, but contact-rich hybrid dynamics make planning difficult and can require seconds, a priori environment knowledge, or offline computation.A prior quadrupedal approach was limited to 0.1 m obstacles at 0.33 m/s, with jumping-controller switching hard-coded.
  • C. Related work: Learning-based navigation uses demonstrations,, reinforcement learning –, or self-supervision [30], while hierarchical methods combine reusable skills through steering and switching,,.Compared with a similar quadrupedal obstacle-course system, this work removes human-designed path and skill selection, pre-mapped environments, and motion-capture dependence; it also trains skills using and steers between them as in.
  • C. Related work: Legged navigation and locomotion commonly rely on elevation maps,,,,, but noise and state-estimation errors produce unclean maps that filtering or teacher-student training can mitigate.The paper instead develops terrain reconstruction for challenging sensory conditions, including multi-resolution scene representation and efficient training-time inference.

II. RESULTS

On ANYmal D, the integrated learning-based pipeline reliably reaches targets across randomized obstacle arrangements without demonstrations, offline computation, or prior environmental knowledge. It also recovers from falls and slips and readapts when obstacles move during execution.

  • Hardware: ANYmal D weighs around 55 kg and uses 12 series elastic actuators capable of producing 85 N m of torque each.The platform is the hardware used for deployment experiments.
  • System implementation: Perception uses six Intel Realsense depth cameras and a Velodyne Puck LiDAR, with asynchronous processing on an NVIDIA Jetson Orin.Locomotion and navigation run synchronously in one onboard node, while perception operates asynchronously across the onboard system.
  • Deployment results: The three learning-based modules reliably enable ANYmal D to reach targets across randomized obstacle arrangements without expert demonstrations, offline computation, or a priori environmental knowledge.The deployment includes trajectories and profiles of speed, selected skills, and joint positions and torques.
  • Deployment results: The system leverages the motors’ full torque capabilities and large joint deflections to reach high speeds and overcome challenging obstacles.Figure 3 compares real-robot and simulation trajectories with speed, selected-skill, joint-angle, and torque profiles.
  • Robustness: The robot recovers from disturbances by standing after falling from a box, completing the course after slipping, and readapting when obstacles are pulled away during execution.These adaptations occur even though all components were trained only with static environments.

A. Locomotion Module

The locomotion module learns separate policies for walking, jumping, climbing up and down, and crouching, each tailored to distinct obstacle types. Across increasing obstacle difficulty, all skills perform well through 90% of training difficulty, after which performance declines, especially for crouching.

  • Jumping and Climbing: Jumping, climbing, and climbing-down policies coordinate footholds, body posture, and leg support to negotiate boxes up to 1 m high or gaps up to 1 m.The climbing-up policy can recover with its knee if a leg slips or misses the edge, preventing a fall back to the ground.
  • Crouching: The crouching policy lowers the robot’s base and adapts its gait to cross passages as narrow as 0.4 m.At low base height, both hip motors help lift the feet while the robot walks toward the target.
  • Walking: The walking policy traverses stairs, slopes, and scattered obstacles, including 40° slopes, 0.25 m steps, and flat-ground running at 2 m/s.A common policy handles these diverse irregular terrains, which resemble settings from prior perceptive legged-locomotion work,.
  • Skill Evaluation: All skills achieve strong success rates through 90% of their respective training difficulty, while crouching degrades fastest beyond that threshold.Jumping, climbing, and climbing-down skills also drop sharply at higher difficulties.

B. Navigation Module

The navigation module selects terrain-adapted sub-goals and switches locomotion policies using 3D perception and knowledge of each skill’s capabilities. It adapts trajectories to obstacle configurations, including high-speed approaches and cases where direct climbing would fail.

  • The planner extracts 3D terrain information from perception’s latent space to select sub-goals and adapt paths to obstacle dimensions.For similar environment configurations, it chooses different paths based on the obstacles’ dimensions.
  • The high-level module selects and controls the appropriate locomotion policy for the terrain, including rapid deceleration and turning on narrow obstacles.This control is particularly important when the robot arrives at high speed and must quickly reach the next obstacle.
  • The navigation module uses each skill’s capabilities and limitations to modify trajectories, avoiding climb-up or climb-down actions that would fail and climbing over tables that are too low.Its behavior changes with obstacle configuration, especially for climbing and crouching skills.
  • The method is compared with manually hard-coded trajectories across three randomly selected, near-maximum-difficulty scenarios using 1000 roll-outs each.The manual baseline hard-codes commands and skills from the obstacle sequence, requiring human expert demonstrations that do not scale when terrain is randomized.

C. Perception module

The perception module reconstructs meaningful scene geometry and navigation latents from noisy, occluded point clouds, operating asynchronously during deployment. Compared with an elevation-map baseline, it handles sparse measurements, overhangs, and state-estimation drift, while reconstruction precision decreases farther from the robot.

  • C. Perception module: The module converts noisy, occluded point clouds into a latent for navigation and a clean reconstruction for locomotion, while operating asynchronously at deployment.Training instead assumes perceptive information, reconstruction, and latent are available exactly at inference time.
  • C. Perception module: Compared with the elevation-map baseline, the network reconstructs sparse scenes more completely, including elevated surfaces and walls that lack direct measurements.It uses edge measurements to infer box tops and correctly identifies a wall, whereas the baseline omits regions above higher boxes.
  • C. Perception module: Reconstruction precision decreases farther from the robot because of lower voxel resolution and noisy obstacle edges, but the refiner improves nearby geometry and produces cleaner stairs.One distant box is estimated approximately 8 cm too wide despite having the correct height.
  • C. Perception module: Auto-regressive feedback preserves occluded geometry under overhangs, whereas the baseline mixes table-top and ground surfaces into an erroneous map during crouching.The network remembers the table’s top surface from earlier observations while the robot approaches.
  • C. Perception module: The network detects and corrects sudden state-estimation drift, while the elevation map leaves robot limbs embedded in the map.This robustness is demonstrated when the robot’s position estimate jumps and in another scenario where the hind leg appears inside the elevation map.

III. DISCUSSION … A. Overview

The paper presents a fully learned, onboard pipeline for agile quadrupedal parkour, while identifying scalability, training complexity, and navigation-convergence limitations. Its evaluation uses structured pallet-box terrains that preserve agile-navigation challenges across simulated and real-world scenarios.

  • III. DISCUSSION: The complete pipeline combines low-level locomotion skills, high-level navigation, and perception to let the robot climb, jump, and select non-trivial paths through complex scenes.The system is designed for highly challenging terrains and enables movement with unprecedented agility.
  • III. DISCUSSION: All modules use neural networks trained in simulation and transferred to hardware, enabling real-time onboard operation without pre-mapping or offline planning.Learning shifts the task’s complexity to training while keeping deployment computation relatively small.
  • A. Current Limitations: The method’s scalability remains untested beyond a limited set of scenarios and a handful of distinct environmental modules.Extending it to collapsed buildings or real parkour courses would require broader perception, navigation, and traversal capabilities.
  • A. Current Limitations: Training is time-consuming because the pipeline contains eight separately tuned, interdependent neural networks whose modifications can require retraining other modules.For example, navigation is tied to a specific perception latent tensor and locomotion policies, while perception must be retrained when skills change.
  • A. Current Limitations: Navigation requires many iterations to converge because reaching the goal involves a sequence of decisions with many failure paths.A curriculum is needed to discover correct behaviors; without it, the robot can get stuck before larger obstacles.
  • A. Overview: The task is defined as reaching a target quickly while navigating and locomoting through structured configurations of pallet-sized boxes.This setup retains the main agile-navigation challenges while remaining feasible, repeatable, and structured.
  • A. Overview: The benchmark includes randomized box arrangements requiring climbing or jumping, a winding parkour line with randomized obstacles, and a straight-line version for real-world deployment.The scenarios vary object dimensions, platform shape, obstacle sequences, parameters, and initial positions; walls and distractors are also added to test generalization.

B. Pipeline · 1) Perception Module:

The pipeline comprises three learning-based modules, with a perception module that reconstructs the robot’s surroundings from noisy, occluded point clouds for downstream planning and control. Its multi-resolution encoder-decoder combines spatially adaptive mapping with temporal evidence accumulation, trained unsupervised on simulated parkour trajectories.

  • B. Pipeline: The pipeline consists of three learning-based modules, including perception, locomotion, and navigation policies.
  • 1) Perception Module:: The perception module provides scene understanding used by navigation and locomotion for path planning, policy selection, foothold placement, and contact decisions.It estimates surrounding terrain from depth-camera and LiDAR point clouds despite sensor noise and heavy occlusion.
  • 1) Perception Module:: A multi-resolution encoder-decoder reconstructs the terrain at high resolution near the robot and lower resolution farther away, balancing locomotion detail against broader scene coverage.The two networks operate at different spatial scales; distant regions need only approximate configurations for planning and policy selection.
  • 1) Perception Module:: The encoder-decoder compresses point clouds, completes missing information, filters noise, and uses coarse-network autoregressive feedback to accumulate evidence across frames.This temporal aggregation can reconstruct occluded structures, such as a table’s top surface when the robot passes underneath.
  • 1) Perception Module:: Measurements become voxel grids with centroid features, and dense 3D convolutions scale to reinforcement-learning batches of 4000 robots but require approximately 45 GB of GPU memory.The dense formulation was chosen because the sparse implementation in did not scale adequately to this setup.
  • 1) Perception Module:: The decoder predicts voxel occupancy probabilities and cell centroids, enabling reconstruction by pruning low-probability cells; omitting skip connections worked well in randomized parkour worlds.Unlike, the method does not expose a skip-connected latent directly to navigation, though this may limit generalization.
  • 1) Perception Module:: The high-resolution network receives coarse-network features and measurements without autoregressive feedback, because temporal information is already encoded in its input.The coarse map uses 12.5 cm voxels over 4 m per axis, while the high-resolution map uses 6.25 cm voxels over 2 m.
  • 1) Perception Module:: The networks are trained unsupervised on simulated data from 2000 trajectories of 100 time-steps, split evenly across parkour scenarios, using occupancy and centroid reconstruction losses.Occupancy uses binary cross-entropy, centroids use Euclidean distance to ground truth, and augmentation follows.

2) Locomotion Module:

The locomotion module exposes separately trained walking, climbing, crouching, and jumping policies through a 50 Hz interface, using proprioception, local elevation maps, and skill-specific commands. Curricula and task-specific training conditions produce robust, transferable behaviors for varied terrain and obstacles.

  • Locomotion Module: The 50 Hz locomotion interface selects among separately trained walking, climbing-up, climbing-down, crouching, and jumping policies that output motor joint-position commands.Each policy receives proprioceptive state, a local terrain map, and an intermediate command; policies share observation and action spaces but use different rewards and termination conditions.
  • Skill Training: Walking is trained on irregular stairs, slopes, and small obstacles, while specialized skills use obstacle-focused curricula and 20% random rough terrain to produce natural gaits and better deployment performance.Walking uses 60% stairs, 20% slopes, and 20% randomized obstacles; the other skills use 80% corresponding obstacles and 20% random rough terrain. Walking scenarios resemble prior legged-locomotion settings,.
  • Skill Training: Jumping spans gaps up to 1 m, while climbing-down and climbing-up curricula reach boxes up to 1 m high.Climbing down terminates on high foot impacts to prevent unsafe simulated jumping; climbing up permits base and knee contacts, encouraging progression from knee-assisted to foot-based climbing.
  • Skill Training: Crouching traverses narrow overhanging passages as low as 0.4 m using a height curriculum, despite its top-view map being unable to distinguish tables from boxes.This ambiguity is acceptable because the policy is trained specifically for such overhanging-obstacle scenarios.

3) Navigation Module: · SUPPLEMENTARY MATERIALS · S1. Observations, actions, and rewards definitions

The navigation module steers the robot toward a target using hierarchical reinforcement learning, combining a 5 Hz navigation policy with frozen 50 Hz locomotion policies. Supplementary materials define the symbols, rewards, observations, and actions used by the locomotion and navigation modules.

  • 3) Navigation Module:: The navigation module guides the robot around terrain to reach the target within the allocated time.
  • 3) Navigation Module:: Hierarchical reinforcement learning runs navigation at 5 Hz above frozen locomotion policies operating at 50 Hz.The navigation policy receives goal, timing, robot-state, and perceptual latent information at each high-level step.
  • 3) Navigation Module:: A modified PPO policy uses Gaussian command outputs and categorical skill-selection probabilities, sampling both during training and using means during deployment.
  • 3) Navigation Module:: Rolling out actual low-level policies lets navigation account for each controller’s operating modes, capabilities, and limitations, unlike simplified inner-loop models such as.This enables decisions such as approaching a lower box when a higher one cannot be climbed and placing targets carefully in narrow passages.
  • S1. Observations, actions, and rewards definitions: The supplementary materials define the symbols used throughout the method.
  • S1. Observations, actions, and rewards definitions: The supplementary materials specify locomotion and navigation rewards, observations, and actions in dedicated tables.

S2. Implementation details · S4. Navigation across long ranges

The method uses massively parallel simulation and symmetry-aware training, while its planner can solve long-range navigation by temporarily moving away from the goal in constrained terrain.

  • S2. Implementation details: Training and perception data collection use Isaac Gym with 4096 parallel agents, while custom Warp CUDA kernels process roughly 140 million sensor rays per timestep.The kernels raycast six depth cameras and LiDAR and directly convert measurements into voxel-grid inputs without memory copies.
  • S2. Implementation details: Position-tracking locomotion training leaves trajectories unconstrained, enabling complex behaviors but producing asymmetric motions that can hinder crossing multiple obstacles.The climbing policy learns to climb forward and may turn around when facing an obstacle backward, creating unfavorable multi-obstacle situations.
  • S4. Navigation across long ranges: The planner can reach remote targets even when solving the route requires temporarily increasing its distance from the goal.This behavior is illustrated in a U-shaped terrain where a direct jump across the wide gap is unavailable.
  • S2. Implementation details: The method augments each environment transition with front-back and left-right symmetric variants for the ANYmal D robot, following the duplication method of.The corresponding representation expands an original state into four symmetric states using the robot’s X and Y symmetries.
  • S2. Implementation details: Because mirrored states can be off-policy and assign low probabilities to transformed actions, the method assigns the original-action probability to all symmetric variants.This is intended to bootstrap learning for randomly initialized policies while exploiting symmetry at convergence.
  • S4. Navigation across long ranges: In a U-shaped terrain, the planner reaches a distant goal by first moving away from it because the wide gap cannot be crossed by jumping.This demonstrates long-range planning under a terrain constraint that rules out the seemingly direct action.

S5. Ablation study of the navigation module’s output · S6. Description of the measurement blind spots · S7. Incorrect terrain reconstructions

The supplementary analysis evaluates navigation-policy action-space components and documents perception blind spots and incorrect terrain reconstructions. These reconstruction errors arise under occlusion but can be corrected as visibility improves without impeding navigation or locomotion.

  • S5. Ablation study of the navigation module’s output: Table S6 compares navigation-policy variants that remove the time command, heading command, or both under the same comparison conditions.The ablation isolates the contributions of timer and heading outputs to low-level policy control.
  • S6. Description of the measurement blind spots: During box climbing, the sensors initially miss the top surface, while limbs create large occluded regions during the climb.The perception module estimates the box height from points along the front-surface edge to reconstruct the unseen top.
  • S6. Description of the measurement blind spots: The camera arrangement also creates blind spots immediately below the robot, limiting perception during locomotion.These blind spots are attributed to the sensors’ placement relative to the robot and its limbs.
  • S7. Incorrect terrain reconstructions: The terrain network sometimes hallucinates a stair behind the first box in an occluded region, probably because of dataset imbalance.The navigation and locomotion modules are not impeded because the network quickly corrects the reconstruction after gaining a better view.
  • S7. Incorrect terrain reconstructions: When the robot crawls underneath the table, the network sometimes inflates the table’s shape.This is another incorrect reconstruction documented alongside the hallucinated stair.
  • S7. Incorrect terrain reconstructions: Incorrect reconstructions occur in measurement blind spots, but the network can recover once the scene becomes more visible.The reported recovery prevents these errors from impeding navigation and locomotion performance.
Loading 2306.14874v1…