PaperlayerView exact revision on arXiv

Source-linked AI summary

ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots

David Hoeller, Nikita Rudin, Dhionis Sako, Marco Hutter

arXiv:2306.14874v1cs.RO

TL;DR

Agile quadrupedal navigation is challenging because robots must use different motions for different obstacles while sensing only a partial scene. This paper combines learned locomotion, navigation, and perception modules; in real-world experiments, the robot crossed consecutive challenging obstacles at speeds up to two meters per second.

  • Problem

    Quadrupedal robots must use different motions and limb contacts for different obstacles, while self-occlusions and limited sensor field of view provide only a partial scene.

    Source: §I · p. 1

  • Method

    The pipeline combines separately trained reinforcement-learning locomotion skills with high-level navigation and perception modules.

    Source: §III · p. 10, 2) Locomotion Module · p. 14

  • Results

    The robot navigates complex scenes by climbing and jumping large obstacles while selecting a non-trivial path toward its target.

    Source: §III · p. 10

  • Takeaways & Limitations

    The task is solved without pre-mapping or offline planning, with required computations performed onboard in real time.

    Source: §III · p. 10

    Scalability to more diverse scenarios remains untested, and the system has been demonstrated only in a limited range of scenarios.

    Source: A. Current Limitations · p. 12

Abstract

from arXiv · show

Performing agile navigation with four-legged robots is a challenging task due to the highly dynamic motions, contacts with various parts of the robot, and the limited field of view of the perception sensors. In this paper, we propose a fully-learned approach to train such robots and conquer scenarios that are reminiscent of parkour challenges. The method involves training advanced locomotion skills for several types of obstacles, such as walking, jumping, climbing, and crouching, and then using a high-level policy to select and control those skills across the terrain. Thanks to our hierarchical formulation, the navigation policy is aware of the capabilities of each skill, and it will adapt its behavior depending on the scenario at hand. Additionally, a perception module is trained to reconstruct obstacles from highly occluded and noisy sensory data and endows the pipeline with scene understanding. Compared to previous attempts, our method can plan a path for challenging scenarios without expert demonstration, offline computation, a priori knowledge of the environment, or taking contacts explicitly into account. While these modules are trained from simulated data only, our real-world experiments demonstrate successful transfer on hardware, where the robot navigates and crosses consecutive challenging obstacles with speeds of up to two meters per second. The supplementary video can be found on the project website: https://sites.google.com/leggedrobotics.com/agile-navigation

I. INTRODUCTION · A. Method overview

The paper presents a three-part learned pipeline for quadrupedal parkour navigation, linking scene perception, obstacle-specific locomotion skills, and skill selection. It addresses the need to navigate dynamic, partially observed terrain in real time while coordinating distinct motions and contacts.

I. INTRODUCTION

  • Parkour coordinates walking, running, climbing, and jumping in timed sequences; the authors use the goal of matching free-runners’ agility to examine robot capabilities and motivate applications such as search and rescue.Source: §I · p. 1
  • Agile navigation requires sensing a changing scene, finding a feasible path and motion sequence, and controlling dynamic maneuvers in real time from partial, noisy sensor data.The task also imposes limited onboard computing and tight latency requirements.Source: §I · p. 1
  • Parkour-like navigation challenges conventional mobile-robot planning because locomotion must vary by obstacle, perception suffers from self-occlusion and limited field of view, and plans must respect robot and controller capabilities.Source: §I · p. 1
  • High-impact forces and contacts on different robot parts can cause substantial state-estimation drift.Source: §I · p. 1

A. Method overview

  • The method splits agile navigation into perception, locomotion, and navigation modules that estimate the scene, provide obstacle-specific skills, and select skills with intermediate commands.The modules are trained in simulation; the full pipeline is deployed on ANYmal D.Source: A. Method overview · pp. 1, 3, Figure 1 · p. 2
  • Five locomotion policies walk on irregular terrain, jump gaps, climb obstacles, and crouch through narrow passages.The locomotion policies train on single obstacles, while perception and navigation train on varied arrangements.Source: A. Method overview · p. 3
  • Randomized training fields include stairs, inclined surfaces, boxes, gaps, and tables, with locomotion policies trained on single obstacles and perception and navigation on varied arrangements.Source: A. Method overview · p. 3
  • After simulation training, all modules are deployed in the real world.Source: A. Method overview · p. 3

B. Contributions · C. Related work

The paper presents a learned quadrupedal navigation system that combines terrain reconstruction, locomotion skills, and skill-aware planning, then demonstrates autonomous sim-to-real operation on challenging courses. Its contributions address limitations in prior locomotion, navigation, and perception approaches.

B. Contributions

  • 2 m/s: the complete system autonomously crosses difficult terrain and reaches targets on real hardware despite fast-motion sensing and state-estimation noise.The locomotion controllers make precise movements on narrow boxes and use the robot’s full range of motion to pass higher obstacles; mapping updates frequently and the planner selects feasible paths using skill capabilities.Source: B. Contributions · p. 3
  • The learned navigation policy plans through intricate scenes from a terrain-reconstruction belief state while selecting among locomotion skills, with inference in milliseconds.The modified PPO [1] uses a hybrid actor with Gaussian low-level commands and categorical skill selection.Source: B. Contributions · p. 3
  • The locomotion skills extend the position-based formulation in with new terrains, heading commands, and symmetry augmentation to improve policy performance.Source: B. Contributions · p. 3
  • The terrain-reconstruction method combines precise near-robot and coarse larger-scale maps, supports efficient large-batch inference, and handles overhanging obstacles.It is inspired by; the paper notes that elevation maps cannot represent full 3D geometry or extrapolate beyond visible data, limiting passage under obstacles and reconstruction of higher obstacle tops.Source: B. Contributions · p. 3, C. Related work · p. 5
  • The authors deploy all modules on ANYmal D and test varied obstacle arrangements indoors and outdoors.The complete pipeline is designed to scale during simulation training and operate in real time on the robot.Source: B. Contributions · p. 3

a) Locomotion

  • Prior locomotion work includes MPC for precise footholds and learned robust locomotion, alongside quadruped jumping and climbing and biped walking and jumping demonstrations.MPC is limited by its model, can fail under slippage or imperfect perception, and assumes foot-contact schedules that exclude other environmental contacts; learned locomotion is not yet exploiting its full potential.Source: C. Related work · p. 3

b) Navigation and Hierarchical Learning

  • Sampling-based navigation is difficult for legged robots’ hybrid contacts, and existing methods simplify the problem through assumptions that constrain feasible motions or deployment.The approach in takes seconds, requires prior environment knowledge, and produces statically stable motions; assumes continuous foot contact, while plans offline, hard-codes jumping, and handles obstacles only up to 0.1 m at 0.33 m/s.Source: C. Related work · pp. 3–4
  • Unlike prior agile quadruped navigation, which requires human-designed paths and skill selection in one pre-mapped environment, this work claims navigation without a priori planning or mapping.Prior learning-based navigation includes demonstrations, reinforcement learning, and self-supervision; requires a global map and cannot plan beneath obstacles. Earlier hierarchical learning combines or switches skills –, while this work trains skills using and learns to steer and switch between them, similar to.Source: C. Related work · pp. 4–5

II. RESULTS

On ANYmal D, the learned modules reached targets across randomized obstacle arrangements at up to 2 m/s; the pipeline also recovered from falls and readapted when obstacles changed during execution.

II. RESULTS

  • Up to 2 m/s, ANYmal D reached targets across randomized obstacle arrangements without demonstrations, offline computation, or prior environment knowledge, selecting suitable skills while using large joint deflections and motor torque.The robot underwent fast accelerations and decelerations, and the deployment profiles show use of the motors’ torque capabilities and large joint deflections to cross obstacles.Source: §II · p. 5, Figure 3 · p. 6
  • At 1.5 m/s, the robot precisely placed footholds on a 0.8 m box smaller than its standing footprint, despite the jumping skill’s training on boxes twice as large.It passed the last step and prepared to jump, demonstrating that the low-level skills handled a more intricate scene than their training scenarios.Source: §II · p. 5
  • In narrow passages, navigation repurposed the jumping skill to turn in place, exploiting a capability learned from randomized-heading jumps; low-level skill switches were smooth and unnoticeable.The skill was selected after the first step down or before the climb, although it was not designed for those specific settings.Source: §II · p. 5
  • After falls and heavy slipping, the robot completed the course; it also rapidly readapted its trajectory when obstacles were moved during execution despite static-environment training.The perception module corrected its output when its belief mismatched current measurements, enabling the pipeline to respond to changes.Source: §II · p. 7

A. Locomotion Module

The locomotion policies learn distinct maneuvers for jumping, climbing, crouching, and walking, with a common walking policy handling varied terrain. All skills perform well through 90% of their respective training difficulty.

a) Jumping

  • For gaps up to 1 m, jumping approaches sideways, launches with three legs, lands with the fourth, then transfers the remaining legs; weight shifts help recover missteps and slippage.The policy keeps its feet at a safe distance from the edge through randomization.Source: A. Locomotion Module · p. 7

b) Climbing Down

  • From boxes up to 1 m high, climbing down lowers the robot’s center of gravity, lands on its front legs with rear knees supported, then uses its knees and shanks as edge hooks.It pushes its feet slowly over the edge until the knees make contact, helping it tolerate small shifts in perceived terrain.Source: A. Locomotion Module · p. 7

c) Climbing Up

  • Climbing up uses a front-foot lift, hind-leg landing, and fourth-leg bracing before bringing the body up; knee contact lets the robot recover from a slipping or missed leg.The policy climbs boxes up to 1 m high and learns to use its knees when needed.Source: A. Locomotion Module · p. 7

d) Crouching

  • To cross a passage at least 0.4 m high, the crouching policy lowers its base while walking and adapts its gait, using both hip motors to lift its feet.Source: A. Locomotion Module · p. 7

e) Walking

  • A common walking policy traverses stairs, slopes, and small obstacles, scales and descends 40° slopes, climbs 0.25 m steps, runs at 2 m/s, and generalizes to unseen terrain.The unseen terrain includes narrow stairs, slopes, and combinations of obstacles.Source: A. Locomotion Module · p. 7

Locomotion skill performance

  • All skills perform well through 90% of their respective training difficulty (Fig. 4F); beyond that, crouching declines fastest, jumping and climbing skills drop sharply, while walking extrapolates.Crouching drops fastest when the passage becomes narrower than the robot’s height; jumping, climbing, and climbing down drop due to the robot’s physical limits.Source: A. Locomotion Module · p. 7

B. Navigation Module

The navigation module selects terrain-dependent routes and coordinates five locomotion skills, adapting its behavior to obstacle geometry and skill limitations rather than relying on fixed demonstrations.

B. Navigation Module

  • The high-level policy controls all five skills by coordinating position, heading, and timing commands, including rapid deceleration and on-the-spot turns before narrow obstacles.Each low-level policy can move freely during its allocated time and must satisfy position and heading commands when that time ends.Source: B. Navigation Module · pp. 7–8
  • As box height rises, the policy increasingly favors longer, safer routes, switching sooner on ascent than descent; on descent it sends the robot directly until 1 m.The faster switch on ascent is attributed to training disturbances having a stronger impact on the climb-up skill; for overly high boxes it avoids direct climbing, and for low tables it climbs over rather than crouching underneath.Source: B. Navigation Module · p. 8, Figure 5 · p. 9
  • Manual hard-coded trajectories succeed in some terrains but fail in others requiring finer control, while learned target placement can increase speed and avoids demonstrations that do not scale across randomized terrain.Demonstrations placing targets at obstacle midpoints take longer to reach the goal than the learned policy’s farther targets.Source: B. Navigation Module · pp. 8, 10, Table I · p. 10

C. Perception module

The perception module turns noisy, occluded measurements into scene reconstructions for navigation and locomotion, using temporal feedback and local refinement to handle difficult terrain and state-estimation drift. Its asynchronous operation on hardware does not appear to affect performance.

C. Perception module

  • Although perception runs asynchronously on hardware, the resulting delays do not seem to affect system performance.During training, reconstruction and latent outputs are assumed to be available at inference time.Source: C. Perception module · p. 10
  • The network reconstructs scene layout from sparse measurements, identifying box tops and vertical walls that the elevation-map baseline misses.The baseline does not represent vertical surfaces and leaves unmeasured regions atop higher boxes unmapped.Source: C. Perception module · p. 10
  • The coarse map is less precise farther from the robot, but the nearby refiner corrects inaccuracies and produces cleaner stairs.In scenario (A), the left box is estimated about 8 cm too wide despite its correct height.Source: C. Perception module · p. 10
  • Autoregressive feedback preserves the table’s top surface under sparse observations, while the baseline produces an erroneous map of the overhang.The network retains information from when the surface was visible during the robot’s approach.Source: C. Perception module · p. 10
  • The network detects sudden position-estimation drift and corrects the map, whereas the elevation map places the robot’s legs inside the map.This behavior is shown for drift in scenarios (B) and (D).Source: C. Perception module · p. 10

III. DISCUSSION … A. Overview

The paper presents a fully learned parkour pipeline combining locomotion skills, navigation, and perception, with simulation training and real-world transfer. It also identifies limits in scenario generalization, component retraining, and navigation-policy convergence, and evaluates the approach in structured box terrains.

III. DISCUSSION

  • The pipeline combines specialized locomotion skills, high-level navigation, and terrain perception to let legged robots climb and jump through complex scenes toward a target.The task requires actuation-limit locomotion, navigation aware of skill capabilities, and reconstruction of terrain topology from partial sensor observations.Source: §III · p. 10
  • All modules use neural networks trained in simulation and transferred to real-world operation, solving the task without pre-mapping or offline planning using onboard real-time computation.The authors argue that learning shifts complexity to training and enables small networks to produce complex behaviors at low computational cost.Source: §III · p. 10

A. Current Limitations

  • The method’s scalability remains untested beyond a limited range of scenarios and modules, leaving its generalization to new settings such as collapsed buildings or real parkour courses uncertain.Broader deployments would require handling more obstacle types, with additional skills, perception data, and navigation training potentially needed.Source: A. Current Limitations · p. 12
  • Training is time-consuming because eight separately tuned networks are interdependent, and changing a perception module, skill, or obstacle can require retraining other components.The navigation module depends on the specific perception latent tensor and locomotion policies it was trained with; simultaneous component training is proposed as a possible future need.Source: A. Current Limitations · p. 12
  • The navigation policy needs many iterations to converge because many decisions can lead to failure, so the authors use a curriculum to prevent getting stuck before larger obstacles.They suggest expert-demonstration pretraining, potentially using brute-force search for candidate solutions, as an alternative.Source: A. Current Limitations · p. 12

A. Overview

  • The agent must reach a target quickly in repeatable environments built from pallet-sized boxes, preserving agile-navigation challenges in a structured task.Terrain dimensions and target or robot starting positions are randomized in some configurations.Source: A. Overview · p. 12
  • Terrain A uses randomized box arrangements requiring climb-and-jump route choices, while terrain B tests traversal of randomized obstacle sequences on a winding platform with only one path.Scenario A tests navigation choices based on skill capabilities and target location; scenario B tests generalization across platform shapes and obstacle arrangements.Source: A. Overview · p. 12
  • Terrain C is a straight-line obstacle course for real-world deployment that prevents walking around obstacles and uses a separate navigation policy.Obstacle sequences and parameters are randomized, with walls and distracting objects beside the line.Source: A. Overview · p. 12

B. Pipeline · 1) Perception Module:

The pipeline comprises three learning-based modules; this section describes perception, which reconstructs terrain from noisy, occluded sensor data for downstream navigation and locomotion decisions.

B. Pipeline

  • The pipeline has three learning-based modules, with supplementary sections S1 and S2 specifying locomotion and navigation policy observations, actions, rewards, and implementation details.Source: B. Pipeline · p. 12

1) Perception Module:

  • The perception module supplies terrain estimates from depth-camera and LiDAR point clouds for path planning, policy selection, foothold placement, and contact decisions.Its inputs are noisy and heavily occluded by obstacles or the robot itself.Source: 1) Perception Module · p. 12
  • A two-network, multi-resolution encoder-decoder balances map size and reconstruction accuracy, using a smaller, higher-resolution map nearby and a broader, lower-resolution map farther away.The nearby detail supports locomotion, while approximate distant scene layouts suffice for navigation planning and policy selection.Source: 1) Perception Module · pp. 12–13
  • The coarse network uses autoregressive feedback to combine transformed prior outputs with current measurements, accumulating evidence to reconstruct occluded scene elements over time.This can recover a table’s top surface after the robot passes beneath it and helps when limbs block camera views during climbing.Source: 1) Perception Module · p. 13
  • The system voxelizes measurements around the robot and applies dense 3D convolutions, which scale to batches of 4,000 robots but require approximately 45 GB of GPU memory.Unlike the sparse implementation in, dense convolutions worked at the scale required by this reinforcement-learning setup.Source: 1) Perception Module · pp. 13–14
  • The decoder predicts voxel occupancy probabilities and cell centroids for point-cloud reconstruction; it omits skip connections, which may limit generalization but worked in randomized parkour worlds.Cells below a user-defined occupancy threshold are pruned.Source: 1) Perception Module · p. 14
  • The coarse map uses 12.5 cm voxels over 4 m per axis, while the high-resolution map uses 6.25 cm voxels over 2 m and combines coarse-network features with point-cloud measurements.The high-resolution network has no autoregressive feedback because its input already contains temporal information.Source: 1) Perception Module · p. 14
  • The perception networks are trained unsupervised on 2,000 simulated trajectories of 100 time steps each, split evenly across parkour scenarios, with occupancy and centroid losses.Data augmentation perturbs point positions, adds random blobs, removes point patches, and noisifies robot position; the authors report it is key to robustness against noise and drift.Source: 1) Perception Module · p. 14

2) Locomotion Module:

The locomotion module exposes five reinforcement-learned motor-command skills at 50 Hz. Their target-directed training and terrain-map inputs are adapted to support robust navigation.

Locomotion module

  • A 50-Hz interface activates policies for walking, climbing up and down, crouching, or jumping, which output motor joint-position commands.The policies receive proprioception, a local terrain map, and an intermediate command.Source: 2) Locomotion Module · p. 14

Training setup

  • Policies share observation and action spaces but use skill-specific rewards and termination, with position, time, and heading targets plus symmetry augmentation for more robust motion.To reduce computational cost, they use 2.5D elevation maps perturbed with point noise and shifts up to 7.5 cm. Walking training uses 60% stairs, 20% slopes, and 20% randomized obstacles; other skills use 80% corresponding obstacles and 20% rough terrain.Source: 2) Locomotion Module · p. 14

Walking

  • Walking trains traversal over irregular terrain including stairs, slopes, and randomly placed small obstacles.Source: 2) Locomotion Module · p. 14

Jumping

Climbing down

  • Climbing down from boxes up to 1 m uses a height curriculum and high-foot-impact termination, preventing simulation-only jumps that could damage the real robot.Source: 2) Locomotion Module · p. 14

Climbing up

  • Climbing up onto boxes up to 1 m uses a height curriculum and reduced base and knee penalties, encouraging progression from knee-assisted climbs to foot use when possible.Source: 2) Locomotion Module · p. 14

Crouching

  • Crouching trains passage through narrow overhangs at least 0.4 m high, using a passage-height curriculum; although the top-down map cannot distinguish tables from boxes, the policy learns to go underneath.Source: 2) Locomotion Module · p. 14

3) Navigation Module: · SUPPLEMENTARY MATERIALS · S1. Observations, actions, and rewards definitions

The navigation module combines a high-level policy with frozen low-level locomotion skills to guide the robot to its goal, while supplementary tables define the symbols, rewards, observations, and actions used by the system.

3) Navigation Module:

  • The navigation policy guides the robot across terrain to reach its target within the allocated time.It operates in a hierarchical reinforcement-learning setup: the navigation policy runs at 5 Hz and the frozen locomotion module at 50 Hz.Source: 3) Navigation Module · pp. 14–15
  • The policy observes the goal’s relative position, remaining time, robot state, and perception latent tensor, then selects a skill and issues local position, heading, and time commands.A sparse distance-to-goal penalty is applied only at the final timestep; episodes also end if the robot falls or contact forces are too high, and a curriculum progressively moves targets farther away.Source: 3) Navigation Module · p. 15
  • The modified PPO actor uses a hybrid output: Gaussian commands and categorical skill probabilities, sampled during training and deployed using their mean and highest-probability skill.The time-dependent command formulation follows.Source: 3) Navigation Module · p. 15
  • Rolling out actual low-level policies during training, rather than simplified kinematic models as in, lets the navigator account for each controller’s capabilities and limitations.This enables behaviors such as choosing lower boxes when one is too high to climb, precise target placement in narrow passages, and favoring climb-down to reduce contact forces; it must also coordinate time, position, and heading commands.Source: 3) Navigation Module · p. 15

S1. Observations, actions, and rewards definitions

  • Supplementary Tables S1–S3 define system symbols and provide locomotion and navigation reward specifications.The supplied table passages identify their topics but do not provide reward values.Source: Table S1 · p. 17, Table S2 · p. 17, Table S3 · p. 17
  • Supplementary Tables S4–S5 document the locomotion and navigation observations and actions.The supplied captions identify the table scope but do not provide the listed definitions.Source: Table S4 · p. 17, Table S5 · p. 17

S2. Implementation details

The policies and perception data are generated in Isaac Gym with custom CUDA processing, while real-robot perception is adapted to reduce sensor delay. Symmetry augmentation addresses asymmetric locomotion behaviors and supports learning symmetric actions.

Implementation details

  • 4096 agents are trained in parallel in Isaac Gym, while custom Warp CUDA kernels process about 140 million camera and LiDAR rays per time step into voxel-grid inputs without copying memory.The kernels raycast six depth cameras and a LiDAR for each robot.Source: S2. Implementation details · p. 17
  • Switching from delayed Realsense point-cloud messages to depth images reduces perception delay from up to 250 ms to 25 ms on the real robot.The node projects depth images to point clouds and merges them with LiDAR measurements.Source: S2. Implementation details · p. 17
  • Position-tracking training permits unconstrained trajectories but can produce asymmetric behavior, such as a climbing policy that only climbs forward and turns around when facing an obstacle backward.This preference can be unfavorable when crossing multiple obstacles that require different skills.Source: S2. Implementation details · p. 17
  • The planner can reach remote targets even when it must first move farther away from the goal.This capability is illustrated in Fig. S2.Source: Figure S2 · p. 18
  • Using the duplication method, the authors augment transitions with front-back and left-right symmetric variants of observations and actions, as illustrated in Fig. S1.They assign the original action's probability to all symmetric variants to bootstrap learning and address poor convergence from off-policy mirrored states.Source: Figure S1 · p. 17, S2. Implementation details · p. 18

S4. Navigation across long ranges … S7. Incorrect terrain reconstructions

Heading and timing commands improve navigation in different terrain conditions, while long-range planning can require moving away from the goal. The supplement also describes perception blind spots and terrain reconstruction errors that the system can recover from or tolerate.

S4. Navigation across long ranges

  • In a U-shaped terrain with a distant goal, the planner recognizes that it cannot jump the wide gap and first moves away from the target.This demonstrates that reaching the goal may require increasing distance from it before proceeding.Source: S4. Navigation across long ranges · p. 18

S5. Ablation study of the navigation module’s output

  • Heading and time outputs improve navigation: heading helps rapid repeated turns, while timing benefits longer terrains by enabling fast motion on simple obstacles and slower motion in risky parts.The comparison tests removing time, heading, or both outputs against the same conditions as the manually coded trajectory comparison.Source: Table S6 · p. 18, S5. Ablation study of the navigation… · p. 18

S6. Description of the measurement blind spots

S7. Incorrect terrain reconstructions

  • When climbing the first box, the network can hallucinate a stair behind it in a measurement blind spot, probably due to dataset imbalance, but corrects the error once it has a better view.This erroneous reconstruction does not impede navigation or locomotion performance.Source: Figure S4 · p. 18, S7. Incorrect terrain reconstructions · p. 19
  • When the robot crawls under a table, the network can inflate its shape because of sparse top-surface measurements and state-estimation drift, especially during crouching.Even in the worst case, the robot stays crouched longer rather than failing to complete the task.Source: Figure S4 · p. 18, S7. Incorrect terrain reconstructions · p. 19

Related papers

Loading 2306.14874v1…