PaperlayerView exact revision on arXiv

Source-linked AI summary

AME-2: Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding

Chong Zhang, Victor Klemm, Fan Yang, Marco Hutter

arXiv:2601.08485v2cs.RO

TL;DR

Agile, generalized legged locomotion remains difficult because existing approaches often trade agility against generalization, interpretability, and robustness to occlusions. AME-2 combines attention-based map encoding with uncertainty-aware neural mapping and teacher–student reinforcement learning, and demonstrates agile, generalized behavior on quadruped and biped robots across challenging terrains.

  • Problem

    Existing locomotion methods often trade off agility, generalization, mapping efficiency, and interpretability, while struggling with occlusions and sparse footholds.

  • Method

    AME-2 uses an attention-based map encoder with learned local elevations and uncertainties, fused with odometry and trained through teacher–student reinforcement learning with online mapping.

  • Results

    The controllers demonstrate strong agility and robust generalization on an ANYmal-D quadruped and LimX TRON1 biped in simulation and real-world experiments.

  • Takeaways & Limitations

    The framework combines agile and generalized locomotion across diverse terrains under the same training setup for quadruped and biped robots.

  • Takeaways & Limitations

    The method uses 2.5-D elevation maps, is not designed for severely degraded perception such as high grass or snow, and can fail in highly dynamic environments with occlusions.

Abstract

from arXiv · show

Achieving agile and generalized legged locomotion across terrains requires tight integration of perception and control, especially under occlusions and sparse footholds. Existing methods have demonstrated agility on parkour courses but often rely on end-to-end sensorimotor models with limited generalization and interpretability. By contrast, methods targeting generalized locomotion typically exhibit limited agility and struggle with visual occlusions. We introduce AME-2, a unified reinforcement learning (RL) framework for agile and generalized locomotion that incorporates a novel attention-based map encoder in the control policy. This encoder extracts local and global mapping features and uses attention mechanisms to focus on salient regions, producing an interpretable and generalized embedding for RL-based control. We further propose a learning-based mapping pipeline that provides fast, uncertainty-aware terrain representations robust to noise and occlusions, serving as policy inputs. It uses neural networks to convert depth observations into local elevations with uncertainties, and fuses them with odometry. The pipeline also integrates with parallel simulation so that we can train controllers with online mapping, aiding sim-to-real transfer. We validate AME-2 with the proposed mapping pipeline on a quadruped and a biped robot, and the resulting controllers demonstrate strong agility and generalization to unseen terrains in simulation and in real-world experiments.

I. INTRODUCTION

AME-2 addresses the trade-off between agile locomotion and generalized, interpretable behavior by combining attention-based map encoding with learned uncertainty-aware mapping. The framework is evaluated on quadruped and biped robots in simulation and real-world experiments, where it demonstrates agility and generalization across diverse terrains.

  • Motivation: Existing perceptive-locomotion systems struggle to combine agility, generalization, mapping efficiency, and interpretability, especially under occlusions, noise, and sparse footholds.Classical pipelines can be computationally heavy and terrain-specific, while end-to-end sensorimotor policies often generalize poorly and encode terrain reasoning implicitly.
  • Proposed framework: AME-2 uses global terrain context and proprioception to attention-weight local map features, forming a terrain-aware representation for reinforcement-learning control.The encoder can downweight less relevant regions and learn different attention and motion patterns across terrains.
  • Proposed framework: The learned mapping pipeline predicts local elevations and per-cell uncertainties from depth observations, then fuses them with odometry into a fast representation robust to noise and occlusions.The resulting egocentric elevations and uncertainties are provided as controller inputs.
  • Training and deployment: A teacher–student training scheme uses ground-truth maps for the teacher and the learned mapping pipeline for the student, with online mapping running in parallel simulation and deployment.This setup avoids relying on idealized or hand-tuned maps during training and supports sim-to-real transfer.
  • Evaluation: AME-2 is evaluated on an ANYmal-D quadruped and LimX TRON1 biped, with controllers showing strong agility and robust generalization across diverse terrains in simulation and on hardware.The mapping pipeline produces real-time elevation maps using onboard sensing and computation.

B. Perceptive Locomotion with Raw Sensor Data

The paper positions AME-2 between explicit mapping and monolithic sensor-to-action policies by using neural networks for mapping while retaining an explicit terrain representation. Its architecture combines map and proprioceptive embeddings through attention to generate joint actions for goal-reaching locomotion.

  • Motivation: Monolithic sensor-to-action policies can be agile but often have limited generalization and interpretability, whereas explicit mapping pipelines may be slower and terrain-specific.AME-2 uses learned mapping rather than a single network that directly converts raw sensor streams to joint commands.
  • Learning-Based Mapping: The learned mapping module predicts local elevation and uncertainty from depth-derived grids and fuses these predictions over time with odometry.This design is intended to suppress sensor noise and represent occluded regions as high uncertainty.
  • Learning-Based Mapping: The system trains goal-reaching policies with PPO in parallel simulation, first using a privileged teacher with ground-truth maps and then a deployable student using neural mapping under teacher supervision.The policies produce joint-level actions for reaching position and heading goals.
  • Policy Architecture and AME-2 Encoder: The AME-2 architecture encodes proprioception and mapping separately, applies attention to map features, and feeds their combined embeddings to an MLP action decoder.The encoder extracts local and global map features, while global context and proprioception condition the weighting of local features.
  • Policy Architecture and AME-2 Encoder: Compared with AME-1, AME-2 additionally computes global features and uses them to weight local features for more generalized locomotion over complex terrains.The paper reports a performance gap between the two encoders in later evaluations.

B. Asymmetric Actor Critic

The asymmetric actor–critic setup uses different information and architectures for teacher-side policy learning and critic optimization, while rewards combine task, stability, and simulation-fidelity objectives.

  • Asymmetric Actor Critic: The critic receives contact states unavailable to the teacher and uses a computationally efficient mixture-of-experts design instead of attention over local features.The teacher is trained to generalize beyond training terrains, whereas the critic need not generalize and therefore avoids the costly attention module.
  • Asymmetric Actor Critic: Teacher–Student RL is used because online mapping makes simulation roughly half as fast and requires significantly more GPU memory than ground-truth mapping.Student training combines PPO, action distillation, and teacher–student map-embedding losses, with the PPO surrogate loss initially disabled.
  • Rewards: The reward design combines task rewards for reaching and standing at goals, regularization rewards for stability, and simulation-fidelity rewards for avoiding near-limit joint states.Task terms cover position tracking, heading tracking, moving toward the goal, and standing at the goal.
  • Rewards: Tracking rewards encourage the desired goal position and heading at episode end without constraining the route, allowing complex locomotion skills to emerge.The moving-to-goal reward is active when the robot is near the goal or moving roughly toward it, subject to the specified velocity bounds.
  • Rewards: The standing reward promotes a static, upright configuration at the goal using foot-contact, base-tilt, and joint-deviation terms.The reward definitions use dfoot, z, and dq to characterize contact, tilt relative to gravity, and deviation from the standing reference.
  • Rewards: Instead of terrain-specific foothold rewards, the method rewards all robot links to enable emergent whole-body contacts across climbing and sparse terrains.This accommodates useful knee contacts and near-edge placements for climbing while avoiding undesirable near-edge placements on sparse terrain.

2) Termination:

Episodes terminate early for unsafe orientation, excessive base collision, damaging thigh impacts, or stagnation away from the goal, and these terminations incur penalties.

  • Termination: Early termination occurs when the projected gravity vector indicates bad orientation, including a flipped robot.The thresholds are |[gb]x| > 0.985, |[gb]y| > 0.7, or [gb]z > 0.0.
  • Termination: A base collision terminates the episode when contact force on the base exceeds the robot’s total weight.This condition is intended to identify severe base impacts during locomotion.
  • Termination: High thigh acceleration during foot contact triggers termination to reduce jump-landing impacts that could damage hardware or shorten its lifespan.The thresholds are informed by typical dog and human biokinetic acceleration data.
  • Termination: Stagnation terminates an episode when movement over the past 5 s is below 0.5 m while the robot remains more than 1 m from the goal.Any listed termination condition ends the episode immediately and applies early termination penalties.

3) Terrains and Curriculum:

Controllers train on primitive terrains and generalize to complex test terrains through curriculum learning, perception-noise scheduling, domain randomization, and randomized map access.

  • Terrains and Curriculum: Controllers are trained on primitive terrains and evaluated for generalization on complex test terrains.The terrain curriculum scales difficulty from easy to hard using an exponential moving average of success rate.
  • Terrains and Curriculum: Environment difficulty advances when the robot reaches the goal and estimated success exceeds 0.5, and decreases when it remains more than 4 m from the goal.After the highest level is passed, the environment resets to a random difficulty.
  • Terrains and Curriculum: During the first 20% of teacher-training iterations, mapping noise increases from zero to its maximum while initial heading expands to random yaw in [−π, π].This jointly increases perception difficulty and variation in the robot’s initial orientation.
  • Domain Randomization: Domain randomization varies robot dynamics, observation quality, and mapping conditions to improve robustness and facilitate sim-to-real transfer.Randomized factors include payload, friction, actuation delays, observation noise, missing depth points, sensor artifacts, and mapping drifts.
  • Domain Randomization: During student training, some environments provide complete maps while others use partial online maps, enabling map reuse when traversing the same terrain repeatedly.Mapping is further corrupted by removing points, assigning random heights with high uncertainties, and simulating drift.

E. Training and Deployment

Training and deployment use parallel simulation, domain-specific map sizes, lightweight onboard inference, and an uncertainty-aware neural mapping pipeline that supplies controller inputs from fused global maps.

  • Training and Deployment: Teacher policies train for 80000 iterations and student policies for 40000 iterations, with the student surrogate loss disabled for the first 5000 iterations.Training costs are approximately 60 RTX-4090-days for ANYmal-D and 30 RTX-4090-days for TRON1.
  • Training and Deployment: The map uses 36×14 cells for ANYmal-D and 18×13 cells for TRON1, both at 8-cm resolution, with centers selected from robot and terrain dimensions.The different map sizes account for the robots’ dimensions and terrain sizes and contribute to the difference in training cost.
  • Training and Deployment: ONNX Runtime deploys the controllers with approximately 2 ms policy inference time on onboard Intel Core i7-8850H CPUs.Actuator dynamics are modeled in simulation and applied torques are clipped using joint torque–velocity constraints.
  • Training and Deployment: The actor clips observed goal distance at 2 m, removes remaining episode time, and randomizes observed yaw beyond that distance to support continuous deployment.The critic instead receives the full relative position, relative-yaw sine and cosine, and remaining episode time.
  • Mapping Pipeline: Each depth frame becomes a local elevation grid, a lightweight Bayesian CNN predicts elevations with uncertainties, and odometry fuses predictions into a global map queried by the controller.The local grid keeps the maximum z-value per cell and assigns a fixed minimum to cells without points.
  • Mapping Pipeline: The mapping pipeline runs in real time on hardware and across thousands of parallel simulation environments, allowing identical mapping during training and deployment.Its design supports rapid updates, parallel simulation, and uncertainty-aware handling of occluded regions.
  • Mapping Pipeline: The global map contains elevation and variance layers and uses a probabilistic winner-take-all update to prevent repeated uncertain observations from becoming over-confident.Updates are stochastic and depend on effective measurement variance and relative precision.
  • Mapping Pipeline: Mapping-model training samples elevation grids from four terrain-mesh categories and random poses using raytracing rather than physical simulation.The mesh set includes locomotion training terrains, stacked boxes, random heightfields, and floating boxes.

B. Training

The mapping model is trained on procedurally sampled terrain grids and augmented observations to reconstruct elevations from noisy, partially observed inputs.

  • B. Training: The mapping model is trained on local elevation grids sampled from locomotion and procedurally generated terrain meshes.Raytracing enables sampling hundreds of thousands of frames per second on one RTX 4090 GPU.
  • B. Training: Training inputs are augmented with noise, cropping, simulated occlusions, clipped elevations, missing points, and outliers.The original ground-truth elevations serve as labels for these diverse partially observable grids.

3) Model and Optimization:

The mapping pipeline combines uncertainty-aware elevation reconstruction, terrain-sensitive training weights, and efficient deployment for simulated and real-time locomotion.

  • 3) Model and Optimization:: The model reconstructs ground-truth elevations with β-NLL loss, predicting both an elevation estimate and its variance.With β = 0.5, the formulation discourages trivial uncertainty inflation and supports high uncertainty when accurate prediction is impossible.
  • 3) Model and Optimization:: Samples are re-weighted by total variation so terrains with larger elevation variations receive greater training emphasis.This prevents flat terrains from dominating batched optimization.
  • 3) Model and Optimization:: A gated residual U-Net preserves clearly observed elevations while selectively replacing noisy or occluded regions.The CNN outputs uncertainty, a raw estimate, and a gating map.
  • 3) Model and Optimization:: 5 ms per frame is the approximate real-hardware mapping latency, including about 2.5 ms for ONNX model inference.In simulation, inference is below 0.3 ms with 1000 parallel ANYmal-D environments and about 3 GB of GPU memory.
  • 3) Model and Optimization:: The system uses different high-frequency odometry solutions for ANYmal-D and TRON1, while removing noisy and delayed linear-velocity observations from the student policy.The stated rationale is that velocity errors can be large despite having little impact on map fusion.
  • 3) Model and Optimization:: The ANYmal-D controller zero-shots the hardest parkour and rubble-pile terrains reported in prior work.This figure is presented as evidence of agility without training on those reported terrains.

VI. RESULTS

AME-2 demonstrates agile locomotion and generalization across quadruped and biped platforms, including unseen, sparse, omnidirectional, and moving terrains.

  • VI. RESULTS: State-of-the-art agility is reported on ANYmal-D and TRON1, measured by the difficulty of terrains they can climb up and down.TRON1 climbs platforms up to 0.48 m and descends up to 0.88 m.
  • VI. RESULTS: Both robots reach peak forward velocities above 1.5 m/s across all terrains while combining agility with terrain-aware locomotion.The passage also reports stronger generalization than existing methods.
  • VI. RESULTS: The ANYmal-D system traverses an unseen parkour course at speeds up to 2 m/s by composing climbing and jumping maneuvers.Neither the controller nor mapping model was trained on that course.
  • VI. RESULTS: The controllers traverse diverse sparse terrains, including many terrains unseen during training, on both ANYmal-D and TRON1.The reported examples include narrow beams, floating blocks, gaps, and mixed terrain sequences.
  • VI. RESULTS: TRON1 performs smooth omnidirectional maneuvers over a 38-cm platform, a gap, a staircase, and rough terrain.The robot is commanded to traverse these terrain elements in sequence.
  • VI. RESULTS: The mapping captures fine-grained gaps and supports on unseen terrains while retaining uncertainty in occluded regions for later geometry integration.The authors contrast this with prior approaches that lack reliable unseen-obstacle inference, concurrent-training efficiency, or explicit uncertainty.
  • VI. RESULTS: After an unsuccessful climb exposes more obstacle geometry, ANYmal-D uses the updated map to succeed on the immediate retrial.The behavior is presented as emergent active perception from partial observations and uncertainty-aware mapping.
  • VI. RESULTS: The mapping module maintains longer-term spatial information, addressing a reported recurrent-controller limitation during climbs onto high boxes.The robot also looks upward while climbing platforms to gain additional terrain information.

2) Loco-Navigation:

The controllers exhibit local navigation, whole-body stabilization, interpretable attention patterns, and reliable generalization across unseen test terrains.

  • 2) Loco-Navigation:: Goal-conditioned controllers align the robot with traversable terrain and avoid obstacles, reducing the need for high-frequency navigation commands.The reported behaviors include turning toward a beam and avoiding an obstacle while reaching a goal.
  • 2) Loco-Navigation:: ANYmal-D uses knee contacts to stabilize itself and traverse challenging terrains, although such contacts may stress hardware designed primarily for foot contacts.The paper presents whole-body contact as beneficial for agility and robustness while identifying a hardware concern.
  • 2) Loco-Navigation:: Impact-reduction behaviors include gentle knee touch-downs by ANYmal-D and support-leg retraction by TRON1 during descents.These behaviors are linked to thigh-acceleration termination thresholds set using human and dog motion data.
  • 2) Loco-Navigation:: Local attention emphasizes fine-grained terrain details, while global features select distinctive points such as obstacle boundaries, platform surfaces, and beam centers.The observed feature patterns persist on test terrains and are reported to help explain generalization.
  • 2) Loco-Navigation:: The controllers traverse or balance on unlocked tiltable platform carts and recover from unfixed tilted stepping stones.These examples are presented as robustness to moving terrains.
  • 2) Loco-Navigation:: The maps visualize estimated elevations with uncertainty lines and use a red–blue colormap to enhance height contrast.Regions never covered by the cameras are excluded from the visualizations.
  • 2) Loco-Navigation:: On unseen test terrains, AME-2 generalizes reliably across all tests, whereas MoE generalizes very little and AME-1 struggles on mixed terrains.The paper attributes AME-1’s weakness to attention based only on proprioception, which misses global context.

B. Student Designs

The student-design evaluation compares AME-2 against recurrent and ablated alternatives, showing strongest overall performance on unseen terrains while revealing a parkour-specific trade-off. The mapping pipeline provides accurate, uncertainty-aware estimates and supports robustness to several perception degradations, but active-perception terrains remain challenging.

  • Student policy comparison: On unseen test terrains, the AME-2 student achieves the best overall success rates among the compared student designs.All student policies perform well on training terrains, with AME-2 slightly higher on sparse terrains.
  • Student policy comparison: The visual recurrent baseline performs slightly better on Test 3 but underperforms on other test terrains containing sparse regions.AME-2 failures on Test 3 often involve obstacle climbing under occlusions, whereas full depth images may provide richer cues and a longer reaction horizon.
  • Learning setup: Removing RL losses weakens generalization to unseen terrains, while adding representation alignment further improves the mixed teacher-supervision and RL design.Direct teacher supervision alone performs well on training terrains but does not produce strong unseen-terrain generalization without RL.
  • Mapping evaluation: The proposed mapping pipeline produces accurate estimates when confident, assigns high uncertainty to occluded regions, and yields higher-quality maps than the temporal recurrent model in practice.The recurrent model produces less meaningful uncertainty maps and loses fine-grained details on unseen terrains.
  • Robustness to visual noise: Under missing points and depth artifacts, policy performance does not drop and can sometimes improve because increased uncertainty induces more conservative behavior.With the front upper camera disabled, performance remains strong on most terrains but struggles on high-obstacle tasks requiring active perception.

VIII. DISCUSSION

The discussion frames AME-2 as a modular alternative that combines learned mapping and attention-based control for agile, generalized locomotion. It also identifies boundaries involving command controllability, contact-pattern diversity, unseen-terrain transitions, perception degradation, and dynamic occlusions.

  • Discussion: AME-2 combines a mapping module with an end-to-end neural controller whose encoder learns global terrain context and local contact-relevant features.This preserves system-level modularity while avoiding explicit online model-based planning.
  • Goal Reaching v.s. Velocity Tracking: Goal-reaching rewards encourage agile behavior and reduce reliance on high-frequency navigation commands, but provide less direct control over intermediate motion.Semantic obstacles that appear geometrically traversable can therefore be troublesome along the commanded path.
  • Whole-Body Skills: For a given terrain type, learned motions tend to use similar contact patterns, leaving broader multi-strategy behavior in higher-DoF systems unresolved.The paper gives humanoid examples requiring different strategies as obstacle scale increases.
  • Towards Higher Success Rates on Unseen Terrains: Unseen terrains produce more failures than training terrains, with many failures occurring during difficult skill transitions such as deceleration followed by climbing.The paper identifies scalable, principled zero-shot learning of such transitions as unresolved.
  • Limitations and Future Work: The system uses 2.5-D elevation maps, is not designed for severely degraded perception, and may fail in highly dynamic scenes with occlusions.Suggested directions include multi-layer maps, robust controllers, and explicit reasoning about moving scene elements.

APPENDIX A TERRAIN PARAMETERS

The appendix defines a curriculum spanning climbing, sparse foothold, rough, stair, box, and obstacle terrains, with randomized physical and sensing conditions. It also specifies observation corruption, student-training degradation, PPO configuration, and the recurrent mapping baseline setup.

  • Terrain curriculum: Dense terrains include rough heightfields, ascending and descending stairs, randomized boxes, and positive or negative obstacles on slopes.Stair slopes increase from 5° to 45° for both robots.
  • Terrain curriculum: The terrain curriculum includes climbing, sparse foothold, dense rough, stair, box, and obstacle environments with progressively varied geometry.Examples include climbing heights, gap distances, beam widths, stair slopes, box heights, and obstacle densities that increase or decrease across training.
  • Physical randomization: Training randomizes payload, actuation delay, friction, and selected robot parameters to expose both robots to physical variation.TRON1 additionally receives random PD-gain and armature biases within ±15% of raw values.
  • Observation noise: Policy observations are corrupted with bounded noise in velocities, gravity, joint states, and map queries, including random map-query drift.Map observations receive up to 0.05 m noise and query drift is randomized within [−0.03, 0.03] m.
  • Student mapping robustness: Student training removes 15% of depth-cloud points, adds 2% artifacts, and corrupts a further 1% of student map observations with high-variance random values.Ten percent of environments also access complete maps with variance 0.0025 m2.
  • Baselines and training configuration: The recurrent mapping baseline receives local grids and ground-truth delta transforms, while the appendix provides its architecture and PPO settings for comparison.Ground-truth transforms replace unavailable proprioceptive transform information used in prior mapping work.
Loading 2601.08485v2…