Source-linked AI summary
Learning robust perceptive locomotion for quadrupedal robots in the wild
Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun, Marco Hutter
TL;DR
Legged robots need exteroceptive foresight for fast terrain adaptation, but real-world depth sensing is often incomplete or unreliable. The paper presents an end-to-end recurrent encoder that combines exteroception with proprioception and reports robust, fast locomotion across challenging environments.
Problem
Reliable exteroceptive perception remains difficult in uncontrolled terrain, while relying solely on proprioception limits locomotion speed.
Method
A recurrent encoder integrates proprioceptive and exteroceptive inputs into a belief state, trained end-to-end to handle noisy, biased, and incomplete terrain observations without heuristics.
Results
The controller achieved substantial improvements in locomotion speed and obstacle traversability over the state of the art while maintaining exceptional robustness.
Takeaways & Limitations
The controller remained robust across diverse natural, underground, and urban environments, including terrain and conditions that degraded exteroception.
Abstract
from arXiv · showhide
Legged robots that can operate autonomously in remote and hazardous environments will greatly increase opportunities for exploration into under-explored areas. Exteroceptive perception is crucial for fast and energy-efficient locomotion: perceiving the terrain before making contact with it enables planning and adaptation of the gait ahead of time to maintain speed and stability. However, utilizing exteroceptive perception robustly for locomotion has remained a grand challenge in robotics. Snow, vegetation, and water visually appear as obstacles on which the robot cannot step~-- or are missing altogether due to high reflectance. Additionally, depth perception can degrade due to difficult lighting, dust, fog, reflective or transparent surfaces, sensor occlusion, and more. For this reason, the most robust and general solutions to legged locomotion to date rely solely on proprioception. This severely limits locomotion speed, because the robot has to physically feel out the terrain before adapting its gait accordingly. Here we present a robust and general solution to integrating exteroceptive and proprioceptive perception for legged locomotion. We leverage an attention-based recurrent encoder that integrates proprioceptive and exteroceptive input. The encoder is trained end-to-end and learns to seamlessly combine the different perception modalities without resorting to heuristics. The result is a legged locomotion controller with high robustness and speed. The controller was tested in a variety of challenging natural and urban environments over multiple seasons and completed an hour-long hike in the Alps in the time recommended for human hikers.
1. INTRODUCTION
Legged robots struggle to use unreliable exteroceptive perception in uncontrolled terrain, limiting robust fast locomotion. This paper introduces a recurrent, end-to-end multimodal controller that combines exteroception with proprioception and is designed to remain robust when terrain sensing degrades.
- Motivation: Legged robots are suited to steep slopes, steps, and gaps, but robust autonomous operation remains important for hazardous and remote environments.Such settings include areas too dangerous or distant for humans and terrain that can impede wheeled or tracked vehicles.
- Motivation: Animals combine visual foresight with proprioception, whereas legged robots had not matched their performance on challenging real-world terrain.Visual terrain prediction supports earlier foothold planning and adaptation to irregular surfaces, slipperiness, and softness.
- Perception challenge: Exteroceptive sensors produce incomplete or unreliable terrain information under low texture, difficult illumination, reflectance, soft vegetation, and pose-estimation errors.These failures make elevation maps uncertain and complicate locomotion control.
- Prior approaches: Conventional planners assume accurate terrain information, while existing outdoor controllers often avoid visual input or rely on heuristic reflex rules.Offline approaches also assume full terrain and robot-state knowledge, whereas online methods continuously replan from onboard maps.
- Approach: The proposed controller uses a recurrent encoder trained end-to-end to integrate proprioception and exteroception into a belief state without heuristics.Privileged teacher learning provides terrain-informed behavior, and imitation learning trains a student using partial, noisy field-available observations.
- Deployment: The deployed policy accepts onboard observations and desired velocity commands, outputs joint target positions, and supports different depth sensors without fine-tuning.Training with noisy, biased, and incomplete elevation maps lets the robot continue walking when mapping fails or sensors are physically broken.
- Contribution: The approach reports substantial improvements in locomotion speed and obstacle traversability over prior work while maintaining exceptional robustness.Its stated contribution is a general framework for combining multimodal perception and handling exteroceptive failures in autonomous machines deployed outdoors.
2. RESULTS
The controller remained robust across diverse natural and urban terrain, using exteroception to anticipate obstacles and adapt locomotion while falling back on proprioception when perception degraded. It completed challenging deployments and experiments without failures while achieving faster locomotion and higher step-traversal capability than the proprioceptive baseline.
- Field deployments: Zero falls were recorded across alpine, forest, underground, and urban deployments, including slippery, snowy, dusty, wet, and poorly illuminated conditions.Robustness persisted when occlusion, high reflectance, and challenging surface properties impeded exteroception.
- Field deployments: ANYmal traversed stairs natively in any direction and orientation without requiring a dedicated stair mode.The controller supported sideways, diagonal, and turning movements on stairways.
- A hike in the Alps: 31 minutes: ANYmal reached the summit during a 2.2 km, 120 m-elevation-gain hike with inclinations up to 38%.The robot completed the route without failure, stopping only to fix a detached shoe and swap batteries, and finished without human help or a fall.
- Step traversal: 30.5 cm: the controller reliably traversed steps, whereas the proprioceptive baseline’s success rate dropped at 20 cm.Anticipating the step, the controller lifted its legs higher and leaned forward to help the hind legs clear it; above 32 cm, it hesitated near the robot’s physical limits.
- Locomotion speed: 1.2 m/s: the controller’s measured flat-ground walking speed exceeded the proprioceptive baseline’s speed.The supplied passage introduces the flat-ground and obstacle-speed comparison, while the reported baseline value is truncated.
- Locomotion speed: 3 rad/s versus 0.6 rad/s: the controller turned five times faster than the proprioceptive baseline.The authors attribute the broader gains to exteroception, which enabled more successful traversal and higher speeds than pure proprioception.
- Belief-state adaptation: The recurrent controller revised terrain beliefs after contact with misleading foam or transparent obstacles and retained corrected information after the foot left the ground.With sensors covered, it also successfully climbed stairs after revising its terrain estimate from contact feedback.
- Belief-state adaptation: On a slippery elevated surface, the controller detected low friction, adapted its stepping behavior, and fell back on proprioception when sliding destabilized the elevation map.The supplied passage states that this fallback continued until the elevation estimate recovered.
3. DISCUSSION
The controller integrates exteroceptive and proprioceptive inputs end-to-end, anticipating terrain when perception is reliable and transitioning to proprioceptive locomotion when it is not. Controlled experiments and field deployments demonstrated robust operation across challenging environments, including an Alpine hike and underground missions.
- The controller transitions smoothly to proprioceptive locomotion when exteroceptive perception is misleading, incomplete, or unavailable.It remains robust without hand-coded rules or heuristics.
- Exteroceptive perception enables faster, smoother locomotion by allowing the robot to anticipate terrain and adapt its gait before contact.
- The controller completed a difficult hour-long Alpine route without human assistance beyond reattaching a shoe and swapping batteries, within the human-recommended completion time.
- Four ANYmals explored over 1700m across tunnel, urban, and cave courses without a single fall during DARPA Subterranean Challenge missions.
- The current elevation-map interface omits raw sensory detail and uses uncertainty only implicitly, so future work could estimate uncertainty explicitly.The authors note that insufficient information near narrow cliffs or stepping stones may lead the robot to assume a continuous surface and step off.
4. MATERIALS AND METHODS
The method uses privileged-learning distillation: a reinforcement-learned teacher supervises a recurrent student that reconstructs hidden state from noisy multimodal observations. The student is deployed zero-shot using elevation-map height samples combined with proprioception.
- The training pipeline first learns a teacher policy with reinforcement learning, then distills it into a student policy for zero-shot sim-to-real transfer.
- The teacher follows random target velocities over randomized terrain and disturbances while accessing noiseless terrain, friction, and disturbance information.
- The student imitates teacher actions without privileged information and constructs a belief state with a recurrent encoder.Behavior-cloning and reconstruction losses train imitation and hidden-state reconstruction.
- At deployment, height samples from a robot-centric elevation map are combined with proprioceptive data to generate actuator commands.
Problem formulation
The control problem is modeled as a partially observable sequential decision process because terrain and external disturbances are hidden. A recurrent belief state summarizes observation history so reinforcement learning can operate despite incomplete state information.
- The environment evolves through states, actions, observations, transition probabilities, and rewards in discrete time.
- When observations equal states the problem is an MDP, whereas hidden terrain and external forces make it a POMDP.
- The reinforcement-learning objective is to find a policy that maximizes expected discounted future reward.
- A belief state built from observation history helps capture unobservable information in a POMDP.Deep reinforcement learning can construct it by stacking past observations or using recurrent architectures.
- Privileged learning first trains a teacher with privileged information and then distills it into a student through supervised learning.
Training environment
Training uses parallel simulated ANYmal-C robots on randomized rough terrain with actuator modeling, varied stair geometries, randomized physical conditions, and termination constraints that shape safe behavior.
- RaiSim simulates multiple ANYmal-C robots in parallel on randomly generated rough terrain with an integrated actuator model.
- The terrain is represented as a parameterized height map.
- Four stair types—standard, open, ledged, and random—are modeled with boxes to avoid exploiting nonvertical height-map risers.
- Each episode randomizes robot physical states, applies external forces and torques, and occasionally reduces foot friction to induce slippage.
- Episodes terminate after body collision, excessive body tilt, or actuator torque-limit violations to shape constraint-satisfying motion.
Teacher policy training
The teacher policy is trained with privileged terrain information to follow velocity commands over randomized terrain while maintaining stable, smooth locomotion. Its architecture combines exteroceptive and privileged encoders with a main MLP, and training uses adaptive curricula and behavior-shaping rewards.
- Teacher policy objective: The teacher policy uses privileged terrain information to follow randomly generated velocity commands over randomized terrain.The command contains longitudinal, lateral, and yaw velocities in the robot body frame.
- Observations: The teacher observes proprioceptive history, height samples around each foot, and privileged terrain and contact states.Proprioception includes body and joint histories, action history, and leg phase; privileged inputs include contact and force information.
- Action space: Each action combines per-leg phase offsets with residual joint-position targets around nominal CPG trajectories.Nominal foot trajectories are converted to joint targets through inverse kinematics.
- Network architecture: The teacher policy uses separate exteroceptive and privileged encoders whose latent representations feed a main MLP.The encoders compress their inputs and support reuse of teacher components by the student policy.
- Rewards: Training rewards command following while penalizing undesired velocity, posture, aggressive joint motion, slippage, and collisions.Additional shaping terms address orientation, torque, velocity, acceleration, foot slippage, and shank or knee collisions.
- Curriculum: Two curricula progressively increase terrain difficulty and the magnitude of selected randomization and reward terms.An adaptive terrain curriculum keeps conditions challenging but achievable, while a second curriculum increases selected factors toward one.
Student policy training
The student policy distills a privileged teacher into a recurrent controller that operates on noisy field-available observations. A gated belief encoder integrates proprioception and exteroception, reconstructs privileged information, and adaptively controls reliance on terrain sensing.
- Distillation: The student policy is distilled from a privileged teacher using only information available on the physical robot.Training uses the same simulation environment while adding noise to student height samples.
- Belief-state estimation: Large exteroceptive noise makes terrain dynamics partially observable, so the policy estimates hidden states from sequential observations.The recurrent encoder combines exteroceptive and proprioceptive sequences into a belief state.
- Student architecture: The recurrent encoder maps noisy student observations and its hidden state to a belief vector used with proprioception to compute actions.The belief vector is trained to match the teacher’s locomotion-relevant latent feature.
- Training objectives: Student training minimizes behavior-cloning and reconstruction losses, with rollouts generated from the student policy.The reconstruction target includes noiseless height samples and privileged information.
- Noise model: Height-sample noise models lateral scan shifts and height perturbations at scan-point, foot, and episode scopes.Gaussian noise is resampled at different frequencies depending on scope, while episodic noise remains fixed across scan points.
- Noise conditions: Three mapping conditions simulate nominal sensing, large map offsets, and near-total terrain-information loss from occlusion or mapping failure.The conditions represent regular operation, pose-estimation or deformable-terrain offsets, and severe mapping failure.
- Gated fusion: A gated encoder learns how much exteroceptive information enters the final belief state and uses the same gate for reconstruction.The gate is produced by neural networks with a sigmoid, and reconstruction encourages veridical environmental information.
- Decoder: The decoder reconstructs privileged information and height samples from the gated representation, and the recurrent unit uses a GRU architecture.The reconstruction loss encourages the belief state to capture information about the environment.
Deployment
The controller is deployed zero-shot on ANYmal C using either LiDAR or stereo depth cameras. A robot-centric elevation map supports sensor-independent terrain representation, while GPU mapping sustains the controller’s fast processing rates.
- Robot deployment: The controller runs zero-shot on ANYmal C with either two LiDAR sensors or four stereo depth cameras.The policy is deployed without fine-tuning and uses onboard sensor observations plus a desired velocity command.
- Runtime pipeline: A robot-centric 2.5D elevation map is built at 20 Hz, while the locomotion policy runs at 50 Hz.Sensor point clouds are registered using estimated robot pose, and heights are sampled from the map.
- Mapping: GPU-parallelized elevation mapping uses Kalman-style updates, drift compensation, and ray casting to maintain fast processing.The implementation was described as crucial for keeping pace with the controller’s locomotion speeds.
SUPPLEMENTARY MATERIALS
The supplementary materials provide sections on training, terrain, observations, rewards, noise, and belief-encoder evaluation, along with additional comparisons, ablations, tables, and videos.
- Supplementary sections: Supplementary sections document nomenclature, exteroception importance, training details, terrain generation, observations, network architecture, rewards, height-sample noise, and belief-encoder evaluation.
- Figures and ablations: Supplementary figures compare the controller with a proprioceptive baseline and analyze the belief encoder.
- Tables and videos: Additional materials include hyperparameter and observation tables, teacher–student action comparisons, and videos covering stairs, baseline comparison, robustness, slippery surfaces, and soft obstacles.
S1. Nomenclature
This nomenclature defines symbols for states, observations, belief and hidden states, latent features, motion quantities, losses, and core functions used throughout the paper.
- s, o, b, and h denote state, observation, belief state, and hidden state, respectively.
- l, v, ω, τ, q, and φ denote leg, linear velocity, angular velocity, joint torque, joint position, and CPG phase.
- ∆φ0 and ck denote the CPG phase base frequency and curriculum factor, while csk denotes the student curriculum factor.
- Lbc and Lre denote the behavior cloning and reconstruction losses.
- The notation (·)p, (·)e, (·)priv, and (·)target identifies proprioceptive, exteroceptive, privileged, and target quantities; (·)t marks time t.
- g(·), p(·), and IK(·) denote the MLP encoder, foot trajectory function, and inverse kinematics function.
S2. Evaluating the importance of exteroception: Additional experiments in simulation
Additional simulation experiments evaluate terrain traversal, training configuration, observations and actions, reward design, and the gated encoder under varying exteroceptive noise. The results show that gating improves use of reliable exteroception while retaining robustness when perception is unreliable.
- Evaluation setup: 300 trials at 0.7 m/s tested whether the controller could traverse 4 m across stepped terrain and stairs without failure.The success rate was compared against a baseline across a wider range of terrain.
- Training details: The simulated training uses PPO for the teacher and curriculum-based student training with gradually increasing height-sample noise.Student training uses 300 environments and 400 trajectory timesteps per iteration; the loss is Lbc + 0.5 · Lre.
- Terrain generation: The ablation compares rough, rough-discrete, large-step, grid-step, step-stair, and standard-stair terrain types with randomized terrain parameters.Large steps vary between heights 0 and 0.4 m; grid steps vary in height and width.
- Observations and actions: The policy receives proprioceptive commands, joint, body, and leg-phase information, while exteroception is represented by circular height samples around each foot.The action contains per-leg phase offsets and joint-position offsets, with target joint positions computed using inverse kinematics.
- Reward design: The reward combines command-following, body-motion, foot-clearance, orientation, torque, joint, collision, and foot-slippage terms.The linear and angular velocity terms encourage following desired horizontal and yaw velocity commands, while torque is penalized to reduce energy consumption.
- Attention-gate ablation: The gated encoder produces smaller teacher–student action differences than the non-gated encoder across terrains under small exteroceptive noise.Under large noise, the two structures perform similarly because the gate blocks the exteroceptive skip connection.
- Attention-gate ablation: The GRU gate policy achieved the best step-traversal success rate under both small-noise and large-noise conditions.The comparison used a constant 0.8 m/s command and 100 trials for each step height.
A Learning curves of student training
The student-training learning curves compare GRU and MLP encoders with and without the proposed attention gate. GRU performs better than MLP, and attention gating improves both encoder types.
- GRU worked better than MLP in all compared encoder settings during student-policy training.
- The attention gate worked better than omitting attention for both GRU and MLP encoders.