Source-linked AI summary
Neural Volumetric Memory for Visual Locomotion Control
Ruihan Yang, Ge Yang, Xiaolong Wang
TL;DR
Challenging locomotion is partially observable because terrain can be hidden from a robot’s current camera view. The paper proposes Neural Volumetric Memory (NVM), which aligns recent visual features into the current robot frame, and reports improved traversal and sim-to-real generalization in simulation and the real world.
Problem
Legged robots need to infer nearby terrain from past observations when relevant terrain is occluded in the current view, while image frame-stacking ignores 3D equivariance.
Source: §1 · p. 1Method
NVM transforms feature volumes from recent depth images into the present robot frame using estimated relative poses, and a teacher-student pipeline trains a visual policy from a privileged teacher.
Source: §1 · p. 2Results
Takeaways & Limitations
The authors conclude that NVM provides reliable surrounding 3D structure information for robot decision-making on complex terrain.
Source: §5 · p. 8With longer histories, the pose encoder has difficulty estimating transformations from the oldest frame, and surplus visual information may be redundant for terrain decisions.
Source: §4.6 · p. 8
Abstract
from arXiv · showhide
Legged robots have the potential to expand the reach of autonomy beyond paved roads. In this work, we consider the difficult problem of locomotion on challenging terrains using a single forward-facing depth camera. Due to the partial observability of the problem, the robot has to rely on past observations to infer the terrain currently beneath it. To solve this problem, we follow the paradigm in computer vision that explicitly models the 3D geometry of the scene and propose Neural Volumetric Memory (NVM), a geometric memory architecture that explicitly accounts for the SE(3) equivariance of the 3D world. NVM aggregates feature volumes from multiple camera views by first bringing them back to the ego-centric frame of the robot. We test the learned visual-locomotion policy on a physical robot and show that our approach, which explicitly introduces geometric priors during training, offers superior performance than more na\"ive methods. We also include ablation studies and show that the representations stored in the neural volumetric memory capture sufficient geometric information to reconstruct the scene. Our project page with videos is https://rchalyang.github.io/NVM .
1. Introduction
Challenging terrain locomotion requires visual information about nearby 3D structure, including terrain hidden from the current view. The paper introduces a pose-aligned volumetric memory and reports that its policy outperforms baselines across simulated environments.
1. Introduction
- Occluded terrain makes locomotion partially observable, requiring the robot to use past observations to infer terrain beneath it.The task includes behaviors such as climbing stairs and stepping over gaps.Source: §1 · p. 1
- Frame-stacking recent images ignores 3D equivariance, leaving a policy to learn around viewpoint changes caused by robot motion.The paper also argues that saving detailed pixel-by-pixel observations is unlike biological memory, motivating an aggregated short-term representation.Source: §1 · pp. 1–2
- NVM extracts feature volumes from recent depth images, transforms them into the present robot frame using estimated relative poses, and fuses them for locomotion decisions.The architecture is designed to encourage SE(3) equivariance and make multi-timestep scene integration and policy learning less difficult.Source: §1 · p. 2
- A two-stage teacher-student pipeline first trains a policy with privileged information, then behavior-clones it using ego-centric depth views and NVM in simulation.The visual stage trains the NVM components and policy end-to-end; an additional self-supervised objective uses novel-view consistency.Source: §1 · p. 2
- The NVM policy outperforms all baselines by a large margin, and modeling the environment’s 3D structure is described as essential.Policies are trained in simulation and transferred to the real world without fine-tuning.Source: §1 · p. 2, Figure 2 · p. 2
2. Related Work
Prior locomotion approaches use proprioception, elevation maps, or camera images, while learned 3D representations provide another route to scene understanding. This work uses volumetric features as short-term memory for decisions rather than view synthesis.
Blind Locomotion Controllers
- Blind controllers trained with reinforcement learning can transfer from simulation and adapt across terrains, but cannot perceive their surroundings.Model-based methods are also described as often failing to generalize to unseen real-world environments.Source: §2 · p. 2
Locomotion with Exteroceptive Perception
- Elevation maps support foot-placement planning, but real-world maps may be noisy or require a complicated hardware pipeline.Other systems use depth readings hierarchically or train visual policies to control joint angles and traverse varied terrains.Source: §2 · pp. 2–3
- Prior visual policies show encouraging results on specific tasks but have not fully exploited environmental 3D structure; this work trains volumetric memory with the policy.NVM aggregates transformed feature volumes from recent observations into a representation for decision-making.Source: §2 · p. 3, Figure 3 · p. 3
Volumetric Cognitive Map
- Learned 3D representations include work that learns voxel features and camera trajectories from video, whereas this paper aggregates volumes for short-term decision-making rather than view synthesis.The cited approach learns without odometry in a self-supervised manner.Source: §2 · p. 3
Teacher-Student Training
- Teacher-student training transfers a policy trained with privileged simulation information into a student policy without that information.The paper relates its pipeline to prior elevation-map-based teacher-student policies and subsequent visual distillation.Source: §2 · p. 3
3. Volumetric Memory for Legged Locomotion
The paper introduces Neural Volumetric Memory (NVM) to integrate depth observations despite camera motion and occluded terrain. It combines pose-aligned 3D features, self-supervised reconstruction, and visuomotor distillation for locomotion control.
3. Volumetric Memory for Legged Locomotion
- NVM aggregates depth observations into a 3D feature volume representing surrounding structure, addressing the difficulty of integrating frames as the robot-mounted camera changes pose.The visual policy uses a forward-looking, ego-centric camera and relies on past observations to infer occluded terrain.Source: §3 · p. 3
3.1. Baking-in SE(3) Equivariance in NVM
- Relative camera poses transform past feature volumes into the present frame, where canonicalized volumes are aggregated to encourage SE(3) equivariance.The method applies translation and rotation transformations before aggregating new observations.Source: §3.1 · pp. 3–4
3.2. Learning NVM via Self-Supervision
- A self-supervised decoder predicts observations in different frames from a canonicalized feature volume, using pixel-wise L1 reconstruction loss to encourage structural understanding and SE(3) equivariance.The reconstruction task assumes the surrounding 3D scene remains static between frames, although perfect prediction is not feasible.Source: §3.2 · pp. 4–5, Figure 4 · p. 4
3.3. Teacher Policy and Visuomotor Distillation
- Because image observations slow training by up to 40 times, the pipeline first trains an elevation-map teacher, then distills its actions into a visual policy using proprioception and depth images.The visual policy is trained by minimizing the L1 distance between teacher and visual-policy actions; its overall loss combines behavior cloning and self-supervision.Source: §3.3 · p. 5
4. Experiments
Across four simulated terrains, NVM outperforms the visual baselines, and in real-world tests it travels farther than LocoTransformer on Stages and Stairs. The experiments also examine baseline variants, reconstructed scene features, and memory-resolution and history-length ablations.
4.1. Simulation Environments
- The four simulated terrains test nearby foot placement on Stages, Stairs, and Stones, and long-term obstacle avoidance on Obstacles.The environments vary stage gaps, stair heights and widths, and stone shapes.Source: §4.1 · pp. 5–6
4.2. Baselines
- Baselines lack SE(3) equivariance priors and include CNN, recurrent, transformer, and multi-step CNN variants, with shared encoder structure for fair comparison.NaiveCNN and LocoTransformer stack recent observations across channels; NaiveCNN-MS extracts features from each depth image before fusion.Source: §4.2 · p. 6
4.3. Simulated Results
- NVM outperforms all baselines across the four simulated environments, especially on Stones, and traverses farther on Obstacles; Teacher is treated as the upper bound.The authors conjecture that 3D understanding helps NVM stay closer to the path favored by the privileged teacher on Obstacles.Source: §4.3 · p. 6, Table 2 · p. 6
- NaiveCNN-MS and NaiveCNN-RNN show no improvement over NaiveCNN, supporting the authors’ interpretation that NVM’s gain comes from explicitly modeling 3D transformations.NaiveCNN-MS processes frames separately, while NaiveCNN-RNN adds a recurrent memory mechanism.Source: §4.3 · pp. 6–7
4.4. Real World Results
- In real-world trials, NVM moved significantly farther than LocoTransformer on Stages and Stairs, and slightly farther on Obstacles.The experiments used five repeats per method and environment.Source: §4.4 · p. 7, Table 4 · p. 7
4.5. Visualization of SSL task
- Reconstructions preserve overall structure in Stairs and Obstacles and decision-relevant regions in Stages and Stones, although distant details are not fully recovered.The authors attribute missing distant detail to large input values being clipped.Source: §4.5 · pp. 7–8, Figure 5 · p. 8
4.6. Ablation Study
- Increasing voxel depth generally improves performance, H = W = 6 works best, and five history frames outperform three while nine reduce performance.The authors hypothesize that longer histories make pose estimation harder and may add redundant information.Source: Table 3 · p. 7, §4.6 · p. 8
5. Conclusion
The conclusion presents Neural Volumetric Memory (NVM) as short-term memory for legged locomotion that combines sequential visual observations using explicit inter-frame transformations. It reports traversal of complex terrain in simulation and the real world, with better sim-to-real generalization.
5. Conclusion
- NVM models translation and rotation between past and present frames to aggregate recent visual observations.The paper says explicit transformation modeling and the self-supervised task encourage SE(3) equivariance in the memory space.Source: §5 · p. 8
- The memory module provides surrounding 3D structure information for robot decision-making.Source: §5 · p. 8
- The authors report complex-terrain traversal in simulation and the real world, with better sim-to-real generalization.Source: §5 · p. 8
1. Detailed Observation Space
The observation space combines depth-image history and robot proprioception, while the privileged teacher additionally receives velocity and randomized environment parameters. The visual input uses five depth frames and includes simulated pixel corruption.
1. Detailed Observation Space
- The robot has 12 joints and uses positional control; proprioception includes joint rotations, joint velocities, previous actions, and projected gravity.Rotations and previous actions cover the past two control steps; projected gravity represents robot orientation.Source: §1 · p. 12
- The privileged teacher observation includes the robot’s linear and angular velocity and eight randomized environment parameters.Source: §1 · p. 12
- The visual policy receives five 64×64 depth frames, with 1% of pixels randomly set to maximum range at each time step to simulate noisy observations.Source: §1 · p. 12
2. Reward for Privileged Teacher Training
The privileged teacher is trained with a shared reward function across environments. Its terms reward forward motion, low torque use, target-height tracking, and gait similarity to reference motions.
2. Reward for Privileged Teacher Training
- The same reward function is used to train the privileged teacher in all environments.Source: §2 · p. 12
- The passages specify target forward speed 0.4 and target relative height above terrain 0.265; AMP assigns higher scores to policy-produced gaits closer to reference motions.Source: §2 · p. 12
- The height reward uses the absolute difference between the robot’s current and target relative heights, with a height coefficient of −2.0.Source: §2 · p. 12
- The AMP reward uses a gait discriminator score from 0 to 1, assigning higher reward to gaits closer to reference motions.The paper states that this setting follows Peng et al [49].Source: §2 · p. 12