Source-linked AI summary
Learning to Explore using Active Neural SLAM
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, Ruslan Salakhutdinov
TL;DR
Exploration in unknown environments requires efficient coverage, but end-to-end learning can be sample-intensive and underperform classical methods. Active Neural SLAM addresses this with modular learned SLAM, global and local policies, and analytical planning. It achieves competitive exploration results in realistic simulated environments and transfers to PointGoal navigation, winning the CVPR 2019 Habitat challenge.
Problem
Exploration in unknown environments must maximize coverage, while end-to-end learning for mapping, state estimation, and planning can require many millions of frames and still underperform classical methods.
Method
Active Neural SLAM modularly combines learned SLAM, global and local policies, and an analytical path planner.
Results
32.701m2/0.948 average absolute and relative coverage versus 24.863m2/0.789 for the best baseline on Gibson Val.
Takeaways & Limitations
The approach demonstrates effective exploration in realistic simulated environments and can be modified for PointGoal navigation, where it won the CVPR 2019 Habitat challenge.
Takeaways & Limitations
The evaluation uses simulated Habitat environments, albeit with real-world reconstructed scenes and collected actuation and sensor noise models.
Abstract
from arXiv · showhide
This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called `Active Neural SLAM'. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies. The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies). Such use of learning within each module retains its benefits, while at the same time, hierarchical decomposition and modular training allow us to sidestep the high sample complexities associated with training end-to-end policies. Our experiments in visually and physically realistic simulated 3D environments demonstrate the effectiveness of our approach over past learning and geometry-based approaches. The proposed model can also be easily transferred to the PointGoal task and was the winning entry of the CVPR 2019 Habitat PointGoal Navigation Challenge.
1 INTRODUCTION
Active Neural SLAM addresses exploration in unknown environments with a modular, hierarchical alternative to costly end-to-end learning. It combines learned components with analytical planning to retain learning’s flexibility, robustness, and ability to exploit environmental regularities.
- Exploration aims to efficiently visit unknown environments, maximizing coverage for target discovery or pre-mapping under time constraints.
- End-to-end exploration learning can require imitation learning and many millions of frames while still underperforming classical methods.
- Active Neural SLAM factors learning across time scales within a modular and hierarchical classical navigation pipeline.
- The architecture combines a learned Neural SLAM module, global policy, local policy, and analytical path planner.
- Experiments in visually and physically realistic Habitat simulations demonstrate effectiveness for exploration, while a modification addressed PointGoal navigation and won the CVPR 2019 Habitat challenge.
2 RELATED WORK
Prior navigation research spans classical mapping and planning, active exploration strategies, learning-based exploration, and modular policy design. Active Neural SLAM connects learning-based policies with classical exploration and navigation components.
- Navigation Approaches: Classical navigation commonly separates mapping from path planning, but dense mapping from RGB images is inefficient or may require specialized scanners.
- Exploration in Navigation: Active SLAM focuses on actively controlling the camera to build maps, often formulating exploration as an approximately solved POMDP.
- Exploration in Navigation: Other exploration methods select vantage points such as frontiers between explored and unexplored regions, while recent work applies learning.
- Exploration in Navigation: The proposed modular policies unify vantage-point exploration with learning-based exploration and report improvements over representative methods from both lines.
- Hierarchical and Modular Policies: Hierarchical reinforcement learning seeks automatically discovered hierarchies, but navigation research often uses hand-defined modular policies interfaced with low-level controllers.
3 TASK SETUP
The task maximizes traversable-area coverage within a fixed time budget using realistic reconstructed environments and robot-derived noise models. The setup explicitly relaxes common assumptions of noiseless motion and known pose.
- The exploration objective is to maximize the total traversable area known within a fixed time budget.
- Experiments use Habitat with Gibson and Matterport scenes based on real-world reconstructions.
- Real-world motion and sensor data are used to relax assumptions of simplistic agent motion and perfect pose knowledge.
- Actuation and Sensor Noise Model: Actuation and sensor noise are defined by differences between commanded or actual poses and sensor-estimated poses.
- Actuation and Sensor Noise Model: The simulator uses forward motion of 25cm and on-the-spot turns of 10 degrees as default navigational actions.
- Actuation and Sensor Noise Model: Separate Gaussian Mixture Models are fitted for actuation and sensor noise, using six models and 600 datapoints per model.
4 METHODS
Active Neural SLAM maintains a learned spatial map and pose estimate, uses a learned global policy to select long-term goals, and applies analytical planning plus a learned local policy for execution.
- Active Neural SLAM consists of Neural SLAM, Global, and Local policy components connected through map estimates and path planning.
- Map Representation: The spatial map is a 2 × M × M matrix whose channels represent obstacle and explored-area probabilities, with 5cm × 5cm cells.
- Neural SLAM Module: Neural SLAM combines RGB observations, sensor readings, prior pose estimates, and prior maps to produce an updated map and current pose estimate.
- Neural SLAM Module: The Mapper predicts egocentric obstacle and explored-area maps, while the Pose Estimator predicts pose from prior estimates and recent egocentric map predictions.
- Global Policy: The Global policy processes map, pose, and visited-location representations to predict a long-term goal.
- Planner and Local Policy: The Planner uses Fast Marching on the spatial map to convert a long-term goal into a short-term goal, which the Local policy pursues from RGB observations.
5 EXPERIMENTAL SETUP
The experiments evaluate Active Neural SLAM and several baselines in Habitat using realistic 3D-reconstructed environments, fixed-budget coverage metrics, unseen-scene splits, and Gibson-to-Matterport transfer. Training combines supervised, reinforcement, imitation, and end-to-end reinforcement-learning procedures.
- Datasets and simulator: Habitat experiments use Gibson and Matterport 3D reconstructions, with Gibson for training and Matterport for domain-generalization experiments.Gibson scenes are mostly offices, whereas Matterport scenes are mostly homes with larger average area.
- Task and metrics: Coverage measures traversable area known to the agent, reported as absolute area in m2 and relative scene coverage (% Cov).A point is known when it lies within the agent’s field of view and less than 3.2m away; training episodes last 1000 steps.
- Evaluation protocol: The validation scenes are disjoint from training scenes, so evaluation tests new environments never seen during training.Gibson validation serves as the comparison set, without being used for hyper-parameter tuning.
- Training: The proposed model trains a Mapper and Pose Estimator with supervision, the Global Policy with reinforcement learning, and the Local Policy with behavioral cloning.All modules are trained simultaneously, with the Global Policy rewarded according to coverage increase.
- Baselines: Baselines include end-to-end reinforcement-learning policies using convolutional or ResNet representations, recurrent units, and an auxiliary depth-prediction task.They use PPO with coverage-increase reward and ground-truth maps during training.
- Reported analyses: Figure 3 plots % Cov over episode progress for ANS and baselines on large scenes, small scenes, and the overall Gibson validation set.The figure organizes comparisons by scene size and aggregate validation performance.
6 RESULTS
Active Neural SLAM outperforms the best baseline on Gibson exploration and generalizes better to Matterport. Ablations and real-world deployment highlight the roles of hierarchical planning, learned local control, and modular transfer.
- Exploration results: 32.701m2/0.948 versus 24.863m2/0.789 for the best baseline on Gibson validation shows stronger exhaustive exploration.The evaluation averages 994 episodes across 14 unseen scenes after training with 10 million frames.
- Exploration results: 73.281m2/0.521 versus 54.775m2/0.378 for the best baseline demonstrates higher domain-generalization performance on Matterport.Models trained on Gibson were evaluated on the Matterport domain, whose larger scenes produce higher absolute coverage but lower percentage coverage.
- Exploration results: ANS nearly completely explores small scenes in around 500 steps, while baselines reach only 85–90% in 1000 steps.On large scenes, the performance gap widens over time because baselines often remain in local areas, whereas ANS remembers explored regions and plans toward distant goals.
- Ablations: Removing the Local Policy slightly reduces performance because it can overcome mapping false positives using RGB observations.For example, it can recognize that a carpet predicted as an obstacle is traversable and walk over it when the short-term goal lies beyond it.
- Ablations: Frontier-based exploration is comparable on small scenes but around 10% lower on large scenes than the learned Global Policy.The learned policy ignores small spaces and selects distant long-term goals, while frontier exploration often spends time on corners or areas behind furniture.
- Ablations: Removing the Pose Estimator drops performance about 1% on small scenes and around 10% on large scenes, while ground-truth pose does not improve baselines.The larger-scene degradation is attributed to accumulated pose errors and drift over longer exploration horizons.
A.1 POINTGOAL RESULTS
Active Neural SLAM transfers effectively to PointGoal navigation, is substantially faster than the strongest RL baseline, and achieved the winning Habitat Challenge results.
- PointGoal transfer: 0.950/0.846 success rate/SPL exceeded the best PointGoal baseline's 0.827/0.730 on the Gibson val set.Both models used 10 million frames; transferred ANS was comparable to ANS trained from scratch.
- Sample efficiency: > 75× speedup was achieved compared with the best RL baseline.The RL model was trained for 75 million frames and achieved 0.678/0.486 Succ/SPL.
- Habitat Challenge: ANS was the winning entry in both RGB and RGB-D tracks among over 150 submissions.It achieved SPL 0.805 on RGB and 0.948 on RGB-D in the Test Challenge set.
C NEURAL SLAM MODULE IMPLEMENTATION DETAILS
The Neural SLAM module predicts egocentric maps from observations, corrects noisy pose estimates, and aggregates transformed predictions into a persistent geocentric map.
- Mapper: The Mapper converts the current RGB observation into an egocentric top-down spatial map predicting obstacles and explored area.It uses ResNet18, fully connected layers, and deconvolutional layers to produce the prediction.
- Pose alignment: A non-learned spatial transformation uses pose change to express the previous map prediction in the current egocentric frame.Its parameters are calculated from pose change rather than learned.
- Pose estimation: The Pose Estimator predicts a relative pose change from transformed consecutive map predictions and adds it to the previous pose estimate.This learned correction aligns recent egocentric predictions despite noisy sensor readings.
- Map aggregation: The current egocentric prediction is transformed into geocentric coordinates and aggregated with the previous map by channel-wise pooling.The resulting map and current pose estimate are outputs of Neural SLAM.
D ARCHITECTURE DETAILS
The architecture combines convolutional and fully connected networks with discretized state inputs and embedding layers for policy processing.
- Neural SLAM architecture: The Mapper uses ResNet18 convolutional layers, two fully connected layers with dropout, and three deconvolutional layers.The Pose Estimator uses three convolutional layers followed by three fully connected layers.
- Global Policy: The Global Policy uses a five-layer convolutional network followed by three fully connected layers.The passage also describes discretized relative distance, angle, and timestep inputs passed through embeddings.
- Input processing: Relative distance, relative angle, timestep, and short-term goals are discretized or embedded before policy processing.Relative angle uses 5-degree bins and timestep uses 30 bins.
E HYPERPARAMETER DETAILS
Training uses parallel environments, replay-based Neural SLAM updates, PPO for the Global Policy, and fixed map and vision ranges across training and evaluation.
- Neural SLAM training: Neural SLAM training uses 72 parallel threads, a FIFO memory of 500000, and 10 updates every 72 environment steps.The batch size is 72 and Adam uses learning rate 0.0001.
- Global Policy training: The Global Policy samples a new goal every 25 timesteps and is trained with PPO using coverage increase as reward.The reward is the increase in coverage in m2 scaled by 0.02.
- Local Policy training: The Local Policy is trained with binary cross-entropy loss and Adam at learning rate 0.0001.No additional optimization details are specified in the passage.
- Spatial settings: Training uses a 24m map while evaluation uses a 48m × 48m map, with a 3.2m Neural SLAM vision range.The map representation is adaptive because its parameters do not depend on map size.
F ADDITIONAL RESULTS
Figure 10 tracks absolute coverage in m2 over episode progress for ANS and baselines across Gibson validation scenes.
- Absolute coverage in m2 is plotted as episodes progress for ANS and baselines on large, small, and overall Gibson Val scenes.