Source-linked AI summary
ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields
Mohammad Mahdi Johari, Camilla Carta, François Fleuret
TL;DR
Dense visual SLAM needs to reconstruct unknown environments while estimating camera poses, yet learning-based methods provide only limited reconstruction accuracy and existing neural implicit systems can be computationally demanding. ESLAM uses TSDF geometry and multi-scale axis-aligned feature planes with shallow decoders for incremental RGB-D SLAM. Across three datasets, it outperforms existing methods in reconstruction and localization accuracy while running up to one order of magnitude faster, without pre-training.
Problem
Recent learning-based dense visual SLAM methods provide meaningful global maps but limited reconstruction accuracy, while existing systems can be computationally demanding and rely on frozen pre-trained MLPs.
Method
ESLAM represents geometry with implicit TSDF and uses multi-scale axis-aligned feature planes decoded into TSDF and RGB values.
Results
ESLAM significantly outperforms existing methods in reconstruction and localization accuracy while running up to one order of magnitude faster.
Takeaways & Limitations
Feature planes avoid cubic model-size growth, while TSDF and plane smoothness support efficient, high-quality reconstruction without explicit smoothness losses.
Takeaways & Limitations
ESLAM allocates substantial computation to retaining previously reconstructed regions because feature updates can affect earlier geometries, leaving more efficient forgetting handling as future work.
Abstract
from arXiv · showhide
We present ESLAM, an efficient implicit neural representation method for Simultaneous Localization and Mapping (SLAM). ESLAM reads RGB-D frames with unknown camera poses in a sequential manner and incrementally reconstructs the scene representation while estimating the current camera position in the scene. We incorporate the latest advances in Neural Radiance Fields (NeRF) into a SLAM system, resulting in an efficient and accurate dense visual SLAM method. Our scene representation consists of multi-scale axis-aligned perpendicular feature planes and shallow decoders that, for each point in the continuous space, decode the interpolated features into Truncated Signed Distance Field (TSDF) and RGB values. Our extensive experiments on three standard datasets, Replica, ScanNet, and TUM RGB-D show that ESLAM improves the accuracy of 3D reconstruction and camera localization of state-of-the-art dense visual SLAM methods by more than 50%, while it runs up to 10 times faster and does not require any pre-training.
ams OSRAM
ESLAM combines TSDF geometry with multi-scale axis-aligned feature planes to improve dense visual SLAM reconstruction and localization efficiency. Experiments across three datasets report stronger accuracy, smoother detail preservation, and substantially faster execution than prior methods.
- ESLAM reconstructs scene details more accurately than iMAP* and NICE-SLAM while running up to ×10 faster.
- ESLAM uses implicit TSDF geometry, which converges faster than volume-density or occupancy representations and produces higher-quality reconstruction.
- Multi-scale axis-aligned feature planes reduce memory-footprint growth with scene side-length from cubic to quadratic.
- Across Replica, ScanNet, and TUM RGB-D, ESLAM is benchmarked against existing methods through performance evaluations and an extensive ablation study.
- Feature-plane smoothness yields higher-quality smooth surfaces without explicit smoothness loss functions.
2. Related Work
NeRF-based dense SLAM methods improve implicit scene representation, but existing systems remain computationally demanding, detail-limited, or dependent on frozen pre-trained components. ESLAM addresses these issues with compact plane-based features decoded directly into TSDF.
- NeRF supports applications including novel-view synthesis, surface reconstruction, dynamic-scene representation, and camera-pose estimation.
- iMAP uses a single large MLP and optimizes camera poses during rendering, while NICE-SLAM stores local voxel-grid features decoded into occupancies with pre-trained MLPs.
- Existing neural implicit SLAM methods offer promising reconstruction quality but remain computationally demanding, capture limited geometry detail, or have limited generalizability to novel scenes.
3. Method
ESLAM represents geometry and appearance with multi-scale axis-aligned feature planes, decodes interpolated features into TSDF and color, and renders RGB-D observations for mapping and tracking. Its TSDF-based losses supervise free space and surface regions while depth and color rendering losses support reconstruction and localization.
- Pipeline: ESLAM processes sequential RGB-D frames using estimated camera poses, samples points along rays, and incrementally predicts scene geometry and appearance.For each sampled point, the system queries TSDF and raw color values and uses SDF-based rendering to estimate pixel depth and color.
- Axis-Aligned Feature Planes: Multi-scale tri-planes store features on perpendicular axis-aligned planes, avoiding the cubic scene-side memory growth of voxel grids.The representation uses coarse and fine feature planes; coarse features support efficient free-space reconstruction, while the plane architecture prevents model size from growing cubically with scene side-length.
- Axis-Aligned Feature Planes: Bilinear interpolation and scale-wise feature aggregation form geometry and appearance inputs that shallow two-layer MLPs decode into TSDF and raw color.Features are interpolated from four nearest neighbors on each plane, summed within each scale, concatenated across scales, and passed to separate decoders.
- SDF-Based Volume Rendering: SDF-based volume rendering converts TSDF values into densities with a learnable sharpness parameter, then uses density weights to render colors and depths.Negative SDF values drive the sigmoid toward one, producing volume density inside surfaces; weighted samples produce each ray’s rendered color and depth.
- Loss Functions: TSDF supervision combines free-space, truncation-region, depth-rendering, and color-rendering losses to train the scene representation.The global objective weights free-space and two truncation-region losses together with rendered depth and color consistency terms.
- Loss Functions: Reducing the mapping weight of tail truncation points yields a smaller effective truncation distance, fewer occlusion artifacts, and higher reconstruction accuracy while retaining the full distance for tracking.The method differentiates points near the surface from those at the truncation-region tail.
4. Experiments
ESLAM is evaluated against NeRF-based dense visual SLAM methods on Replica, ScanNet, and TUM RGB-D, measuring reconstruction, localization, scalability, and runtime. Across these evaluations, it reports more accurate and stable localization and reconstruction, faster processing, and compact scene-size scaling.
- Experimental setup: ESLAM is compared with iMAP and NICE-SLAM on Replica, ScanNet, and TUM RGB-D using reconstruction, localization, and runtime evaluations.The experiments use the same scenes as NICE-SLAM and include qualitative and quantitative analyses.
- Replica: ESLAM outperforms baselines on Replica for reconstruction and localization accuracy, with lower variance across five independent runs.The lower variances indicate greater stability and robustness in the reported experiments.
- Replica: ESLAM reconstructs Replica scene details more accurately, produces fewer artifacts, and yields higher-quality reconstructed colors.Color quality is reported as an additional qualitative result rather than the paper’s main focus.
- ScanNet: ESLAM achieves more accurate and more stable ScanNet localization, with no large drifting reported in qualitative trajectory comparisons.ScanNet localization is evaluated using mean and RMSE of ATE over five runs per scene.
- ScanNet: ESLAM produces more precise ScanNet geometry and detailed textures, while qualitative geometry evaluation is used because the dataset’s ground-truth meshes are incomplete.The incomplete meshes prevent the paper from providing quantitative ScanNet geometry comparisons.
- Runtime and scalability: ESLAM is significantly faster than previous methods, uses a smaller model than NICE-SLAM, and avoids cubic model-size growth with scene side-length.Runtime and model-size comparisons use room0 from Replica and scene0000 from ScanNet on an NVIDIA GeForce RTX 3090 GPU.
5. Conclusion
ESLAM combines feature planes and implicit TSDF modeling to improve neural implicit SLAM efficiency, reconstruction, and localization. The method retains prior geometry through keyframes, at a computational cost that remains an open optimization target.
- ESLAM replaces voxel grids with axis-aligned feature planes, preventing model size from growing cubically with scene side-length.
- TSDF geometry modeling enables efficient, high-quality surface reconstruction.
- ESLAM outperforms existing methods in reconstruction and localization accuracy while running up to one order of magnitude faster.
- Updating feature planes for new geometry can affect previously reconstructed regions, so ESLAM retains prior keyframes using substantial computation.
- More efficient handling of forgetting could further reduce ESLAM's frame processing time.
Supplementary Materials for ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of
The supplementary materials identify the paper's authors and publication record. They list affiliations with Idiap, EPFL, and the University of Geneva.
- Mohammad Mahdi Johari is affiliated with the Idiap Research Institute and EPFL.
- François Fleuret is affiliated with the University of Geneva and EPFL.
- The supplementary material identifies the paper as arXiv:2211.11704v2, dated 3 Apr 2023.
1. Further Implementation Details
The implementation uses multi-scale feature planes, shallow decoders, alternating mapping and tracking losses, and dataset-specific sampling and optimization settings. Evaluation extracts meshes from TSDF volumes with standardized culling.
- Representation and decoders: The implementation uses a 6 cm truncation distance, multi-resolution geometry and appearance planes, 32 channels per plane, and two-layer MLP decoders.Coarse planes use 24 cm resolution; fine geometry and appearance planes use 6 cm and 3 cm, respectively.
- Optimization: Mapping and tracking use different loss coefficients selected through grid search.
- Dataset-specific settings: Replica and ScanNet use different stratified-sampling and mapping settings, with more samples and iterations for ScanNet.
- Mapping and tracking: Mapping starts every 4 input frames and jointly optimizes feature planes, MLP decoders, and camera poses over W = 20 keyframes.
- Optimization: Adam optimizes feature planes, decoders, and selected-keyframe camera poses with separately specified learning rates.
- Mesh evaluation: Evaluation builds a TSDF volume, extracts meshes with marching cubes, and applies only standardized camera-frustum and occlusion culling.The same mesh culling is applied to previous approaches for fairness.
2. Ablation Study
The ablation study tests robustness to depth quality, keyframe policies, and architectural design choices across Replica and ScanNet. ESLAM's reconstruction and localization are less sensitive to reduced depth resolution.
- Study scope: The experiments evaluate robustness across different settings and validate ESLAM's architectural design choices.
- Robustness to Depth Quality: ESLAM's reconstruction and localization are less sensitive to input-depth-map resolution.
- Keyframe Policy: NICE-SLAM only slightly benefits from adopting ESLAM's keyframe updating policy.
- Our Design Choices: Design ablations compare shared, coarse-only, fine-only, and concatenated feature-plane configurations across Replica and ScanNet.
- Our Design Choices: Additional ablations remove importance sampling, RGB inputs, separate truncation-region losses, or joint camera-pose optimization during mapping.
3. Additional Qualitative Analysis
On Replica, ESLAM preserves scene details better than iMAP* and NICE-SLAM, producing more accurate meshes with fewer artifacts.
- ESLAM produces more accurate meshes with fewer artifacts than iMAP* and NICE-SLAM on Replica.
4. Per-Scene Breakdown of the Results
Across all Replica scenes, ESLAM outperforms previous approaches and shows lower run-to-run variance, indicating more stable performance.
- ESLAM outperforms iMAP* and NICE-SLAM in every Replica scene.The per-scene comparison evaluates both reconstruction and localization accuracy.
- Lower experimental variances indicate that ESLAM is more stable from run to run.
5. Effect of Frame Processing Time
Increasing optimization iterations improves ESLAM’s reconstruction and camera trajectory accuracy but increases frame processing time. Even the 10× iteration setting remains faster than NICE-SLAM, while producing smoother geometry.
- Iteration settings: ESLAM uses 15 mapping iterations and 8 tracking iterations by default; ESLAM x2 doubles both, while ESLAM x10 uses 150 and 80.
- Accuracy–runtime trade-off: More optimization iterations increase frame processing time while improving scene reconstruction and camera trajectory accuracy.
- Geometry quality: Increasing optimization iterations produces more accurate geometry reconstruction and smoother surfaces, although the qualitative difference is subtle.
- Evaluation: The evaluation reports reconstruction accuracy, reconstruction completion, and ATE RMSE, alongside average frame processing time.