Source-linked AI summary
GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction
Youmin Zhang, Fabio Tosi, Stefano Mattoccia, Matteo Poggi
TL;DR
Real-time dense SLAM must avoid tracking drift and reconstruction distortion while maintaining accurate, high-fidelity geometry. GO-SLAM addresses this with global pose optimization, loop closing, full bundle adjustment, and continuous implicit mapping, achieving state-of-the-art performance across datasets and camera modalities. Its main scope boundary is the limited flexibility of existing discrete representations and the global-consistency limitations identified in prior NeRF-based SLAM.
Problem
Real-time online reconstruction remains challenging because camera motion, tracking errors, and limited representation flexibility can compromise accurate, high-fidelity 3D geometry.
Method
GO-SLAM jointly performs global pose optimization with loop closing and online full bundle adjustment while continuously updating a neural implicit 3D reconstruction.
Results
GO-SLAM achieves state-of-the-art tracking and dense mapping performance across various datasets, including long monocular trajectories without depth information.
Takeaways & Limitations
The framework provides globally consistent real-time reconstruction with monocular, stereo, or RGB-D input.
Takeaways & Limitations
Discrete point, surfel, and volumetric representations lack flexibility in shape extraction, while prior NeRF-based SLAM lacks online loop closing and full bundle adjustment.
Abstract
from arXiv · showhide
Neural implicit representations have recently demonstrated compelling results on dense Simultaneous Localization And Mapping (SLAM) but suffer from the accumulation of errors in camera tracking and distortion in the reconstruction. Purposely, we present GO-SLAM, a deep-learning-based dense visual SLAM framework globally optimizing poses and 3D reconstruction in real-time. Robust pose estimation is at its core, supported by efficient loop closing and online full bundle adjustment, which optimize per frame by utilizing the learned global geometry of the complete history of input frames. Simultaneously, we update the implicit and continuous surface representation on-the-fly to ensure global consistency of 3D reconstruction. Results on various synthetic and real-world datasets demonstrate that GO-SLAM outperforms state-of-the-art approaches at tracking robustness and reconstruction accuracy. Furthermore, GO-SLAM is versatile and can run with monocular, stereo, and RGB-D input.
1. Introduction
GO-SLAM addresses the difficulty of achieving accurate, globally consistent real-time reconstruction during online camera motion. It combines global pose optimization, efficient loop closure, and continuously updated implicit reconstruction for monocular, stereo, and RGB-D inputs.
- Online 3D reconstruction must balance high fidelity, spatial resolution, real-time performance, and accurate handling of camera motion.
- Existing discrete point, surfel, and volumetric representations lack flexibility in shape extraction, limiting high-fidelity reconstruction.
- NeRF-based SLAM provides continuous, arbitrary-resolution representations but can accumulate camera-tracking drift and reconstruction distortion without global online optimization.
- GO-SLAM globally optimizes poses using complete frame history, efficient loop closure, and online full bundle adjustment.
- Its instant implicit reconstruction continuously updates the 3D model with newly optimized global geometry for real-time reconstruction.
- GO-SLAM is designed for joint robust pose estimation and dense reconstruction with monocular, stereo, or RGB-D cameras.
2. Related Work
Prior dense SLAM systems use discrete representations and increasingly adopt neural implicit fields for continuous reconstruction. GO-SLAM extends this direction with online loop closing and full bundle adjustment to support globally consistent mapping.
- Dense SLAM representations include depth points, height maps, surfels, and volumetric models, with some systems adding global bundle adjustment and loop closure.
- NeRF-based visual SLAM enables scene completeness, continuous arbitrary-resolution modeling, and operation with RGB-D or RGB-only inputs.
- The architecture combines front-end tracking with keyframe initialization and loop closing, back-end tracking with full bundle adjustment, and instant mapping.
- Prior related approaches lack online loop closing and full bundle adjustment, restricting globally consistent 3D reconstruction.
3. Method
GO-SLAM combines keyframe-based tracking, loop closing, online full bundle adjustment, and instant implicit mapping to maintain globally consistent poses and 3D reconstruction in real time.
- Tracking with Global Optimization: GO-SLAM performs global pose optimization through loop closing and online full bundle adjustment over keyframe graphs.Loop closing detects historical revisits, while full bundle adjustment refines poses online in a separate thread.
- Tracking with Global Optimization: The front end accepts monocular, stereo, or RGB-D video and creates keyframes when average optical flow exceeds a threshold.A recurrent RAFT-based update operator compares each new frame with the latest keyframe.
- Tracking with Global Optimization: Integrating loop closing and full bundle adjustment produces a more complete and accurate model than either component alone in the qualitative ScanNet example.The combined system also achieves real-time performance.
- Tracking with Global Optimization: The tracking objective jointly updates camera poses and dense per-pixel depths by minimizing confidence-weighted flow reprojection errors.A differentiable Dense Bundle Adjustment layer uses damped Gauss-Newton optimization for the local keyframes.
- Instant Mapping: The instant mapping thread updates the global implicit reconstruction using optimized geometry while selecting keyframes to preserve consistency and efficiency.It snapshots poses and depths, includes the latest two and unmapped keyframes, and optimizes the scene over sampled pixels.
- Instant Mapping: GO-SLAM renders each selected keyframe through stratified and depth-focused ray samples, with hash-encoded SDF and color networks predicting the scene representation.Predicted color and depth are accumulated using unbiased volume rendering.
4. Experimental Results
GO-SLAM is evaluated across synthetic and real-world datasets, input modalities, and ablations, showing robust tracking, globally consistent reconstruction, and real-time performance. Its strongest results arise from combining online loop closing, full bundle adjustment, and continuous implicit-model updates.
- Evaluation setup: GO-SLAM is evaluated on TUM RGB-D, EuRoC, ETH3D-SLAM, ScanNet, and Replica using monocular, stereo, and RGB-D inputs.Trajectory accuracy is evaluated with ATE RMSE, while reconstruction uses Accuracy, Completion, Completion Ratio, F-score, and Depth L1.
- TUM RGB-D: GO-SLAM achieves the lowest average trajectory error among deep-learning monocular methods on challenging TUM RGB-D sequences with online loop closing and full BA.Traditional point-based SLAM fails on most challenging sequences, while GO-SLAM is also evaluated with RGB-D input.
- EuRoC: On EuRoC, GO-SLAM produces trajectories comparable to state-of-the-art stereo SLAM and accurate predictions across all monocular sequences.Its reconstruction is described as more complete, smoother, and cleaner than DROID-SLAM with TSDF-Fusion, while NeRF-based SLAM without online BA fails in most monocular cases.
- ETH3D-SLAM: On ETH3D-SLAM, GO-SLAM outperforms published point-, surfel-, and voxel-based RGB-D systems in successful trajectories under a maximum ATE RMSE of 2cm.Compared with BundleFusion, GO-SLAM yields smoother reconstruction and photometrically convincing rendering.
- Replica: Replica results show comparable accuracy to existing RGB-D and concurrent monocular methods, while GO-SLAM runs at 8 FPS using 18 GB maximum GPU memory.The paper reports better depth-rendering quality than comparable systems lacking dense reconstruction results.
- Ablation study: Ablations show that largest-change keyframe selection, integrated geometric and RGB losses, loop closing, and full BA improve globally consistent reconstruction and pose estimation.Loop closing reduces drift with negligible speed reduction; full BA improves global pose estimation but slows the system, while frame skipping causes minimal degradation.
5. Conclusions
GO-SLAM is presented as a real-time SLAM algorithm for globally consistent reconstruction across monocular, stereo, and RGB-D inputs. It combines loop closure, online full bundle adjustment, and continuously updated NeRF-based reconstruction.
- GO-SLAM achieves globally consistent reconstruction with monocular, stereo, or RGB-D input.
- The system explicitly detects loop closures and performs online full bundle adjustment to minimize trajectory error.
- Its NeRF-based reconstruction provides an efficient, compact, multi-resolution representation that is continuously updated with newly optimized global geometry.
- Experiments demonstrate robust tracking and dense mapping in large-scale scenes, especially along long monocular trajectories without depth information.
- The experiments report state-of-the-art performance on various datasets.
– Supplementary Material
The supplementary material identifies the authors and their institutional affiliation at the University of Bologna, Italy.
- The listed authors are Youmin Zhang, Fabio Tosi, Stefano Mattoccia, and Matteo Poggi.
- The authors are affiliated with the Department of Computer Science and Engineering (DISI).
- The department is located at the University of Bologna, Italy.
A.1. Rendering Network Details.
The rendering architecture uses separate signed-distance-function and color networks with multi-resolution hash encodings. Training uses AdamW with different learning rates for hash encodings and network parameters.
- The rendering networks consist of one SDF network and one color network.Figure A presents the rendering-network details.
- Multi-resolution hash encodings use feature dimension 2, 219 maximum entries per level, and 16 levels spanning resolutions from 16 to 4096.
- The SDF and color networks use 32 and 64 hidden units, respectively.
- AdamW is used with learning rates of 10−2 for learnable hash encodings and 10−3 for SDF and color-network parameters.
B. More Qualitative Results
Additional qualitative results are provided for ScanNet, EuRoC, and Replica. The listed scenes include scene0000_00, scene0054_00, and scene0181_00.
- Qualitative results are provided on the ScanNet dataset.The results appear in Figure B.
- Qualitative results are provided on the EuRoC dataset.The results appear in Figure C.
- Qualitative results are provided on the Replica dataset, including scene0000_00, scene0054_00, and scene0181_00.The results appear in Figure D.
GT Mesh NICE-SLAM
Across ScanNet and EuRoC, GO-SLAM produces cleaner, more complete, globally consistent reconstructions than NICE-SLAM and DROID-SLAM, while operating in real time with monocular or RGB-D input.
- On ScanNet, GO-SLAM predicts globally consistent 3D reconstruction with monocular or RGB-D input at real-time speed, whereas NICE-SLAM suffers accumulated trajectory errors and model distortion.
- On EuRoC, GO-SLAM reconstructions are much cleaner and more complete than DROID-SLAM reconstructions obtained with TSDF-Fusion.
- On Replica, GO-SLAM produces dense 3D reconstruction with monocular or RGB-D input at real-time speed, while NICE-SLAM takes hours at much less than 1 FPS.