PaperlayerView exact revision on arXiv

Source-linked AI summary

SG^2SLAM: Spatial Geometry-Informed 3D Gaussian Splatting SLAM for Monocular Outdoor Navigation

Ziang Guo, Didar Seyidov, Yixiao Zhou, Xuefeng Zhang, Ali Alridha Abdulkarim, Chen Min, Zufeng Zhang, Dzmitry Tsetserukou

arXiv:2610.36312v1cs.RO

TL;DR

Monocular outdoor 3DGS SLAM commonly recovers geometry through per-frame 2D–3D association, and the paper identifies this as a gap. SG2SLAM combines streaming spatial geometry with ray-conditioned 3DGS and joint chunk optimization. It surpasses existing outdoor-navigation baselines on KITTI and Waymo Open.

  • Problem

    Monocular outdoor 3DGS SLAM commonly recovers geometry through 2D–3D association, a cross-dimensional process that the paper identifies as a central gap.

    Source: §I · p. 1

  • Method

    SG2SLAM streams chunk-level spatial geometry, supervises ray-conditioned 3DGS primitives, and jointly optimizes chunk fusion over spatial geometry and poses.

    Source: §I · p. 1

  • Results

    SG2SLAM surpasses existing outdoor-navigation baselines, with average ATE RMSE of 6.24 m on 11 KITTI sequences and 0.63 m on 9 WOD sequences.

    Source: B. Experimental Results · p. 7, §V · p. 8

  • Takeaways & Limitations

    The paper presents streaming joint 3D–3D association as a paradigm for mapping and tracking in monocular outdoor navigation.

    Source: §V · p. 8

Abstract

from arXiv · show

Monocular outdoor navigation requires robust camera motion estimation and a dense, navigable map, where 3D Gaussian Splatting SLAM serves this task through an explicit differentiable parameterization and closed-form projection. However, typically, the geometry is recovered by associating 2D pixels with 3D points, so a modular SLAM system absorbs tracking error into the map, leading to cascading degradation. To address these gaps, we present SG^2SLAM, which couples streaming spatial geometry with a ray-conditioned 3DGS kernel. Streaming spatial geometry replaces the per- frame cross-dimensional correspondence with a chunk-level metric prior that supervises tracking and mapping from one source. A ray-conditioned kernel extends each primitive with a viewing-direction subspace that senses the monocular camera viewpoints. The joint optimization over spatial geometry and poses ensures a globally consistent map and trajectory outcome. Experiments on the KITTI and Waymo Open datasets show that our SG^2SLAM surpasses the per-frame association baselines in the monocular outdoor navigation task. Code will be made publicly available upon acceptance.

I. INTRODUCTION

The introduction identifies 2D–3D association and photometric ambiguity as challenges for monocular outdoor 3DGS SLAM. SG2SLAM instead uses streaming spatial geometry, ray-conditioned primitives, and joint chunk fusion; the related work shows that these choices address gaps persisting across monocular SLAM, 3D foundation-model, and 3DGS-SLAM approaches.

I. INTRODUCTION

  • In modular monocular SLAM, cross-dimensional matching errors can enter the map and degrade subsequent matching, while photometric ambiguity can leave geometrically harmful artifacts for navigation.The introduction names floating Gaussians, broken thin structures, and inconsistent surface normals, which can pollute depth used for collision checking and traversability.Source: §I · p. 1
  • SG2SLAM replaces per-frame matching with streaming spatial geometry, ray-conditioned 3DGS primitives, and joint global optimization of chunks over geometry and poses.The stated contributions include improved frontend tracking and backend mapping relative to pairwise 2D matching baselines, and closed-form rendering intended to remove grazing-angle blending that may produce floating primitives.Source: §I · pp. 1–2

A. Classical and Learning-Based Monocular SLAM

  • Classical and learning-based monocular SLAM methods recover depth indirectly through 2D-to-3D association, whose representation affects association accuracy and cost.The related-work discussion spans feature-based, photometric, dense-flow, and sparse-patch systems.Source: A. Classical and Learning-Based… · p. 2

B. SLAM with 3D Foundation Models

  • 3D foundation-model SLAM predicts 3D structure from images, but cited approaches still rely on submap registration, sliding windows, or pairwise alignment to recover geometric similarity.The passage describes systems including MASt3R-SLAM, SLAM3R, VGGT-SLAM, and VGGT-Long.Source: B. SLAM with 3D Foundation Models · p. 2

C. 3D Gaussian Splatting SLAM

  • Existing 3DGS-SLAM systems use varied tracking and mapping strategies, yet their image-dominated maps leave the cross-dimensional cost of 2D–3D matching in place.The passage also describes depth-sensor systems, external trackers, learned monocular trackers, loop closure, and chunk-based methods.Source: C. 3D Gaussian Splatting SLAM · p. 2

III. METHODOLOGY

SG2SLAM combines streaming spatial geometry with monocular-depth-aware 3DGS and joint optimization across chunks. Its frontend streams pose, point-map, and confidence predictions in overlapping frame chunks to provide geometry for mapping and fusion.

III. METHODOLOGY

  • The system couples a spatial geometry prior with monocular-depth-aware 3DGS, using photometric loss, metric-depth supervision, point consistency, and joint Sim(3) optimization across chunks.The joint global pose-graph solve optimizes chunk fusion over spatial geometry and poses to mitigate scale-induced drift.Source: §III · p. 2

A. Streaming Spatial Geometry Prior

  • A frozen reconstruction network processes a sliding window and predicts local camera poses, dense point maps, and per-pixel confidence for frames in the window.The point maps are predicted on normalized pixel rays, with range represented in log space.Source: A. Streaming Spatial Geometry Prior · p. 2

1) Chunking Setup:

  • The input sequence is divided into overlapping chunks, each streamed through a forward network call with attention restarting at the chunk’s first frame.The stride is the chunk length minus the overlap, giving sequences of different lengths identical coverage.Source: 1) Chunking Setup · p. 3

2) Metric Scale Correction:

Metric scale correction uses a second depth network to scale each chunk’s local reconstruction. The resulting anchor scale is applied to chunk poses and point clouds to produce metric reconstructions.

2) Metric Scale Correction:

  • A metric depth network supplies range estimates, and the ratio between its ranges and the reconstruction prior provides the chunk’s scale factor.The anchor scale is obtained for the chunk’s first frame and held across the chunk.Source: 2) Metric Scale Correction · p. 3
  • The ratio is filtered using a robust spread based on the median, with 1.4826·MAD converting median absolute deviation to Gaussian standard deviation.The cited condition retains ratios within the stated robust spread.Source: 2) Metric Scale Correction · p. 3
  • A uniform dilation applies the anchor scale to relative poses and point clouds, yielding a metric reconstruction for each chunk.The passage marks the scaled point cloud with a superscript and identifies metric poses and point clouds as frontend outputs.Source: 2) Metric Scale Correction · p. 3

B. Monocular Depth-Aware 3DGS

SG2SLAM extends each Gaussian with spatial and viewing-direction subspaces, then conditions its rendered behavior on camera rays. Monocular spatial geometry supervises this conditioning and supports consistency across views.

B. Monocular Depth-Aware 3DGS

  • Each primitive’s mean and covariance span spatial dimensions and a conditioning viewing-direction subspace, alongside base opacity and color.The covariance uses a block lower-triangular Cholesky factor.Source: B. Monocular Depth-Aware 3DGS · p. 3
  • The spatial covariance block combines a first-order rotation with anisotropic scales, while free sub-diagonal blocks couple spatial and conditioning subspaces.The per-dimension shape parameter is activated as ζ_i = 4 exp(ζ̃_i), with damped exponents controlling conditioning strength.Source: B. Monocular Depth-Aware 3DGS · p. 3

1) Ray-Conditioned Kernel:

  • At rendering, each primitive is queried with the camera’s unit viewing ray, and its response uses a regression coefficient damped by the shape parameter.The conditional statistics interpolate between view-independent and fully conditioned behavior; opacity smoothly vanishes for rays inconsistent with the conditioning subspace.Source: 1) Ray-Conditioned Kernel · p. 4

2) Rasterization:

The rasterizer uses a ray-conditioned kernel for color and depth, while map optimization jointly refines Gaussian primitives and keyframe poses using photometric, depth, and geometric constraints.

2) Rasterization:

  • The ray-conditioned kernel replaces unbounded Gaussian tails with a polynomially decaying profile that vanishes at σ_i = 1, eliminating grazing-angle blending artifacts while retaining a closed-form derivative.Color and depth are composited front-to-back by alpha-blending; expected depth uses the same weights as color and remains differentiable through the kernel.Source: 2) Rasterization · p. 4

3) Map Optimization:

  • Keyframes are selected by counts of finite, confident points, and their metrically anchored point maps are appended with scales initialized from nearest-neighbor distances.The initialization uses the mean distance to each point’s three nearest neighbors so primitive footprints match local prior sampling density.Source: 3) Map Optimization · p. 4
  • Windowed bundle adjustment jointly optimizes primitive parameters and keyframe poses using photometric, monocular metric-depth, and geometric-regularization terms.Depth supervision is weighted by the intersection of valid pixels and the depth network’s confidence.Source: 3) Map Optimization · p. 4
  • Keyframe geometry is tied to the world frame to prevent pose error from propagating into the map, while cumulative chunk optimization refines geometry without disturbing built maps.Relative-pose terms penalize discrepancies between estimated inter-keyframe transforms and their priors.Source: 3) Map Optimization · p. 4

C. Global Joint Trajectory and Chunk Fusion

Each locally anchored chunk is placed in a global frame through a similarity transform, and the chunk placements are jointly optimized. A closed-form fit between matched point sets supplies a scale estimate tied to spatial density.

C. Global Joint Trajectory and Chunk Fusion

  • Each refined chunk is a self-consistent reconstruction in a local anchor frame and is fused into the global frame using a similarity transform.The first chunk is fixed to the identity, leaving 7(K−1) parameters for K chunks to optimize.Source: C. Global Joint Trajectory and Chunk… · pp. 4–5
  • A closed-form similarity fit between matched point sets recovers scale from their centered coordinates and the rotation aligning the sets.Applied to point clouds, the scale carries the density ratio of the matched spatial regions.Source: C. Global Joint Trajectory and Chunk… · p. 5

1) Chain Constraints:

Overlapping frames provide constraints for the relative similarity between adjacent chunks. The method combines camera-center alignment and point-cloud refinement while keeping scale fixed during the latter.

1) Chain Constraints:

  • Adjacent chunks use shared-frame camera centers to constrain their relative similarity, with edge scale combining the shared-center fit and chunk metric anchors by geometric mean.The camera-center chain scale is independent of point-cloud sampling-density bias because the spatial prior supplies one center per frame.Source: 1) Chain Constraints · p. 5
  • Spatial geometry across the shared window supplies rotation and translation, with calibrated camera centers used to seed alignment between chunks.The shared frame set is the intersection of the chunks’ frame sets.Source: 1) Chain Constraints · p. 5
  • Refinement aligns shared-frame point clouds using trimmed correspondences while pinning the scale, leaving rotation and translation free to adjust.The scale is fixed to the estimated inter-chunk value.Source: 1) Chain Constraints · p. 5

2) Loop Constraints:

Loop constraints use revisits between non-adjacent chunks to make accumulated drift observable and correctable. Joint optimization combines chain and loop measurements, then fuses chunk placements into a continuous trajectory.

2) Loop Constraints:

  • A revisit between non-adjacent chunks closes a pose-graph cycle, making the drift between observations observable and correctable.The rotation consensus uses view orientations when near-coincident centers provide no scale to recover.Source: 2) Loop Constraints · pp. 5–6
  • Loop-edge scale is estimated from median per-pixel range ratios within a valid range band, then combined across matches using retrieval scores.The median is taken over pixels observing one patch to avoid density bias in cloud-to-cloud alignment.Source: 2) Loop Constraints · pp. 5–6

3) Joint Fusion and Optimization:

  • The global solve combines chain and loop similarity measurements, weighting direct loop-scale measurements highly and chain edges weakly because their scales inherit chunk error.The residual accounts for measurement uncertainty; a loop edge can settle scale error accumulated across preceding chain boundaries.Source: 3) Joint Fusion and Optimization · p. 6
  • Solved chunk placements are blended across frame overlaps using weighted means for scale and translation and a weighted chordal mean for rotation, yielding a continuous fused trajectory.The transition is spread across the full overlap; frames covered by one chunk retain that chunk’s placed pose.Source: 3) Joint Fusion and Optimization · p. 6

A. Datasets and Experimental Setup.

Experiments evaluate outdoor monocular tracking and mapping on KITTI and Waymo Open, with additional tracking tests on a self-collected driving sequence. The setup uses streaming frame chunks and reports rendering quality on KITTI.

A. Datasets and Experimental Setup.

  • The experiments run on an Nvidia RTX 5090 GPU with 32 GB VRAM, using 200 streaming frames per chunk.The frontend uses 30 overlapping frames.Source: A. Datasets and Experimental Setup · pp. 6–7
  • Experiments use the KITTI Odometry training sequences 00–10 and nine Waymo Open perception-split sequences, alongside a self-collected driving sequence for tracking without loop closure.The KITTI and Waymo tests use monocular color streams; the self-collected sequence uses an automotive monocular camera with ground truth from a sensor suite.Source: A. Datasets and Experimental Setup · p. 7
  • KITTI mapping performance is evaluated through rendering quality of 3DGS checkpoints on keyframes.Table I is identified as rendering performance on KITTI.Source: A. Datasets and Experimental Setup · p. 7, Table I · p. 7

B. Experimental Results

SG2SLAM outperforms existing baselines on outdoor monocular tracking benchmarks, and reports rendering-quality results on KITTI. Qualitative comparisons, ablations, and the conclusion describe its mapping behavior, design effects, and scope for future robustness work.

B. Experimental Results

  • 6.24 m average ATE RMSE on 11 KITTI sequences and 0.63 m on 9 Waymo Open sequences; SG2SLAM surpasses existing baselines in outdoor navigation.Without loop closure, it averages 4.62 m ATE across KITTI Sequence 07 and the authors’ driving sequence, exceeding GigaSLAM by 21%.Source: B. Experimental Results · p. 7, Table III · p. 7
  • 22.70 dB average PSNR, 0.86 average SSIM, and 0.41 average LPIPS are reported for KITTI 3DGS rendering without post color refinement.The reported evaluation uses a panoramic aspect ratio.Source: B. Experimental Results · p. 7
  • The per-frame cost comparison identifies backend chunk fusion as the dominant wall-clock cost, while the reconstruction and metric-depth models set the frontend budget.The comparison is shown on a logarithmic scale.Source: B. Experimental Results · p. 7

C. Qualitative Results

  • Qualitative KITTI comparisons show rendered RGB and depth images without post color refinement, and the authors report high photorealistic and depth fidelity.The comparisons follow tracking and mapping.Source: C. Qualitative Results · p. 8

D. Ablation Study

  • Ablations report that the ray-conditioned kernel improves mapping metrics, while chunk size affects both frontend and backend through the amount of prior noise propagated into fusion and mapping.The ablations also indicate that frame-wise tracking errors may accumulate along a run.Source: D. Ablation Study · p. 8, Table IV · p. 8

V. CONCLUSION

  • SG2SLAM dispenses with per-frame 2D–3D association and pairwise image matching; the conclusion identifies robustness under unstructured conditions such as rainy nights and wild forests as future work.The stated evaluation includes KITTI, the authors’ real-world driving sequence, and Waymo Open.Source: §V · p. 8

Related papers

Loading 2610.36312v1…