PaperlayerView exact revision on arXiv

Source-linked AI summary

LingGS: Decoupling Feed-Forward Geometry Prediction from Camera Tracking for High-Fidelity Gaussian SLAM

Da Li, Yihan Zang, Ivan Viola, Zeyu Wang, Dominik Engel

arXiv:2610.36853v1cs.CVcs.GR

TL;DR

Monocular Gaussian SLAM must estimate pose and build a scene from RGB while balancing reconstruction quality and online computation. LingGS uses a frozen streaming model for depth, an explicit geometry solver for camera tracking, and incremental Gaussian mapping. It reports improved rendering and geometric accuracy across indoor and outdoor datasets, with solver gains across streaming backbones.

  • Problem

    Monocular Gaussian SLAM must jointly recover camera pose, scene geometry, and appearance from RGB under a limited computational budget, while obtaining consistent dense depth and poses is challenging.

    Source: §1 · p. 1

  • Method

    LingGS uses a frozen streaming 3D model as a virtual RGB-D sensor and an explicit geometric solver that combines image features with predicted depth for camera tracking.

    Source: §1 · p. 1, §5 · p. 6

  • Results

    Across indoor and outdoor datasets, LingGS improves rendering quality and geometric accuracy; its solver reduces ATE by up to 96.7% relative to native camera heads across evaluated backbones.

    Source: §4.3 · p. 5, §5 · p. 6

  • Takeaways & Limitations

    The solver’s improvements across multiple streaming backbones support its broader applicability to camera tracking with streaming geometry models.

    Source: §4.3 · p. 5, §5 · p. 6

    The system does not address loop closure or dynamic scenes, and ORB matching can compromise tracking in repetitive or weakly textured environments.

    Source: §5 · p. 6

Abstract

from arXiv · show

Achieving high-quality reconstruction and responsive camera tracking remains challenging for monocular RGB Gaussian splatting SLAM under a limited optimization budget. We present an online system that addresses this challenge by combining learned geometric priors with explicit geometric camera tracking, which can substantially outperform learned camera prediction in terms of tracking accuracy. A frozen streaming 3D foundation model serves as a virtual RGB-D sensor, providing depth estimation from monocular video. An explicit geometric solver then estimates camera motion through feature reprojection, and a streaming Gaussian mapper integrates the resulting observations into a dense scene representation. We further show that geometric tracking improves camera pose accuracy across representative streaming geometry backbones, demonstrating the broader applicability of our approach. Extensive experiments on indoor and outdoor datasets show that our method significantly improves reconstruction quality over existing approaches, achieving about 7 FPS on a single RTX 4090 and delivering 2-3 times the throughput of the evaluated prior SLAM systems.

1 Introduction

Monocular Gaussian SLAM must recover camera pose, scene geometry, and appearance from RGB under a limited online budget. LingGS combines streaming depth prediction with explicit geometric tracking and Gaussian mapping to improve reconstruction and tracking.

1 Introduction

  • The challenge is to obtain consistent dense depth and camera poses together within a limited online computational budget.Depth determines Gaussian initialization, while poses align observations across frames; poor initialization requires refinement and inaccurate poses can blur the scene.Source: §1 · p. 1
  • Streaming 3D foundation models provide temporally consistent dense depth and camera poses, but their predicted poses can cause cross-frame misalignment.LingGS uses the depth predictions as dense surface estimates and replaces learned camera prediction with an explicit solver combining 2D features and predicted depth.Source: §1 · p. 1
  • The system achieves higher rendering quality than previous baselines across indoor and outdoor datasets, with the lowest depth error and competitive tracking error; it requires no additional training and runs at ∼7 FPS on a single RTX 4090 GPU.Source: §1 · p. 1
  • Across evaluated datasets, the system improves rendering PSNR by up to 3.1 dB and the geometry solver reduces trajectory error by up to about 97% relative to native camera heads.The solver is also reported to provide better tracking than learned streaming camera heads.Source: §1 · p. 1

2 Related Work

Prior Gaussian SLAM work addresses geometry and tracking through sparse visual methods, learned dense priors, feed-forward prediction, or explicit optimization. LingGS focuses on streaming consistent depth for camera tracking and Gaussian mapping.

2 Related Work

  • Without learned dense geometric priors, visual-tracking methods recover surfaces from multi-view observations and parallax, leaving weakly constrained regions to interpolation or later refinement.MonoGS tracks by photometric alignment with its evolving Gaussian map, while Photo-SLAM combines ORB-SLAM3 with Gaussian mapping.Source: §2 · p. 2
  • Learned dense geometry enriches tracking and mapping, but dense tracking and iterative refinement add computation to the online pipeline.Methods use dense depth, pointmaps, map-to-tracker feedback, optical flow, or reliability masks to improve geometric estimation.Source: §2 · p. 2
  • Feed-forward prediction can reduce mapping burden, but some methods require intensive retraining, incur high tracking cost, process frames in batches, or lose consistent geometry.StreamGS (Y. Li et al., 2025) and Flash-Mono (Z. Zhang et al., 2026) predict poses and Gaussian attributes before limited map refinement but require intensive retraining.Source: §2 · p. 2
  • Several methods combine predicted geometry with explicit pose estimation, while LingGS focuses on streaming consistent depth for tracking and Gaussian mapping.GigaSLAM and KiloGS-SLAM (Yu et al., 2026) use predicted depth with geometric solvers; the passage notes scale drift in these approaches.Source: §2 · p. 2

3 Method

LingGS combines a frozen streaming geometry model with explicit geometric camera tracking and an online Gaussian mapper. The solver uses predicted depth directly alongside image reprojection, while the mapper initializes and refines the scene from incoming observations and stored keyframes.

3 Method

  • The system uses initial frames to establish fixed camera intrinsics, then combines predicted depth and image features for pose estimation and Gaussian mapping.The mapper selects keyframes, inserts Gaussians using Laplacian guidance, and performs 30 refinement steps with RGB and depth losses and keyframe replay.Source: §3 · p. 2, Figure 1 · p. 3

3.1 Geometry Solver

  • The geometry solver estimates each world-to-camera pose from RGB features and predicted depth, retaining image reprojection while replacing virtual stereo with a direct depth residual.It keeps map points fixed during pose refinement, uses a constant-relative-uncertainty model for predicted depth, and alternates optimization with outlier reclassification.Source: §3.1 · pp. 2–3

3.2 Gaussian Mapper

  • The Gaussian mapper initializes scene elements by back-projecting sampled pixels with predicted depth, then prioritizes insertion where rendered structure underrepresents observed high-frequency detail.The insertion score is a heuristic for missing or underresolved detail, not a complete measure of photometric disagreement; the map is refined using keyframe replay.Source: §3.2 · p. 4

4 Experiment

Experiments evaluate reconstruction, tracking, geometry, and throughput across four indoor and outdoor datasets. LingGS leads on rendering and depth accuracy, while ablations show that explicit geometric tracking improves pose estimates across backbones at modest computational cost.

4.1 Setup and Evaluation

  • Evaluation covers 7-Scenes, Oxford Spires, RE10K, and TUM-D at 518×392 resolution, using NVS, trajectory, rendered-depth, and end-to-end FPS metrics.All methods run on one RTX 4090; Photo-SLAM receives up to 200 mapping iterations while retaining concurrent tracking and mapping, at about 6 FPS.Source: §4.1 · p. 4

4.2 Novel View Synthesis and Tracking

  • LingGS achieves the best novel-view-synthesis metrics on every dataset, with competitive trajectory accuracy: best ATE on Oxford Spires and TUM-D and second-best on 7-Scenes and RE10K.Qualitative comparisons also report finer structures and sharper appearance than several baselines, which show blurring, artifacts, or geometric inconsistency.Source: §4.2 · pp. 4–5, Table 1 · p. 6
  • Against S3PO-GS, depth error falls by 11.8% on 7-Scenes, 42.1% on TUM-D, and 28.9% on Oxford Spires.The method has the lowest mean L1 depth error on all three datasets; OTF has the highest despite competitive rendering quality.Source: §4.2 · p. 5, Table 2 · p. 6

4.3 Ablation study

  • Against the backbone camera head, the solver reduces ATE by 87.6% and translational RPE by 93.9%, adding 6.6 ms of tracking time; throughput decreases from 7.04 to 6.40 FPS.Removing the depth residual reduces PSNR by 3.45 dB and increases ATE from 73.8 to 162.1 cm; the solver also outperforms virtual stereo and point-to-plane ICP on reported pose metrics.Source: §4.3 · p. 5
  • Replacing the backbone camera head with the geometry solver reduces ATE by 31.0–96.7%, translational RPE by 64.3–97.1%, and rotational RPE by 55.9–78.8% across four backbones.Tracking takes 16.1 ms per frame, an additional 5.5–11.9 ms over the camera heads.Source: §4.3 · p. 5
  • Rendering quality improves by 0.93 dB from 20 to 30 refinement steps and about 0.88 dB from 30 to 75, then changes only marginally beyond 100 steps.The system uses 30 steps as its speed-oriented operating point; Figure 2 shows means over six Oxford Spires sequences and marks that default against saturation around 75–100 steps.Source: Figure 2 · p. 5, §4.3 · p. 6

5 Conclusion

LingGS combines learned geometric priors with explicit camera tracking and incremental Gaussian mapping for online reconstruction. Experiments across streaming backbones support the solver’s broader applicability, while the system retains limitations in loop closure, dynamic scenes, and difficult feature-matching environments.

5 Conclusion

  • LingGS combines a frozen streaming model’s depth predictions, explicit geometric pose estimation, and incremental Gaussian mapping for high-fidelity reconstruction.The solver jointly minimizes image reprojection and direct depth residuals.Source: §5 · p. 6
  • Experiments across multiple streaming backbones support the broader applicability of the geometric solver.Source: §5 · p. 6
  • The system does not address loop closure or dynamic scenes, and ORB feature matching can compromise tracking in repetitive or weakly textured environments.The authors identify learned priors as a direction for addressing these limitations.Source: §5 · p. 6

A.1 Relation to ORB-SLAM3’s Virtual-Stereo Model

ORB-SLAM3 adapts its stereo reprojection model to RGB-D input by constructing a virtual right-image coordinate from observed depth. This parameterization induces depth sensitivity in the pixel-weighted cost, unlike the paper’s direct-depth formulation with an explicit depth-dependent weight.

A.1 Relation to ORB-SLAM3’s Virtual-Stereo Model

  • ORB-SLAM3 constructs a virtual right-image coordinate from valid RGB-D depth and reuses its stereo reprojection model without requiring a physical right image.The RGB image serves as the left image.Source: §A.1 · pp. 6–7
  • The virtual-stereo residual’s third component combines horizontal reprojection error with inverse-depth discrepancy, so its full cost is not an independent scalar depth penalty.Source: §A.1 · p. 8
  • Near a consistent estimate, virtual stereo locally weights squared depth residuals with a coefficient scaling as z^-4 at fixed calibration and feature pyramid level.This dependence comes from residual parameterization, not an explicitly assigned depth information weight.Source: §A.1 · p. 8
  • The paper’s direct-depth formulation instead uses a direct depth residual with an explicitly specified z^-2 weight, based on an assumed constant relative depth uncertainty.The observed depth and corresponding weight remain fixed during pose-only optimization; the paper evaluates direct-depth, virtual-stereo, and reprojection-only tracking in Section 4.3.Source: §A.1 · p. 8

AI use statement

The authors report using generative AI tools for research feedback, implementation assistance, and manuscript work, with the resulting contributions independently reviewed and verified.

AI use statement

  • Generative AI assisted with methodology feedback, experiments, implementation, brainstorming, code editing, literature search, and manuscript readability; authors independently reviewed and verified the outputs.Source: AI use statement · p. 8

Ethics statement

The work uses publicly available research datasets and reports no human-subject studies or personally identifiable information collection.

Ethics statement

  • The authors report no human-subject studies or personally identifiable information collection and identify no specific ethical concerns beyond general considerations for computer vision and 3D reconstruction.Source: Ethics statement · p. 8

Reproducibility statement

The paper states that its source code and configuration files will be released upon acceptance and describes materials provided to support reproducibility.

Reproducibility statement

Related papers

Loading 2610.36853v1…