PaperlayerView exact revision on arXiv

Source-linked AI summary

PanoGS-SLAM: Panoramic 3D Gaussian Splatting SLAM

Yongqi Mao, Hao Shi, Yufan Zhang, Zhonghua Yi, Xiangfei Guo, Kaiwei Wang

arXiv:2609.17387v1cs.CV

TL;DR

Narrow-FoV Gaussian SLAM can suffer from weak pose observability and unstable photometric optimization during rapid motion and large viewpoint changes. PanoGS-SLAM performs differentiable rendering and pose optimization directly in the spherical domain, adding distortion-aware photometric optimization and depth-guided Gaussian initialization. It consistently outperforms geometric and Gaussian-based baselines in tracking and rendering quality while converging rapidly, and controlled experiments show monotonic gains as angular coverage increases.

  • Problem

    Most 3DGS-based SLAM systems use narrow-FoV pinhole cameras, limiting pose observability and destabilizing photometric optimization under rapid motion and large viewpoint changes.

  • Method

    PanoGS-SLAM performs joint differentiable rendering and pose optimization in the spherical domain with a sphere-consistent photometric loss and depth-guided Gaussian initialization.

  • Results

    PanoGS-SLAM consistently achieves better tracking accuracy and rendering quality than geometric and Gaussian-based baselines, with stable pose tracking in 15 iterations versus approximately 100 for MonoGS.

  • Takeaways & Limitations

    Controlled FoV experiments show that increasing angular coverage monotonically improves conditioning, convergence stability, and robustness in differentiable Gaussian-based SLAM.

Abstract

from arXiv · show

Real-time dense SLAM is a core capability for robotics applications that require robust localization and high- quality mapping in dynamic or fast-changing environments. Recent 3D Gaussian Splatting (3DGS)-based SLAM methods have shown promising performance, but most are designed for narrow-FoV pinhole cameras, where limited angular coverage weakens pose observability and often leads to unstable photo- metric optimization under rapid motion and large viewpoint changes. We present PanoGS-SLAM, the first panoramic dense SLAM system built on 3D Gaussian Splatting. Our method per- forms differentiable rendering and pose optimization directly in the spherical domain, enabling omnidirectional photometric constraints for more stable tracking. To improve geometric consistency and robustness, we introduce (1) a sphere-consistent photometric loss that compensates for the area distortion of equirectangular projection, and (2) a depth-guided Gaussian initialization strategy that stabilizes incremental mapping in newly observed regions. Extensive experiments on both real and synthetic panoramic benchmarks (PALVIO and SynPano) show that PanoGS-SLAM consistently outperforms geometric and GS-based baselines in tracking accuracy and rendering quality, while achieving fast front-end convergence and real-time perfor- mance. In addition, controlled field-of-view experiments reveal a clear monotonic improvement in optimization conditioning and convergence stability as angular coverage increases, high- lighting the fundamental role of sensing geometry in shaping the optimization landscape of differentiable Gaussian-based SLAM. The source code will be made publicly available.

I. INTRODUCTION

PanoGS-SLAM addresses the limitations of narrow-FoV Gaussian SLAM by optimizing panoramic observations directly on the sphere. It combines sphere-consistent photometric optimization and depth-guided Gaussian insertion, with experiments showing stronger accuracy, rendering quality, and convergence.

  • Motivation: Narrow-FoV pinhole sensing weakens rotational observability, couples translation and rotation, and produces poorly conditioned optimization under rapid motion or viewpoint changes.Limited angular coverage concentrates image gradients within a narrow viewing cone, reducing the convergence basin.
  • Evaluation: Across SynPano and PALVIO, PanoGS-SLAM achieves the best tracking accuracy and superior rendering quality against geometric and Gaussian-based baselines.It can outperform existing baselines by up to an order of magnitude and reaches stable tracking in 15 iterations, approximately one-seventh of MonoGS’s requirement.
  • Panoramic 3D Gaussian SLAM Framework: PanoGS-SLAM is the first panoramic 3DGS SLAM system to jointly optimize camera tracking and Gaussian maps directly in the spherical domain.The framework avoids decomposing panoramas into perspective crops and maintains continuous gradients over the viewing sphere.
  • Sphere-Consistent Photometric Optimization: The method compensates for equirectangular area distortion with a sphere-consistent photometric loss that improves gradient balance and tracking stability.The objective prevents polar regions from dominating the loss function.
  • Depth-Guided Gaussian Initialization: Depth-guided Gaussian insertion improves geometric consistency in newly observed regions and stabilizes incremental mapping.The strategy uses depth information to strengthen geometric priors during map growth.
  • Field-of-View Analysis: Controlled FoV experiments show that wider angular coverage monotonically improves conditioning, convergence behavior, and robustness in Gaussian-based dense SLAM.The experiments identify field of view as a factor governing the optimization properties of differentiable SLAM.

II. RELATED WORK

Prior work includes geometric panoramic SLAM and Gaussian-based reconstruction, but omnidirectional 3DGS had not been integrated into an online, globally consistent SLAM framework. PanoGS-SLAM addresses this gap by jointly optimizing poses and Gaussian parameters.

  • Omnidirectional SLAM: Omnidirectional images provide richer geometric constraints and generally improve tracking robustness during rapid motion and abrupt scene transitions.Prior systems extended sparse SLAM to panoramic cameras through unified omnidirectional modeling and uncertainty handling.
  • Omnidirectional 3DGS: Existing omnidirectional Gaussian methods focus on offline reconstruction and do not address online camera pose estimation or global consistency.This leaves integration of omnidirectional 3DGS into globally consistent SLAM unresolved.
  • Research Gap: PanoGS-SLAM concurrently optimizes camera poses and Gaussian parameters within a unified omnidirectional 3DGS-based SLAM framework.This directly targets the gap between panoramic Gaussian rendering and online globally consistent SLAM.

III. METHOD

PanoGS-SLAM processes monocular panoramic sequences with one unified 3D Gaussian representation for tracking and mapping. Its pipeline combines depth preprocessing, front-end pose optimization, and incremental back-end reconstruction.

  • Architecture: The system takes monocular panoramic sequences as input and maintains a unified 3D Gaussian representation for tracking and mapping.The representation supports both pose estimation and scene reconstruction within one pipeline.
  • Architecture: The pipeline comprises preprocessing for initial depth estimation, a tracking front end for pose optimization, and a mapping back end for incremental reconstruction.These modules organize the system’s initialization, camera tracking, and scene-growth stages.

A. Gaussian Splatting

The method uses 3D Gaussian Splatting as an explicit scene representation whose projected Gaussians are efficiently composited for image synthesis. Each Gaussian carries geometry, opacity, covariance, and view-dependent appearance.

  • Gaussian Representation: Each 3D Gaussian is parameterized by a world-space mean, opacity, and covariance matrix, forming the explicit scene representation.The Gaussian set is denoted as {G_i}^N and models the scene without explicit geometric surfaces.
  • Appearance and Projection: Spherical harmonics model view-dependent color, while each 3D Gaussian is projected onto the image plane before splatting.The projection produces a 2D covariance used by the rasterization process.
  • Rendering: 3DGS synthesizes each pixel by compositing Gaussians along the viewing direction through a rasterization-based pipeline.Traversing 2D Gaussians on the image plane exploits scene sparsity for efficient rendering.

B. Camera Model

PanoGS-SLAM replaces perspective projection with spherical projection, mapping 3D Gaussians onto an equirectangular plane for omnidirectional rendering. Covariance transformation accounts for local spherical deformation, including pole stretching.

  • Spherical projection maps camera-frame 3D Gaussian means to longitude–latitude coordinates and then to equirectangular pixel coordinates.The projection uses the panoramic image width and height to convert spherical coordinates into pixel locations.
  • The projection Jacobian propagates covariance information and captures geometric deformation from Euclidean space to the spherical domain.This transformation stretches Gaussians appropriately near the poles of the panoramic image.
  • The resulting 2D Gaussians are rasterized and blended on the equirectangular plane using the standard Gaussian splatting procedure.

C. SLAM Front-end

The front-end estimates camera pose through panoramic photometric optimization on the sphere and uses keyframes to support efficient backend mapping. A latitude-aware loss corrects equirectangular area distortion and improves tracking stability.

  • Panoramic pose optimization uses omnidirectional coverage to provide smooth photometric gradients across the view, improving stability and efficiency.The camera pose is optimized while the Gaussian map remains fixed, with optimization initialized from the previous-frame pose.
  • The Panoramic Loss weights each pixel by the cosine of its spherical latitude to compensate for equirectangular area distortion.This prevents polar regions from being overweighted and makes error accumulation consistent with spherical geometry.
  • Area-consistent error accumulation significantly improves tracking stability during panoramic pose optimization.
  • Keyframes are selected using Gaussian co-visibility and camera pose displacement, while opacity-guided insertion supports newly observed regions.

D. SLAM Back-end

The backend jointly optimizes poses and Gaussian parameters over a keyframe window while using sampled non-keyframes for cross-view consistency. Depth-guided initialization builds and expands the map with balanced, adaptively placed Gaussians.

  • Backend optimization jointly updates camera poses and Gaussian parameters using keyframes plus two randomly sampled non-keyframe views.The additional views help alleviate global forgetting and enforce cross-view consistency, with the panoramic loss reused for supervision.
  • A pretrained panoramic depth estimator supplies the initial depth map used to construct the first Gaussian map.Gaussians are generated by downsampling RGB-D observations during map initialization.
  • Equal-area unit-sphere sampling reduces polar Gaussian redundancy and produces a more balanced spatial distribution across the panorama.
  • Opacity-guided insertion assigns higher Gaussian density to under-reconstructed regions and initializes new Gaussian depths from rendered depth.

A. Experimental Setting

The evaluation compares PanoGS-SLAM with geometric and Gaussian-based baselines on synthetic SynPano and real PALVIO panoramic data. It measures trajectory accuracy and rendering quality under differing camera models and challenging motion.

  • Experiments compare PanoGS-SLAM against classical, wide-FoV, and Gaussian-based SLAM baselines, with MonoGS serving as the primary comparison.The baselines include ORB-SLAM3, VINS-Mono, P2U-SLAM, LF-VISLAM, MonoGS, and Photo-SLAM.
  • SynPano provides controlled 360°×180° synthetic panoramas, while PALVIO contains real 360°×[40°,120°] sequences with aggressive 6-DOF motion and rapid viewpoint changes.
  • PanoGS-SLAM achieves substantially better ground-truth trajectory alignment than the baselines, whereas MonoGS exhibits unstable tracking and inconsistent trajectory scale.
  • VINS-Mono and LF-VISLAM have unavailable SynPano results because they require IMU data that the dataset does not provide.
  • Virtual pinhole views are extracted from the equatorial region at 120°×60° FoV to reduce resampling artifacts and support fair cross-camera evaluation.
  • Trajectory accuracy is evaluated with aligned ATE RMSE, while reconstruction quality uses PSNR, SSIM, and LPIPS.

B. Quantitative Evaluation

PanoGS-SLAM improves tracking, rendering, field-of-view robustness, and pose-optimization convergence over the evaluated baselines. Its spherical representation and associated optimization strategies produce more stable trajectories, higher-quality reconstructions, and faster convergence.

  • Tracking Accuracy: PanoGS-SLAM consistently achieves the lowest ATE RMSE across evaluated sequences and maintains resilient pose constraints during rapid viewpoint changes.The method is especially advantageous in SynPano room4 and room5, where full panoramic coverage supports convergence in textureless regions.
  • Rendering Quality: PanoGS-SLAM consistently outperforms Gaussian-based baselines in rendering quality, including novel-view comparisons where MonoGS shows trajectory-driven distortions and Photo-SLAM exhibits artifacts and overfitting.It also outperforms baselines on individual pinhole views, while panoramic renderings remain geometrically consistent despite lower PSNR from the larger field of view.
  • Ablative Analysis: The sphere-consistent panoramic loss and depth-guided Gaussian initialization improve geometric consistency and reduce trajectory errors during incremental mapping.The panoramic loss limits polar noise amplification, while DGIS improves the initial map and newly inserted Gaussians.
  • Effect of Field-of-View: Trajectory accuracy improves monotonically as field of view increases, with challenging sequences showing a critical performance turning point above 200°.Under identical field-of-view settings, PanoGS-SLAM consistently outperforms MonoGS across all four scenes.
  • Pose Optimization Convergence: PanoGS-SLAM converges within 15 pose-optimization iterations at 7 FPS, whereas MonoGS shows convergence trends only after approximately 100 iterations.The convergence-basin analysis attributes this stability to a broader, better-conditioned attraction basin with consistent gradients across the viewing sphere.

V. CONCLUSION

PanoGS-SLAM jointly performs camera tracking and 3D Gaussian map optimization directly in the spherical domain. It combines a sphere-consistent photometric objective with depth-guided Gaussian initialization and identifies a connection between sensing geometry and numerical robustness.

  • V. CONCLUSION: PanoGS-SLAM jointly performs camera tracking and 3D Gaussian map optimization directly in the spherical domain.The framework addresses equirectangular distortion with a sphere-consistent photometric objective and improves incremental mapping with depth-guided Gaussian initialization.
Loading 2609.17387v1…