PaperlayerView exact revision on arXiv

Source-linked AI summary

KFS$^4$LAM: Adaptive KeyFrame Selection with Structure-Aware Mapping for Endoscopic Gaussian Splatting SLAM

Xinwei Ju, Zhehua Mao, Tianyi Song, Danail Stoyanov, Sophia Bano, Francisco Vasconcelos

arXiv:2610.22505v1cs.CV

TL;DR

Endoscopic Gaussian-SLAM depends on which observations are retained, but non-uniform motion and challenging anatomy make conventional keyframe criteria unreliable. KFS4LAM couples motion-guided keyframe selection with fold-supervised mapping and restricted temporal deformation. Across C3VD and C3VD-v2, the paper reports improved trajectory–reconstruction trade-offs across backbones without increasing keyframe density.

  • Problem

    Endoscopic scenes and non-uniform camera motion challenge fixed-interval or simple pose- and overlap-based keyframe selection, although observation choice affects tracking and reconstruction.

    Source: §I · p. 1

  • Method

    KFS4LAM combines motion-guided adaptive keyframe selection with fold-boundary supervision and fold-conditioned restricted temporal deformation in Gaussian mapping.

    Source: §I · p. 2, C. Structure-Aware Restricted Temporal… · p. 3

  • Results

    Experiments on C3VD and C3VD-v2 show an improved trajectory–reconstruction trade-off across Gaussian-SLAM backbones without increasing keyframe density.

    Source: §VII · p. 8

  • Takeaways & Limitations

    Matched experiments with SplaTAM and EndoGSLAM indicate that the reported benefits are not specific to one Gaussian-SLAM backbone.

    Source: §VII · p. 8

    Fold supervision relies on an external predictor that may be less reliable for weak or occluded boundaries or anatomy outside its training distribution.

    Source: B. Limitations · p. 8

Abstract

from arXiv · show

Gaussian-splatting Simultaneous Localization and Mapping (SLAM) augments conventional SLAM with a dense, renderable 3D scene representation, enabling high-quality reconstruction and novel-view synthesis. However, its localization, reconstruction, and rendering performance depends strongly on which observations are retained for mapping. Existing systems commonly rely on fixed temporal intervals or simple pose- and overlap-based criteria for keyframe selection. In colonoscopy, non-uniform camera motion, repetitive anatomy, specularities, and visual artifacts can make these criteria unreliable, leading to either redundant keyframes or missed informative viewpoints. We present KFS$^4$LAM (KeyFrame Selection with Structure-aware mapping for endoscopic Gaussian Splatting SLAM), which couples motion-guided adaptive keyframe selection with structurally supervised mapping and restricted temporal deformation. Accumulated motion guides adaptive keyframe selection, with complementary cues suppressing redundant or unreliable candidates. Density-aware mapping with anatomical contour supervision (colon folds) emphasizes structurally informative regions, while restricted temporal deformation accommodates local tissue dynamics without unconstrained geometric drift. We evaluate KFS$^4$LAM against Gaussian-SLAM baselines on C3VD and C3VD-v2, together with cross-backbone and component ablations. KFS$^4$LAM achieves strong overall performance across trajectory, rendering, and depth metrics, with pronounced gains on more challenging C3VD-v2 dataset. In the matched comparison with SplaTAM on C3VD-v2, it reduces rotational relative pose error by 20.3% and depth RMSE by 38.4%, while improving PSNR by 0.78 dB. Cross-backbone experiments further show that the framework is not tied to a single underlying SLAM architecture. Code and visualization results are available at https://xinweiju.github.io/KFSLAM/.

I. INTRODUCTION

Endoscopic SLAM must select observations carefully because camera motion and visual information vary, while deformable, repetitive, and weakly textured scenes challenge conventional methods. KFS4LAM couples motion-guided adaptive keyframes with structure-aware mapping and restricted deformation, and reports improved localization and map quality across datasets and backbones.

I. INTRODUCTION

  • Non-uniform camera motion makes consecutive frames vary in information: slow motion yields near-redundant views, while rapid viewpoint changes can reveal new scene content within a few frames.Endoscopic scenes also feature deformable tissue, restricted field of view, specular reflections, variable illumination, and repetitive or texture-poor anatomy.Source: §I · p. 1
  • Existing endoscopic Gaussian-SLAM systems use temporal spacing or pose/overlap-driven keyframes, while appearance and correspondence cues can misrepresent viewpoint novelty.The introduction identifies comparatively less attention to adaptive observation selection and persistent anatomical structure in mapping.Source: §I · p. 1
  • KFS4LAM reports consistent localization and map-quality improvements, with larger benefits on challenging C3VD-v2, without higher keyframe density.Component and cross-backbone ablations indicate the gains are not attributable to increased mapping effort or one particular Gaussian-SLAM backbone.Source: §I · p. 2
  • Accumulated camera motion proposes keyframes, while complementary image- and feature-level cues suppress unreliable or redundant candidates.The mapping design combines density-aware mapping, colon-fold boundary supervision, and fold-conditioned restricted temporal deformation.Source: §I · p. 2
  • The framework is evaluated on C3VD and C3VD-v2 with component and matched cross-backbone ablations.The evaluation is described as demonstrating consistent improvements across different Gaussian-SLAM backbones.Source: §I · p. 2

II. METHODS

KFS4LAM combines adaptive observation selection and structure-aware mapping for deformable endoscopic Gaussian-SLAM. Its RGB-D input and fold-response maps support a time-conditioned Gaussian representation used for rendering, mapping, and frame-wise tracking.

A. Overview and Problem Formulation

  • KFS4LAM couples adaptive keyframe selection with structure-aware mapping; predicted colon folds provide supervision and a soft prior for restricted temporal deformation.The keyframe policy selects observations for subsequent mapping.Source: A. Overview and Problem Formulation · p. 2
  • The framework retains the underlying SLAM backbone’s RGB-D input setting and estimates camera poses.The passage states the pose-estimation objective, but the supplied text does not provide a complete formulation.Source: A. Overview and Problem Formulation · p. 2
  • Per-pixel fold-response maps are predicted from RGB images by a pretrained fold-segmentation model.The maps provide the fold information used in structure-aware mapping.Source: A. Overview and Problem Formulation · pp. 2–3
  • Each Gaussian includes canonical mean, covariance, color, opacity, a fold response, and parameters for restricted temporal displacement.Replacing the canonical mean with its time-conditioned mean defines the map used for rendering, mapping, and frame-wise tracking.Source: A. Overview and Problem Formulation · p. 3

B. Adaptive Keyframe Selection

Keyframe selection separates motion-based candidate proposal from asymmetric visual suppression. Motion proposes observations; visual cues may reject ordinary candidates but cannot independently trigger insertion.

B. Adaptive Keyframe Selection

  • Accumulated frame-to-frame translation is used as a motion proxy, with the proposal threshold adapted to recent motion.The threshold uses the median positive translation step over recent 16 frame pairs; the reference interval is 8 and the minimum motion threshold is 0.03.Source: B. Adaptive Keyframe Selection · p. 3
  • Ordinary proposals require accumulated motion to meet the adaptive threshold within the permitted gap, while protected proposals are generated at the maximum gap or near sequence end.The configured minimum and maximum gaps are 4 and 16 processed frames.Source: B. Adaptive Keyframe Selection · p. 3
  • Visual suppression can reject ordinary proposals using low-texture checks or similarity to recent retained keyframes, but cannot trigger insertion.The policy compares ordinary proposals with up to three recent retained keyframes; protected proposals bypass suppression.Source: B. Adaptive Keyframe Selection · p. 3

C. Structure-Aware Restricted Temporal Mapping

KFS4LAM combines fold-boundary supervision with restricted temporal deformation to preserve anatomical structure while accommodating local tissue motion. Tracking uses a fixed time-conditioned map; structural supervision and deformation regularization act during mapping.

C. Structure-Aware Restricted Temporal Mapping

  • Predicted fold structure supplies image-space supervision and a soft spatial prior that relaxes the deformation-magnitude penalty in higher-response regions.The deformation terms regularize magnitude, temporal variation, and global mean displacement.Source: C. Structure-Aware Restricted Temporal… · p. 3
  • Fold-boundary supervision penalizes rendered boundaries far from observed fold boundaries without requiring every observed boundary to match.The Boundary-DT loss updates Gaussian geometry, opacity, and temporal deformation; stored fold attributes and mapping pose are detached.Source: C. Structure-Aware Restricted Temporal… · p. 3
  • Only Gaussian means are time-conditioned, using gradual linear and smooth nonlinear displacement terms that vanish at the first frame and anchor canonical geometry.The nonlinear term is half-sine; the deformation regularizer uses fold response to modulate its magnitude weighting.Source: C. Structure-Aware Restricted Temporal… · p. 3
  • During tracking, the camera pose is optimized against the fixed time-conditioned map; fold-boundary supervision and deformation regularization apply only during mapping.The tracking objective uses Gaussian rendering and the backbone’s native tracking objective.Source: C. Structure-Aware Restricted Temporal… · p. 4

D. Optimization and Adaptive Mapping Budget

The per-frame mapping budget adapts to the replay-pool size created by adaptive keyframe selection. Larger-than-reference pools receive additional mapping iterations, within specified minimum and maximum bounds.

D. Optimization and Adaptive Mapping Budget

  • The per-frame mapping budget adapts to the number of retained observations to accommodate the non-uniform replay pool produced by adaptive keyframe selection.The policy compares the available pool, including the current observation, with the nominal fixed-gap replay-pool size.Source: D. Optimization and Adaptive Mapping… · p. 4
  • Larger-than-reference replay pools receive additional mapping iterations, with the budget clipped between 25 and 75 iterations.The setup uses a reference gap of 8 and sets N0 and Nmin to 25, Nmax to 75, smax to 3, and p to 1 for both backbones.Source: D. Optimization and Adaptive Mapping… · p. 4

A. Datasets and Baselines

The evaluation uses registered, undistorted RGB-D sequences from C3VD and C3VD-v2 and compares KFS4LAM on two reconstruction backbones with established Gaussian-SLAM baselines. It reports trajectory, rendering, depth, and qualitative reconstruction metrics.

A. Datasets and Baselines

  • The benchmarks comprise 14 C3VD sequences with 5,810 frames and 10 C3VD-v2 sequences with 4,511 frames.Both datasets use registered and undistorted RGB-D sequences.Source: A. Datasets and Baselines · p. 4

B. Experimental Settings

  • KFS4LAM is instantiated on EndoGSLAM and SplaTAM, alongside comparisons with GaussianSLAM, Endo-2DTAM, MonoGS, EndoGSLAM, and SplaTAM.Baselines retain their native tracking, mapping, and keyframe mechanisms; KFS4LAM variants use adaptive gaps of 4, 8, and 16 processed frames, against fixed intervals of 8 for EndoGSLAM and SplaTAM.Source: A. Datasets and Baselines · p. 4, B. Experimental Settings · p. 4

C. Evaluation Protocol

  • Evaluation reports Sim(3)-aligned ATE and ARE, consecutive-frame RPE-t and RPE-r, and macro-averaged RPE, PSNR, SSIM, LPIPS-Alex, and depth RMSE.ATE and ARE are reported as both macro and sequence-length-weighted averages; qualitative evaluation includes RGB, depth, and Sim(3)-aligned spatial reconstruction.Source: C. Evaluation Protocol · pp. 4–5

IV. RESULTS

KFS4LAM improves global trajectory consistency and geometric reconstruction, with its advantages more pronounced on C3VD-v2. Its rendering remains competitive, although the relative-pose and image-space metrics show that gains are not uniform across every measure.

A. Trajectory Accuracy

  • On C3VD-v2, KFS4LAM’s weighted ARE falls from 53.00° for EndoGSLAM to 43.13°, while its principal tracking advantage is improved global trajectory consistency.KFS4LAM has the lowest weighted ATE and macro and weighted ARE, and the second-lowest macro ATE; SplaTAM retains the lowest RPE-t and GaussianSLAM the lowest RPE-r, with KFS4LAM second on both.Source: A. Trajectory Accuracy · p. 5

B. Rendering and Geometric Reconstruction

  • On C3VD-v2, KFS4LAM reduces depth RMSE from 4.95 mm to 3.05 mm versus SplaTAM, a 38.4% reduction, while PSNR rises by 0.78 dB.SSIM also improves, while LPIPS remains slightly better for SplaTAM; the reported pattern emphasizes geometric consistency alongside competitive image-space rendering.Source: B. Rendering and Geometric… · p. 5
  • On C3VD, KFS4LAM achieves the best SSIM, LPIPS, and depth RMSE, while ranking second in PSNR.On C3VD-v2, MonoGS has the highest PSNR on both datasets but very large depth errors, showing that photometric quality alone does not characterize geometric fidelity.Source: B. Rendering and Geometric… · p. 5
  • Qualitative comparisons show more pronounced C3VD-v2 differences: KFS4LAM preserves more coherent cavity geometry while baselines show appearance artifacts, cavity distortion, or inconsistent depth.The MonoGS comparison illustrates that plausible RGB synthesis can coexist with substantially inaccurate depth.Source: B. Rendering and Geometric… · p. 5

V. ABLATION STUDY

The component ablations show that adaptive keyframe selection, structural supervision, and restricted deformation contribute differently, with the full combination improving several C3VD-v2 measures. KFS4LAM redistributes a comparable keyframe budget over time rather than retaining more observations.

A. Component Contributions

  • The full SplaTAM-based KFS4LAM reduces C3VD-v2 mATE, mARE, RPE-r, and depth RMSE compared with Structure-Guided Mapping.The comparison combines adaptive selection, structural supervision, and restricted temporal deformation.Source: Table III · p. 6, A. Component Contributions · p. 6
  • On C3VD-v2, adding deformation to fixed-keyframe Structure-Guided Mapping improves the trajectory–reconstruction balance, including mARE, RPE-r, and depth RMSE.Without deformation, Motion-and-Structure-Guided improves mATE, PSNR, SSIM, and depth RMSE over Structure-Guided Mapping while using fewer keyframes.Source: Table III · p. 6, A. Component Contributions · p. 6
  • Adaptive Mapping (DynMap) remains close to SplaTAM on C3VD and does not consistently improve C3VD-v2, while Motion Proposal Only improves individual trajectory measures, especially C3VD-v2 mATE.Motion-Guided KFS produces mixed trajectory and rendering changes, particularly on C3VD-v2.Source: A. Component Contributions · p. 6, Table III · p. 6
  • SplaTAM + MonoGS KFS yields a sparser keyframe set but inconsistent gains, unlike the more consistent improvements of KFS4LAMSplaTAM.The full-model gains are achieved without a higher keyframe ratio.Source: A. Component Contributions · p. 6, B. Keyframe Allocation · p. 6

B. Keyframe Allocation

  • KFS4LAM’s mean keyframe ratio is 12.33% on C3VD and 12.47% on C3VD-v2, below fixed-interval SplaTAM’s 13.08% and 12.90%.Across the illustrated backbones, selection density varies locally, with sparse and dense periods rather than fixed-interval sampling.Source: B. Keyframe Allocation · p. 6
  • On representative C3VD-v2 frames, both KFS4LAM backbone variants show more coherent cavity structure and suppress appearance and depth artifacts.The improvement is particularly visible on c2 sigmoid t2 v1.Source: Figure 3 · p. 6

C. Cross-Backbone Generalization

Matched experiments transfer KFS4LAM’s selection and mapping components to SplaTAM and EndoGSLAM without backbone-specific redesign, with reconstruction benefits across both. The results also show metric-specific trade-offs and suggest that selection and mapping components work complementarily.

C. Cross-Backbone Generalization

  • The selection policy, structural supervision, and restricted temporal deformation transfer to SplaTAM and EndoGSLAM through their native tracking and mapping interfaces, without backbone-specific redesign.Only integration into the backbones’ native interfaces differs.Source: C. Cross-Backbone Generalization · p. 6
  • On C3VD-v2, SplaTAM mARE changes from 59.49° to 44.81° and depth RMSE from 4.95 to 3.05 mm; EndoGSLAM RPE-r changes from 1.22° to 0.93° and depth RMSE from 14.89 to 11.93 mm.Trajectory gains are not uniform: SplaTAM has lower RPE-t on C3VD-v2, while EndoGSLAM has lower mATE on C3VD-v2 and lower mARE on C3VD.Source: C. Cross-Backbone Generalization · p. 6
  • Figures 3 and 5 show more coherent cavity structure and improved alignment with the visible reference surface in examples from both reconstruction backbones.Figure 4 illustrates integration of the observation policy into both tracking pipelines.Source: C. Cross-Backbone Generalization · p. 7, Figure 4 · p. 7, Figure 5 · p. 7

A. Interpretation and Evaluation Implications

  • Controlled ablations indicate that structure-aware mapping benefits fixed-keyframe selection, while the full model benefits from combining mapping components with adaptive observation allocation.The proposed components provide complementary mechanisms for reconstruction consistency as the retained observations change.Source: A. Interpretation and Evaluation… · p. 7
  • The larger gains on C3VD-v2 may reflect greater sensitivity to observation allocation and scene variation under more challenging acquisition conditions.This is a plausible interpretation, not an independently verified explanation; fold-boundary supervision adds an anatomical constraint, while restricted deformation allows limited geometric flexibility.Source: A. Interpretation and Evaluation… · p. 7
  • MonoGS achieves the highest PSNR on both datasets despite substantially larger depth errors, so an improvement in one metric does not imply improvement in others.The paper argues for evaluating trajectory consistency, local pose accuracy, rendering, and geometric reconstruction together.Source: A. Interpretation and Evaluation… · p. 7

B. Limitations

The authors identify computation, dependence on predicted anatomy, deformation-model constraints, and limited evaluation breadth as boundaries of the current formulation. They call for broader validation and methods that jointly address selection, mapping, and computation.

B. Limitations

  • Adaptive mapping increases optimization effort when the retained replay pool is denser, making KFS4LAM a quality–computation trade-off rather than a runtime improvement.A fixed-compute evaluation would more strictly separate adaptive mapping from keyframe selection.Source: B. Limitations · p. 7
  • Fold supervision depends on an external predictor and may be less reliable for weak or occluded boundaries or anatomy outside its training distribution.The authors also say evaluation on additional clinical data and more diverse acquisition conditions is needed to establish broader generalization.Source: B. Limitations · p. 8

VII. CONCLUSION

  • The current formulation is limited by dependence on externally predicted anatomy, adaptive mapping computation, and a restricted deformation model.Future work should examine broader clinical validation, explicit computational budgets, and more expressive yet anatomically regularized deformation.Source: §VII · p. 8
  • The authors conclude that keyframe selection should be considered together with mapping because retained observations change the spatial and temporal constraints available for reconstruction.They identify validation across broader clinical datasets and acquisition conditions as future work.Source: §VII · p. 8

Related papers

Loading 2610.22505v1…