PaperlayerView exact revision on arXiv

Source-linked AI summary

CLoSeR: Closing the Loop for Long-Context Streaming Reconstruction

Moyang Li, Zihan Zhu, Wei Zhang, Marc Pollefeys, Daniel Barath

arXiv:2610.01927v1cs.CVcs.RO

TL;DR

Long-context streaming reconstruction models accumulate tracking drift, while existing loop-closure approaches face noisy point-cloud alignment and scale inconsistency. CLoSeR adds loop-conditioned pose estimation to a globally consistent-scale streaming backbone and optimizes poses on SE(3). Experiments report reduced drift and consistent kilometer-scale geometry, with strong benchmark comparisons.

  • Problem

    Streaming reconstruction models accumulate tracking drift over long trajectories without explicit constraints on revisited regions, while submap methods rely on noisy registration and scale-aware optimization.

    Source: §1 · p. 2, §3.1 · p. 4

  • Method

    CLoSeR uses loop-conditioned windows on the LoGeR streaming backbone to estimate loop-relative poses, then optimizes frame poses on SE(3) with sequential and loop constraints.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    CLoSeR substantially outperforms LoGeR and LoGeR* on kilometer-scale VBR and KITTI sequences, and outperforms other baselines on DROID-W while remaining competitive with in-the-wild methods.

    Source: §4 · pp. 6–7

  • Takeaways & Limitations

    The experiments support SE(3) pose graph optimization for this globally consistent-scale framework, avoiding the Sim(3) or SL(4) optimization used by prior submap methods.

    Source: §6 · p. 9

    When the frontend odometry completely fails, including under highly dynamic environments or aggressive camera motion, CLoSeR cannot recover the correct trajectory.

    Source: §5 · p. 9

Abstract

from arXiv · show

Feedforward foundation models have recently shown remarkable 3D reconstruction capabilities. However, existing models exhibit large tracking drift in long-context streaming reconstruction due to error accumulation. In this paper, we revisit loop closure with streaming reconstruction foundation models to enable accurate, drift-free, kilometer-scale reconstruction. Specifically, our method detects loop candidates through global descriptor retrieval, and constructs loop-conditioned windows to estimate the relative poses between looped frames. Given the observation that our adopted streaming reconstruction backbone produces a globally consistent scale, we optimize all frame poses on the SE(3) manifold with sequential and loop closure constraints, avoiding the pose graph optimization on the Sim(3) or higher-dimensional SL(4) manifolds employed in prior works. Extensive experiments show that our method reduces drift and produces consistent geometry on kilometer-scale sequences, significantly outperforming the state of the art. Code is available at https://github.com/MoyangLi00/CLoSeR.git.

1 Introduction

Long-context streaming reconstruction still accumulates tracking drift, while submap-based loop closure depends on noisy point-cloud registration and scale-aware pose optimization. CLoSeR uses a streaming backbone’s globally consistent scale and noncontiguous window inference to recover loop-relative poses and optimize on SE(3).

1 Introduction

  • Traditional reconstruction uses multi-view geometric constraints, while loop closure and pose graph optimization have been studied to correct drift on long trajectories and large scenes.Pipelines also fuse IMU or LiDAR data for more robust estimation under challenging conditions.Source: §1 · pp. 1–2
  • VGGT jointly predicts multi-frame geometry and poses, but quadratic self-attention and non-causal inference limit long-sequence scaling and streaming.Earlier feedforward models use pairwise alignment to handle multiple frames.Source: §1 · p. 2
  • Streaming models propagate information across windows but still drift without explicit revisit constraints; submap methods add loop closure but rely on noisy point-cloud registration and scale-aware optimization.Submap pose graphs use Sim(3) or SL(4) to absorb scale or projective ambiguity.Source: §1 · p. 2
  • CLoSeR adds loop closure to streaming feedforward reconstruction, using loop-conditioned windows to recover relative poses and SE(3) optimization instead of prior Sim(3) or SL(4) approaches.The method targets accurate kilometer-scale reconstruction from monocular videos.Source: §1 · p. 2

2 Related Works

Related work spans classical SfM and SLAM, feedforward reconstruction, and approaches for long sequences. Existing streaming models still accumulate drift, while submap-based systems and streaming systems use different strategies to extend reconstruction to longer contexts.

2 Related Works

  • SfM research ranges from staged keypoint matching, pose estimation, and joint refinement to deep feature and matching components and end-to-end differentiable optimization.Traditional pipelines estimate sparse point clouds and camera poses from image collections.Source: §2 · pp. 2–3
  • Classical SLAM uses feature tracking or photometric residuals, with sensors and loop closure addressing robustness and drift; newer learning-based and neural-scene systems retain iterative optimization and sophisticated pipelines.Feedforward models instead predict scene geometry and camera poses in a single forward pass.Source: §2 · p. 3
  • Feedforward models progress from pairwise reconstruction to multi-frame and streaming approaches, but existing streaming methods still struggle with drift accumulation in long-context reconstruction.Submap alignment extends VGGT, while other methods update global state or use Test-Time Training layers to propagate information across windows.Source: §2 · p. 3

3 Method

CLoSeR adds loop closure to a streaming reconstruction backbone, using its globally consistent scale to estimate loop-relative poses and refine the sequence on SE(3).

3.3 Pose Graph Optimization

  • CLoSeR augments LoGeR with loop closure and refines all frame poses on SE(3) using sequential and loop constraints.The backbone's globally consistent scale makes SE(3) optimization sufficient; a Huber loss suppresses the influence of outlier loop constraints.Source: §3 · p. 3, §3.3 · p. 5

3.1 Preliminaries

  • LoGeR propagates information across chunks with sliding-window attention and TTT, but lossy memory and missing revisit constraints leave long trajectories vulnerable to tracking drift.Its cross-chunk poses remain in a single global reference frame with consistent scale.Source: §3.1 · p. 4

3.2 Loop Detection and Alignment

  • SALAD descriptors retrieve prior-frame loop candidates using similarity and temporal-separation thresholds, retaining each current frame’s top-k matches.The descriptors are compared by cosine similarity.Source: §3.2 · pp. 4–5
  • For each connected prior window, CLoSeR forms a loop-conditioned window from Nwin/2 current and Nwin/2 prior frames, then estimates loop-relative poses in one forward pass.It selects the most similar participating frames when enough are available and adds temporally adjacent frames when needed.Source: §3.2 · p. 5

3.3 Pose Graph Optimization

  • Figure 3 contrasts LoGeR’s misaligned duplicate geometry with CLoSeR’s globally consistent reconstruction after revisits.The figure highlights the comparison regions with red dashed boxes.Source: Figure 3 · p. 6

4 Experiments

CLoSeR is evaluated against optimization-based and feedforward methods across four reconstruction benchmarks, including kilometer-scale and dynamic sequences. Results report stronger tracking, more coherent loop-region geometry, and benefits from the method’s design choices.

Benchmarks and Metrics

  • Evaluation covers VBR, KITTI, Oxford Spires, and DROID-W, with comparisons against optimization-based SLAM systems and feedforward reconstruction methods.ATE is the tracking metric, and estimated trajectories are aligned to ground truth using Sim(3) Umeyama alignment.Source: §4 · p. 6

Quantitative Results

  • On kilometer-scale VBR and KITTI, CLoSeR substantially outperforms LoGeR and LoGeR* [Zhang et al., 2026].Tables 1 and 2 report ATE RMSE comparisons; the supplied passages provide no cell values.Source: §4 · p. 6, Table 1 · p. 7, Table 2 · p. 7
  • On DROID-W, CLoSeR substantially outperforms baselines other than dynamics-aware DROID-W, while Oxford Spires results are reported as best among methods.CLoSeR remains competitive on dynamic sequences despite lacking dynamics-specific handling.Source: §4 · p. 7, Table 3 · p. 8, Table 4 · p. 8

Qualitative Comparisons

  • At loop regions, CLoSeR produces coherent geometry and trajectories where LoGeR shows duplicate or fragmented structures and erroneous poses.Figure 4 compares estimated trajectories with ground truth after Sim(3) Umeyama alignment.Source: §4 · pp. 7–8, Figure 4 · p. 9

Ablation Study

  • Ablations show that denser sequential constraints and Huber loss improve performance, while Sim(3) and SL(4) pose graphs reduce accuracy relative to the full SE(3) method.The authors attribute the scale-optimization ambiguity to the backbone’s globally consistent scale; the full method achieves the best ablation performance.Source: §4 · pp. 8–9, Table 5 · p. 8

5 Limitations

CLoSeR reduces accumulated drift but depends on the feedforward frontend continuing to produce odometry rather than failing completely.

Limitations

  • CLoSeR cannot recover the correct trajectory when the underlying odometry fails completely.The stated limitation is frontend failure, not merely accumulated drift.Source: §5 · p. 9
  • The limitation arises when LoGeR is pushed beyond its operating conditions, including highly dynamic environments or aggressive camera motion.Source: §5 · p. 9
  • The method is described as substantially reducing drift over kilometer-scale trajectories, not as recovering trajectories after complete odometry failure.Source: §5 · p. 9

6 Conclusion

CLoSeR adds loop closure to a feedforward streaming backbone for uncalibrated monocular reconstruction without retraining. Its globally consistent scale supports loop-conditioned relative-pose estimation and SE(3) pose-graph optimization.

Conclusion

  • CLoSeR targets accurate kilometer-scale reconstruction from uncalibrated monocular videos without retraining.Source: §6 · p. 9
  • Its loop-conditioned windows recover relative poses between looped frames directly in the backbone’s global coordinate system.Source: §6 · p. 9
  • The framework uses SE(3) pose-graph optimization, avoiding the Sim(3) or SL(4) optimization used by prior submap-based methods.Source: §6 · p. 9
  • Future work aims to improve robustness to motion blur, low-texture regions, and exposure variations, and to explore multi-agent reconstruction and online deployment.Source: §6 · p. 9

A.1 Experimental Setting

Evaluations use LoGeR as the base model and a roughly uniform input pixel budget, with common window and loop-detection settings except for VBR’s larger window.

Experimental Setting

  • LoGeR is the evaluation base; images are downsampled to about 2.55 × 10^5 pixels, with dataset-specific resolutions.The resulting resolutions are 924 × 266 for KITTI, 672 × 378 for VBR and DROID-W, and 588 × 434 for Oxford Spires.Source: §A.1 · p. 14
  • Window size is 32 with overlap 3, except VBR uses a window size of 48.Source: §A.1 · p. 14
  • Loop detection uses a temporal separation threshold of four window sizes, similarity threshold 0.7, and the top five similar pairs; pose optimization sets Nseq = 4.Source: §A.1 · p. 14

A.2 Additional Results

CLoSeR produces more geometrically consistent reconstructions and accurate trajectories across the reported sequences, while competing methods show alignment errors or drift.

A.2 Additional Results

  • CLoSeR produces more geometrically consistent reconstructions by constraining revisited regions, enabling drift-free reconstruction.LoGeR shows noticeable drift across all sequences because it lacks explicit constraints on revisited regions.Source: §A.2 · p. 14
  • CLoSeR robustly recovers accurate trajectories across all sequences, while VGGT-Long and VGGT-SLAM err substantially on VBR and Oxford Spires.LoGeR shows noticeable drift across all sequences.Source: §A.2 · p. 14
  • Table 6 reports runtime breakdown, average windows, FPS, peak GPU memory, and trajectory accuracy on KITTI.Source: Table 6 · p. 14

A.3 Runtime and GPU Memory Analysis

CLoSeR nearly halves LoGeR's trajectory error while incurring marginal runtime overhead and reducing peak GPU memory usage.

A.3 Runtime and GPU Memory Analysis

  • ATE RMSE falls from 25.44 m for LoGeR to 12.66 m for CLoSeR.The paper describes this as nearly halving trajectory error.Source: §A.3 · p. 14
  • CLoSeR runs at 7.12 FPS versus LoGeR's 7.60 FPS, a 6.3% throughput reduction, while peak GPU memory falls from 16.65 GiB to 15.69 GiB, a 5.8% reduction.Source: §A.3 · p. 14
  • Loop-closure detection and additional loop-closure windows for model inference account mainly for the added cost.Experiments used one NVIDIA RTX 4090 GPU and 4 AMD EPYC 9554 CPU cores.Source: §A.3 · p. 14

A.4 Analysis of Cross-Window Scale Consistency

CLoSeR has substantially lower relative scale error in both regular and loop-closure windows, and its loop-window construction incorporates relative pose estimation into the streaming framework.

A.4 Analysis of Cross-Window Scale Consistency

  • Relative scale error compares each window's fitted scale with a sequence reference averaged over regular windows.The evaluation fits Sim(3) transformations to decoded poses and ground truth; VGGT-SLAM 2.0 instead uses segment-length ratios for its two-frame loop-closure submaps.Source: §A.4 · p. 15
  • CLoSeR achieves substantially lower relative scale error for both regular and loop-closure windows.The reported averages are across KITTI windows.Source: §A.4 · p. 15
  • Regular-window scale consistency benefits from LoGeR's long-context memory, while loop-window consistency benefits from the loop-closure window construction.The construction integrates relative pose estimation into LoGeR's streaming framework rather than using a separate estimator or feedforward module.Source: §A.4 · p. 15
  • The qualitative comparisons show more accurate geometry and trajectories for CLoSeR, with LoGeR exhibiting ghosting and pose-related artifacts.The trajectory comparison overlays ground truth and estimated trajectories after Sim(3) Umeyama alignment [Umeyama, 1991].Source: Figure 5 · p. 16, Figure 6 · p. 17

Related papers

Loading 2610.01927v1…