PaperlayerView exact revision on arXiv

Source-linked AI summary

LDSO: Direct Sparse Odometry with Loop Closure

Xiang Gao, Rui Wang, Nikolaus Demmel, Daniel Cremers

arXiv:1808.01111v1cs.CV

TL;DR

Direct monocular odometry is robust in feature-poor scenes but faces difficulties reusing marginalized information and detecting loop closures. LDSO adapts DSO to favor repeatable corner features, uses BoW and depth-based Sim(3) constraints, and fuses them in a pose graph. Experiments report retained tracking robustness and accuracy, substantially reduced drift, and performance comparable to feature-based methods without global bundle adjustment.

  • Problem

    Direct sliding-window odometry must reuse map information without retaining all past images or reusing measurements already encoded in marginalization priors, while loop closure requires repeatable features with depth estimates.

  • Method

    LDSO favors repeatable corner features in DSO, detects loops with ORB descriptors and BoW, estimates Sim(3) constraints from feature depths, and fuses them with a co-visibility pose graph.

  • Results

    LDSO retains the odometry frontend’s tracking accuracy and robustness while significantly reducing rotation, translation, and scale drift, with performance comparable to feature-based methods without global bundle adjustment.

  • Takeaways & Limitations

    Loop closure and pose-graph optimization extend DSO into a monocular SLAM system that preserves direct-method robustness while correcting global drift.

Abstract

from arXiv · show

In this paper we present an extension of Direct Sparse Odometry (DSO) to a monocular visual SLAM system with loop closure detection and pose-graph optimization (LDSO). As a direct technique, DSO can utilize any image pixel with sufficient intensity gradient, which makes it robust even in featureless areas. LDSO retains this robustness, while at the same time ensuring repeatability of some of these points by favoring corner features in the tracking frontend. This repeatability allows to reliably detect loop closure candidates with a conventional feature-based bag-of-words (BoW) approach. Loop closure candidates are verified geometrically and Sim(3) relative pose constraints are estimated by jointly minimizing 2D and 3D geometric error terms. These constraints are fused with a co-visibility graph of relative poses extracted from DSO's sliding window optimization. Our evaluation on publicly available datasets demonstrates that the modified point selection strategy retains the tracking accuracy and robustness, and the integrated pose-graph optimization significantly reduces the accumulated rotation-, translation- and scale-drift, resulting in an overall performance comparable to state-of-the-art feature-based systems, even without global bundle adjustment.

I. INTRODUCTION

LDSO extends direct monocular odometry into a loop-closing SLAM system by adapting DSO’s point selection and adding feature-based loop detection with pose-graph optimization. The approach preserves direct tracking robustness while reducing accumulated drift.

  • Motivation: Direct sliding-window odometry creates integration challenges because past keyframe images must be retained and marginalized information must not be reused.LDSO addresses this by using DSO for locally consistent odometry and loop closure with backend pose-graph optimization for drift correction.
  • Motivation: Direct methods estimate motion and correspondences through photometric alignment, offering accuracy and robustness when images lack explicit corner features.This contrasts with indirect methods that rely on fixed correspondences between repeatable discrete features.
  • Approach: LDSO adapts DSO’s point selection to favor repeatable corner features while retaining robustness in feature-poor environments.These features support conventional bag-of-words loop closure detection.
  • Backend: LDSO computes Sim(3) pose constraints from matched-point depth estimates using pose-only bundle adjustment and point-cloud alignment, then fuses them with DSO’s co-visibility graph.The graph incorporates relative poses extracted from DSO’s sliding-window optimization.
  • Results: The evaluation reports preserved tracking accuracy and robustness, significantly reduced rotation, translation, and scale drift, and performance comparable to feature-based methods without global bundle adjustment.The implementation is also made publicly available.
  • Results: Fig. 1 shows LDSO trajectories and point clouds aligning better after loop closure on TUM-Mono sequence 31.The sequence returns to its starting location, making the accumulated-drift correction visually apparent.

II. RELATED WORK

Related work includes feature-based monocular SLAM, visual-inertial and stereo systems, and direct systems with appearance-based loop detection. LDSO differs by reusing repeatable frontend features and their depth estimates for loop closure constraints.

  • Feature-based SLAM: Feature-based SLAM systems commonly combine frontend tracking, local mapping, BoW loop detection, pose-graph optimization, and sometimes global bundle adjustment.ORB-SLAM is cited as an influential example of this multi-level map-optimization design.
  • Sensor-aided systems: Visual odometry has been extended to SLAM using inertial sensors, stereo cameras, map-based updates, and marginalization-window estimators.These approaches address robustness or observability through additional sensors or alternative estimator designs.
  • Direct SLAM: LSD-SLAM uses FAB-MAP for large-loop candidate generation, but extracts separate features and computes constraints through direct alignment of semi-dense depth maps.Its loop-closure process therefore does not reuse feature matches from the odometry frontend.

III. LOOP CLOSING IN DSO

LDSO extends DSO with a global pose graph and loop-closing module while retaining DSO’s sliding-window direct odometry formulation. The design addresses the difficulty of global photometric optimization by using pose constraints and loop detection alongside the local window.

  • DSO Framework: DSO jointly optimizes keyframe poses and inverse-depth points in a 5–7-keyframe sliding window using photometric error.The active window estimates each new frame’s initial pose by directly aligning projected active 3D points, and may add the frame to local bundle adjustment.
  • DSO Framework: The sliding window forms a local co-visibility graph, but marginalized keyframes and points are removed from the local optimization.
  • Global Optimization: Global photometric bundle adjustment is impractical because it requires storing all images and handling DSO’s large point set.
  • Loop Closing: LDSO adds a global pose graph and uses BoW with ORB features selected from DSO points to propose and validate loop candidates.
  • Loop Closing: The loop-closing module estimates relative Sim(3) constraints for validated candidates and connects them with pose information from the sliding window.

B. Point Selection with Repeatable Features

LDSO modifies DSO’s pixel selection to include repeatable corner features while preserving extra high-gradient pixels for weakly textured environments. Corners support both tracking and loop detection, whereas non-corners remain tracking-only points.

  • Tracking and Repeatability: When too few corners are available, LDSO adds extra DSO-style pixels to remain robust in feature-less or weakly textured areas.
  • Corner Selection: LDSO reserves part of DSO’s selected pixels for corners detected with the Shi–Tomasi score.The remaining pixels continue to be selected using DSO’s original strategy; DSO selects 2000 pixels by default.
  • Corner Selection: ORB descriptors are computed only for the selected corners and packed into a BoW model for loop closure detection.
  • Tracking and Repeatability: The frontend tracks with both corners and non-corners, while non-corners are not used for feature matching between keyframes.

C. Loop Candidates Proposal and Checking

LDSO proposes loop candidates through a BoW database and verifies them geometrically before estimating a Sim(3) transformation. The estimation combines 2D reprojection information with 3D points carrying depth.

  • Candidate Proposal: Loop candidates are retrieved from the DBoW3 database and restricted to keyframes outside the current sliding window.
  • Candidate Checking: ORB feature matches are verified with RANSAC PnP, which provides an initial SE(3) transformation estimate.
  • Sim(3) Estimation: The Sim(3) constraint minimizes a cost combining 2D reprojection errors for points without depth and 3D geometric errors for points with depth.
  • Sim(3) Estimation: Scale is estimated from the 3D term, while the 2D term improves rotation and translation when depth estimates are noisy.

D. Sliding Window and Sim(3) Pose Graph

LDSO fuses local sliding-window information with a global pose graph by converting window information into pairwise relative pose observations. The graph optimization is constrained to preserve the current frame’s local estimate.

  • Sliding Window: The sliding-window state contains keyframe poses in twist coordinates and points represented by inverse depth.
  • Sliding Window: DSO’s windowed optimization uses Levenberg–Marquardt, with a Hessian approximated as JᵀWJ+λI and an increment solving Hδx=−b.
  • Sparse Optimization: The Hessian has an arrow-like sparse structure, with the point block Hdd diagonal, enabling sparse bundle-adjustment computations.
  • Pose Graph Construction: DSO’s marginalization prior acts as a hyperedge over the keyframes inside the window, whereas traditional pose graphs use pairwise keyframe observations.
  • Pose Graph Construction: LDSO approximates constraints inside the marginalization window with pairwise relative poses computed from the frontend’s current global pose estimates.
  • Pose Graph Optimization: The current frame’s pose is fixed during pose-graph optimization, so optimization primarily modifies the older global trajectory.

IV. EVALUATION

LDSO is evaluated on three popular public datasets in a monocular setting.

  • The evaluation covers TUM-Mono, EuRoC MAV, and KITTI Odometry datasets under monocular operation.

A. The TUM-Mono Dataset

The TUM-Mono evaluation tests whether corner-focused point selection preserves DSO’s visual-odometry accuracy and examines loop-closed trajectories qualitatively.

  • Dataset and protocol: TUM-Mono contains 50 indoor and outdoor monocular sequences with photometric calibration but no full ground-truth trajectories.The camera returns to its starting point in every sequence, making accumulated drift suitable for evaluation.
  • Dataset and protocol: The study compares random, original DSO, and proposed point-selection strategies using 10 forward and 10 backward runs per sequence.Accumulated translational, rotational, and scale drift are computed from keyframe trajectories.
  • Results: Corner-feature integration does not reduce the original DSO visual-odometry accuracy, while random selection causes more frequent tracking failures.Among successfully tracked sequences, random selection does not appear to increase errors.
  • Results: Qualitative TUM-Mono results include Sim(3)-closed trajectories and a reconstructed map.These examples are shown in Figures 6 and 7.

B. The EuRoC MAV Dataset

The EuRoC MAV evaluation compares LDSO with DSO and ORB-SLAM2 using monocular RMSE, while the KITTI discussion examines loop closure for large-scale monocular operation.

  • EuRoC MAV evaluation: EuRoC MAV provides 11 stereo-image sequences with synchronized IMU readings and ground-truth camera trajectories.The methods are evaluated in monocular settings using root-mean-square error.
  • EuRoC MAV evaluation: Each EuRoC sequence is run 10 times forward and 10 times backward for LDSO, DSO, and ORB-SLAM2.Results are reported in Figures 8 and 9.
  • KITTI evaluation: Monocular DSO and ORB-SLAM visual odometry suffer severe accumulated drift on KITTI, motivating evaluation of loop closure in the monocular system.Stereo cameras and IMUs are identified as alternative ways to address this large-scale drift.
  • KITTI evaluation: On KITTI sequences containing loops, LDSO substantially improves DSO and reaches accuracy comparable to ORB-SLAM2.The comparison uses Absolute Trajectory Errors after Sim(3) alignment; ORB-SLAM2 uses global bundle adjustment, whereas LDSO uses pose-graph optimization.

D. Runtime Evaluation

The evaluation reports the computational effect of LDSO’s additional feature processing and defines the displayed trajectory and error visualizations used in the paper.

  • Runtime analysis: Loop closure is occasional and pose-graph optimization runs in a single thread, so neither substantially affects main-thread computation time.The principal main-thread addition is feature extraction and descriptor computation.
  • Evaluation visualizations: Figure 4 shows color-coded alignment errors by run and dataset sequence, using 10 forward and 10 backward runs.Each square represents one run’s alignment error.
  • Evaluation visualizations: Figure 5 plots the number of runs below each accumulated translational, rotational, and scale-drift threshold.The x-axis is the error threshold and the y-axis is the count of runs below it.
  • Evaluation visualizations: Figure 6 contrasts the visibly drifting original DSO trajectory with the loop-closed trajectory on TUM-Mono.Figure 7 shows the LDSO map before and after loop closure for sequence 33.
  • Evaluation visualizations: Figures 8 and 9 report full-trajectory RMSE distributions after Sim(3) alignment to ground truth.Figure 8 covers named TUM-Mono sequences, while Figure 9 uses threshold counts on EuRoC MAV.
  • Runtime analysis: LDSO performs feature extraction and descriptor computation only for keyframes rather than every frame.This differs from feature-based approaches and accounts for the added point-selection cost.

V. CONCLUSION

LDSO integrates loop closure and global map optimization into the fully direct DSO system. Its adapted point selection preserves odometry robustness and accuracy while reducing global drift without global bundle adjustment.

  • LDSO integrates loop closure and global map optimization into the fully direct DSO system.
  • The approach is evaluated using Sim(3)-aligned trajectories on Kitti sequences 00, 05, and 07, which contain closed loops.
  • DSO’s point selection is adapted to include repeatable features, enabling ORB descriptors and bag-of-words models for loop closure detection.
  • The modified point selection retains the original odometry frontend’s robustness and accuracy while the backend reduces global rotation, translation, and scale drift.
Loading 1808.01111v1…