PaperlayerView exact revision on arXiv

Source-linked AI summary

PL-SLAM: a Stereo SLAM System through the Combination of Points and Line Segments

Ruben Gomez-Ojeda, David Zuñiga-Noël, Francisco-Angel Moreno, Davide Scaramuzza, Javier Gonzalez-Jimenez

arXiv:1705.09479v2cs.CV

TL;DR

Traditional stereo visual SLAM can degrade when low-textured scenes provide too few reliable point features. PL-SLAM combines points and line segments throughout stereo localization and mapping, including loop closure, and reports robust real-time performance across varied environments, especially low-textured ones.

  • Problem

    Low-textured environments often lack sufficient reliable point features, degrading traditional stereo visual SLAM performance.

  • Method

    PL-SLAM is a real-time stereo SLAM system that combines point and line-segment features across visual odometry, mapping, bundle adjustment, and bag-of-words loop closure.

  • Results

    PL-SLAM reports superior accuracy and robustness in most dataset sequences compared with point-only and line-only systems, while running in real time.

  • Takeaways & Limitations

    Combining points and line segments supports robust localization in low-textured environments and produces geometrically meaningful maps.

Abstract

from arXiv · show

Traditional approaches to stereo visual SLAM rely on point features to estimate the camera trajectory and build a map of the environment. In low-textured environments, though, it is often difficult to find a sufficient number of reliable point features and, as a consequence, the performance of such algorithms degrades. This paper proposes PL-SLAM, a stereo visual SLAM system that combines both points and line segments to work robustly in a wider variety of scenarios, particularly in those where point features are scarce or not well-distributed in the image. PL-SLAM leverages both points and segments at all the instances of the process: visual odometry, keyframe selection, bundle adjustment, etc. We contribute also with a loop closure procedure through a novel bag-of-words approach that exploits the combined descriptive power of the two kinds of features. Additionally, the resulting map is richer and more diverse in 3D elements, which can be exploited to infer valuable, high-level scene structures like planes, empty spaces, ground plane, etc. (not addressed in this work). Our proposal has been tested with several popular datasets (such as KITTI and EuRoC), and is compared to state of the art methods like ORB-SLAM, revealing a more robust performance in most of the experiments, while still running in real-time. An open source version of the PL-SLAM C++ code will be released for the benefit of the community.

I. Introduction

PL-SLAM addresses the loss of feature-based SLAM performance in low-textured environments by combining keypoints and line segments in a stereo system. It extends this combination across localization, mapping, loop closure, and the resulting geometric map.

  • Low-textured environments often lack enough reliable keypoints, reducing accuracy and sometimes causing feature-based SLAM systems to fail.
  • Planar low-textured environments can still provide line segments, which complement keypoints and support more versatile, robust, and stable SLAM.The combined map contains both 3D points and segments, providing more structural information.
  • Existing line-feature approaches face higher computational costs and segment-specific tracking difficulties, with several methods limited to highly structured environments.
  • PL-SLAM combines point and line-segment features for real-time stereo localization and mapping, including a bundle-adjustment process that handles both feature types.
  • A bag-of-words approach combines point and line-segment descriptions to detect loop closures and perform place recognition.

III. PL-SLAM Overview

PL-SLAM organizes stereo visual odometry, local mapping, and loop closure into separate threads while representing both point and line-segment landmarks in its map. Keyframes, covisibility relationships, and background optimization support continuous tracking and real-time operation.

  • PL-SLAM uses visual odometry, local mapping, and loop closure as separate threads, with mapping and loop closure running in the background after keyframe insertion.This distribution allows continuous visual-odometry tracking while background modules process new keyframes.
  • The map contains keyframes, 3D point and line-segment landmarks, a covisibility graph, and a spanning tree.
  • Point landmarks store 3D positions, while line landmarks store directions and the 3D coordinates of their endpoints.
  • Local mapping connects keyframes sharing at least 20 landmark observations and optimizes their poses and observed landmark positions.
  • Loop closure retrieves visually similar frames using bag-of-words descriptors and accepts candidates when their surrounding local sequences are also similar.

IV. Feature Tracking

Feature tracking follows both keypoints and line segments through stereo frames, reconstructing their 3D positions before estimating motion from projection errors. ORB, LSD, and LBD provide complementary point and line-feature detection and matching.

  • The stereo visual-odometry front end tracks points and segments, computes their 3D positions and covariances, and projects them into the new camera pose.
  • ORB keypoints are matched efficiently using binary descriptors and mutual-best-match filtering to reduce outliers.
  • LSD extracts precise, repeatable line segments, while LBD supplies binary descriptors for stereo matching and frame-to-frame tracking.
  • The line-feature pipeline retains more structural lines, enabling more consistent maps that combine points and lines.

C. Motion Estimation

PL-SLAM estimates camera motion by minimizing robust projection errors for tracked keypoints and line segments, then uses motion uncertainty to decide when to insert keyframes. Local bundle adjustment optimizes the connected keyframes and their landmarks.

  • Camera ego-motion is estimated through robust Gauss-Newton minimization of keypoint and line-segment projection errors.A Pseudo-Huber loss and two-step minimization are used to handle outliers.
  • The estimated incremental motion is represented in se(3), with covariance approximated by the inverse Hessian of the final cost-function iteration.
  • Keyframe insertion compares entropy from motion between the previous and current keyframes with entropy from the previous keyframe and its first consecutive frame.
  • A frame becomes a new keyframe when α falls below the experimental threshold 0.9.
  • After keyframe insertion, bundle adjustment optimizes the local map formed by covisible keyframes and their observed landmarks.

A. Keyframe Insertion

When a keyframe is inserted, PL-SLAM updates the local map by refining keyframe poses and optimizing point and line-segment landmarks with bundle adjustment.

  • Keyframe insertion: Each inserted keyframe triggers local-map optimization after visual odometry selects it.The system first refines the relative pose between the current and previous keyframes using geometrically consistent feature associations.
  • Map construction: The map stores keyframe poses, pose uncertainties, new 3D landmarks, image coordinates, descriptors, and additional observations.Landmark representations include both points and line segments.
  • Bundle adjustment: Local bundle adjustment jointly optimizes keyframe poses, 3D point positions, and line-segment endpoints.The local map contains covisible keyframes sharing at least 20 landmarks and all landmarks observed by them.
  • Residuals: Point projection errors use 2D observation-to-projection distances, while line errors measure projected endpoints against their corresponding infinite image lines.Line endpoints are used because endpoint positions can shift along a line or become occluded.
  • Optimization: Levenberg-Marquardt recursively updates the optimization variables until convergence, after which local keyframe and landmark positions are updated.Observation errors affect only their associated keyframe and landmark blocks, enabling local Hessian construction.
  • Robustness: Point and line errors are weighted by inverse covariance matrices, although identity covariances with robust weights are more effective in practice.Landmarks with fewer than 3 observations are removed as less meaningful.

VI. Loop Closure

PL-SLAM performs loop-closure detection with a bag-of-words representation that combines binary descriptors from keypoints and line segments.

  • Visual representation: The bag-of-words approach summarizes keypoint and line-segment descriptors into word vectors using an offline vocabulary.Word vectors from captured images are stored in a database for later similarity search.
  • Loop closure: Descriptors from both feature types are used for data association and loop-closure detection.This extends bag-of-words place recognition beyond point-only descriptions.
  • Detection pipeline: The system first retrieves similar frames and then detects loop closures and estimates relative pose between candidate keyframes.The loop-closure stage includes candidate detection followed by pose correction.

A. Loop Closure Detection

PL-SLAM combines keypoint- and line-based similarity searches to improve loop-closure robustness, balancing feature strength and image dispersion.

  • Dual-feature retrieval: Separate visual vocabularies and databases retrieve similar images for keypoints and line segments in parallel.The two results are merged because different scenes may be described more distinctly by one feature type.
  • Similarity behavior: ORB-only similarity can produce false positives at corridor corners, whereas line-only similarity avoids those specific false positives.ORB similarities are generally more distinctive for nonsimilar images than line-only similarities.
  • Score fusion: The combined similarity score weights keypoint and line results according to feature strength and spatial dispersion.Strength depends on feature counts, while dispersion uses coordinate variances; line dispersion uses segment midpoints.
  • Score fusion: Equal importance is assigned to keypoints and segments through the 0.5 weighting factors, although the balance can be tuned to the environment.The score combines normalized counts and dispersion values for both feature types.
  • Evaluation: The proposed fusion strategy achieved better overall precision-recall results than single-feature and single-criterion alternatives across all evaluated datasets.Compared alternatives used only keypoints, only lines, or both features with only strength or only dispersion.
  • Verification: Candidate loop closures undergo relative-pose recovery and feature matching to reject remaining false positives.The procedure also searches for new correspondences between the current keyframe and local map.

2) Estimating the Relative Motion:

PL-SLAM validates loop-closure candidates using uncertainty, pose-change, and inlier-ratio criteria after relative-motion estimation.

  • Validation criteria: A candidate is rejected when the covariance matrix maximum eigenvalue is not below 0.01.This criterion detects potentially ill-conditioned estimates associated with many outliers.
  • Validation criteria: Estimated translation and rotation must remain below 0.50 meters and 3.00 degrees, respectively.Large pose changes between visually similar frames indicate likely incorrect loop closures.
  • Validation criteria: The estimated relative motion must retain an inlier ratio above 50%.The threshold limits the influence of outliers and incorrect place-recognition associations.

B. Loop Correction

PL-SLAM corrects loop-induced drift by optimizing keyframe poses in a pose graph and then updating associated landmarks and local maps.

  • Loop correction formulates pose-graph optimization with keyframes as nodes and essential-graph and spanning-tree constraints as edges.The loop error is defined as an se(3) difference between an estimated keyframe transformation and the currently observed transformation.
  • For regular edges, the initial loop-graph error is zero because the reference and first-step estimated transformations coincide.
  • The pose-graph problem is solved with g2o to obtain optimized poses for keyframes connected by loop-closure edges.
  • After optimization, PL-SLAM updates keyframe and landmark poses and merges local maps from both sides of the loop.Matched landmarks are fused first, followed by searches for correspondences among unmatched landmarks.

VII. Experimental Validation

KITTI experiments show that line segments add little accuracy in highly textured urban scenes, while PL-SLAM generally slightly outperforms point-only and ORB-SLAM approaches. Loop closures correct drift in trajectories, but line detections can introduce noisy map elements.

  • PL-SLAM was evaluated on all 11 KITTI sequences with available ground truth and compared against stereo ORB-SLAM and point- and line-only systems.Experiments used relative keyframe-position errors on an Intel Core i5-6600 CPU without GPU parallelization.
  • In highly textured urban KITTI sequences, line segments barely improve accuracy because detected points are generally sufficient for SLAM operation.
  • PL-SLAM performs slightly better than point-only SLAM and ORB-SLAM for most datasets, particularly in rotation estimation, whereas line-only SLAM performs worse outdoors and sometimes fails.
  • The KITTI-07 map captures building structure with line segments but also contains noisy measurements where detected lines lack clear structural meaning.
  • Loop closures correct drift in long KITTI trajectories, while sequences without loop closures retain drift despite similar relative translation and rotation errors.KITTI-00 contains many loop closures, KITTI-07 contains a closing loop, and KITTI-08 contains none.

B. EuRoC MAV dataset

EuRoC experiments show that combining points and line segments benefits motion estimation in indoor and structured scenes, while maps represent structured objects alongside noisy features from difficult regions.

  • The EuRoC dataset contains 11 stereo MAV sequences across two indoor rooms and one industrial environment with varying speed, illumination, and texture.
  • Line-segment maps depict structured objects such as checkerboards and boxes, while non-structured regions contain smaller, noisier segments.These maps are described as geometrically richer than point-only maps and suitable for extracting high-level meaningful information.
  • Including line segments substantially benefits camera-motion estimation and robustness in indoor and structured scenarios.Both point-only and line-only systems perform worse than PL-SLAM, while ORB-SLAM loses feature tracking in several sequences.
  • Low-textured stereo sequences show that line-based methods can robustly estimate the camera path when point-based approaches fail or perform worse.The sequences were recorded in a room equipped with an OptiTrack system providing trajectory ground truth.

D. Performance

PL-SLAM combines point and line-segment features to improve robustness in low-textured environments while retaining real-time operation. Compared with ORB-SLAM2, it is not claimed to be more accurate in common environments, and its comparisons use a different pose-error metric than the original ORB-SLAM2 paper.

  • Computational performance: The visual odometry thread runs in real time for lower-resolution datasets and nearly in real time on KITTI, while outperforming ORB-SLAM2 in processing speed.Local bundle adjustment runs at around 20 Hz in parallel, and loop closure is the most time-consuming step but is also computed in parallel and not every frame.
  • System design: PL-SLAM includes line segments alongside point features, enabling robust camera localization where keypoint-only methods perform poorly or fail.The system also incorporates binary line descriptors for loop closure and a frame-to-frame visual odometry thread.
  • Accuracy and robustness: PL-SLAM and ORB-SLAM2 perform similarly in common environments, with slightly higher accuracy for ORB-SLAM2 and lower computational burden for PL-SLAM.The paper claims better performance for PL-SLAM specifically in low-textured scenarios, where point features decrease substantially.
  • Accuracy and robustness: PL-SLAM achieves better performance in low-textured environments, where the number of point features dramatically decreases.These environments can contain planar elements rich in linear shapes, motivating the complementary use of point and segment features.
  • Evaluation caveat: The reported ORB-SLAM2 comparison uses keyframe-to-keyframe pose errors rather than the metric used in the original ORB-SLAM2 paper.Therefore, the paper cautions that differences between these results and the original publication reflect metric differences.
  • Future work: Future work includes better keypoint front-ends and refined maps produced by filtering structural lines with SfM or Multi-Stereo techniques.These directions target feature-tracking speed and more meaningful information about structured parts of the environment.
Loading 1705.09479v2…