Source-linked AI summary
PL-VINS: Real-Time Monocular Visual-Inertial SLAM with Point and Line Features
Qiang Fu, Jialong Wang, Hongshan Yu, Islam Ali, Feng Guo, Yijia He, Hong Zhang
TL;DR
Line-feature VINS can add scene constraints, but the computational cost of LSD extraction is a bottleneck for real-time performance. PL-VINS combines modified LSD with point, line, and IMU optimization. It reports lower localization error than VINS-Mono at the same pose update rate.
Problem
Line features add scene-structure constraints, but LSD's high computational cost is a bottleneck for real-time VINS.
Source: §I · p. 1Method
PL-VINS uses modified LSD line extraction and fuses point, line, and IMU information with Plücker-coordinate line residuals.
Source: §I · p. 2Results
PL-SLAM's localization error is 16% lower without loop closure and 12% lower with loop closure than VINS-Mono.
Source: E. Sliding Window Optimization with… · p. 6Takeaways & Limitations
PL-VINS produces more accurate localization than VINS-Mono at the same pose update rate.
Source: §VI · p. 8
Abstract
from arXiv · showhide
Leveraging line features to improve localization accuracy of point-based visual-inertial SLAM (VINS) is gaining interest as they provide additional constraints on scene structure. However, real-time performance when incorporating line features in VINS has not been addressed. This paper presents PL-VINS, a real-time optimization-based monocular VINS method with point and line features, developed based on the state-of-the-art point-based VINS-Mono \cite{vins}. We observe that current works use the LSD \cite{lsd} algorithm to extract line features; however, LSD is designed for scene shape representation instead of the pose estimation problem, which becomes the bottleneck for the real-time performance due to its high computational cost. In this paper, a modified LSD algorithm is presented by studying a hidden parameter tuning and length rejection strategy. The modified LSD can run at least three times as fast as LSD. Further, by representing space lines with the Pl\"{u}cker coordinates, the residual error in line estimation is modeled in terms of the point-to-line distance, which is then minimized by iteratively updating the minimum four-parameter orthonormal representation of the Pl\"{u}cker coordinates. Experiments in a public benchmark dataset show that the localization error of our method is 12-16\% less than that of VINS-Mono at the same pose update frequency. %For the benefit of the community, The source code of our method is available at: https://github.com/cnqiangfu/PL-VINS.
I. INTRODUCTION
PL-VINS addresses the challenge of adding line features to monocular VINS without sacrificing real-time performance. It combines faster line detection and line-feature optimization with point and IMU information, and reports higher localization accuracy than VINS-Mono at the same pose update frequency.
I. INTRODUCTION
- The method represents space lines as infinite Plücker-coordinate lines and minimizes point-to-line reprojection residuals by iteratively updating a four-parameter orthonormal representation.The paper contrasts this with endpoint-based lines, whose endpoints are difficult to extract and track across viewpoint changes and occlusions.Source: §I · p. 2
- PL-VINS is presented as the first real-time optimization-based monocular VINS method with point and line features.The paper motivates line features as additional scene-structure constraints, while prior work had emphasized accuracy and robustness rather than real-time performance.Source: §I · pp. 1–2
- The modified LSD runs at least three times as fast as LSD through hidden-parameter tuning and length rejection.The paper targets dominant lines and rejects short features that are difficult to match or may disappear in subsequent frames.Source: §I · pp. 1–2
- Points, lines, and IMU information are fused in an optimization-based sliding window for pose estimation.This is the paper's stated feature-fusion approach.Source: §I · p. 2
- PL-VINS reports higher localization accuracy than VINS-Mono on EuRoc at the same pose update frequency.The cited comparison is against the state-of-the-art point-based VINS-Mono method.Source: §I · p. 2
II. RELATED WORK
The related work distinguishes filter-based from optimization-based VINS and reviews point-only and geometric-feature extensions. It positions PL-VINS within optimization-based methods, where LSD's computational cost is a real-time concern.
A. Filter-based VINS methods
- Filter-based VINS includes MSCKF, introduced as a multi-state constraint Kalman filter framework, and OpenVINS, an EKF-based platform later extended with line or plane features.The cited extensions incorporate line or plane features to improve localization accuracy.Source: A. Filter-based VINS methods · p. 2
B. Optimization-based VINS methods.
- Optimization-based VINS jointly optimizes visual and IMU residuals, while point-based VINS-Mono is a strong localization benchmark that can struggle when point extraction is unreliable.The paper focuses on optimization-based approaches despite their high nonlinear-optimization computation cost.Source: B. Optimization-based VINS methods · p. 2
- Point-and-line VINS methods jointly use point, line, and IMU residuals, but their use of LSD for line extraction creates a real-time bottleneck.Line and plane features are pursued because they provide additional constraints on scene structure; this paper focuses on real-time point-and-line methods.Source: B. Optimization-based VINS methods · pp. 2–3
III. SYSTEM OVERVIEW
PL-VINS combines point and line features in an optimization-based monocular visual-inertial SLAM system built on VINS-Mono. It preprocesses camera and IMU measurements, initializes a map, then estimates motion in a tightly coupled sliding-window VIO.
III. SYSTEM OVERVIEW
- PL-VINS integrates point and line features into an optimization-based monocular visual-inertial SLAM system based on VINS-Mono.The camera–IMU extrinsic parameters are assumed known in advance.Source: §III · p. 3
A. Measurement Preprocessing
- For each image, the system detects, tracks, and refines point and line features concurrently, while pre-integrating IMU measurements between consecutive frames.Points use Shi-Tomasi detection, KLT tracking, and RANSAC-based epipolar inlier identification; lines use modified LSD, LBD descriptors, and KNN matching.Source: A. Measurement Preprocessing · p. 3
- Initialization estimates up-to-scale poses, space points, and lines across several frames, then aligns the graph with the pre-integrated IMU state.Source: A. Measurement Preprocessing · p. 3
B. Local Visual-Inertial Odometry
- The local VIO thread triangulates point and line correspondences and jointly minimizes measurement residuals over a fixed-size sliding window.Points use inverse depth and lines use Plücker coordinates; each new frame triggers marginalization of the oldest window frame.Source: B. Local Visual-Inertial Odometry · p. 3
C. Loop Closure
PL-VINS modifies LSD for pose estimation by tuning hidden parameters and rejecting short line features. The system also selects keyframes and runs loop closure following VINS-Mono.
C. Loop Closure
- A frame becomes a keyframe when parallax exceeds a threshold or tracked-feature count falls below a threshold; loop-closure search then follows VINS-Mono.Source: C. Loop Closure · p. 3
A. Line Feature Detection
- The modified LSD runs at least three times as fast as LSD by tuning hidden parameters and rejecting short lines.The change targets LSD's high computational cost and many detections that are difficult to match or may disappear in later frames.Source: A. Line Feature Detection · p. 4
- The length-rejection threshold uses ratio η, and the method sets η = 0.125 within the suggested [0.1, 0.15] range.Higher η rejects longer lines and affects description and matching time.Source: Figure 5 · p. 4, A. Line Feature Detection · p. 4
- Decreasing the hidden parameters s and d reduces detection time with practically negligible accuracy loss; the selected values are s = 0.5 and d = 0.6.These settings differ from defaults 0.8 and 0.7, respectively.Source: A. Line Feature Detection · p. 4, Figure 4 · p. 4
B. Line Tracking and Inlier Refinement
PL-VINS describes and matches detected lines, refines line inliers, and represents reconstructed space lines with Plücker coordinates and a minimal orthonormal form for optimization.
B. Line Tracking and Inlier Refinement
- Line features are described with LBD and matched with KNN, then refined to identify inliers.The stated checks require Hamming distance greater than 30 and matching-line angle below 0.1 rad.Source: B. Line Tracking and Inlier Refinement · p. 5
C. Space Lines
- Space lines are initialized by triangulating image-plane correspondences and represented with Plücker coordinates.The coordinates comprise a plane normal and a direction vector.Source: C. Space Lines · p. 5
- Because Plücker coordinates are over-parameterized by the constraint n_w^T d_w = 0, optimization uses a minimal four-parameter orthonormal representation.The representation is expressed using U and W rotation matrices and computed through QR decomposition.Source: C. Space Lines · p. 5
D. Line Reprojection Residual Model
The line reprojection residual compares an observed line feature with the projection of its corresponding space line. It is modeled as a point-to-line distance.
D. Line Reprojection Residual Model
- The space line is transformed into camera coordinates and projected into the image plane to obtain a projection line.The transformation uses rotation and translation, and the projection uses the line projection matrix.Source: D. Line Reprojection Residual Model · p. 5
- For each observed space line, the reprojection error is the point-to-line distance from the observed feature midpoint to its projected line.The Jacobian is obtained by the chain rule.Source: D. Line Reprojection Residual Model · p. 5
E. Sliding Window Optimization with Point, Line and IMU
The sliding-window objective extends VINS-Mono with a line reprojection residual alongside prior, IMU, point, and loop-closure terms. The state is optimized iteratively, including line updates in a four-parameter orthonormal representation.
E. Sliding Window Optimization with Point, Line and IMU
- The sliding-window state includes IMU body position and orientation, acceleration and gyroscope biases, and point and line features.The passage also defines point inverse distance and a four-parameter orthonormal representation for each 3D line.Source: E. Sliding Window Optimization with… · p. 5
- The objective combines prior, IMU, point, line, and loop-closure residual terms in the sliding window.The line term is added to the four terms already used in VINS-Mono.Source: E. Sliding Window Optimization with… · pp. 5–6
- Line reprojection error is minimized by iteratively updating the minimum four-parameter orthonormal representation of Plücker coordinates.The objective uses a Huber norm to suppress outliers.Source: E. Sliding Window Optimization with… · p. 6
- Localization error is reported as 16% lower without loop closure and 12% lower with loop closure than VINS-Mono.The comparison is based on real reproduction tests; the supplied table caption identifies Table II as an RMSE ATE comparison.Source: E. Sliding Window Optimization with… · p. 6, Table II · p. 6
V. EXPERIMENTS
The experiments evaluate PL-VINS for localization accuracy and real-time performance on the EuRoC benchmark.
V. EXPERIMENTS
- PL-VINS is evaluated on EuRoC using Ubuntu 18.04, ROS Melodic, and a low-power Intel Core i7-10710U CPU at 1.10 GHz.The evaluation covers localization accuracy and real-time performance.Source: §V · p. 6
A. Accuracy Comparison
The experiments report lower localization error for PL-VINS than VINS-Mono overall, with an exception in one ATE sequence. RPE comparisons also favor PL-VINS over VINS-Mono and PL-VIO.
A. Accuracy Comparison
- ATE is lower for PL-VINS except on V1-02-medium, with a maximum reduction from 0.375 to 0.270 on MH-04-difficult.This is reported as an approximately 28% reduction relative to VINS-Mono.Source: A. Accuracy Comparison · p. 6
- Loop closure reduces accumulated error across all five sequences; on MH-04-difficult, VINS-Mono's error falls from 0.375 to 0.220.The passage presents VINS-Mono as an example of the loop-closure effect.Source: A. Accuracy Comparison · p. 6
- Localization error is reported as 16% lower without loop closure and 12% lower with loop closure than VINS-Mono.The conclusion attributes the improvement to line features in point-based VINS.Source: A. Accuracy Comparison · p. 7
- The trajectory and feature-map illustrations compare motion paths and reconstructed space features on the reported sequences.Fig. 7 identifies the additional space lines reconstructed by PL-VINS on MH-04-Difficult.Source: A. Accuracy Comparison · p. 7
- PL-VINS records 15 best RPE results, compared with 5 for VINS-Mono and 1 for PL-VIO.The evaluation uses fixed 1-second and non-fixed time-delta settings; the reported counts combine both.Source: A. Accuracy Comparison · p. 7
B. Real-Time Analysis
On MH-04-difficult, PL-VINS meets the paper's real-time criterion alongside VINS-Mono, while PL-VIO does not. The reported preprocessing time is lower for PL-VINS than for PL-VIO.
B. Real-Time Analysis
- VINS-Mono and PL-VINS run at 10 Hz, while PL-VIO fails the real-time criterion.The criterion requires over 10 Hz pose-update frequency while providing accurate localization.Source: B. Real-Time Analysis · p. 7
- PL-VINS takes approximately 46 ms in measurement preprocessing, versus approximately 100 ms for PL-VIO.VINS-Mono takes at most approximately 42 ms in local VIO; the paper attributes PL-VINS's efficiency relative to PL-VIO to modified LSD.Source: B. Real-Time Analysis · p. 7
VI. CONCLUSIONS
PL-VINS combines modified LSD with line constraints in an optimization-based monocular VINS and produces more accurate localization than VINS-Mono at the same pose update rate. The authors note that line features are not fully exploited and identify stereo and more challenging environments as future work.
VI. CONCLUSIONS
- PL-VINS is presented as the first real-time optimization-based monocular VINS method with point and line features, using modified LSD with hidden-parameter tuning and length rejection.Source: §VI · p. 7
- PL-VINS produces more accurate localization than SOTA VINS-Mono at the same pose update rate.Source: §VI · p. 8
- Line features are not fully exploited: their constraints are not used in loop closure, and frame-to-model matching is proposed to use older lines visible in the current frame.The proposed strategy would maintain a local map of space lines and establish correspondences between that model and the current frame.Source: §VI · p. 8
- Future work will extend monocular vision to stereo and test more challenging settings, including large-scale, low-texture, and low-light indoor scenes.These experiments are intended to further study the utility of line-feature constraints.Source: §VI · p. 8