Source-linked AI summary
CamVox: A Low-cost and Accurate Lidar-assisted Visual SLAM System
Yuewen Zhu, Chunran Zheng, Chongjian Yuan, Xu Huang, Xiaoping Hong
TL;DR
Camera–lidar calibration in uncontrolled scenes remains challenging, and low-cost lidar offers a way to combine camera detail with depth and range. CamVox adapts Livox lidar to ORB-SLAM2 and uses its non-repeating scans for automatic calibration. Its trajectories are reported close to ground truth and significantly better than the compared frameworks, with real-time onboard operation.
Problem
Camera–lidar calibration remains challenging, and the paper identifies no open-source algorithm for calibrating the sensors in uncontrolled scenes.
Source: §II · p. 3Method
CamVox adapts Livox lidar to ORB-SLAM2, using synchronized, IMU-corrected lidar input and Livox’s non-repeating scan pattern for automatic camera–lidar calibration.
Source: §I · p. 1Results
CamVox trajectories are close to ground truth and significantly outperform Livox Horizon LOAM and VINS-mono, while running in real time onboard.
Source: C. Comparison of trajectory · p. 6, D. Timing results · p. 6Takeaways & Limitations
The authors present CamVox as a low-cost, out-of-the-box SLAM solution for the robotics and sensor research community.
Source: §V · p. 6
Abstract
from arXiv · showhide
Combining lidar in camera-based simultaneous localization and mapping (SLAM) is an effective method in improving overall accuracy, especially at a large scale outdoor scenario. Recent development of low-cost lidars (e.g. Livox lidar) enable us to explore such SLAM systems with lower budget and higher performance. In this paper we propose CamVox by adapting Livox lidars into visual SLAM (ORB-SLAM2) by exploring the lidars' unique features. Based on the non-repeating nature of Livox lidars, we propose an automatic lidar-camera calibration method that will work in uncontrolled scenes. The long depth detection range also benefit a more efficient mapping. Comparison of CamVox with visual SLAM (VINS-mono) and lidar SLAM (LOAM) are evaluated on the same dataset to demonstrate the performance. We open sourced our hardware, code and dataset on GitHub.
I. INTRODUCTION
CamVox adapts Livox lidar to ORB-SLAM2, combining camera information with lidar depth and range. It uses Livox’s non-repeating scans for automatic camera–lidar calibration in uncontrolled scenes and reports accurate trajectories and large-scale mapping.
I. INTRODUCTION
- CamVox combines camera input with Livox lidar in an ORB-SLAM2-based visual SLAM system.The paper presents it as a low-cost lidar-assisted system.Source: §I · p. 1
- Its preprocessing synchronizes lidar and camera data, uses IMU data to correct scan distortions, and transforms lidar points into camera frames.The system also uses Livox’s non-repeating scan pattern for automatic camera–lidar calibration in uncontrolled scenes.Source: §I · p. 1
- The authors report that CamVox supports large-scale mapping with higher efficiency and robust operation in strong sunlight outdoors.They also describe its lidar point-cloud accuracy and range as superior to other depth cameras.Source: §I · p. 1
- The paper evaluates CamVox against mainstream frameworks and reports very accurate trajectory results.It also open-sources the hardware, code, and dataset as an out-of-the-box solution.Source: §I · p. 1
- The example map’s long side is about 200 meters, and its figure includes a robot trajectory, dense RGB-D map, and magnified point-cloud detail.The caption identifies the magnified view as demonstrating point-cloud quality.Source: §I · p. 1, Figure 1 · p. 1
II. RELATED WORK
Prior work addresses camera scale and depth limits, lidar’s lower angular resolution and lack of loop closure, and the cost and calibration challenges of sensor fusion. Livox’s low-cost, non-repeating scans offer a potential route to dense point clouds and calibration in uncontrolled scenes.
II. RELATED WORK
- Monocular ORB-SLAM cannot recover real-world scale accurately, while ORB-SLAM2 adds stereo and RGB-D depth support.Stereo long-depth accuracy depends on a long baseline and is susceptible to calibration changes; RGB-D cameras are sunlight-sensitive and typically range less than 10 meters.Source: §II · pp. 1–2
- Lidar provides direct spatial measurements and can be robust in dynamic environments, but its lower angular resolution can hinder performance in tunnels or hallways, and LOAM lacks loop closure.LeGO-LOAM adds loop closure and feature-extraction changes; other camera–lidar fusion approaches include loosely coupled and tightly coupled systems.Source: §II · p. 2
- Livox’s prism-based, non-repeating scans can build high-definition point clouds as they accumulate; Horizon reaches 64 lines at 100 ms and detects up to 260 m under strong sunlight.The paper notes that the scan pattern may help camera–lidar calibration where traditional multiline lidar lacks precision between lines.Source: §II · p. 2
- Camera–lidar calibration remains challenging, with prior approaches using targets or manual effort and no open-source algorithm identified for uncontrolled scenes.The paper points to Livox’s non-repeating scan pattern as a possible solution.Source: §II · pp. 2–3
III. CAMVOX FRAMEWORK
CamVox builds on ORB-SLAM2’s RGB-D model, preprocessing separate RGB-D inputs and adding automatic calibration for uncontrolled scenes. It uses lidar-assisted visual keyframes for local mapping and ORB-SLAM2’s bundle adjustment and loop closure for pose-graph optimization.
III. CAMVOX FRAMEWORK
- CamVox builds on ORB-SLAM2’s RGB-D model, with separate RGB-D input preprocessing and automatic calibration for uncontrolled scenes.Its platform includes a camera, Livox Horizon, and IMU; GPS/RTK is used for ground-truth estimation.Source: §III · p. 3, Figure 3 · p. 3
- Lidar-assisted visual keyframes generate local maps, while ORB-SLAM2’s backend uses pose-graph optimization, bundle adjustment, and loop closure.The paper describes this lightweight optimization as contributing to robustness.Source: §III · p. 3
- The platform figure shows the robot-mounted CamVox hardware, an additional RGB-D camera for comparison, and example RGB and lidar-derived depth images.The close-up identifies the camera, Livox Horizon, and IMU.Source: Figure 3 · p. 3
A. Hardware and software
CamVox combines a rolling-shutter camera, Livox Horizon lidar, and IMU in a synchronized system whose software adds RGBD preprocessing and automatic camera–lidar calibration to ORB-SLAM2. IMU-corrected lidar depth supplies longer-range RGBD input than the Realsense D435 in the reported comparison.
A. Hardware and software
- The camera, Livox Horizon lidar, and IMU are hard-synchronized by a 10 Hz trigger; the IMU runs at 200 Hz, and the lidar outputs continuously timestamped points.The system also records GPS-RTK data for ground-truth comparison and mounts a Realsense D435 for comparison.Source: A. Hardware and software · p. 3
- A preprocessing thread combines synchronized camera and IMU-corrected lidar data into a unified RGBD frame, alongside ORB-SLAM2’s major threads and a camera–lidar calibration thread.Calibration can start automatically when the robot is detected to be stationary or through human interaction.Source: A. Hardware and software · p. 3, Figure 4 · p. 3
- IMU-measured motion transforms lidar points from their individual scan times into the lidar coordinate frame at the camera-trigger time, correcting scan distortion.Fig. 5 contrasts a raw moving scan with the IMU-corrected point cloud.Source: A. Hardware and software · p. 4, Figure 5 · p. 4
- CamVox reliably detects depth-associated camera features beyond 100 meters, while the Realsense RGBD camera does not detect beyond 10 meters and suffers from sunlight noise.The system uses close keypoints with associated depth below 130 meters.Source: A. Hardware and software · p. 4, Figure 6 · p. 4
C. Calibration
CamVox calibrates lidar to camera in uncontrolled scenes by accumulating scans while stationary and matching projected lidar edges to camera contours. An improved ICP cost and iterative optimization refine the extrinsic parameters, which are evaluated before potential update.
C. Calibration
- Calibration can run in uncontrolled scenes after the robot is detected to be stationary, accumulating lidar scans for a few seconds to form a dense depth image for correspondence with the camera image.The method addresses the need to recalibrate after possible misalignment and avoids motion blur during accumulation.Source: C. Calibration · p. 4
- The method projects lidar depth and reflectivity onto the imaging plane using initial extrinsics, then compares the resulting contours with camera-image contours and optimizes extrinsics with an improved ICP cost.The optimized cost is evaluated against the previous value to decide whether to update the extrinsic parameters.Source: C. Calibration · p. 4, Figure 7 · p. 4
- After histogram equalization, Canny edges from lidar depth and reflectivity are combined, short or cluttered interior edges are filtered, and shared camera–lidar edges are matched by nearest-edge distance.Edges shorter than 200 pixels are removed.Source: C. Calibration · p. 5
- The ICP cost adds a degree-of-mismatch term to address wrong matches in which very few points contribute, and coordinate descent iteratively optimizes roll, pitch, and yaw.The authors report that a value of 10 for the weighting factor is a good starting default.Source: C. Calibration · p. 5
IV. RESULTS
The evaluation covers automatic calibration, the effect of the close-keypoint depth threshold, and CamVox trajectory performance relative to other SLAM frameworks. It also includes a time analysis.
IV. RESULTS
- The evaluation presents CamVox trajectory results in comparison with mainstream SLAM frameworks.The passage does not report the comparison outcome.Source: §IV · p. 5
- The evaluation examines the results of automatic calibration.Source: §IV · p. 5
- The evaluation studies the effect of the depth threshold used for close keypoints and includes a time analysis.Source: §IV · p. 5
A. Automatic calibration results
Automatic calibration brings the lidar–camera overlay close to the best manual result and is reported to work robustly across varied uncontrolled scenes.
A. Automatic calibration results
- The automatic calibration cost was 6.11, close to the best manual result of 5.88 and below the uncalibrated value of 7.95.Fig. 8 compares the uncalibrated, automatically calibrated, and best manual overlays.Source: A. Automatic calibration results · p. 5, Figure 8 · p. 5
- Calibration was triggered after an initial overlay misalignment of more than 2 degrees.The uncalibrated overlay is shown in Fig. 8(a), followed by automatic calibration in Fig. 8(b).Source: A. Automatic calibration results · p. 5, Figure 8 · p. 5
- The image-like calibration scheme worked robustly on most uncontrolled scenes, with evaluated scene costs around or below 6.Fig. 8 includes outdoor roads, outdoor artificial structures, and indoor underexposed structures.Source: A. Automatic calibration results · p. 5, Figure 8 · p. 5
B. Evaluation of depth threshold for keypoints
Increasing the close-keypoint depth threshold expands the map and mapped features; CamVox uses 130 meters as its default threshold.
B. Evaluation of depth threshold for keypoints
- Raising the close-keypoint depth threshold from 20m to 130m significantly increased mapping scale and mapped features.The lidar detects up to 260 meters, providing many fused-frame keypoints that can be treated as close.Source: B. Evaluation of depth threshold for… · p. 5
- Matching-point counts increased as more frames were captured during the first 100 frames at 10 FPS.The evaluation tracked matching points as a function of time after CamVox started.Source: B. Evaluation of depth threshold for… · p. 5
- A larger threshold tracked more features initially, and CamVox sets the default close-keypoint threshold to 130 meters.The authors say the additional initial matches help start robust localization and mapping.Source: B. Evaluation of depth threshold for… · p. 5
C. Comparison of trajectory
On the SUSTech dataset, CamVox was reported to track close to ground truth and significantly outperform Livox Horizon LOAM and VINS-mono.
C. Comparison of trajectory
- The trajectory comparison used the SUSTech dataset and evo tools.The evaluation compared CamVox with two mainstream SLAM frameworks and ground truth.Source: C. Comparison of trajectory · p. 5
- Fig. 9 compares reconstructed maps across close-keypoint thresholds and plots matching-point counts by frame sequence and threshold.The threshold-based matching-count plot evaluates the beginning of SLAM at the fifth frame.Source: Figure 9 · p. 6
- CamVox was very close to ground truth and significantly outperformed Livox Horizon LOAM and VINS-mono.Fig. 10 shows the three trajectories alongside ground truth.Source: C. Comparison of trajectory · pp. 5–6, Figure 10 · p. 6
D. Timing results
CamVox ran in real time on the tested onboard computer, while automatic calibration took about 58 seconds and ran outside routine motion processing.
D. Timing results
- CamVox performed in real time on the onboard Manifold 2C with a 4-core Intel Core i7-8550U processor.The timing analysis is identified as Table II.Source: D. Timing results · p. 6, Table II · p. 6
- Automatic calibration took about 58 seconds to finish.Table II presents the framework timing analysis.Source: D. Timing results · p. 6, Table II · p. 6
- The calibration thread ran occasionally while the robot was stationary, and parameter updates could be deferred.The authors state this timing would not affect real-time performance.Source: D. Timing results · p. 6
V. CONCLUSION AND PERSPECTIVE
CamVox is a low-cost lidar-assisted visual SLAM framework combining camera angular resolution with lidar depth and range. The paper reports automatic calibration in uncontrolled scenes, evaluations across several measures, and real-time operation on an onboard computer.
- CamVox combines camera angular resolution with lidar depth and range in a low-cost visual SLAM framework.Source: §V · p. 6
- The Livox lidar working principle supports an automatic calibration algorithm designed for uncontrolled scenes.Source: §V · p. 6
- The framework was evaluated for calibration accuracy, close-keypoint depth-threshold classification, and trajectory comparison.Source: §V · p. 6
- CamVox runs in real time on an onboard computer.Source: §V · p. 6
- The authors propose CamVox as a potential low-cost, out-of-the-box solution for robotics and sensor research.Source: §V · p. 6
Related papers
This paper cites
- ORB-SLAM: a Versatile and Accurate Monocular SLAM System2015
- ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras2016
- ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM2020
- VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator2017