PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual-Inertial Monocular SLAM with Map Reuse

Raul Mur-Artal, Juan D. Tardos

arXiv:1610.05949v2cs.ROcs.CV

TL;DR

Visual-inertial odometry lacks loop closure and map reuse, allowing drift to accumulate during repeated motion in the same environment. The paper introduces tightly coupled monocular Visual-Inertial ORB-SLAM with rapid IMU initialization, and reports metric-scale recovery, centimeter precision, and no drift accumulation during revisiting.

  • Problem

    Visual-inertial odometry cannot close loops or reuse maps, so trajectory drift accumulates even when the sensor repeatedly moves through the same environment.

  • Method

    The paper combines keyframe-based visual-inertial SLAM, online map reuse, real-time metric loop closure, and initialization of scale, gravity, velocity, and IMU biases.

  • Results

    Typically 1% scale-factor error and centimeter precision are achieved, while revisiting sequences show no drift accumulation.

  • Takeaways & Limitations

    Map reuse enables zero-drift localization in already mapped areas, with better accuracy than the compared stereo visual-inertial odometry during continual localization.

Abstract

from arXiv · show

In recent years there have been excellent results in Visual-Inertial Odometry techniques, which aim to compute the incremental motion of the sensor with high accuracy and robustness. However these approaches lack the capability to close loops, and trajectory estimation accumulates drift even if the sensor is continually revisiting the same place. In this work we present a novel tightly-coupled Visual-Inertial Simultaneous Localization and Mapping system that is able to close loops and reuse its map to achieve zero-drift localization in already mapped areas. While our approach can be applied to any camera configuration, we address here the most general problem of a monocular camera, with its well-known scale ambiguity. We also propose a novel IMU initialization method, which computes the scale, the gravity direction, the velocity, and gyroscope and accelerometer biases, in a few seconds with high accuracy. We test our system in the 11 sequences of a recent micro-aerial vehicle public dataset achieving a typical scale factor error of 1% and centimeter precision. We compare to the state-of-the-art in visual-inertial odometry in sequences with revisiting, proving the better accuracy of our method due to map reuse and no drift accumulation.

I. INTRODUCTION

The paper addresses visual-inertial odometry’s inability to close loops and reuse maps, which causes unbounded drift during revisiting. It introduces tightly coupled Visual-Inertial ORB-SLAM and an initialization procedure for monocular metric estimation.

  • Visual-inertial odometry computes incremental motion but cannot close loops or reuse mapped environments, so drift accumulates during continual revisiting.
  • Visual-Inertial ORB-SLAM is presented as a keyframe-based system that metrically closes loops in real time and reuses its online-built map.It detects large loops using place recognition, then applies lightweight pose-graph optimization followed by full bundle adjustment.
  • The initialization estimates structure, keyframe poses, scale, gravity, velocity, and gyroscope and accelerometer biases before tightly coupled optimization begins.It starts from monocular ORB-SLAM, estimates gyroscope bias, then solves for scale and gravity before full bundle adjustment.
  • The system fuses monocular camera frames with IMU measurements using a calibrated pinhole camera model and IMU preintegration.The IMU measurements include acceleration and angular velocity affected by sensor biases and gravity.

III. VISUAL-INERTIAL ORB-SLAM

Visual-Inertial ORB-SLAM tracks pose, velocity, and IMU biases at frame rate while combining visual reprojection and IMU constraints. Its prior-based tracking is reset after map changes so optimization remains consistent with the updated map.

  • Tracking: Tracking estimates sensor pose, velocity, and IMU biases at frame rate, enabling reliable camera-pose prediction.
  • Tracking: When the map changes, tracking discards the previous prior and links the current frame directly to the last keyframe.The old prior is invalid after a Local Mapping or Loop Closing update.
  • Tracking: Current-frame optimization minimizes matched-point reprojection error together with an IMU error term.After map updates, the current frame is linked by an IMU constraint to the last keyframe.
  • Tracking: The tracking objective combines visual reprojection and preintegrated IMU information with robust weighting.The reprojection term uses keypoint locations, map points, and their information matrices; the IMU term uses preintegration and bias-random-walk information.
  • Tracking: The resulting state estimate and Hessian become a prior for optimizing the next frame, which is then marginalized after joint optimization.

B. Local Mapping

Local Mapping performs bundle adjustment over a temporally selected keyframe window and visible map points, while fixing an outer window. The visual-inertial formulation adds IMU-state optimization and retains temporal keyframes needed for motion constraints.

  • Local Mapping: Local bundle adjustment optimizes the last N keyframes and all points observed by them, while connected outer keyframes contribute cost but remain fixed.Keyframe N+1 is always included in the fixed window to constrain IMU states.
  • Local Mapping: Visual-Inertial ORB-SLAM retrieves its local window by temporal keyframe order, unlike original ORB-SLAM’s covisibility-graph retrieval.
  • Local Mapping: The local bundle-adjustment cost combines IMU and reprojection errors, adding 9 velocity and bias states per keyframe.A smaller local window is required for real-time performance because the visual-inertial optimization is more complex.
  • Local Mapping: Keyframe culling is constrained so redundant-keyframe removal does not leave consecutive keyframes too far apart in time for useful IMU constraints.Longer temporal separation weakens the information provided by IMU measurements.

C. Loop Closing

Loop Closing detects revisited places through keyframe place recognition, validates matches geometrically, and reduces accumulated trajectory error with a scalable pose-graph optimization.

  • Loop Closing: Place recognition matches a recent keyframe to a past keyframe, then validates the match with a rigid transformation aligning their matched points.
  • Loop Closing: Loop correction uses pose-graph optimization because ignoring structure reduces complexity and provides good convergence in large maps.
  • Loop Closing: The visual-inertial system optimizes loop closures over 6 degrees of freedom because scale is observable.The pose graph ignores IMU information and does not optimize velocities.

IV. IMU INITIALIZATION

The initialization estimates the variables needed for visual-inertial full bundle adjustment from keyframes processed by a monocular SLAM system. It decomposes this task into four simpler subproblems.

  • The method initializes scale, gravity direction, velocity, and gyroscope and accelerometer biases for visual-inertial full bundle adjustment.
  • Initialization proceeds through gyroscope bias estimation, scale and gravity approximation, accelerometer bias estimation with refinement, and velocity estimation.

A. Gyroscope Bias Estimation

Gyroscope bias is estimated by comparing IMU gyroscope integration with relative orientations from consecutive monocular-SLAM keyframes. The resulting bias is then used to correct acceleration preintegration.

  • The method optimizes a constant gyroscope bias using orientations from all pairs of consecutive keyframes.The bias-change assumption makes the estimated bias constant during this optimization.
  • Relative keyframe orientations are obtained from ORB-SLAM, while the corresponding gyroscope integrations are computed between consecutive keyframes.
  • The gyroscope-bias objective is solved with Gauss-Newton initialized at zero bias.
  • After bias estimation, velocity and position preintegrations compensate acceleration measurements for the gyroscope bias.
  • Because monocular SLAM produces an arbitrary-scale camera trajectory, the initialization introduces a scale factor between camera and IMU coordinates.
  • Scale and gravity are estimated from relations across three consecutive keyframes, avoiding explicit estimation of all keyframe velocities.
  • Stacking the relations yields a 3(N −2) × 4 linear system solved by SVD for the scale factor and gravity vector, requiring at least 4 keyframes.

C. Accelerometer Bias Estimation, and Scale and Gravity Direction Refinement

Accelerometer bias is incorporated after an initial scale and gravity estimate, with gravity magnitude added to improve observability. A linear system then refines scale, gravity direction, and accelerometer bias.

  • Including accelerometer bias directly can make the estimation system ill-conditioned because gravity and accelerometer bias are difficult to distinguish.
  • The method introduces gravity magnitude G as additional information to increase observability.
  • Gravity direction is represented using an inertial reference with ˆgI = {0, 0, −1}, then related to the previously computed world gravity direction.
  • The gravity-alignment rotation is parameterized by two angles around the inertial x and y axes because rotation around z has no effect on gravity.
  • A perturbation δθ and first-order approximation refine the gravity direction while accounting for accelerometer bias.
  • Three-consecutive-keyframe relations eliminate velocities and form a linear system for scale, gravity-direction correction, and accelerometer bias.
  • The system contains 3(N −2) equations and 6 unknowns, and its condition number assesses whether the motion makes all variables observable.

D. Velocity Estimation

Once scale, gravity, and bias are known, the method computes velocities for all keyframes and obtains the latest keyframe velocity from the IMU velocity relation.

  • Known scale, gravity, and bias allow velocities for all keyframes to be computed using the velocity relation.
  • The velocity of the most recent keyframe is computed using the velocity relation in equation (3).

E. Bias Reinitialization after Relocalization

After long-term relocalization, the system reinitializes gyroscope and accelerometer biases using vision-localized frames while retaining known scale and gravity.

  • Gyroscope biases are reinitialized by solving (9) after place-recognition relocalization.
  • Accelerometer bias is estimated from a simplified (19) because scale and gravity are already known.Twenty consecutive vision-only localized frames are used to estimate both biases.

V. EXPERIMENTS

Experiments on EuRoC evaluate initialization, trajectory accuracy, map reuse, and comparison with stereo visual-inertial odometry. The system achieves metric-scale monocular localization with centimeter-level precision and avoids drift accumulation when revisiting mapped scenes.

  • B. SLAM Evaluation and Comparison to State-of-the-Art: The EuRoC evaluation covers 11 MAV sequences across two rooms and an industrial environment, with synchronized stereo images, IMU measurements, and ground truth.
  • A. IMU Initialization: 15 seconds of MAV exploration always gave accurate initialization on EuRoC, reliably estimating gravity, biases, scale, and velocity.Initialization was evaluated on V2 01 easy, where variables converged between 10 and 15 seconds.
  • B. SLAM Evaluation and Comparison to State-of-the-Art: Scale error was typically below 1%, with typical precision of 3cm in 30m^2 rooms and 8cm in 300m^2 industrial environments.The reported trajectory errors use raw ground truth; ideal true-scale and final full-BA results are also shown for comparison.
  • B. SLAM Evaluation and Comparison to State-of-the-Art: Visual-inertial SLAM processed V2 03 difficult, recovered metric scale, and avoided scale drift, while incurring higher bundle-adjustment cost than vision-only SLAM.Visual-inertial full BA typically converged in 15 iterations over 7 seconds, versus 5 iterations in under 1 second for vision-only full BA.
  • B. SLAM Evaluation and Comparison to State-of-the-Art: With map reuse, RMSE was 0.037m, 0.027m, and 0.076m for V1, V2, and MH, with scale errors of 1.2%, 0.1%, and 0.2%.Revisiting sequences showed no drift accumulation because their RMSE was no larger than for individual sequences.
  • B. SLAM Evaluation and Comparison to State-of-the-Art: Relative pose error grew with traveled distance for stereo visual-inertial odometry but did not accumulate for the monocular SLAM system using map reuse.The monocular method failed on V1 03 difficult, while the stereo method succeeded there; short-path accuracy was comparable.

VI. CONCLUSIONS

The proposed tightly coupled Visual-Inertial SLAM system closes loops in real time and reuses maps for zero-drift localization in previously mapped areas. It recovers monocular metric scale precisely and outperforms stereo visual-inertial odometry during continual revisiting.

  • Real-time loop closure and map reuse enable zero-drift localization in already mapped areas.
  • The monocular system recovers metric scale with high precision.
  • During continual localization in the same environment, it achieves better accuracy than state-of-the-art stereo visual-inertial odometry.
  • Zero-drift localization is particularly relevant to virtual and augmented reality, where user viewpoints must remain stable within the same workspace.
  • Stereo or RGB-D cameras may improve accuracy and robustness while simplifying IMU initialization because scale is known.
  • The main weakness is that monocular IMU initialization depends on successful monocular SLAM initialization.
Loading 1610.05949v2…