Source-linked AI summary
Visual-Inertial Navigation: A Concise Review
Guoquan Huang
TL;DR
VINS have become important for applications using increasingly ubiquitous visual and inertial sensors, but the field lacks a contemporary VINS-focused review. This paper surveys the field, emphasizing state estimation, related work, and open challenges, to help researchers and engineers track and understand existing systems more efficiently.
Problem
The literature lacks a contemporary review specializing in VINS, making related work difficult for researchers and engineers to find and understand.
Method
The paper provides a concise but complete survey focused on VINS state estimation, important related work, and remaining challenges.
Results
The paper presents a review of VINS research covering the field's state-estimation literature and related systems.
Takeaways & Limitations
The review is intended to help researchers and engineers track and understand state-of-the-art VINS algorithms and systems more efficiently.
Takeaways & Limitations
Current VINS are not robust enough for long-term, large-scale, safety-critical deployments, particularly beyond small-scale friendly environments.
Abstract
from arXiv · showhide
As inertial and visual sensors are becoming ubiquitous, visual-inertial navigation systems (VINS) have prevailed in a wide range of applications from mobile augmented reality to aerial navigation to autonomous driving, in part because of the complementary sensing capabilities and the decreasing costs and size of the sensors. In this paper, we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work -- which is unfortunately missing in the literature while being greatly demanded by researchers and engineers -- in the hope to accelerate the VINS research and beyond in our society as a whole.
1 Introduction
VINS address drift-prone pose estimation in GPS-denied settings by combining visual and inertial sensing. This review responds to the lack of a contemporary VINS-focused survey by organizing key estimation work and remaining challenges.
- Motivation: INS estimate 6DOF platform poses in GPS-denied environments using measurements from a rigidly connected 6-axis IMU.Low-cost, lightweight MEMS IMUs have enabled localization for mobile devices and micro aerial vehicles.
- Motivation: VINS remain challenging because low-quality sensors exacerbate motion drift accumulated without global information.Existing approaches include EKF, UKF, smoothing, and windowed optimization methods for visual-inertial SLAM and odometry.
- Review gap: The literature lacks a contemporary review specializing in VINS, making important related work difficult for researchers and engineers to find and understand.Broader SLAM surveys do not specifically cover VINS.
- Scope and contributions: The paper offers a concise but complete VINS review focused on state estimation, interprets important related work, and discusses remaining challenges.Its stated goal is to help researchers and engineers track and understand state-of-the-art VINS algorithms and systems more efficiently.
2 Visual-Inertial Navigation
Canonical VINS fuse IMU propagation with camera feature measurements in an EKF framework to estimate platform motion and feature positions. The formulation models sensor dynamics, noise, calibration, linearization, covariance propagation, and measurement updates.
- EKF formulation: The canonical EKF state contains IMU states and feature positions, with IMU gyroscope and accelerometer measurements driving state propagation.Static feature dynamics are combined with IMU motion dynamics.
- IMU kinematic model: The IMU model represents attitude with a unit quaternion and includes global position, velocity, gyroscope bias, accelerometer bias, and bias-driving white Gaussian noise.The gyroscope measures rotational velocity in the IMU frame, while acceleration is represented in the global frame.
- EKF propagation: The continuous-time error model is linearized around the current estimate, then discretized for covariance propagation using a state-transition matrix and discrete-time noise covariance.The transition matrix may be computed numerically or analytically before applying the EKF covariance update.
- IMU kinematic model: Quaternion errors use a multiplicative angle-error representation, providing a minimal 3×1 attitude error and a 3×3 covariance representation for attitude uncertainty.The error quaternion describes the small rotation aligning the true and estimated attitudes.
- Camera measurement model: A calibrated perspective camera provides measurements by projecting static 3D corner features, while camera–IMU rotation and translation define the sensor-frame transformation.When calibration is imperfect, these parameters can be included in the state and estimated jointly with poses.
- Camera measurement model: Linearized camera residuals and measurement Jacobians feed the standard EKF update to revise state estimates and error covariance.The camera observes visual corner features concurrently with estimating platform ego-motion.
3 State Estimation
VINS state estimation fuses IMU and camera measurements to track platform motion through filtering or optimization, with different computational and accuracy trade-offs. Visual processing further distinguishes indirect feature-based methods from direct intensity-based methods.
- VINS state estimation optimally fuses IMU measurements and camera images to provide motion tracking of the sensing platform.
- Filtering-based vs. Optimization-based Estimation: MSCKF tightly couples quaternion-based inertial propagation with an efficient EKF update while marginalizing feature measurements through Jacobian null-space projection.This reduces computational cost but prevents later feature relinearization, yielding performance-degrading approximations.
- Filtering-based vs. Optimization-based Estimation: Observability-constrained MSCKF variants enforce correct linearized VINS observability properties to improve filter consistency.
- Filtering-based vs. Optimization-based Estimation: Batch optimization solves nonlinear least-squares problems with relinearization to reduce error, but incurs high computational cost and commonly uses bounded sliding windows.
- Loosely Coupled vs. Tightly Coupled Fusion: Tightly coupled approaches fuse visual and inertial measurements within one process, achieving higher accuracy than loosely coupled fusion.
- VIO vs. SLAM: VIO and SLAM differ chiefly in loop-closure use: loop closures can bound VINS localization error but are difficult to integrate efficiently without inconsistent assumptions.
- Direct vs. Indirect Visual Processing: Indirect methods use tracked point features and geometric reprojection constraints, whereas direct methods use raw pixel intensities and more image information.Direct methods have been applied to dynamic motion and low-texture environments, while indirect methods remain more mature and robust in practice.
- IMU Preintegration: IMU preintegration reduces repeated inertial reintegration during optimization, while later continuous-time work addressed gaps in closed-form preintegration theory.
4 Sensor Calibration
VINS requires accurate spatial and temporal calibration to fuse camera and IMU measurements correctly. Calibration may be performed offline with batch trajectory models or online during sensor operation.
- Calibration Requirements: Accurate rigid-body transformation and timing calibration are critical for correctly fusing camera and IMU motion information.Hardware triggering, transmission delays, and clock synchronization errors can make sensor timestamps disagree.
- Offline Calibration: Offline calibration methods often require additional aids such as fiducial tags and batch processing.
- Offline Calibration: Kalibr represents the sensor trajectory with continuous-time B-splines to batch-calibrate multi-sensor extrinsics and intrinsics from inertial residuals.
- Online Calibration: Online calibration estimates calibration parameters during sensor operation, improving practical robustness and ease of use, but some methods omit temporal calibration.
5 Observability Analysis
Observability analysis tests whether measurements determine system states and parameters without ambiguity, guiding consistent VINS estimator design. Linearized VINS ideally preserves four unobservable directions, but standard EKF linearization can violate this structure.
- Observability Fundamentals: Observability examines whether available measurements suffice to estimate states and parameters without ambiguity.For observable systems, the observability matrix is invertible and relates closely to the Fisher information or covariance matrix.
- Nonlinear Analysis: Nonlinear VINS analyses use system trajectories and continuous symmetries to study indistinguishability and derive observable or unobservable quantities.
- Linearized VINS: The linearized VINS observability matrix ideally has a nullspace spanning four directions over the estimation interval.
- Linearized VINS: The nullspace includes global translation and global rotation about the gravity vector.
- Consistency Implications: Standard EKF linearization produces a three-dimensional rather than four-dimensional unobservable subspace, causing inconsistent gains from nonexistent information.FEJ, OC-VINS, and R-VIO were developed to address observability and consistency issues.
6 Discussions and Conclusions
The review frames VINS as a rapidly advancing field whose expanding literature motivates a consolidated reference. It concludes that persistent localization, semantic understanding, dynamic-object tracking, cooperation, and alternative sensors remain open challenges.
- Discussion: The growing VINS literature and sensor-specific trade-offs make it difficult for practitioners to track the state of the art or develop algorithms from scratch.
- Contribution: The review provides a concise but complete synthesis of state estimation, interprets important related work, and discusses remaining challenges.
- Persistent Localization: Current VINS provide accurate 3D motion tracking but are not robust enough for long-term, large-scale, safety-critical deployments in small-scale friendly environments.
- Semantic Localization and Mapping: Semantic localization remains limited because current systems primarily use handcrafted geometric features, while semantic understanding is only sparsely explored.
- High-Dimensional Object Tracking: Dynamic complex environments require real-time detection, representation, and tracking of moving objects alongside high-precision localization.
- Distributed Cooperative VINS: Real-time distributed cooperative VINS remains challenging despite preliminary cooperative VINS and mapping research.
- Alternative Aiding Sensors: Alternative aiding sensors may better suit particular conditions, including acoustic sonars underwater, LiDAR in poor lighting, and event cameras for dynamic motions.