Source-linked AI summary
LBDU-VIO: Learned Bias Dynamics and Uncertainty for Visual-Inertial Odometry with Unreliable Vision
Qizhi Guo, Junning Lyu, Defu Lin, Shaoming He
TL;DR
VIO relies on IMU propagation between visual updates, but conventional random-walk bias and fixed-noise models can mismatch when vision is unreliable. LBDU-VIO adds learned continuous-time bias dynamics and adaptive IMU uncertainty to MSCKF propagation. On EuRoC and TUM-VI, it reports accuracy gains over representative baselines, including lower mean APE on each benchmark than the strongest comparable baseline described.
Problem
Conventional VIO uses random-walk IMU biases and fixed noise parameters, although IMU modeling errors can accumulate when visual information is unreliable.
Source: §I · p. 1Method
LBDU-VIO augments MSCKF with learned continuous-time bias dynamics and an IMU uncertainty model that predicts motion-adaptive noise covariances for propagation.
Source: §I · pp. 1–2Results
On EuRoC and TUM-VI, LBDU-VIO reports accuracy gains over representative baselines; on EuRoC, it reduces mean APE by 13.6% relative to VIO-IPNet.
Source: B. VIO Evaluation Under Nominal… · pp. 7–8, §VI · p. 11Takeaways & Limitations
Experiments on EuRoC and TUM-VI suggest that refining inertial propagation offers a practical path toward robust VIO, especially during visual outages.
Source: §VI · p. 11
Abstract
from arXiv · showhide
Visual-inertial odometry (VIO) for aerial robots relies on high rate inertial measurement unit (IMU) propagation between visual updates. However, conventional multi state constraint Kalman filters (MSCKFs) use random walk bias assumptions and fixed noise parameters, which can limit robustness when visual information is unreliable. To address this problem, we propose LBDU-VIO, a learning-augmented MSCKF with learned continuous time bias dynamics and an IMU uncertainty model. A neural ordinary differential equation (ODE) models continuous time bias dynamics to propagate the filter's bias states, replacing their random walk model. The IMU uncertainty model predicts motion adaptive measurement noise covariances for covariance propagation. Both models are trained with pose supervision without direct labels. Experiments on real world EuRoC and TUM-VI benchmarks show lower errors than representative visual-inertial baselines, including a 25.1% reduction in mean relative position error compared with S-MSCKF on EuRoC sequences with 10s visual outage.
I. INTRODUCTION
VIO depends on accurate inertial propagation when vision is unreliable, but conventional bias and noise assumptions can mismatch low-cost IMUs. LBDU-VIO combines learned continuous-time bias dynamics and adaptive uncertainty with visual updates.
I. INTRODUCTION
- When visual information is unreliable, IMU modeling errors can accumulate into large drift, while conventional VIO uses random-walk biases and fixed noise parameters.Learning-based motion corrections also commonly predict a single output from a fixed window rather than model the underlying time-varying IMU errors.Source: §I · p. 1
- LBDU-VIO couples continuous-time IMU bias dynamics with visual updates and predicts motion-adaptive gyroscope and accelerometer noise covariances for propagation.The approach is evaluated on real-world EuRoC and TUM-VI benchmarks under nominal and unreliable visual conditions.Source: §I · pp. 1–2
A. Learning Inertial Motion Models
- Learning-based inertial motion methods regress outputs such as displacement, velocity, orientation, or window-level relative pose, rather than explicitly modeling the IMU error process.The cited methods include IONet, RoNIN, TLIO, AirIO, OriNet, RNIN-VIO, and learned IMU preintegration.Source: A. Learning Inertial Motion Models · p. 2
B. Learning Inertial Bias and Noise Models
Prior approaches often use motion-invariant bias and noise models or learn corrections outside the IMU-rate propagation step. The paper motivates integrating learned bias dynamics and uncertainty into that step to refine both state and covariance propagation.
B. Learning Inertial Bias and Noise Models
- Conventional IMU error models use random-walk biases and fixed white-noise statistics, while learned denoising can combine systematic bias with stochastic noise into one correction.Explicit bias methods may update intermittently or require ground-truth bias labels; the cited continuous-time approach models bias but not uncertainty propagation.Source: B. Learning Inertial Bias and Noise… · pp. 2–3
C. Estimator Integration of Learned Inertial Models
- LBDU-VIO places learned bias dynamics and predicted gyroscope and accelerometer noise covariances inside the IMU-rate state update, replacing random-walk bias and fixed-noise models.The learned quantities shape the predicted mean and covariance before visual updates.Source: C. Estimator Integration of Learned… · p. 3
III. PROBLEM STATEMENT
- Because visual measurements are vulnerable to degradation, the problem statement motivates refining inertial propagation to improve both the propagated state and its uncertainty.Source: §III · p. 3
A. Preliminaries
- The UAV state includes orientation, velocity, position, and gyroscope and accelerometer biases, with high-rate IMU predictions and lower-rate visual corrections.The IMU measurement model distinguishes the two biases from zero-mean measurement noises with covariance matrices.Source: A. Preliminaries · p. 3
B. Problem Formulation
The formulation estimates the IMU state from inertial propagation and available visual observations, including periods without vision. It uses learned continuous-time bias dynamics and an IMU uncertainty model within MSCKF.
B. Problem Formulation
- The formulation treats gyroscope and accelerometer biases as deterministic state variables governed by learned continuous-time dynamics, rather than assigning one bias estimate per window.Source: B. Problem Formulation · p. 3
- The learned vector field describes bias evolution, while the IMU uncertainty model separately supplies noise covariance matrices for estimating the state through inertial propagation and visual updates.Source: B. Problem Formulation · p. 3
- The estimator combines learned IMU bias dynamics and an IMU uncertainty model within MSCKF for state and covariance propagation under unreliable visual observations.The pipeline corrects raw IMU measurements with bias estimates and uses predicted measurement-noise covariances in covariance propagation before visual updates.Source: B. Problem Formulation · p. 3, Figure 2 · p. 4
IV. METHODOLOGY
LBDU-VIO integrates learned bias dynamics and adaptive IMU noise covariances into MSCKF propagation, with visual measurements providing multi-state constraint updates. Its learned bias model represents evolution within each window rather than a single piecewise-constant estimate.
IV. METHODOLOGY
- LBDU-VIO propagates MSCKF state and covariance from raw IMU measurements using learned bias dynamics and adaptive noise covariances, then applies visual multi-state constraint updates.The visual updates follow propagation using the learned models.Source: §IV · p. 4
A. Learned Continuous Time Bias Dynamics
- A neural-network vector field models continuous bias evolution within each window, unlike a single estimate that makes bias piecewise constant.The bias serves as the neural ODE state, and the vector field also depends on external IMU input.Source: A. Learned Continuous Time Bias Dynamics · p. 4, Figure 3 · p. 4
- The learned bias trajectory is numerically integrated at the IMU rate, yielding an updated bias estimate at each IMU sample.Because IMU inputs are discrete, they are interpolated so the ODE solver can evaluate dynamics between samples.Source: A. Learned Continuous Time Bias Dynamics · p. 4
B. Learned IMU Uncertainty Model and Differentiable Covariance Propagation
The learned uncertainty model estimates gyroscope and accelerometer noise covariances from local IMU windows. These covariances are used to propagate preintegration error uncertainty alongside rotation, velocity, and position increments.
B. Learned IMU Uncertainty Model and Differentiable Covariance Propagation
- A shared feature from a local IMU window is decoded into gyroscope and accelerometer noise covariance estimates at each IMU sample.The temporal encoder uses a convolutional front end followed by two gated recurrent unit layers.Source: B. Learned IMU Uncertainty Model and… · pp. 4–5
- Separate decoders map the shared feature to three-axis sensor noise variances, which are made positive and assembled into diagonal covariance matrices.The uncertainty model is supervised through motion errors using differentiable preintegration and covariance propagation.Source: B. Learned IMU Uncertainty Model and… · p. 5
- Preintegration accumulates rotation, velocity, and position increments from the fixed window start, and propagates their associated error covariance at each IMU step.The increments are expressed in the body frame at the window start.Source: B. Learned IMU Uncertainty Model and… · p. 5
C. Hierarchical Training Strategy
The training strategy separates gyroscope bias, accelerometer bias, and uncertainty-model learning into three stages. Pose-derived motion residuals supervise the bias dynamics without ground-truth bias labels.
C. Hierarchical Training Strategy
- Gyroscope bias dynamics are trained first using orientation supervision and a curvature-regularized pose mean squared error objective.The regularizer penalizes differences between consecutive bias derivatives to discourage rapid variation.Source: 1) Stage 1: Gyroscope Bias Dynamics · p. 5
- With gyroscope dynamics frozen, accelerometer bias dynamics are trained using velocity and position supervision with a regularized mean squared error objective.The inertial dynamics and bias derivatives are integrated jointly, so motion losses supervise the result without ground-truth bias labels.Source: 2) Stage 2: Accelerometer Bias Dynamics · p. 5
- After both bias models are frozen, the uncertainty model is trained on supervised IMU segments using mean negative log likelihood.The loss weights residuals by inverse covariance and penalizes large predicted variances.Source: 3) Stage 3: IMU Uncertainty Model · p. 5
D. LBDU-VIO Formulation
LBDU-VIO replaces the MSCKF's bias random walks with learned bias dynamics and uses learned sensor noise covariances in covariance propagation. The filter retains visual MSCKF updates to correct its IMU state.
D. LBDU-VIO Formulation
- The learned bias dynamics replace the gyroscope and accelerometer bias random walks in the linearized error-state model.The filter propagates its bias estimate, then uses corrected IMU measurements to propagate attitude, velocity, and position.Source: D. LBDU-VIO Formulation · p. 6
- The uncertainty model supplies gyroscope and accelerometer measurement noise covariances for error-state covariance propagation, while bias process noise covariances remain fixed.The propagation advances covariance between consecutive IMU samples.Source: D. LBDU-VIO Formulation · p. 6
- The MSCKF visual update uses a landmark-eliminated visual residual to correct the IMU error state and initialize the next propagation step.The update also reduces covariance by the Kalman-gain and innovation-covariance term.Source: D. LBDU-VIO Formulation · p. 6
V. EXPERIMENTS
The experiments evaluate the proposed method on public visual-inertial benchmarks, including nominal conditions, visual outages, and visual distortion.
V. EXPERIMENTS
- The evaluation covers EuRoC and TUM-VI benchmarks, nominal visual updates, visual outages, visual distortion, and ablations of the learned models.The paper also includes a three-dimensional trajectory comparison on TUM-VI during visual-outage evaluation.Source: §V · p. 6
A. Experimental Setup
The setup uses EuRoC and TUM-VI train–test splits, specifies learned-model and estimator configurations, and evaluates global accuracy and local drift with absolute and relative errors.
1) Datasets:
- EuRoC uses six training and five test sequences; TUM-VI uses three training and three test indoor room sequences with full ground-truth coverage.EuRoC recordings use synchronized stereo 752 × 480 images at 20 Hz and IMU measurements at 200 Hz; the six TUM-VI indoor room sequences use a BMI160 IMU at 200 Hz.Source: 1) Datasets · p. 6
2) Implementation Details:
- The bias models use Euler integration at the 200 Hz IMU rate, while the estimator maintains up to 10 camera poses and updates visually at 20 Hz.The uncertainty model replaces fixed gyroscope and accelerometer measurement-noise covariances with sample-dependent predictions; bias-process covariances remain fixed at datasheet values.Source: 2) Implementation Details · p. 6
3) Evaluation Metrics:
- Global trajectory accuracy uses AOE and APE, while local drift over fixed-length path segments uses ROE and RPE.AOE and ROE are reported in degrees; APE and RPE are reported in meters.Source: 3) Evaluation Metrics · pp. 6–7
4) Comparison Methods:
The comparison includes four representative filter-based and learning-based baselines. Under nominal visual updates, LBDU-VIO leads overall on the reported EuRoC and TUM-VI metrics, with sequence-level exceptions.
4) Comparison Methods:
- The comparison covers S-MSCKF and MSCEqF filter-based methods alongside VIO-IPNet and Brossard et al.'s learning-based methods.The learning-based baselines are retrained on the corresponding EuRoC and TUM-VI training splits and coupled with unmodified OpenVINS; S-MSCKF uses stereo.Source: 4) Comparison Methods · p. 7
B. VIO Evaluation Under Nominal Conditions
- 0.133 m mean APE and 1.690° mean AOE: LBDU-VIO ranks first overall on EuRoC, reducing mean APE by 13.6% and mean AOE by 4.0% versus VIO-IPNet.It has the lowest APE on four of five sequences and the lowest AOE on three, but does not lead both metrics on every sequence.Source: B. VIO Evaluation Under Nominal… · p. 7
- Across the evaluated EuRoC path lengths, LBDU-VIO has the lowest mean RPE except on MH 02, where VIO-IPNet leads at the two longer lengths.Fig. 4 plots relative translation drift on the five EuRoC test sequences; MH 02's advantage shifts to VIO-IPNet as path length increases.Source: B. VIO Evaluation Under Nominal… · p. 7, Figure 4 · p. 8
- On TUM-VI, LBDU-VIO reduces mean APE by 24.1% and mean AOE by 23.2% versus VIO-IPNet, the closest monocular baseline with complete results.It ranks first in both metrics on room2, room4, and room6; compared with stereo S-MSCKF, it reduces mean APE and AOE by 52.8% and 57.1%.Source: B. VIO Evaluation Under Nominal… · p. 8
C. Robustness Under Visual Outage
LBDU-VIO is evaluated under visual outages and intermittent image distortion. It outperforms the strongest baseline on aggregate outage metrics across tested durations and has lower reported errors in the TUM-VI trajectory and visual-distortion evaluations.
1) Effect of Outage Duration on EuRoC:
- The EuRoC outage evaluation removes camera messages for 1, 2, 3, 5, or 10 s beginning 50 s into each of five sequences while retaining the IMU stream.The protocol evaluates IMU-only propagation during outages and system behavior after visual measurements resume.Source: 1) Effect of Outage Duration on EuRoC · p. 8
- Across five EuRoC sequences, LBDU-VIO has the lowest mean RPE and ROE at every tested outage duration, reducing RPE by 13.2%–26.7% and ROE by 4.4%–17.5% versus the best baseline.The table selects the best baseline separately for each metric and duration; its means use fixed 1 m path increments over the complete output trajectory.Source: 1) Effect of Outage Duration on EuRoC · p. 8, Table IV · p. 8
- At a 10 s EuRoC outage, mean RPE is 0.140 m, 25.1% below the best baseline’s 0.187 m; the ROE advantage narrows to 4.4%.At that duration, Proposed has mean ROE of 0.306° versus 0.320° for the best baseline; long IMU-only propagation remains challenging.Source: 1) Effect of Outage Duration on EuRoC · p. 9
2) Trajectory Comparison on TUM-VI:
- On TUM-VI room4 with a 10 s outage, LBDU-VIO’s mean APE is 0.1349 m, 29.8% below VIO-IPNet, and is lower than all three baselines in every run.Just before visual updates resume, its XY position error is 0.66 m versus 1.00 m for VIO-IPNet.Source: 2) Trajectory Comparison on TUM-VI · p. 9, Figure 6 · p. 9
D. Robustness Under Visual Distortion
- Under intermittent distortion on EuRoC V2 02 medium, LBDU-VIO has the lowest median AOE and APE RMSEs across three runs, reducing them by 22.5% and 33.3% versus the respective best baselines.Its median AOE RMSE is 1.378° versus 1.779° for Brossard; median APE RMSE is 0.101 m versus 0.152 m for VIO-IPNet.Source: D. Robustness Under Visual Distortion · pp. 9–10, Table V · p. 10
E. Evaluation and Ablation of the Learned Components
The learned bias and uncertainty modules are evaluated separately and together, alongside open-loop gyroscope integration. The conclusion reports accuracy gains on EuRoC and TUM-VI, especially during visual outages.
E. Evaluation and Ablation of the Learned Components
- In open-loop gyroscope integration, the learned bias model has a mean room4 orientation error of 2.3°, near 2° and below the Linear model’s approximately 5°; the same ordering holds on room2 and room6.The evaluation integrates from ground-truth initial attitude and compares Raw IMU, Linear, and learned-bias streams.Source: E. Evaluation and Ablation of the… · p. 10, Figure 8 · p. 11
- Adding learned bias with fixed noise reduces mean APE from 0.249 to 0.158 m and mean RPE from 0.103 to 0.069 m; adding learned uncertainty with learned bias further reduces them to 0.111 m and 0.045 m.The uncertainty model reduces both metrics on all five sequences, and the complete configuration has the lowest APE and RPE in every reported column.Source: E. Evaluation and Ablation of the… · p. 10, Table VI · p. 11
VI. CONCLUSION
- The authors report consistent accuracy gains over representative baselines on real-world EuRoC and TUM-VI benchmarks, especially during visual outages when estimation relies mainly on IMU measurements.They identify generalization across longer outages and less structured visual degradation as future work.Source: §VI · p. 11