PaperlayerView exact revision on arXiv

Source-linked AI summary

The TUM VI Benchmark for Evaluating Visual-Inertial Odometry

David Schubert, Thore Goll, Nikolaus Demmel, Vladyslav Usenko, Jörg Stückler, Daniel Cremers

arXiv:1804.06120v3cs.CVcs.RO

TL;DR

Visual-inertial odometry needs diverse, accurately synchronized datasets with ground truth for comparing competing methods. The paper introduces TUM VI, a calibrated stereo-and-IMU benchmark, and evaluates state-of-the-art algorithms, finding that long and visually challenging sequences still produce significant drift.

  • Problem

    High-quality visual-inertial datasets with accurate synchronization, ground truth, modern imaging, and varied scenes are limited.

  • Method

    The paper constructs TUM VI with calibrated high-resolution stereo images, synchronized IMU measurements, aligned motion-capture ground truth, and raw and calibrated data.

  • Results

    Even the best-performing evaluated algorithms show significant drift on long and visually challenging sequences.

  • Takeaways & Limitations

    The dataset is challenging enough to serve as a benchmark for further visual-inertial odometry research.

Abstract

from arXiv · show

Visual odometry and SLAM methods have a large variety of applications in domains such as augmented reality or robotics. Complementing vision sensors with inertial measurements tremendously improves tracking accuracy and robustness, and thus has spawned large interest in the development of visual-inertial (VI) odometry approaches. In this paper, we propose the TUM VI benchmark, a novel dataset with a diverse set of sequences in different scenes for evaluating VI odometry. It provides camera images with 1024x1024 resolution at 20 Hz, high dynamic range and photometric calibration. An IMU measures accelerations and angular velocities on 3 axes at 200 Hz, while the cameras and IMU sensors are time-synchronized in hardware. For trajectory evaluation, we also provide accurate pose ground truth from a motion capture system at high frequency (120 Hz) at the start and end of the sequences which we accurately aligned with the camera and IMU measurements. The full dataset with raw and calibrated data is publicly available. We also evaluate state-of-the-art VI odometry approaches on our dataset.

I. INTRODUCTION

The paper introduces TUM VI to address the limited availability of diverse, accurately synchronized visual-inertial datasets with ground truth. The benchmark combines calibrated stereo imagery, IMU measurements, and evaluations of VI algorithms.

  • The benchmark responds to shortcomings in existing datasets, including limited scene variability, outdated image specifications, and inaccurate or absent synchronization.
  • TUM VI provides diverse indoor and outdoor sequences for evaluating visual-inertial odometry.
  • Its IMU supplies 3-axis accelerometer and gyroscope measurements at 200 Hz, with hardware synchronization to the cameras.
  • Motion-capture ground truth is recorded at 120 Hz at sequence beginnings and endings and aligned with sensor measurements.

III. SENSOR SETUP

The sensor setup uses synchronized stereo cameras and an IMU designed to provide calibrated, photometrically consistent image measurements. Exposure control is coordinated across both cameras using an external illuminance measurement.

  • The setup contains two monochrome cameras, an IMU, infrared markers, and a luminance sensor integrated around a common sensor platform.
  • Each camera produces 1024x1024 monochrome images at 20 Hz using a global shutter and 16-bit intensity representation.
  • The fisheye lenses provide a 195° diagonal field of view, slightly reduced horizontally and vertically by the sensor size.
  • Both cameras use the same exposure time, yielding matched brightness and a shared exposure-time center for corresponding images.
  • A least-squares fit maps illuminance measurements to exposure times and sets both cameras’ exposure using the latest measurement.

C. IMU

The benchmark calibrates camera, IMU, and motion-capture relationships so raw measurements can be aligned and used with ground truth. Calibration includes timing, geometric transforms, sensor corrections, and bias handling.

  • Motion-capture coverage is limited to one room, so ground truth is available at sequence starts and ends rather than throughout longer trajectories.
  • The dataset supplies raw and calibrated sensor data because the raw measurements require calibration before use.
  • Camera–IMU and IMU–MoCap extrinsics, sensor time shifts, biases, axis scaling, and misalignment are estimated during calibration.
  • MoCap poses are transformed into calibrated IMU-frame ground truth using hand-eye-calibrated relative poses.
  • Each sequence’s MoCap–IMU offset is estimated by aligning angular velocities with a 100 µs grid search followed by parabola fitting.
  • Raw accelerations and angular velocities are corrected for axis and scale misalignment and initial biases.
  • Approximate bias precalibration is reasonable for this hardware because a large bias is coarsely reproducible across sensor restarts, although accurate sequence-level bias estimation remains required.

C. IMU Noise Parameters

The paper estimates IMU noise and bias parameters from Allan deviation under white-noise and random-walk assumptions. Different integration-time regimes identify the corresponding noise scales.

  • The IMU noise model assumes white measurement noise with standard deviation σw and slowly varying bias modeled as a random walk with standard deviation σb.
  • Allan deviation σAllan(τ) is analyzed as a function of integration time τ to estimate the IMU’s intrinsic noise parameters.
  • The white-noise parameter σw is obtained from the Allan-deviation relation at τ = 1 s.
  • The bias parameter is obtained from the corresponding relation at τ = 3 s.
  • Log-log Allan-deviation fits use slopes −1/2 and 1/2 in separate integration-time ranges to determine σw and σb, respectively.

D. Photometric Calibration

The dataset provides vignette calibration for direct methods, using an image formation model in which scene intensity is modulated by exposure time and per-pixel vignetting before response conversion.

  • Vignette calibration is provided to enable intensity matching for direct methods.The calibration uses code from the TUM MonoVO dataset.
  • I(x) = G(tV(x)B(x)) models image formation from scene intensity, vignetting, exposure time, and camera response.
  • With a linear response function, the model simplifies to I(x) ∝ tV(x)B(x).The resulting vignette calibration is a PNG containing values between 0 and 1 for each pixel.
  • The calibration procedure alternates optimization of wall texture and a non-parametric vignette function from images taken at varied viewpoints.

V. DATASET

The dataset combines diverse evaluation and calibration sequences with dedicated recordings for sensor noise, camera vignette, camera–IMU calibration, and motion-capture coverage.

  • Calibration sequences separately target camera intrinsics and stereo extrinsics, camera–IMU calibration, and vignette calibration.The camera–IMU calibration sequences include rapid motions.
  • Allan deviation analysis estimates white-noise and bias parameters from different integration-time regions.The white-noise fit uses slope −1/2, while the bias fit uses slope 1/2.
  • The calibration recordings include a small exposure to avoid motion blur and excitation of all 6 degrees of freedom in front of an AprilTag grid.
  • The imu-static sequence contains 111 hours of stationary IMU data for estimating noise and random-walk parameters.
  • Evaluation sequences include room, corridor, magistrale, outdoors, and slides scenes.Slides includes motion through a closed tube with no visual features.

B. Format

The benchmark distributes each sequence in raw and calibrated ROS bags, with calibrated files incorporating geometric, temporal, outlier, and IMU-data processing plus a reduced-resolution option.

  • 1) ROS Bag Files:: Each sequence is provided as one raw ROS bag and two calibrated ROS bags.All three contain the dataset’s core camera, IMU, and motion-capture topics.
  • 1) ROS Bag Files:: Raw bags preserve recorded data before hand-eye calibration, time-shift correction, or IMU calibration.
  • 1) ROS Bag Files:: The raw topics include both camera image streams, IMU measurements, and raw motion-capture transforms.
  • 1) ROS Bag Files:: The raw camera frame ID stores exposure time in nanoseconds.
  • 1) ROS Bag Files:: Calibrated bags align motion-capture poses with the IMU frame through hand-eye calibration.
  • 1) ROS Bag Files:: Calibrated bags remove outlier motion-capture poses with a median position filter and synchronize their timestamps to IMU time.
  • 1) ROS Bag Files:: IMU data in calibrated bags is processed according to the paper’s calibration equations.Two calibrated versions provide full resolution or quarter resolution, with the latter reducing each dimension by half.

2) Calibration Files:

The dataset supplies geometric calibration files and defines trajectory metrics for evaluating tracking accuracy over globally aligned trajectories and short-term pose changes.

  • 2) Calibration Files:: Geometric calibration files contain camera intrinsics for different models and relative poses between cameras and the IMU.They are obtained from processed calibration bags using the Kalibr toolbox.
  • The absolute trajectory error is the root mean squared difference between aligned tracked and ground-truth 3D positions.It uses an optimal SE(3) alignment.
  • Absolute trajectory error uses every tracked pose for which ground truth is available.Ground truth is generally available at sequence starts and ends, and throughout some sequences.
  • Relative pose error measures the accuracy of pose changes over a small time interval Δ.Its evaluation excludes Δ poses at the end of each tracked segment.
  • The relative pose error uses the 3D translational component of the relative pose.

B. Results

The benchmark evaluates several state-of-the-art visual-inertial odometry systems and exposes both their strengths and failure modes. Results indicate that the dataset remains challenging because long and visually difficult sequences produce substantial drift.

  • The evaluation covers ROVIO, OKVIS, VINS-Mono, and BASALT on the benchmark sequences.
  • Most algorithms track sequences containing intervals with no visual features, including the slides sequences, but some diverge during traversal.
  • The benchmark reports divergence when end-segment ATE exceeds 2 m, while values above 1000 m are displayed as “X”.ATE remains informative because divergence most often occurs near the end of a sequence.
  • OKVIS, VINS-Mono, and BASALT perform mostly well but struggle on some longer outdoor sequences, while ROVIO is more prone to drift and divergence.
  • VINS-Mono diverges on most outdoor sequences, typically after returning to the motion-capture room and switching from forward motion to fast rotations.
  • Median ATE values across all sequences are below 0.1 m for start segments and below 0.5 m for end segments.

APPENDIX

The appendix derives Allan-variance relationships for estimating IMU white noise from averaged measurements. It models measurement noise as independent Gaussian errors while accounting for accelerometer bias and gravity.

  • For measurements gi spaced by τ0, the appendix defines an average over n consecutive measurements beginning at gi.
  • The Allan variance is used to characterize successive averaged measurements, with M denoting the total number of measurements.
  • The measurement model assumes independent Gaussian noise ϵi with variance σ^2w,τ0.
  • For a resting IMU, measurements may include bias and gravity, so they are not necessarily centered at zero.This is especially relevant for accelerometer measurements.
  • Because averaged noise terms are zero-mean and uncorrelated, the cross term vanishes and the Allan deviation follows from the resulting variance expression.
  • Estimating σw uses σA(1 s), obtained by fitting σA(τ) in a log-log plot with a fixed slope of −1/2.The fit therefore determines only the straight-line offset.

3) Bias random walk:

The appendix derives the large-sample Allan deviation for IMU bias random walk. Its asymptotic log-log behavior has a positive slope of +1/2.

  • Continuing from the preceding Allan-variance expression, the appendix derives the bias-random-walk form for the Allan deviation.
  • For large n, retaining only the leading power yields the asymptotic Allan deviation.
  • The resulting Allan-deviation curve has slope +1/2 in a log-log plot.
Loading 1804.06120v3…