PaperlayerView exact revision on arXiv

Source-linked AI summary

TLIO: Tight Learned Inertial Odometry

Wenxin Liu, David Caruso, Eddy Ilg, Jing Dong, Anastasios I. Mourikis, Kostas Daniilidis, Vijay Kumar, Jakob Engel

arXiv:2007.01867v3cs.ROcs.CVcs.LGeess.SP

TL;DR

IMU-only pedestrian navigation suffers drift from sensor bias and noise, while camera-based systems face tracking, power, and privacy constraints. TLIO predicts 3D displacement and uncertainty from IMU segments and tightly fuses them with an EKF. The resulting system outperforms velocity-integration methods on position estimates and an AHRS filter on orientation estimates, within the scope of its training data.

  • Problem

    IMU integration drifts because sensor bias and noise accumulate, motivating IMU-only estimation that avoids visual-inertial tracking, power, and privacy constraints.

  • Method

    TLIO uses a neural network to regress 3D displacement and uncertainty, then tightly fuses these outputs with raw-IMU EKF propagation for full-state estimation.

  • Results

    TLIO outperforms velocity-integration approaches on position estimates and a model-based AHRS attitude filter on orientation estimates.

  • Takeaways & Limitations

    A learned pedestrian-motion prior combined with an EKF enables low-drift pose estimation and calibration using an IMU alone.

  • Takeaways & Limitations

    The system is limited by the scope of its training data, and unusual motions cause system failure.

Abstract

from arXiv · show

In this work we propose a tightly-coupled Extended Kalman Filter framework for IMU-only state estimation. Strap-down IMU measurements provide relative state estimates based on IMU kinematic motion model. However the integration of measurements is sensitive to sensor bias and noise, causing significant drift within seconds. Recent research by Yan et al. (RoNIN) and Chen et al. (IONet) showed the capability of using trained neural networks to obtain accurate 2D displacement estimates from segments of IMU data and obtained good position estimates from concatenating them. This paper demonstrates a network that regresses 3D displacement estimates and its uncertainty, giving us the ability to tightly fuse the relative state measurement into a stochastic cloning EKF to solve for pose, velocity and sensor biases. We show that our network, trained with pedestrian data from a headset, can produce statistically consistent measurement and uncertainty to be used as the update step in the filter, and the tightly-coupled system outperforms velocity integration approaches in position estimates, and AHRS attitude filter in orientation estimates.

I. INTRODUCTION

The paper targets IMU-only pedestrian state estimation as an alternative to visual-inertial odometry, whose image tracking, power, and privacy constraints limit some applications. It combines learned 3D displacement uncertainty with tightly coupled filtering to estimate full state from pedestrian IMU data.

  • Visual-inertial odometry can fail under extreme lighting or camera blockage, while frequent image processing creates power costs and camera use raises privacy concerns.
  • Strapdown pedestrian navigation accumulates sensor errors because the IMU kinematic model provides only relative state estimates.
  • Prior neural approaches demonstrated accurate 2D trajectory reconstruction by estimating average velocity or displacement from pedestrian IMU segments.
  • TLIO regresses 3D displacement and covariance, then tightly couples these outputs with an EKF estimating position, orientation, velocity, and IMU biases.
  • 27% lower average yaw drift and 33% lower average position drift were achieved than the best RoNIN velocity-concatenation baseline on the test dataset.
  • Unlike traditional PDR, the approach avoids step-counting, stride, or gait detection, and unlike common deep-learning methods, it operates in 3D without an external ZUPT-based orientation estimator.

III. SYSTEM DESIGN

TLIO uses a convolutional network to predict short-timescale 3D displacement and uncertainty from gravity-aligned IMU segments, while an EKF propagates raw IMU dynamics and fuses those predictions as updates. The design explicitly addresses uncertainty modeling and the dependence created by using IMU data in both components.

  • Network and filter: A convolutional network predicts 3D relative displacement and uncertainty between two time instants from the intervening IMU segment, without initial velocity.
  • Network and filter: The EKF estimates 3D position, velocity, orientation, IMU biases, and a sparse set of past poses.
  • Network and filter: Raw IMU samples drive EKF propagation, while network displacement outputs provide measurement updates in a local gravity-aligned frame.
  • Robustness and uncertainty: Using IMU data directly and through the network violates EKF independence assumptions, so training adds bias and gravity-direction perturbations to reduce error propagation.
  • Robustness and uncertainty: The network outputs displacement and uncertainty vectors from N × 6 gravity-aligned IMU inputs, with uncertainty parameterizing diagonal covariance entries.
  • Robustness and uncertainty: Training uses both Mean Square Error and Gaussian Maximum Likelihood losses, with the latter modeling displacement under the regressed Gaussian distribution.
  • Robustness and uncertainty: The diagonal covariance assumption decouples axes and constrains uncertainty-ellipse principal axes to the gravity-aligned frame.

B. Data Collection and Implementation Details

The system is trained and evaluated using a multi-device headset dataset covering varied pedestrian activities and motion patterns. Inputs use overlapping IMU windows transformed to a gravity-aligned frame, with augmentation for yaw and sensor-state perturbations.

  • Data collection: The dataset contains more than 400 sequences totaling 60 hours of pedestrian data from multiple devices and more than five people.
  • Data collection: Recorded activities include walking, standing still, kitchen organization, playing pool, and ascending or descending stairs.
  • Input preparation: The final system uses overlapping windows of N = 200 IMU samples at 200 Hz, rotated into a gravity-aligned frame based on the window’s initial orientation.
  • Input preparation: Training applies random horizontal rotations for yaw invariance and perturbs sensor bias and gravity direction to reduce sensitivity to filter-input errors.
  • Optimization: Optimization pretrains with LMSE for 10 epochs before switching to LML, requiring approximately four hours on an NVIDIA DGX computer.

V. STOCHASTIC CLONING EXTENDED KALMAN FILTER

The filter uses a tightly coupled stochastic-cloning EKF that combines network-derived pairwise displacement constraints with IMU-based state propagation. Its sliding-window state contains cloned past poses and a current state represented with manifold-aware error variables.

  • Filter structure: The EKF integrates network-predicted displacements with a statistical IMU model using stochastic cloning.Displacement estimates constrain pairs of past states, so the filter maintains a sliding window of poses.
  • State definition: The full filter state concatenates m cloned past states, ξ1 through ξm, with the current state s.Each ξi is a past cloned state, while s is the current state.
  • State definition: Each past cloned state stores rotation and position, whereas the current state additionally contains velocity and gyroscope and accelerometer biases.The state components are expressed in the world frame for velocity and position, with IMU-frame-to-world rotation.
  • Error representation: The filter uses error-based linearization on the manifold of a minimal rotation parameterization.Rotation error uses the SO(3) logarithm map, and the complete error-state has dimension 6m + 15.

B. IMU model

The IMU model propagates state using noisy, biased acceleration and angular-velocity measurements under a simplified inertial kinematic model. Gyroscope and accelerometer biases evolve as random walks, while covariance propagation accounts for random inputs and sensor noise.

  • Sensor model: The IMU measurements are modeled as true angular velocity and acceleration corrupted by additive bias and Gaussian noise.The measurement equations are ω = ωtrue + bg + ng and a = atrue + ba + na.
  • Bias model: Gyroscope and accelerometer biases follow random-walk processes over the IMU sampling interval.Their discrete evolution uses ηgd and ηad.
  • Propagation model: The filter propagates the current state with IMU data through a kinematic motion model.When an update is due, propagation and stochastic cloning occur together in one step.
  • Propagation model: The propagation model assumes uniform gravity and ignores Coriolis forces and Earth curvature.
  • Covariance propagation: Linearized error propagation uses a random-input vector containing sensor noise and bias random-walk noise.The propagated state covariance incorporates the corresponding noise covariance matrices.
  • Implementation: Multiple propagation steps can be combined to reduce computational cost.

2) State Augmentation:

State augmentation increases the EKF state by cloning the current pose at measurement-update frequency. The augmented state is propagated jointly, and older cloned states are later removed by marginalization.

  • State Augmentation: State augmentation is performed at the measurement-update frequency through propagation with cloning.
  • State Augmentation: The augmentation transition combines a copy operation with propagation of the augmented and current states.
  • State Augmentation: Each augmentation increases the state-vector dimension by 6.The partial propagation matrices cover rotation and position only.
  • State Augmentation: Old past states are pruned during the marginalization step.

D. Measurement Update

The measurement update fuses network-predicted 3D displacements expressed in a local gravity-aligned frame with the cloned-state EKF. The design avoids injecting yaw information and rejects unreliable or geometrically singular updates.

  • Measurement design: The measurement function uses 3D displacement in a local gravity-aligned frame rather than the world frame to preserve yaw unobservability.A world-frame displacement measurement would imply heading observability at the filter level.
  • Measurement design: The network supplies displacement between states i and j together with Gaussian uncertainty covariance.Input IMU samples are rotated to the gravity-aligned frame anchored at pose i.
  • Linearization: The linearized measurement matrix has nonzero 3 × 3 blocks only for the rotation error at i and the position errors at i and j.
  • Geometric caveat: Updates are discarded when the headset wearer looks straight up or down, where cos β = 0 causes a singularity.These cases are reported as unlikely and absent from the dataset.
  • Kalman update: The Kalman gain updates both the state and covariance using the measurement Jacobian and network covariance.The rotation update uses the ⊕ operator's exponential-map rule.
  • Outlier rejection: 11.345 is the χ2 rejection threshold corresponding to the 99th percentile with 3 degrees of freedom.Updates are discarded when the normalized innovation error exceeds this threshold.
  • Covariance handling: The network covariance is scaled by 10 to compensate for temporal correlation between measurements.

E. State size, Marginalization and Initialization

The filter marginalizes states before each update, with retained state size determined by the network displacement window and update frequency. Evaluation defines translation and yaw metrics against visual-inertial ground truth.

  • E. State size, Marginalization and Initialization: Past states before each update are marginalized from the filter state and covariance.The retained state count depends on the network displacement duration window and update frequency.
  • E. State size, Marginalization and Initialization: At 20 Hz with a 1 s displacement window, the filter retains at most 21 past states.
  • E. State size, Marginalization and Initialization: The evaluation uses 37 trajectories lasting 3–7 minutes, comparing TLIO and 3D-RONIN against a state-of-the-art VIO estimate treated as ground truth.
  • E. State size, Marginalization and Initialization: Absolute Translation Error measures whole-trajectory position RMSE relative to ground truth.
  • E. State size, Marginalization and Initialization: Relative Translation Error measures local position closeness over a 1 s window after removing initial-window yaw drift.
  • E. State size, Marginalization and Initialization: Translation drift is normalized by traveled distance, while yaw evaluation uses Absolute Yaw Error and Yaw-DR in degrees per hour.

B. System Performance

Across the test set, TLIO consistently outperforms the decoupled 3D-RONIN estimator on all reported metrics. Its advantages include better 3D position and lower yaw drift.

  • B. System Performance: TLIO consistently performs better than 3D-RONIN on all metrics across the entire test set.
  • B. System Performance: TLIO achieves better 3D position estimates than 3D-RONIN’s average-velocity integration approach.The reported result attributes robustness to the network’s covariance output and χ2-test outlier rejection.
  • B. System Performance: TLIO has smaller yaw drift than 3D-RONIN without hand-engineered heuristics or step detection.The result indicates that the EKF estimates gyroscope biases alongside position and velocity.

VII. COMPONENTS AND VARIATION STUDIES

The study examines causal network window choices, covariance consistency, and robustness to perturbed IMU inputs. Longer inputs reduce sample MSE but increase error correlation, while perturbation training improves robustness.

  • VII. COMPONENTS AND VARIATION STUDIES: The causal network variations change IMU frequency, displacement interval, and total input duration without using future data.The evaluated reference model is 200hz-1s-1s.
  • 1) Does using more data help?:: Lower MSE does not imply lower trajectory ATE because longer overlapping inputs can produce more correlated displacement errors.The authors observe similar trajectory ATEs across different window variants and use 200hz-1s-1s thereafter.
  • 1) Does using more data help?:: Models using more input data have lower MSE but greater temporal correlation of displacement errors.
  • 2) Consistency of learned covariance:: Over 99% of error points fall within the network’s 3σ region, with 0.70% outside bounds on x and y and 0.47% on z.Across approximately 60,000 test-set samples, the output uncertainty grows with error and is often slightly conservative.
  • 3) Sensitivity Analysis:: Training with bias and gravity perturbations improves robustness to accelerometer bias and gravity-direction errors.The selected system model uses 200hz-1s-1s with both perturbations.

B. EKF System

TLIO combines a learned 3D displacement-and-uncertainty model with a tightly coupled EKF to estimate pedestrian inertial state. Learned covariance and frequent updates improve position-related metrics over direct displacement concatenation and 3D-RONIN, while the system also supports full-state estimation.

  • Uncertainty ablation: 27% and 31% improvements over 3D-RONIN-mse are reported for average yaw and position drift, respectively.The comparison concerns the average yaw and position drift metrics.
  • Uncertainty ablation: Regressed covariance significantly improves ATE and drift over fixed-covariance variants and makes the filter more robust to initialization failure.A fixed-covariance system loses track on one dataset, whereas adaptive uncertainty improves robustness for different inputs.
  • Update frequency: High-frequency measurements reduce ATE and drift, and TLIO outperforms 3D-RONIN across the tested system configurations.The filtering approach remains superior across the experimented network models and update frequencies.
  • EKF formulation: TLIO fuses learned 3D displacement and regressed uncertainty in a tightly coupled EKF that estimates the full state.The proposed system jointly estimates position, orientation, velocity, and IMU biases from pedestrian IMU data.
  • Update frequency: High-frequency updates produce the best ATE and drift despite temporal correlation, while increasing update frequency also increases yaw jitter.The yaw-jitter trend is reported using the RYE-1s metric.
Loading 2007.01867v3…