PaperlayerView exact revision on arXiv

Source-linked AI summary

RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methods

Hang Yan, Sachini Herath, Yasutaka Furukawa

arXiv:1905.12853v1cs.CVcs.RO

TL;DR

Inertial navigation lacks broad evidence for natural smartphone use because existing methods impose restrictive motion constraints. RoNIN addresses this with a large natural-motion benchmark and neural architectures using normalized coordinates and robust losses. The methods outperform competing approaches on RIDI and RoNIN, while poor device-orientation estimates remain a major limitation.

  • Problem

    Existing inertial-navigation algorithms require constraints such as foot attachment or rigid device placement that are incompatible with everyday smartphone use.

  • Method

    RoNIN provides a large natural-motion benchmark and neural models based on ResNet, LSTM, or TCN backbones with coordinate normalization and robust velocity losses.

  • Results

    RoNIN outperforms competing approaches on the RIDI and RoNIN datasets with significant margins, including natural complex motions that previous methods cannot handle.

  • Takeaways & Limitations

    The benchmark and methods establish a foundation for evaluating data-driven inertial navigation under natural smartphone handling and challenging motions.

  • Takeaways & Limitations

    Performance degrades significantly when device-orientation estimates are poor, particularly on the OXIOD dataset.

Abstract

from arXiv · show

This paper sets a new foundation for data-driven inertial navigation research, where the task is the estimation of positions and orientations of a moving subject from a sequence of IMU sensor measurements. More concretely, the paper presents 1) a new benchmark containing more than 40 hours of IMU sensor data from 100 human subjects with ground-truth 3D trajectories under natural human motions; 2) novel neural inertial navigation architectures, making significant improvements for challenging motion cases; and 3) qualitative and quantitative evaluations of the competing methods over three inertial navigation benchmarks. We will share the code and data to promote further research.

1. Introduction

Inertial navigation estimates a moving subject’s position and orientation from IMU measurements, but conventional methods rely on constraints that everyday smartphone use violates. RoNIN advances data-driven research with a large natural-motion benchmark, neural architectures, and evaluations across three benchmarks.

  • Motivation: Inertial navigation estimates positions and orientations using only a sequence of IMU sensor measurements.IMUs are energy-efficient, work in locations such as pockets, and are widely available in smartphones.
  • Motivation: Most existing algorithms assume foot attachment, rigid body attachment, or forward walking, conflicting with everyday smartphone usage.These assumptions support zero-speed updates or fixed motion directions in device coordinates.
  • Motivation: Data-driven approaches learn direct motion parameters from paired IMU data and ground-truth trajectories, relaxing these constraints.An example is supervised learning of velocity vectors from IMU history.
  • Contributions: More than 42.7 hours of IMU and ground-truth 3D motion data from 100 human subjects form the paper’s benchmark.The acquisition protocol allows natural smartphone handling during day-to-day activities.
  • Contributions: The paper introduces neural architectures for challenging motions and evaluates baselines and state-of-the-art methods qualitatively and quantitatively on three benchmarks.The authors state that code and data will be shared to promote further research.

2. Related Work

Prior inertial-navigation methods use physics-based integration, heuristic motion assumptions, or data-driven priors. Existing datasets differ in scale, placements, tracking systems, and subject counts.

  • Algorithm categories: Physics-based navigation subtracts gravity and double-integrates acceleration, but sensor biases quickly accumulate without additional constraints.Foot-mounted IMUs can use zero-speed updates when a foot touches the ground.
  • Algorithm categories: Heuristic methods exploit repetitive human motion through assumptions about rigid attachment, fixed direction, and step-proportional travel distance.They produce impressive results when these assumptions hold in controlled environments.
  • Algorithm categories: RIDI regresses device-frame velocity and expands operation to complex motions, while IONet regresses velocity magnitude and heading-change rate without external orientation information.Both methods represent data-driven priors in different ways.
  • Datasets: RIDI used 10 subjects and 2.5 hours across four phone placements, whereas OXIOD used five subjects and 14.7 hours with Vicon tracking across four placements.The datasets include hand, bag, pocket, leg, body, or trolley configurations.

3. The RoNIN dataset

The RoNIN dataset targets scale, diversity, and fidelity through extensive natural-motion recordings. Its two-device protocol separates freely handled IMU measurements from body-mounted trajectory ground truth and supports body-heading estimation.

  • Scale and diversity: RoNIN contains 42.7 hours across 276 sequences in three buildings, 2.9 times OXIOD’s duration.The dataset also includes 100 human subjects, 20 times OXIOD’s subject count, using three Android devices.
  • Fidelity: Subjects naturally handle devices in bags, deep pockets, or by hand while walking, sitting, or wandering.This fidelity is intended to reflect day-to-day activities rather than constrained placements.
  • Acquisition protocol: A harness-mounted 3D-tracking phone supplies body trajectories while subjects freely handle a separate phone collecting IMU data.The protocol changes the target from the IMU phone’s trajectory to the body’s trajectory.
  • Acquisition protocol: The dataset introduces body-heading estimation because body and device orientations can differ arbitrarily depending on phone carrying.Device orientation is easier to estimate with standard sensor fusion than body heading.
  • Data quality: The processing pipeline records IMU and 3D pose data at 200Hz, with ground-truth trajectory drift below 0.3m after 10 minutes.System orientation drift is reduced below 10 degrees and used as training ground truth.
  • Evaluation split: The dataset reserves 15 subjects as a separate group to test generalization to unseen human subjects.The other 85 subjects are divided into training, validation, and testing subsets.

4. Robust Neural Inertial Navigation (RoNIN)

RoNIN regresses motion from IMU histories using heading-agnostic coordinates, multiple neural backbones, and robust velocity losses. A related LSTM-based model estimates body heading when stationary periods make recent IMU data ambiguous.

  • RoNIN design: RoNIN combines coordinate-frame normalization with robust velocity losses to regress motion from noisy IMU histories.The architecture uses ResNet, LSTM, or TCN backbones.
  • Coordinate frame normalization: Heading-agnostic coordinates align the Z axis with gravity, avoiding the singularities and discontinuities associated with stabilized IMU coordinates.The same coordinate frame represents input IMU data and output velocity data.
  • Backbone architectures: RoNIN provides ResNet, LSTM, and TCN variants with architectures suited to framewise or sequence-based velocity prediction.The ResNet uses 200-frame IMU windows, while TCN uses dilated causal convolutions with a 253-frame receptive field.
  • Robust velocity loss: Because differentiating high-rate poses produces noisy velocities, RoNIN introduces losses designed to improve motion-learning signal-to-noise ratio.The losses operate on integrated or strided motion targets rather than relying only on instantaneous velocity labels.
  • Robust velocity loss: The strided velocity loss trains RoNIN ResNet to predict the positional difference over 200 frames, equivalent to one second.This replaces instantaneous velocity prediction with a one-second positional-difference target.
  • Body heading: For body heading, the LSTM predicts sine and cosine values over long unrolled sequences because stationary periods provide little current heading information.The model is unrolled for 1,000 steps and updates only when the initial velocity exceeds 0.1 m/s.

5. Evaluations: Preliminaries

The evaluation compares proposed and competing inertial-navigation methods across three datasets, using dataset-specific training, orientation, and trajectory-alignment protocols. Experiments are implemented in PyTorch with distinct sampling and optimization settings for the RoNIN variants.

  • The proposed architectures are implemented in PyTorch, with ResNet, LSTM, and TCN training samples drawn using variant-specific sequence lengths and randomized offsets.
  • Optimization uses ADAM with variant-specific batch sizes, learning rates, learning-rate reductions, convergence durations, and dropout keep probabilities.
  • Evaluations compare four competing methods across the RIDI, OXIOD, and RoNIN datasets.
  • NDI converts gravity-subtracted accelerations to the global frame using device orientations and integrates them twice to estimate positions.
  • PDR detects footsteps and advances position along device heading by a predefined 0.67m distance per step.
  • RIDI uses separate models by phone placement for RIDI and OXIOD, but a unified model trained on 10% of RoNIN because its Support Vector Regression does not scale to larger data.
  • PDR, RIDI, and RoNIN use IMU-estimated device orientations for testing, while OXIOD training uses Vicon ground-truth orientations because estimates are severely corrupted.
  • RoNIN estimates global-frame trajectories directly, whereas IONet receives five-second ICP alignment and other methods use five-second trajectory alignment before evaluation.

6. Evaluations

The evaluations compare position and body-heading estimation across datasets, visualize trajectories, and test architectural components. RoNIN performs strongly on challenging natural motions, while results expose sensitivity to device-orientation quality and training coverage.

  • Evaluation setup: The study evaluates position estimation with ATE and RTE across five methods and three datasets, plus body-heading estimation on RoNIN.ATE measures whole-trajectory RMSE; RTE averages RMSE over fixed one-minute intervals, with proportional scaling for shorter sequences.
  • Position evaluation: RoNIN outperforms competing approaches on RIDI and RoNIN, and previous methods fail on the natural complex motions in RoNIN.IONet accumulates heading-estimation errors during integration, while RoNIN LSTM and TCN are slightly better than ResNet but require 3 to 4 times more training time.
  • Position evaluation: RIDI and RoNIN struggle on OXIOD because they rely on inaccurate device-orientation estimates; better bias calibration and compass-free APIs are expected to help.Selected visualizations likewise attribute large errors on some OXIOD sequences to errors in the provided device orientations.
  • Ablation study: Coordinate frame normalization and robust velocity losses improve ATE and RTE overall, with the lowest errors generally occurring when both features are enabled.Coordinate frame normalization appears to have the larger impact; one TCN ATE case is a small exception.

7. Discussions

The paper establishes a foundation for data-driven inertial navigation through a diverse benchmark, new neural architectures, and evaluations across three datasets, while identifying device orientation dependence as its major limitation.

  • The paper contributes a large, diverse benchmark of IMU-motion data collected during real day-to-day activities.Its stated foundation combines this benchmark with new neural architectures and evaluations of competing methods across three inertial navigation datasets.
  • Novel neural inertial navigation architectures make significant improvements on challenging motion cases.
  • The approach relies on device orientation estimates, and performance degrades significantly when those orientations are poor.The paper identifies this dependence as the major limitation and a focus of future work.

Supplementary Material

The supplementary material documents the data acquisition protocol and adds qualitative and quantitative evaluations, with accompanying video illustrating complex phone-handling motions.

  • The supplementary document details device preparation, data collection, and ground-truth preparation.
  • It provides additional qualitative and quantitative evaluations in Figures 2, 3, and 4.
  • A supplementary video demonstrates a complex motion trajectory with varied phone-handling motions.

1. Device preparation

The device-preparation pipeline uses paired Android phones, calibrated raw IMU data, and repeated sensor calibration, while disabling magnetometers because indoor disturbances make them unreliable.

  • An in-house Android app connects the tracking and IMU phones through Bluetooth for coarse synchronization and centralized control.
  • The method uses calibrated raw IMU data because system-provided linear accelerations and gyroscopes differ across manufacturers.
  • Accelerometer calibration estimates biases and scale sensitivities against gravity, with each smartphone recalibrated once per day.
  • Gyroscope biases are estimated from ten seconds of static readings before and after each sequence.
  • Magnetometers are calibrated before each sequence but disabled for orientation estimation because indoor magnetic disturbance remains highly unreliable.Using noisy magnetometer data to reduce long-term drift is left as future work.

2. Data collection

Data collection aligns two phones in time and space, using a tracking phone for ground truth and an IMU phone handled freely during 4–10-minute sequences.

  • 2. Data collection: Each sequence lasts 4–10 minutes, with unconstrained IMU-phone handling; testers only must avoid blocking the tracking phone’s camera.
  • 2. Data collection: Bluetooth provides coarse clock synchronization, followed by signal correlation during a rigid horizontal rotation to estimate the precise time difference.
  • 2. Data collection: The system uses local frames for both phones and separate global frames, then estimates the fixed rotation between global frames so IMU data and ground truth share coordinates.
  • 2. Data collection: The global-frame rotation is decomposed using device rotations from Tango and Android’s game rotation vector, with phone alignment obtained by attaching screens together.
  • 2. Data collection: With flat screens perpendicular to device Z axes, the inter-device rotation reduces to a 2D local-Z rotation whose angle is solved from the gravity vectors.The gravity vectors are expressed in the two local frames.
  • 2. Data collection: The initial inter-device rotation estimate is computed for each frame while the phones are held steady and then averaged.

3. Ground-truth preparation

Ground-truth preparation combines site-specific visual safeguards, sensor-fusion orientation estimates, and filtering to reduce trajectory and orientation drift. The resulting orientation estimates remain difficult to observe when phones are pocketed or bagged.

  • Visual tracking is protected against texture-poor locations by surveying sites and adding checkerboard patterns.
  • Area Learning reduces long-term trajectory drift through pre-scanned environments and loop-closure relocalization, but relocalization can introduce discontinuities or fail outside surveyed areas.
  • A kinematic filter combines raw accelerations, angular rates, and ground-truth positions to estimate optimal 6DoF states and reduce rotational drift.
  • Android sensor fusion typically drifts 10 to 20 degrees over 10 minutes, while filtering reduces drift to 5 to 10 degrees.
  • Phone orientations are unobservable in pockets or bags, motivating endpoint rotational-drift evaluation under an assumption of stable Tango orientations.
Loading 1905.12853v1…