PaperlayerView exact revision on arXiv

Source-linked AI summary

CoWear: Towards Collaborative Wearable Inertial Localization with Multiple Smart Devices

Zhuoyuan She, Zhanpeng Zhang, Yapeng Chen, Jiale Wang, Xin Xie, Ming Xia

arXiv:2610.20596v1cs.RO

TL;DR

Existing inertial-localization benchmarks rarely provide synchronized heterogeneous wearables with device-specific motion ground truth, despite consumers carrying devices at different body locations. CoWear pairs each wearable with its own motion-capture trajectory and formulates collaboration around a common target. Experiments report consistent localization gains from common-target multi-device fusion across target roles.

  • Problem

    Existing inertial-localization benchmarks rarely provide synchronized heterogeneous wearables with device-specific motion ground truth, despite devices observing motion from different body locations.

    Source: §I · p. 1

  • Method

    CoWear records smartphone, smartwatch, and smart-glasses IMUs with device-specific motion-capture trajectories and predicts a selected common target’s displacement from synchronized observations.

    Source: §I · pp. 1–2, A. Data Collection · p. 3

  • Results

    Common-target multi-device fusion consistently improves localization across different target roles, with uncertainty-aware fusion and more diverse device placements further improving collaborative performance.

    Source: §VI · p. 8, D. Discussion · p. 8

  • Takeaways & Limitations

    CoWear establishes a benchmark for studying collaboration among spatially distinct consumer wearables and provides a foundation for broader collaborative wearable sensing.

    Source: §VI · p. 8

    Direct late fusion is not uniformly reliable: PDR and RoNIN do not outperform their strongest individual-device estimates.

    Source: B. Benchmark Results · p. 6

Abstract

from arXiv · show

Consumer smart devices observe the same human motion from different body locations, yet existing inertial-localization benchmarks rarely provide synchronized heterogeneous wearables with device-specific motion ground truth. We present CoWear, a dataset and reference framework for collaborative inertial localization using a smartphone, smartwatch, and smart glasses. The dataset contains over 30 participants, over 300 walking sessions, and more than 30 hours and 100 km of aggregate device trajectories, with each device individually tracked by motion capture. To enable collaboration across distinct body motions, we train role-specific estimators to predict the displacement of a common reference device over synchronized temporal windows. Their predictions are transformed to the world frame using gyroscope integration and fused according to learned uncertainty. Experiments using phone, glasses, and watch as reference targets show that multi-device fusion consistently improves both displacement and trajectory rollout accuracy over individual-device estimation. CoWear provides a dataset and reproducible baseline for studying collaborative inertial localization across heterogeneous consumer wearables.

I. INTRODUCTION

CoWear addresses the mismatch between motion at a wearable’s body location and the locomotion a localization system aims to recover. It pairs synchronized heterogeneous wearables with device-specific motion-capture ground truth and a common-reference collaborative localization baseline.

I. INTRODUCTION

  • Different body-worn devices share global locomotion but also undergo distinct local motions, so device motion may diverge from the trajectory localization aims to recover.A representative RIDI example shows substantial device motion alongside much smaller changes in the human trajectory.Source: §I · p. 1, Figure 1 · p. 1
  • CoWear records smartphone, smartwatch, and smart-glasses inertial data with independently tracked, device-specific 6-DoF motion-capture ground truth.This lets each wearable be studied as either a motion source or a common localization reference.Source: §I · p. 1
  • For each synchronized window, role-specific estimators predict one reference device’s displacement; gyroscope integration transforms predictions to the world frame before learned-uncertainty fusion.The common-target formulation avoids averaging trajectories belonging to different physical carriers.Source: §I · pp. 1–2

II. RELATED WORK

Prior work advances single-device inertial localization, provides varied supervision, and studies body-distributed sensing, but does not primarily address collaborative localization across heterogeneous consumer wearables. CoWear pairs each device’s inertial stream with its own motion-capture trajectory and supplies synchronized multi-device walking data.

II. RELATED WORK

  • Existing inertial-localization methods generally treat each sensing stream independently, while body-distributed sensing works pursue goals other than collaborative localization across heterogeneous consumer devices.Prior datasets vary in sensing and supervision; several pair measurements and trajectory references from different physical carriers, leaving device-to-reference motion implicit.Source: §II · p. 2

III. COWEAR DATASET

  • CoWear preserves each wearable’s distinct motion by pairing its IMU stream with the motion of that same physical device.Its construction includes synchronized sensing, device-specific motion-capture tracking, and temporal alignment and curation.Source: §III · pp. 2–3

A. Data Collection

  • CoWear uses smartphone, smartwatch, and smart-glasses measurements with independent device trajectories, and includes over 30 participants and over 300 walking sessions.The reported scale exceeds 30 hours and 100 km of aggregate device trajectories, not participant walking time or distance.Source: A. Data Collection · p. 3

B. Data Curation and Structuring

CoWear aligns independent device and motion-capture clocks by matching rotational motion, then puts all streams on a common temporal reference and curates synchronized session records.

B. Data Curation and Structuring

  • Temporal offsets are estimated by maximizing correlation between onboard and motion-capture-derived angular-velocity magnitudes.Using magnitudes makes alignment insensitive to differences between the IMU and motion-capture coordinate frames.Source: B. Data Curation and Structuring · p. 3
  • The smartphone first aligns the recording and motion-capture timelines; the same procedure then compensates for residual glasses and smartwatch offsets.This places all IMU measurements and device trajectories on a common temporal reference.Source: B. Data Curation and Structuring · p. 3
  • After alignment, streams are resampled to 100 Hz, sessions with motion-capture gaps longer than 2 s are discarded, and shorter position and orientation gaps are interpolated.Position gaps use linear interpolation and orientation gaps use SLERP; session records retain each device’s individual trajectory.Source: B. Data Curation and Structuring · p. 3

IV. BENCHMARK TASKS AND COLLABORATIVE LOCALIZATION FRAMEWORK

CoWear defines single-device and collaborative localization tasks using synchronized wearable observations and device-specific motion-capture supervision. In the collaborative task, observations from different wearables estimate one selected physical target so their displacements can be compared and fused.

A. Benchmark Task Formulation

  • For each temporal window, observations retain their sensing device’s coordinates and motion characteristics while sharing supervision from the same physical target.The formulation distinguishes the observing device r, target q, and window k.Source: A. Benchmark Task Formulation · pp. 3–4
  • Single-device localization predicts the observing device’s own displacement, while collaborative localization uses multiple wearables to estimate one selected physical target.Each wearable has its own motion-capture trajectory for device-aligned supervision; directly averaging trajectories of different physical carriers is not physically well-defined.Source: A. Benchmark Task Formulation · pp. 3–4
  • CoWear compares representative inertial-motion datasets across sensing setup, scale, supervision, and joint-device support.The table caption identifies the comparison’s scope; the supplied passages do not report its cell values.Source: Table I · p. 4

B. Collaborative Localization Framework

The collaborative framework processes each wearable independently to estimate the displacement of a shared target. It transforms predictions into the world frame and estimates their reliability for later fusion.

B. Collaborative Localization Framework

  • Each wearable’s native-frame inertial sequence is encoded to predict the common target’s displacement in the observing device’s local frame.The motion representation uses a 128-D feature space followed by a two-layer unidirectional LSTM.Source: B. Collaborative Localization Framework · p. 4
  • Gyroscope-propagated device orientation transforms each predicted local displacement into a world-frame prediction.Rollouts start from an initial orientation, and the IMU sampling interval and exponential map are used in orientation propagation.Source: B. Collaborative Localization Framework · p. 4
  • An uncertainty head estimates reliability per world-frame direction using the motion representation and current device orientation.The training objective uses local displacement supervision and a heteroscedastic Gaussian objective in the world frame; uncertainty is learned from localization residuals rather than manually specified fusion weights.Source: B. Collaborative Localization Framework · pp. 4–5

C. Uncertainty-Aware Multi-Device Fusion

The fusion stage combines estimates of the same target displacement in the same world frame, then accumulates fused displacements into a trajectory. Its rule assumes conditionally independent Gaussian estimates.

C. Uncertainty-Aware Multi-Device Fusion

  • Fusion combines predictions of the same physical displacement over the same window and in the same world frame.Each prediction is modeled as a Gaussian estimate.Source: C. Uncertainty-Aware Multi-Device Fusion · p. 5
  • Under a conditional-independence assumption, device predictions are fused in information form, with diagonal covariance allowing different contributions along each world axis.Unavailable wearables are excluded from the set of devices used for that window.Source: C. Uncertainty-Aware Multi-Device Fusion · p. 5
  • The fused displacement is accumulated into the target trajectory using only its initial position and each observing device’s initial orientation.No intermediate ground-truth position or orientation corrects the rollout.Source: C. Uncertainty-Aware Multi-Device Fusion · p. 5

A. Experimental Setup

The experiments use session-level splits and compare single-device localization with benchmark late fusion and common-target models. Trajectories are evaluated horizontally, with ATE reported across test sessions and common-target displacement RMSE reported at the event level.

A. Experimental Setup

  • The 363 complete synchronized sessions are split into 217 training, 31 validation, and 115 test sessions, with no session windows crossing splits.Hyperparameters and checkpoints are selected using only training and validation data.Source: A. Experimental Setup · p. 5
  • Trajectory rollouts use calibrated initial orientation and gyroscope integration without intermediate ground-truth position or orientation correction.Motion events are segmented by peak–trough detection on combined acceleration magnitude, with shared boundaries across wearable streams.Source: A. Experimental Setup · p. 5
  • Single-device comparisons include Weinberg PDR, RIDI, RoNIN, TLIO, and CoWear LSTM, with learning baselines retrained on CoWear using the same split.CoWear LSTM is presented as a lightweight reference baseline, not a new single-device state-of-the-art method.Source: A. Experimental Setup · p. 5
  • Benchmark late fusion uniformly combines independently estimated phone, watch, and glasses self-trajectories after alignment to a common origin.The fused trajectory is evaluated against the phone target.Source: A. Experimental Setup · p. 5
  • Horizontal-world evaluation reports median and mean ATE across 115 test sessions, plus horizontal displacement RMSE at the event level for common-target analysis.Each session trajectory receives equal weight regardless of duration, and errors are in meters.Source: A. Experimental Setup · p. 5

B. Benchmark Results

The benchmark compares single-device localization and uniform late fusion across heterogeneous wearable placements, finding strong learning-based performance alongside method-dependent differences across devices. Adding wearable observations through uniform late fusion is not reliably beneficial.

B. Benchmark Results

  • Learning-based methods substantially outperform classical peak–trough PDR across the three wearable roles, while the same method behaves differently across placements.The benchmark is intended to expose these differences, not establish CoWear LSTM as a new state-of-the-art model.Source: B. Benchmark Results · pp. 5–6, Table II · p. 6
  • Uniform late fusion benefits RIDI, TLIO, and CoWear LSTM, but PDR and RoNIN do not outperform their strongest individual-device estimates.This pattern holds under both median and mean ATE.Source: B. Benchmark Results · p. 6
  • CoWear LSTM is a compact reference baseline for examining method behavior across wearable roles and under simple multi-device fusion, not a new state-of-the-art claim.Table II reports single-device localization and uniform late-fusion performance on the 115 test sessions.Source: B. Benchmark Results · pp. 5–6
  • Across phone, watch, and glasses placements, methods show different amounts of drift, directional deviation, and trajectory-shape distortion.Figure 4 provides representative trajectories with ground truth and predictions, and reports per-session ATE in each panel.Source: B. Benchmark Results · p. 6, Figure 4 · p. 6

C. Collaborative Localization Analysis

The collaborative analysis maps observations from different wearables to a common physical target and compares fusion strategies. Information fusion improves localization across phone, glasses, and watch targets, while different-hand observations also achieve lower error than same-hand observations.

C. Collaborative Localization Analysis

  • The observing device need not match the target: phone observations can estimate glasses and watch motion, and other-device observations sometimes beat the target device’s own estimate.Device-aligned ground truth supports supervision against a common physical target.Source: C. Collaborative Localization Analysis · pp. 6–7
  • Median ATE falls from 1.440 m to 1.173 m for the phone target, 1.408 m to 1.162 m for glasses, and 1.543 m to 1.164 m for the watch after information fusion.The improvement over the strongest individual branch is consistent across all three targets and under both median and mean ATE.Source: Table III · p. 7, C. Collaborative Localization Analysis · p. 7
  • Across representative sequences, the strongest individual observer varies, while the fused estimate remains closer to the target trajectory.Figure 5 shows common-target trajectories from individual wearable observations and information fusion; fusion has lower per-session ATE in both examples.Source: C. Collaborative Localization Analysis · p. 7, Figure 5 · p. 7
  • Information fusion achieves the lowest median and mean ATE for all three targets; uniform fusion also improves over every individual common-target estimate.Information fusion weights each motion event using predicted uncertainty, while fixed-weight fusion uses validation-set reliability weights.Source: C. Collaborative Localization Analysis · pp. 7–8, Table IV · p. 7
  • Different-hand watch–phone configurations achieve lower median and mean ATE for all three targets than same-hand configurations.The authors describe this as consistent with more spatially diverse observations providing more complementary information.Source: C. Collaborative Localization Analysis · p. 8

D. Discussion

CoWear maps heterogeneous wearable observations to a common physical target before fusion, and experiments show consistent localization gains with uncertainty-aware fusion. It complements single-device benchmarks while focusing on indoor walking, three wearable roles, and more constrained smartphone placement than fully unconstrained daily use.

D. Discussion

  • CoWear complements conventional single-device inertial benchmarks by enabling controlled study of cross-device correspondence and collaboration through synchronized, device-aligned trajectories.Source: D. Discussion · p. 8
  • The benchmark focuses on indoor walking with three wearable roles, and smartphone placement is more constrained than in fully unconstrained daily use.Source: D. Discussion · p. 8

VI. CONCLUSION

  • CoWear maps observations from different body locations to the motion of the same physical target before fusion because device self-trajectories are not directly interchangeable.This preserves spatial correspondence between heterogeneous observations and physical motion.Source: §VI · p. 8, D. Discussion · p. 8
  • Common-target fusion consistently improves localization across target roles, while uncertainty-aware fusion and more diverse device placements further improve collaborative performance.Uncertainty-aware fusion exploits event-dependent differences in observer reliability.Source: §VI · p. 8, D. Discussion · p. 8

Related papers

Loading 2610.20596v1…