Source-linked AI summary
Rotation-Invariant Multi-IMU Activity Recognition under Independent Per-Location Orientation Shifts
Seungyeol Baek, Yoonbyung Chai, Yonghyeon Lee, Sungjoon Choi, Sungho Suh
TL;DR
Self-administered multi-IMU HAR faces independent orientation offsets across body locations, while augmentation and calibration approaches have sampling or reference-frame limitations. TRI-HAR applies shared SO(3)-equivariant processing and invariant projection separately to each location, then fuses invariant features. Across four benchmarks, it preserves macro-F1 under fixed independent per-location rotations and outperforms rotation-augmented baselines without rotational augmentation.
Problem
Self-administered multi-IMU HAR must handle independent orientation offsets caused by reattaching sensors across sessions, which conventional scalar models do not structurally handle.
Method
TRI-HAR reshapes inertial streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection per IMU location, and fuses the invariant location features.
Results
Across four public multi-IMU benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift.
Takeaways & Limitations
The results support architecture-level rotation invariance for calibration-light, self-administered wearable HAR.
Takeaways & Limitations
Efficient on-device deployment remains future work because the current robustness-first architecture has substantial encoder and invariant-projection cost.
Abstract
from arXiv · showhide
Human Activity Recognition (HAR) with self-administered wearables, such as at-home rehabilitation and exercise monitoring, often requires reattaching inertial measurement units (IMUs) across sessions. In multi-IMU settings, this can induce independent orientation offsets across body locations, a deployment shift that conventional scalar HAR models do not structurally handle. Existing remedies rely on rotation augmentation, whose robustness depends on sampled transformations, or calibration and orientationnormalization pipelines requiring additional reference-frame assumptions or explicit procedures. We present Truly Rotation-Invariant HAR (TRI-HAR), a rotation-invariant framework that makes robustness to independent per-location IMU orientation offsets a structural model property. TRI-HAR reshapes accelerometer and gyroscope streams into triaxial vectors, applies a shared SO(3)-equivariant backbone and invariant projection to each IMU location, and fuses the resulting invariant features for activity classification. Across four multi-IMU benchmarks, TRI-HAR preserves macro-F1 under fixed independent per-location SO(3) rotations and outperforms rotation-augmented baselines under this target shift without requiring rotational augmentation.
1 Introduction
Self-administered multi-IMU HAR is vulnerable to independent orientation shifts when sensors are reattached, while existing augmentation and calibration remedies have structural or procedural limitations. TRI-HAR addresses this shift by preserving per-location rotations until invariant projection and is evaluated across four benchmarks.
- Motivation: Independent orientation offsets across reattached body-worn IMUs can substantially degrade HAR models without built-in rotational robustness.The challenge is especially relevant to at-home rehabilitation and exercise monitoring.
- Existing approaches: Rotation augmentation depends on sampled transformations, while calibration and normalization add reference-frame or procedural assumptions.These assumptions may be difficult to satisfy in uncontrolled self-administered settings.
- TRI-HAR: TRI-HAR treats accelerometer and gyroscope measurements as triaxial vectors and encodes each physical IMU location with a shared equivariant-to-invariant encoder.Invariant location features are concatenated in fixed body-location order for classification.
- TRI-HAR: Preserving separate rotation actions until invariant projection enables robustness to independent per-location SO(3) offsets without rotational augmentation or test-time orientation calibration.The design makes this robustness a structural property of the model.
- Evaluation: TRI-HAR is evaluated on four public multi-IMU HAR benchmarks against supervised baselines with and without matched per-location rotation augmentation.The evaluation also measures host-side latency on 3-IMU and 5-IMU streams.
2 Related Work
Prior HAR work addresses orientation variability mainly through augmentation, calibration, or normalization, while equivariant architectures encode rotational symmetry directly. These approaches differ in whether robustness depends on sampled transformations or additional deployment assumptions.
- Augmentation: HAR augmentation methods range from signal-space rotations to synthetic IMU generation and large-scale representation learning.Examples include CrossHAR and oneHAR for broader generalization across sensor positions and orientations.
- Augmentation: Augmentation-based robustness remains sampling-dependent because many schemes cover restricted rotation families rather than arbitrary SO(3) rotations.Robustness therefore depends on which transformations are synthesized during training.
- Calibration and normalization: Calibration, canonicalization, and orientation-normalization pipelines map measurements to a consistent reference frame but may require static intervals or reliable magnetic references.Such engineering assumptions may not hold in practical wearable deployments.
- Equivariant architectures: SO(3)-equivariant architectures encode rotational symmetry directly, with examples including Tensor Field Networks, 3D Steerable CNNs, and Vector Neurons.Vector Neurons enable equivariant variants of backbones such as PointNet and DGCNN.
3 Method
TRI-HAR processes triaxial IMU streams separately for each physical location through a shared equivariant-to-invariant encoder, then fuses invariant location features for classification. This per-location projection preserves robustness to independent orientation shifts while retaining fixed location identity and multi-location patterns.
- Location-grouped representation: TRI-HAR reshapes accelerometer and gyroscope channels into triaxial vector streams grouped by physical IMU location.Rotations act on each stream’s final 3D vector dimension, and each location contains S_loc streams.
- Shared equivariant backbone: A shared FER-VN-DGCNN backbone processes each location group as a dynamic graph over time in an SO(3)-equivariant latent space.Vector Neurons provide rotation-predictable vector-valued channels, while the graph connects feature-similar time-step motion states within one location.
- Invariant projection: The invariant projection uses a learned co-rotating frame to convert equivariant features into rotation-invariant time-step representations.Because the latent feature and learned frame transform under the same lifted rotation, their inner products remain unchanged.
- Location features: Global max and average pooling aggregate invariant time-step features into one invariant representation h_l for each location using the shared encoder E_θ.The same encoder weights are evaluated separately on every location group rather than learning location-specific equivariant encoders.
- Per-location invariant fusion: TRI-HAR concatenates location features in a fixed order before classification, allowing activity prediction to use multi-location patterns in invariant feature space.Invariant projection occurs within each location before fusion, so each local rotation is canceled independently; replacing every input by R_l·X^(l) leaves y-hat unchanged.
4 Experiments
The evaluation uses four public multi-IMU HAR benchmarks with cross-subject splits, matched supervised baselines, and fixed independent per-location rotation tests. It also measures live host-side latency in tested 3-IMU and 5-IMU streaming setups.
- Datasets and preprocessing: The evaluation covers PAMAP2, DSADS, Opportunity, and RealDISP using dataset-specific sampling rates, window lengths, subjects, classes, and sensor locations.The experiments retain accelerometer and gyroscope channels, with dataset-specific preprocessing and exclusions.
- Baselines: TRI-HAR is compared with five supervised HAR baselines under the same preprocessing, cross-validation splits, and test-rotation protocol.The baselines are MC-CNN, DeepConvLSTM, MLP-HAR, TinyHAR, and SA-HAR.
- Evaluation protocol: Cross-subject evaluation uses leave-one-subject-out folds for PAMAP2, DSADS, and Opportunity, plus a five-fold grouped split for RealDISP.The RealDISP split separates Ideal and Self-displacement subject groups from Mutual-displacement samples.
- Rotation setup: The rotation protocol evaluates original test data and fixed independent per-location SO(3) offsets, while loc-sample independently augments each location during training.Each test rotation is held fixed across the held-out fold, and augmented rotations are resampled for every training window.
- Runtime evaluation: Live host-side latency is measured over 1000 windows for native-rate 3-IMU and 5-IMU streams using p99 model and window-to-label latency.The tested stream rates are 33 Hz for the PAMAP2-derived setting and 25 Hz for the DSADS-derived setting; real-time feasibility uses the window-update period as the threshold.
5 Results
Across benchmark and deployment-shift evaluations, TRI-HAR preserved macro-F1 under independent per-location rotations and achieved the strongest reported robustness. Its location-wise invariance introduced runtime cost but remained feasible for host-side real-time streaming.
- Benchmark comparison: TRI-HAR preserved macro-F1 under fixed independent per-location SO(3) rotations across all four datasets and achieved the highest shifted-test macro-F1 without rotation augmentation.Its relative invariance error was below 10^-10; on unrotated data, it was best on PAMAP2, DSADS, and RealDISP but trailed the strongest scalar baseline on Opportunity.
- RealDISP mutual-displacement: On the RealDISP Mutual-displacement fold, TRI-HAR exceeded MLP-HAR and TinyHAR by 7.74 and 6.80 macro-F1 points, respectively.Compared with corresponding five-fold means under I, non-augmented baselines fell 10.94–31.89 points, versus 6.7 points for TRI-HAR.
- RealDISP mutual-displacement: Mutual-displacement combines orientation and positional changes, so it is not an isolated test of rotation robustness.The reported advantage therefore concerns a combined physical placement shift rather than orientation alone.
- RealDISP mutual-displacement: Rotation augmentation improved four of five baselines by 1.4–6.9 points but decreased MLP-HAR by 2.9 points, whereas TRI-HAR remained highest without augmentation.The result supports structural invariance under the tested combined shift while avoiding the observed augmentation-induced degradation for MLP-HAR.
- Location-wise fusion: Replacing location-wise fusion with one joint invariant projection caused TRI-HAR-joint to lose 16.80–50.95 macro-F1 points under SO(3)loc-fix, while TRI-HAR lost none.The control comparison attributes robustness to canceling each location-specific rotation before feature fusion.
- Runtime: TRI-HAR’s per-location encoder was slower than scalar baselines, yet its CPU p99 window-to-label latency remained 67.61 ms for 3 IMUs and 85.78 ms for 5 IMUs.The corresponding update periods were 37.6 and 28.9 times larger, supporting host-side real-time streaming inference.
6 Conclusion
TRI-HAR applies rotation-invariant processing separately to each physical IMU location before fixed-order fusion. Across four multi-IMU benchmarks, it preserved macro-F1 under independent per-location SO(3) rotations and outperformed rotation-augmented baselines.
- TRI-HAR applies a shared SO(3)-equivariant backbone and invariant projection separately to each physical IMU location before fixed-order fusion.
- Across four multi-IMU benchmarks, TRI-HAR preserved macro-F1 under fixed independent per-location SO(3) rotations.
- TRI-HAR outperformed rotation-augmented baselines under the independent per-location rotation shift.
- The results support architecture-level rotation invariance for calibration-light, self-administered wearable HAR.
- Efficient on-device deployment remains future work.