PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Field-Deployable Wearable IMU Systems for Occupational Activity Recognition

Ihsan Ul Haq, James Brusey, Farnad Nasirzadeh, Chandan Karmakar

arXiv:2610.31160v1cs.LG

TL;DR

High aggregate VTT-ConIoT accuracies leave uncertainty about subject-independent performance and individual-worker variability. The paper systematically evaluates configurations under LOSO and tests calibration and augmentation; fusion and longer windows improve accuracy, while augmentation outcomes differ by method.

Abstract

from arXiv · show

Wearable inertial measurement unit (IMU) systems for construction worker human activity recognition (HAR) routinely report aggregate accuracies above 95 %, rarely under genuine subject-independent (leave-one-subject-out (LOSO)) evaluation, masking the per-individual variance that determines real-world deployment performance. Using the VTT-ConIoT benchmark, we conduct, to our knowledge, the first systematic configuration sensitivity analysis of this dataset: six classifiers across five sensor placements and three window sizes under a single LOSO protocol, a personalised-calibration pilot, and a validated comparison of tabular GAN-based and standard time-series augmentation. Per-subject LOSO accuracy for the main multi-sensor, 16-class configuration ranges from 61.9 % to 88.7 % (mean 77.5 %), a 27 pp spread, though all data are laboratory-collected. Multi-sensor fusion and longer windows both improve accuracy; standard augmentation gives small, consistent gains, while Conditional Tabular GAN (CTGAN) augmentation actively harms accuracy with a demonstrated distributional-fidelity mechanism; and personalised calibration recovers 16.8 pp for the weakest participants from just 30 % of their own data. These findings provide evidence-grounded configuration guidance for scaling occupational body sensor networks towards field deployment

I. INTRODUCTION

The paper questions whether high aggregate VTT-ConIoT accuracies reflect reliable performance across individual workers. It proposes a systematic LOSO analysis of per-subject variability, configuration choices, and augmentation.

I. INTRODUCTION

  • Wearable IMUs with supervised machine learning are presented as a scalable, low-cost route to continuous in-field ergonomic monitoring for construction workers.The motivation is work-related musculoskeletal disorders associated with repetitive lifting, awkward postures, and prolonged whole-body vibration.Source: §I · p. 1
  • Prior VTT-ConIoT studies reported accuracy up to ≈99 %, but most lacked genuine subject-independent evaluation and none reported per-subject variability.The authors therefore consider the conclusion that classification is nearly solved premature.Source: §I · p. 1
  • The paper asks whether high aggregate accuracy is stable across workers and how placement, window length, classifier choice, and synthetic augmentation affect field-relevant performance.The authors note that a system may perform unreliably for specific individuals despite high aggregate accuracy in a controlled setting.Source: §I · p. 1
  • Its contributions are a per-subject LOSO decomposition, a configuration sensitivity map, a personalised-calibration pilot, and a comparison of GAN-based and standard augmentation.The configuration map covers sensor placement, window size, six classifier families, and a hyperparameter-tuning check.Source: §I · p. 1

II. RELATED WORK

Prior VTT-ConIoT results vary in evaluation rigor and configuration, while per-subject distributions and systematic configuration or augmentation trade-offs remain unexamined. Broader HAR work also suggests that placement rankings depend on protocol, classifier, and window length.

A. Construction Worker HAR on VTT-ConIoT

  • The original VTT-ConIoT study reported LOSO SVM accuracy of 78 %/89 % for 16-/6-class tasks at 5 s and 84 %/94 % at 10 s.Its per-subject breakdown was 39 % to 62 % (mean 52 %) only for the simpler single-sensor 6-class baseline; the comparable 16-class decomposition was absent.Source: A. Construction Worker HAR on VTT-ConIoT · pp. 1–2
  • Subsequent VTT-ConIoT studies reported up to ≈99 % accuracy, but comparable subject-independent results were lower and none reported per-subject distributions.Reported protocols included unspecified-subject-independence k-fold studies,, grouped 5-fold versus LOSO, simplified single-sensor LOSO, and a subject-blind split.Source: A. Construction Worker HAR on VTT-ConIoT · p. 2
  • Earlier studies left construction-specific configuration trade-offs and augmentation unexamined, despite broader HAR evidence on placement sensitivity and window-size effects.Prior work also found GAN augmentation did not reliably outperform standard time-series transforms,,.Source: A. Construction Worker HAR on VTT-ConIoT · p. 2

III. DATASET AND METHODS

The study uses VTT-ConIoT, a controlled-laboratory dataset of construction activities collected from 13 participants with three simultaneously worn IMUs. It extracts statistical features from overlapping windows and standardises them within each training fold.

A. VTT-ConIoT Dataset

  • VTT-ConIoT contains 13 healthy adults performing 16 construction activities in a controlled laboratory, recorded with three IMUs at the hip, back, and shoulder/hand.At the main 5 s configuration, there are 11,385 approximately balanced windows.Source: A. VTT-ConIoT Dataset · p. 2

B. Preprocessing and Feature Extraction

  • The pipeline retains 21 accelerometer, gyroscope, and acceleration-norm channels, then extracts 147 statistical features from 2 s, 5 s, and 10 s sliding windows.The 100-sample stride yields 50 %, 80 %, and 90 % overlap, respectively; features are standardised using training-fold parameters only.Source: B. Preprocessing and Feature Extraction · p. 2

C. Classifiers

The study compares six default classifiers and checks whether tuning changes that comparison. Results use participant-wise LOSO across sensor configurations, with feature scaling fit on training folds only.

C. Classifiers

  • Six classifiers were evaluated with default scikit-learn settings, with a nested training-participant-only search checking tuned-versus-default performance for four models.The classifiers were RF, ET, XGB, LDA, LR, and RBF-kernel SVM; the search covered RF, XGB, SVM, and LR.Source: C. Classifiers · p. 2

D. Evaluation Protocol and Sensor Configurations

  • Evaluation used 13-fold participant-wise LOSO across five sensor subsets, holding out one participant and fitting feature standardisation on training data only.The subsets were all sensors, back, hand, trouser/hip, and trouser accelerometer-only.Source: D. Evaluation Protocol and Sensor… · p. 2

E. Additional Validation Experiments

The main multi-sensor, 16-class configuration shows substantial participant-level variation under LOSO evaluation. The additional-validation protocol also tests whether temporally disjoint data from the held-out participant can support personalised calibration.

E. Additional Validation Experiments

  • Calibration retrained each LOSO fold with 0%, 10%, 20%, or 30% of the held-out participant’s own temporally disjoint data, split by activity.Evaluation used the remaining disjoint portion; the protocol was designed to avoid window-overlap leakage.Source: E. Additional Validation Experiments · p. 2

A. Per-Subject LOSO Decomposition

B. Sensor Configuration and Classifier Comparison

At the 5 s window, multi-sensor fusion leads the tested sensor configurations across classifiers, while nested tuning makes little difference to defaults. A class-weighted, regularised 1-D CNN performs below the classical classifiers under the same LOSO protocol.

B. Sensor Configuration and Classifier Comparison

  • For SVM, all-sensor fusion gains 9.6 pp over the best single sensor, Back; restricting trouser sensing to accelerometers reduces accuracy by 11.3 pp.Back and hand perform comparably as single sensors, and both clearly outperform trouser.Source: B. Sensor Configuration and Classifier… · p. 3
  • Tuning changed results by at most 1.1 pp over defaults, and RF tuning did not outperform defaults.The nested search was used to check whether defaults were a materially unfair basis for classifier comparison.Source: B. Sensor Configuration and Classifier… · p. 3
  • 55.7% mean accuracy (macro-F1 51.9%) was achieved by the class-weighted, regularised 1-D CNN, below the classical classifiers under the same protocol.The paper contrasts this result with ≈99% deep-learning figures reported elsewhere for this dataset, which were not obtained under genuine subject-independent evaluation.Source: B. Sensor Configuration and Classifier… · p. 3

C. Window Size Sensitivity

Under LOSO evaluation with all sensors, accuracy rises with longer windows for every classifier. The paper frames 5 s as a balance between accuracy and feedback latency, while 2 s is the fastest tested option with an accuracy cost.

C. Window Size Sensitivity

  • For SVM, accuracy rises from 68.8% (2 s) to 77.5% (5 s) to 82.2% (10 s), a 13.4 pp gain from 2 s to 10 s.Fig. 2 plots LOSO accuracy against window length for all six classifiers using all sensors.Source: C. Window Size Sensitivity · p. 3, Figure 2 · p. 3
  • Every classifier improves monotonically with window length; 5 s offers a strong balance between accuracy and feedback latency for real-time ergonomic alerts.The paper says 2 s is the only tested window close to the fastest response, at a real accuracy cost; longer windows can capture complete activity cycles and produce more stable statistical estimates.Source: C. Window Size Sensitivity · p. 3

D. Synthetic Data Augmentation

On a representative six-fold subset, standard augmentation gives small, mostly positive accuracy changes, while CTGAN reduces accuracy for every classifier. Feature-level tests find synthetic samples distinguishable from real data, consistent with loss of IMU-window autocorrelation.

D. Synthetic Data Augmentation

  • On the shared six-fold subset, standard augmentation changes accuracy by −0.1 pp to 0.7 pp, while CTGAN changes it by −0.1 pp to −5.5 pp across classifiers.The CTGAN comparison used a 6-of-13-fold representative subset; standard augmentation was also checked on all 13 folds, with changes from −0.1 pp to 0.4 pp.Source: D. Synthetic Data Augmentation · p. 3, Table II · p. 4
  • All 294 feature comparisons distinguish synthetic from real data at p < 0.05, consistent with CTGAN losing IMU-window autocorrelation by treating time steps as independent tabular rows.A three-seed check on the weakest and strongest participants varied by only a few points.Source: D. Synthetic Data Augmentation · p. 3

E. Personalised Calibration

Personalised calibration improves accuracy for the weakest participants, while the results also favour simple augmentation over CTGAN and recommend reporting participant-level performance. Hyperparameter tuning adds at most a small gain over defaults.

E. Personalised Calibration

  • 16.8 pp: among the four weakest participants, mean accuracy rose from 68.4 % to 85.2 % after adding 30 % of each participant’s own data.The added data were temporally disjoint from the test data.Source: E. Personalised Calibration · p. 4

A. Cross-Subject Variability and Deployment Guidance

  • Report per-subject distributions, prefer multi-sensor fusion, and favour 5 s to 10 s windows for deployment guidance.The mean accuracy conceals substantial variation across participants.Source: A. Cross-Subject Variability and… · p. 4

B. Augmentation, Personalisation, and Model Choice

  • Jitter, scaling, and time-warping offer a low-cost augmentation step, whereas CTGAN actively degrades accuracy.The distributional-fidelity analysis provides a likely explanation for CTGAN’s effect.Source: B. Augmentation, Personalisation, and… · p. 4
  • Hyperparameter tuning yielded at most 1.1 pp over defaults, supporting the fairness of the classifier comparison.The results also report personalised calibration narrowing the gap for the weakest participants.Source: B. Augmentation, Personalisation, and… · p. 4

VI. CONCLUSION

Under a consistent LOSO protocol, the analysis finds substantial variation in participant accuracy and identifies configuration choices, augmentation effects, and calibration results. The results support a robust baseline using multi-sensor fusion, 5 s to 10 s windows, and properly scaled classical classifiers.

VI. CONCLUSION

  • 27 pp: the main multi-sensor, 16-class configuration spans 61.9 % to 88.7 % per-subject accuracy, with a 77.5 % mean.The conclusion contrasts this spread with ≈99 % headline figures reported elsewhere for the benchmark.Source: §VI · p. 4
  • Multi-sensor fusion, a 5 s to 10 s window, and properly scaled classical classifiers form a robust baseline under the protocol.Standard augmentation gives small, consistent gains, while CTGAN harms accuracy.Source: §VI · p. 4
  • Personalised calibration recovers 16.8 pp for the weakest participants using 30 % of their own data.A from-scratch CNN baseline remains behind the classical ensembles under the same protocol.Source: §VI · p. 4

Related papers

Loading 2610.31160v1…