Source-linked AI summary
Characterizing the Performance Gap in Human Activity Recognition for Older Adults
Hossein Khayami, Sungjin Hwang, Eshed Ohn-Bar, David E. Conroy, Amanda Lazar, Eun Kyoung Choe, Hernisa Kacorri
TL;DR
Most wearable HAR benchmarks feature younger adults, leaving uncertain whether benchmark gains generalize to older adults. The paper evaluates architectures and representations across younger- and older-adult datasets, finding that age-diverse self-supervised features narrow—but do not eliminate—the performance gap.
Problem
Most HAR datasets with wrist-worn recordings and reliable labels represent younger adults, leaving limited evidence about whether benchmark progress generalizes to older adults.
Source: §1 · p. 1, §2.1 · p. 2Method
The study evaluates HAR architectures and feature representations on younger-adult RealWorld and older-adult MyMove data using LOSO and cross-dataset transfer.
Source: §1 · p. 2, §3.3 · p. 3Results
Takeaways & Limitations
Benchmark performance alone incompletely characterizes wearable HAR progress; representation quality, particularly age-diverse self-supervised pretraining, improves older-adult performance while largely preserving younger-adult performance.
Source: §1 · p. 2, §5 · p. 6Age is not isolated as the sole source of the gap because age remains entangled with collection protocol, labeling methodology, and free-living versus scripted behavior.
Source: §5.1 · p. 6
Abstract
from arXiv · showhide
Human activity recognition (HAR) from wrist-worn accelerometers is increasingly used for health and behavioral tracking. Yet, most wearable HAR models are developed and evaluated on datasets dominated by younger adults, leaving it unclear whether benchmark progress generalizes across age groups. In this work, we leverage MyMove, our carefully annotated, free-living older-adult HAR dataset (mean age 71), to evaluate deep-learning architectures and training regimes under both leave-one-subject-out and cross-dataset transfer. We find that improvements on younger-adult benchmarks fail to transfer equally to data collected from older adults, resulting in a persistent and often widening performance gap. However, richer representations, particularly frozen self-supervised features pretrained on the age-diverse UK Biobank dataset, substantially improve performance on data from older adults and consistently narrow the performance gap, at modest cost to younger-adult performance, though disparities remain. These findings suggest that benchmark gains and architectural scaling alone provide an incomplete picture of progress in wearable HAR, and broader advances may require representations that better capture population diversity, alongside personalized adaptation to individual movement patterns and routines.
1 Introduction
The paper asks whether younger-adult benchmark gains generalize to older adults, whose free-living wrist-worn HAR data remain scarce. Across LOSO and cross-dataset evaluation, gains transfer unevenly, while age-diverse self-supervised representations narrow the gap.
1 Introduction
- The study compares six HAR architectures under within-dataset LOSO and cross-dataset transfer, and varies feature extractors while holding the classifier head fixed.The comparison includes raw inputs, random and supervised models, handcrafted features, and self-supervised representations, including a backbone pretrained on age-diverse UK Biobank data.Source: §1 · p. 2
- Richer, age-diverse self-supervised representations narrow the LOSO gap from 41.4% to 12.1% and the zero-shot gap from 32.4% to 19.3%, with modest costs to younger-adult performance.The reported younger-adult costs are 2% and 4%, respectively.Source: §1 · p. 2
- Improvements on younger-adult benchmarks fail to transfer proportionally to older-adult data, producing a persistent and sometimes widening performance gap.Under LOSO, the gap ranges from 23.9% to 44.6% across architectures, and top-ranked younger-adult models do not reliably rank among the best for older adults.Source: §1 · p. 2
- The authors argue that benchmark-only evaluation and architecture rankings provide an incomplete account of progress on older-adult data.Their contribution shifts attention toward representation quality and age-diverse self-supervised pretraining.Source: §1 · p. 2
2 Related Work
Prior HAR research spans datasets, neural architectures, and cross-dataset evaluation, but evidence for age-related transfer remains limited. This section situates the study within those three areas.
2.1 HAR Datasets & Population Representation
- Free-living, wrist-worn HAR datasets with reliable labels and substantial older-adult representation remain extremely limited.Existing older-adult datasets have limitations such as dense sensor setups or non-wrist placements, while common benchmarks largely use younger adults in controlled settings.Source: §2.1 · p. 2
2.2 Evolution of Neural Architectures for HAR
- Because architectures and evaluation protocols vary, the authors rank models by younger-adult performance to test whether benchmark-defined progress transfers to older adults.HAR architectures have been adapted from other machine-learning domains, and reported performance depends on training procedures, datasets, and evaluation protocols.Source: §2.2 · p. 2
2.3 Generalization & Evaluation Across Datasets
- Prior cross-dataset work often treats domain shift as aggregate, while population shift—especially age-related transfer—and labeling effects are underexplored.The study compares architectures across younger- and older-adult datasets under both in-distribution and cross-dataset settings.Source: §2.3 · p. 2
3 Experimental Setup
The experimental setup tests whether benchmark gains generalize across younger- and older-adult populations, using within-dataset and cross-dataset evaluation. It also isolates representation quality by varying features under a common classifier head.
3.1 Datasets
- MyMove provides free-living smartwatch recordings from 13 older adults, while UK Biobank supplies age-diverse unlabeled data used for self-supervised pretraining.MyMove labels combine thigh-sensor predictions and participant verbal reports; younger MyMove participants are excluded from quantitative evaluation because their data lack Lying and Running.Source: §3.1 · p. 3
3.2 Architectures
- Six end-to-end models span feedforward, convolutional-recurrent, efficient, and attention-based architectures.The set includes MLP, DeepConvLSTM, MobileHART, AttnTCN, Transformer, and LIMU-BERT.Source: §3.2 · p. 3
- A common MLP head enables comparison of six input representations, from raw windows and random weights to handcrafted, supervised, and self-supervised features.The representation configurations include frozen and fine-tuned ResNet-SSL backbones and a 32-dimensional handcrafted feature set.Source: §3.2 · p. 3, Table 2 · p. 4
3.3 Evaluation Protocol
- The study evaluates younger- and older-adult data using leave-one-subject-out and cross-dataset protocols.LOSO holds out one participant within RealWorld or MyMove; zero-shot evaluation trains on WISDM and tests directly on both populations.Source: §3.3 · p. 3, §3 · p. 3
- Pooled macro F1 gives each activity class equal weight, addressing the strong class imbalance in the older-adult dataset.Sitting and Standing comprise 87% of older-adult windows, while Lying and Running each account for less than 2%.Source: §3.3 · pp. 3–4
4 Results
Across in-distribution and cross-dataset evaluations, gains on RealWorld younger adults do not translate proportionally to MyMove older adults. Richer representations, particularly self-supervised pretraining, narrow but do not eliminate the disparity.
4 Results
- Across in-distribution and cross-dataset evaluations, younger-adult gains do not transfer proportionally to MyMove older adults, while richer representations progressively narrow the disparity.Source: §4 · p. 4
4.1 Benchmark Gains Do Not Close the Gap
- Younger-adult benchmark gains fail to transfer equally to older adults, and architectures ranked highest on younger adults are not reliably highest on older adults.The gap persists across LOSO and zero-shot cross-dataset transfer; benchmark gains can provide an incomplete picture of progress.Source: Figure 2 · p. 4, Figure 3 · p. 4, §4.1 · p. 4
4.2 The Gap Persists Across Protocols
- LOSO gap narrows from 41.4% to 12.1%, while the zero-shot cross-dataset gap narrows from 32.4% to 19.3% as representation quality rises.Source: Figure 4 · p. 5, §4.2 · p. 5
- 69.5% macro F1: the frozen SSL-pretrained ResNet is the strongest LOSO representation configuration for older adults.The evaluated ladder progresses from raw inputs and random weights through handcrafted and learned representations.Source: Figure 4 · p. 5, §4.2 · p. 5
- The frozen SSL-pretrained backbone concentrates predictions along the diagonal, with broader class separation gains for older adults; remaining errors are mostly between neighboring classes.UMAP projections also show limited separation, including for Standing and Walking.Source: Figure 5 · p. 5, §4.2 · pp. 5–6, Figure 6 · p. 6
- 38.4% to 65.1%: older-adult performance rises under zero-shot transfer as representations improve, while younger-adult performance stays stable or improves modestly.The frozen SSL-pretrained ResNet reaches 84.4%, 4.1% below the best result with handcrafted features; under LOSO, younger-adult performance decreases by 2%.Source: §4.2 · p. 5
5 Discussion
Across within-dataset and cross-dataset evaluation, younger-adult benchmark gains did not transfer proportionally to older adults, while richer representations substantially reduced the gap. The study also finds that greater movement variability among older adults and differences in activity-label meaning complicate interpretation of that gap.
5 Discussion
- Across LOSO and zero-shot cross-dataset evaluation, younger-adult benchmark gains did not transfer proportionally to older adults, while richer representations substantially improved older-adult performance and largely preserved younger-adult performance.The gap persisted and sometimes widened; richer representations reduced it.Source: §5 · p. 6
- The study’s findings extend earlier reports of asymmetric age transfer and suggest younger-adult benchmark evaluation can mask disparities while appearing to show steady progress.The discussion relates this pattern to accessibility-oriented visual question answering, where benchmark gains did not translate proportionally to blind users [7].Source: §5 · p. 6
5.1 Limitations
- Feature-mean differences between age groups were comparable to or smaller than differences between datasets, and age is not isolated from collection and labeling differences.Wrist placement, sampling rate, and activity taxonomy were matched, but protocol, labeling methodology, and scripted versus free-living behavior differed; age-outlier participants were insufficient to separate these factors conclusively.Source: §5.1 · p. 6, §5.2 · p. 6
- Shared activity labels can represent different behaviors across populations: MyMove Walking included shopping and dog walking, while younger-participant Walking was almost entirely leisure walking.The discussion notes that even longitudinal study would not isolate age cleanly because activity repertoires, label meanings, hardware, and methods change over time.Source: §5.1 · p. 6
5.2 Interpreting the Gap
- Older adults showed greater individual variability in walking intensity and cadence features, suggesting movement-pattern diversity rather than shifts in average movement is a key challenge.Younger adults instead showed greater variability in movement smoothness, anisotropy, and PC1 zero-crossing interval.Source: §5.2 · p. 6
- Richer representations reduced the gap more than increasing end-to-end architectural complexity, motivating age-diverse unlabeled pretraining followed by adaptation to HAR tasks.The authors identify continued self-supervised pretraining, domain adaptation, and test-time adaptation as promising directions.Source: §5.2 · p. 6
- Because activity repertoires and movement patterns vary by person and context, future wearable systems may benefit from broadly pretrained representations combined with personalized, teachable models.The authors note that disparities remain and fully population-general HAR may remain challenging under current paradigms.Source: §5.2 · p. 6