Source-linked AI summary
Exploring Contrastive Learning in Human Activity Recognition for Healthcare
Chi Ian Tang, Ignacio Perez-Pozuelo, Dimitris Spathis, Cecilia Mascolo
TL;DR
Healthcare-oriented HAR is constrained by difficult-to-collect labeled sensor data, motivating methods that learn from unlabeled data. This work adapts SimCLR with sensor time-series transformations and finds promising performance, including gains over supervised and semi-supervised baselines, while showing that transformation choices matter substantially.
Problem
HAR applications face limited labeled datasets because sensor data collection is difficult and often relies on restrictive protocols.
Method
The study adapts the SimCLR contrastive learning framework to sensor-based HAR using time-series transformations and evaluates it on open HAR data.
Results
SimCLR achieved up to 0.020 higher average F1 than fully supervised training, with fine-tuned rotation reaching the highest reported F1 score of 0.942.
Takeaways & Limitations
The results indicate potential for transferring contrastive learning to HAR and other health data, while transformation choices substantially affect performance.
Takeaways & Limitations
The relationship between transformation functions and performance was inconsistent across evaluation protocols, motivating evaluation on more datasets and transformations.
Abstract
from arXiv · showhide
Human Activity Recognition (HAR) constitutes one of the most important tasks for wearable and mobile sensing given its implications in human well-being and health monitoring. Motivated by the limitations of labeled datasets in HAR, particularly when employed in healthcare-related applications, this work explores the adoption and adaptation of SimCLR, a contrastive learning technique for visual representations, to HAR. The use of contrastive learning objectives causes the representations of corresponding views to be more similar, and those of non-corresponding views to be more different. After an extensive evaluation exploring 64 combinations of different signal transformations for augmenting the data, we observed significant performance differences owing to the order and the function thereof. In particular, preliminary results indicated an improvement over supervised and unsupervised learning methods when using fine-tuning and random rotation for augmentation, however, future work should explore under which conditions SimCLR is beneficial for HAR systems and other healthcare-related applications.
1 Introduction
HAR supports healthcare and well-being applications, but labeled sensor data is difficult to collect and often tied to restrictive laboratory protocols. This study adapts SimCLR contrastive learning to sensor-based HAR to address these labeling constraints.
- Labeled sensor data is difficult to collect because sensor streams are abstract and data collection usually requires laboratory protocols.Those fixed protocols further limit generalizability in healthcare applications.
- Contrastive learning uses similar and dissimilar sample pairs to learn representations from data with fewer labeling requirements.It has shown strong performance in visual representation learning and may generalize better than manually designed tasks.
- The study adapts SimCLR and sensor time-series augmentations to evaluate contrastive learning on open HAR datasets.The authors describe this as an initial exploration of contrastive learning effectiveness for sensor-based HAR.
- Different transformation combinations produced significant performance differences, while SimCLR representations outperformed fully supervised training after fine-tuning.The study presents these findings as evidence of contrastive learning’s potential for HAR and other data modalities.
2 Related work
Related work uses unlabeled data through semi-supervised and self-supervised learning, but contrastive learning remains less established for wearable HAR than for computer vision. Earlier HAR work found gains from transformation discrimination but limited transfer benefits with a different unlabeled dataset.
- Semi-supervised and self-supervised learning seek to increase training data diversity and quantity by using unlabeled data.
- Contrastive learning methods aim to capture intrinsic data structure without ad-hoc heuristics, including approaches such as CPC and MoCo.
- SimCLR generates positive pairs with transformations and uses other transformed views in the batch as negative pairs for contrastive learning.This design reduces memory demands and permits flexible batch sizes.
- HAR’s healthcare relevance is established, while labeled datasets remain limited because collecting this type of data is difficult.
- Wearable HAR has fewer self-supervised studies than computer vision, and transformation discrimination previously improved supervised performance but showed little or no benefit with another unlabeled dataset.
3 Method
The method adapts modality-agnostic SimCLR to sensor-based HAR by creating transformed views of time-series data, encoding them, projecting representations, and optimizing NT-Xent contrastive loss. Eight transformations are composed probabilistically, with a lightweight TPN encoder used for HAR.
- SimCLR comprises a probabilistic transformation function, base encoder, projection head, and NT-Xent contrastive loss.The transformation function creates two views of each sample as a positive pair, while the encoder and projection head map representations for contrastive learning.
- The HAR adaptation replaces visual augmentations with eight sensor time-series transformations composed by selecting a subset and applying functions in different orders.
- The transformations include Gaussian noise, random scaling, 3D rotation, signal inversion, time reversal, section permutation, time warping, and channel shuffling.
- The implementation uses a lightweight TPN base encoder, a three-layer fully connected projection head, and the unchanged NT-Xent loss.
4 Evaluation and results
The evaluation uses MotionSense accelerometer data and compares linear and fine-tuned protocols across SimCLR transformation combinations. Fine-tuning with rotation achieved the strongest reported SimCLR result, while SimCLR also improved over fully supervised training.
- 4.1 Evaluation protocol: The MotionSense evaluation used accelerometer data from 24 subjects performing six activities, producing 6630 overlapping windows.
- 4.1 Evaluation protocol: Figure 1 reports average weighted F1 scores across five independent runs for combinations of transformation functions, with diagonal entries representing single transformations.
- 4.1 Evaluation protocol: Models were trained on 19 subjects and evaluated on five unseen subjects using linear and fine-tuned protocols with weighted F1 as the metric.
- 4.2 Evaluation Results: SimCLR improved performance by up to 0.020 over fully supervised models, whose average F1 score was 0.922.The reproduced self-supervised transformation-identification method achieved an average F1 score of 0.923.
5 Discussion and Conclusion
The study adapted SimCLR to sensor-based HAR and found promising performance relative to fully supervised and semi-supervised methods. Results also show that transformation functions can substantially affect performance, with effects varying across evaluation protocols.
- SimCLR slightly outperformed fully supervised and semi-supervised methods in the evaluation, indicating promise for transferring contrastive learning to sensor-based HAR.
- Different transformation functions affected model performance, sometimes significantly, and their effects were inconsistent across evaluation protocols.
- The findings support exploring SimCLR for mobile sensing and other health data because the method is described as modality-agnostic.
- Further evaluation across more datasets and transformations is needed to clarify when SimCLR benefits HAR systems.