PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Contact Sensors: Deep learning with Pseudo-Labeling for remote Photoplethysmography

Bhargav Acharya, Barbara Hammer, Hanna Drimalla

arXiv:2609.10026v1cs.LG

TL;DR

Deep-learning rPPG methods require costly, synchronized contact-sensor labels, motivating alternatives based on signal-processing pseudo-labels. The paper systematically compares these signals across synchronization conditions and finds that pseudo-labels can outperform poorly synchronized contact labels, although label quality affects cross-dataset generalization.

  • Problem

    Deep-learning rPPG methods depend on labor-intensive datasets containing videos synchronized with contact-sensor PPG, despite signal-processing methods not requiring such labels.

  • Method

    The study systematically compares deep-learning models trained with Finger-PPG, POS-derived Pseudo-PPG, and temporally aligned Synced-PPG across within-dataset and cross-dataset evaluations.

  • Results

    Pseudo-PPG outperforms poorly synchronized Finger-PPG, while well-synchronized datasets show no significant within-dataset difference and mixed cross-dataset results affected by label quality.

  • Takeaways & Limitations

    Signal-processing methods can generate valid training labels for deep-learning rPPG models, reducing dependence on labor-intensive synchronized data collection while maintaining competitive performance.

  • Takeaways & Limitations

    Assessing Pseudo-PPG quality in the current framework still requires ground truth, limiting scalability.

Abstract

from arXiv · show

Heart rate is a critical biomarker of health, and remote photoplethysmography (rPPG) enables its contactless estimation from video data for telemedicine applications. Recent advancements in deep learning based rPPG methods achieve state-of-the-art results, outperforming classical signal-processing methods in complex scenarios. However, deep learning methods depend on datasets with precise synchronization between videos and ground truth signals collected via contact sensors, whereas signal-processing-based methods do not. To address this dependence on labeled datasets, which are labor-intensive to collect, we investigate under which circumstances pseudo-labels extracted using unsupervised signal-processing methods can replace contact sensors labels for training deep learning methods. Our systematic evaluations found that for datasets with imperfect synchronization, the pseudo-label approach outperforms supervised training on contact sensors. For datasets with good synchronization, results are mixed: within-dataset evaluation shows no significant difference between training methods, while cross-dataset evaluation favors supervised training. However, removing a single outlier participant significantly improves the pseudo-label approach's cross-dataset performance, highlighting the importance of label quality. These results demonstrate that signal-processing methods can generate valid training signals for deep learning models, reducing dependency on labor-intensive dataset collection while maintaining competitive performance.

1. Introduction

rPPG offers contactless physiological measurement, but deep learning methods depend on costly, precisely synchronized contact-sensor labels. This work evaluates signal-processing-derived pseudo-labels as an alternative training signal.

  • Deep learning rPPG methods require large datasets pairing videos with synchronized contact-device PPG, making annotation labor-intensive and costly.
  • Existing datasets often combine imperfect synchronization with physiological differences between finger-based labels and face-extracted signals.
  • Face-mounted sensors could reduce site mismatch, but they are labor-intensive and obscure facial features during realistic video recording.
  • The study evaluates automatically extracted POS pseudo-labels for weakly supervised deep-learning training to reduce dependence on contact-sensor labels.
  • Experiments compare Finger-PPG, video-derived Pseudo-PPG, and temporally aligned Synced-PPG in within-dataset and cross-dataset evaluations.

2. Related Work

rPPG research has progressed from unsupervised signal processing to supervised deep learning, but synchronized contact sensors remain a scalability bottleneck. This work systematically tests signal-processing pseudo-labels as the sole training signal for deep models.

  • Prior rPPG work primarily used signal-processing methods without supervised learning, followed by deep architectures that improved performance through end-to-end temporal modeling.
  • Supervised deep-learning rPPG methods achieve state-of-the-art results but require synchronized contact PPG sensors, creating a scalability bottleneck.
  • Weakly supervised learning uses cheaper, noisier supervision, with pseudo-labeling treating model predictions on unlabeled data as surrogate labels.
  • Earlier rPPG studies incorporated pseudo-labels within contrastive or self-supervised frameworks, but their use as the sole training signal had not been systematically evaluated.
  • This work directly tests POS-extracted pseudo-labels with mean squared error, without requiring an initial labeled dataset, contrastive learning, or specialized loss functions.

3. Methods

The method compares established CNN-based rPPG architectures trained with contact, pseudo-, and temporally aligned signals. A four-stage pipeline extracts faces, predicts signals, filters them, and estimates heart rate spectrally.

  • The study trains TS-CAN and Physnet to isolate training-signal effects from architectural innovations in large transformer models.
  • TS-CAN combines a temporal-shift branch using diff-normalized frames with an appearance branch using standardized video input.
  • Physnet uses a 3D convolutional encoder-decoder to process spatial and temporal dimensions jointly, with diff-normalized input frames.
  • Models are trained with Finger-PPG, POS-derived Pseudo-PPG, or Synced-PPG, which temporally aligns finger signals to pseudo-signals through cross-correlation.
  • The pipeline detects and crops faces, predicts rPPG signals, applies a 0.75–3.0 Hz bandpass filter, and estimates heart rate from Welch power-spectral-density peaks.

4. Experiments

The experiments compare training signals for deep-learning rPPG models across within-dataset and cross-dataset evaluations using CHILL and PURE. Performance is assessed with participant-level MAE and Wilcoxon signed-rank tests.

  • Training signals: The study generates Finger-PPG, Pseudo-PPG, and Synced-PPG training signals from rPPG recordings for systematic comparison.Pseudo-PPG is extracted from facial video using POS, while Synced-PPG temporally aligns contact-sensor PPG to the facial signal.
  • Experimental setup: CHILL and PURE provide complementary evaluation settings, combining motion, illumination, physiological-state, and synchronization challenges.CHILL uses software synchronization, whereas PURE uses externally ensured hardware synchronization.
  • Evaluation protocol: 10-fold participant-level cross-validation reserves participants for testing and prevents participant overlap between training, validation, and test sets.Cross-dataset experiments train on one complete source dataset and test on the complete target dataset.
  • Metrics and statistics: Mean absolute error compares predicted heart rates with contact-device ground truth, while Wilcoxon signed-rank tests compare paired participant-level error distributions across training signals.The null hypothesis is that error distributions do not differ across training signals.
  • Implementation: Experiments implement Physnet and TS-CAN with waveform MSE loss across all three training signals.The models are adapted from rPPG-Toolbox and use the respective preprocessing procedures before video chunking and rescaling.

5. Results

Results differ by dataset and evaluation direction. Pseudo-PPG is competitive or superior in several comparisons, but its cross-dataset advantage depends on the source and target datasets and model architecture.

  • Within-dataset results: On CHILL within-dataset evaluation, Pseudo-PPG outperformed Finger-PPG and Synced-PPG for both Physnet and TS-CAN.Synced-PPG also significantly outperformed Finger-PPG for TS-CAN, but not Physnet.
  • Within-dataset results: On PURE within-dataset evaluation, training signal choice did not significantly affect performance for either Physnet or TS-CAN.Pseudo-PPG and POS also showed no significant performance difference for either architecture.
  • Cross-dataset results: When trained on CHILL and tested on PURE, Pseudo-PPG significantly outperformed Finger-PPG for both deep-learning methods.Synced-PPG significantly outperformed Finger-PPG for TS-CAN but not Physnet, and Pseudo-PPG did not significantly differ from POS.
  • Cross-dataset results: When trained on PURE and tested on CHILL, Pseudo-PPG significantly underperformed Finger-PPG and Synced-PPG for TS-CAN, while Physnet showed no significant training-signal effect.Pseudo-PPG also significantly underperformed POS for both Physnet and TS-CAN.

6. Sensitivity Analysis

The sensitivity analysis tested whether removing a participant with noisy pseudo-labels changes TS-CAN cross-dataset performance. Excluding participant 9 improved Pseudo-PPG performance but worsened Finger-PPG performance, while Synced-PPG showed no significant difference.

  • Participant 9 produced substantially higher POS estimation error than other PURE participants, resulting in noisy Pseudo-PPG labels.
  • Excluding participant 9 significantly reduced MAE for TS-CAN trained on Pseudo-PPG but significantly increased MAE for TS-CAN trained on Finger-PPG.The analysis evaluated cross-dataset performance after retraining on PURE without participant 9.
  • Training on Synced-PPG showed no significant performance difference before versus after participant 9 was excluded.
  • After exclusion, Pseudo-PPG training remained significantly worse than Finger-PPG training in the cross-dataset comparison.

7. Discussion

The discussion finds that pseudo-label effectiveness depends on synchronization and label quality. Pseudo-PPG can outperform poorly synchronized contact labels, but noisy labels can impair cross-dataset generalization and currently require ground truth for quality assessment.

  • With imperfect synchronization, Pseudo-PPG provides superior training signals compared with poorly synchronized Finger-PPG.The discussion relates this finding to prior strong generalization from POS-extracted labels on a dataset lacking accurate ground truth.
  • On CHILL, Synced-PPG significantly outperforms Finger-PPG, while Pseudo-PPG outperforms Synced-PPG because its morphology more closely matches video-extracted signals.The comparison indicates that both temporal alignment and signal morphology affect deep-learning performance.
  • For well-synchronized data, training-signal choice does not significantly affect within-dataset performance, but TS-CAN fails to generalize cross-dataset when trained on Pseudo-PPG.Removing one participant with poor POS quality significantly improves generalization, linking performance to pseudo-label quality.
  • Assessing Pseudo-PPG quality in the current framework still requires ground truth, which limits scalability.The authors identify ground-truth-free quality assessment as a needed direction for automatic training-sample selection.

8. Conclusion

The conclusion finds that signal-processing methods can generate valid pseudo-labels for training deep-learning rPPG models. This can reduce dependence on labor-intensive synchronized data collection when pseudo-labels are sufficiently reliable.

  • Pseudo-labels provide superior training signals when well-synchronized ground truth is unavailable and can match well-synchronized Finger-PPG performance when they are high quality.
  • Signal-processing methods can generate valid training labels for deep-learning models, enabling rPPG development without dependence on labor-intensive synchronized data collection.
Loading 2609.10026v1…