Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

1 Introduction

• Camera-based measurement uses weak facial color changes to obtain rPPG signals from which heart rate, respiration frequency, and heart-rate variability can be measured.It requires off-the-shelf cameras rather than professional biomedical sensors and is not constrained by physical distance.Source: §1 · pp. 1–2
• Supervised deep-learning rPPG methods require large datasets of face videos paired with ground-truth physiological signals measured by synchronized contact sensors.The authors note that obtaining ground-truth signals is expensive even though face videos are comparatively easy to collect in quantity.Source: §1 · p. 2
• The earlier unsupervised baseline uses downsampled and reconstructed rPPG as negative and positive samples, but requires two video passes, trails supervised methods, and is noise-sensitive.It uses a triplet loss to pull reconstructed and anchor signals together and push downsampled and anchor signals apart.Source: §1 · p. 2
• A 3DCNN generates multiple rPPG signals across each video's spatiotemporal locations, pulling same-video signals together and pushing different-video signals apart.The approach is motivated by spatial and short-term temporal similarity, cross-video dissimilarity, and a heart-rate range constraint.Source: §1 · pp. 2–3
• Contrast-Phys outperforms the previous unsupervised baseline and achieves performance very close to supervised rPPG methods across five datasets, including RGB and NIR videos.The authors also report faster running speed and greater noise robustness than the previous unsupervised baseline.Source: §1 · p. 3

Video-Based Remote Physiological Measurement

• Traditional rPPG methods use handcrafted procedures, while deep-learning methods use 2D or 3D CNN architectures and require face videos paired with ground-truth physiological signals.Earlier methods also used spatial-temporal signal maps from different facial areas as inputs to 2D CNNs.Source: Video-Based Remote Physiological… · p. 3
• Gideon and Stent’s unsupervised method avoids ground-truth signals but falls behind some supervised methods, lacks robustness to external noise, and has unsatisfactory running speed.Source: Video-Based Remote Physiological… · p. 3

Contrastive Learning

• Unlike contrastive methods that learn abstract features for downstream tasks, this work uses contrastive learning to produce rPPG signals directly without ground-truth physiological signals.Source: Contrastive Learning · pp. 3–4

rPPG Spatial Similarity

• Signals from different facial regions have similar waveforms and power spectrum densities, with matching peaks in the illustrated PSDs.Phase differences are removed by transforming waveforms to PSDs, and amplitude differences can be canceled by normalization.Source: rPPG Spatial Similarity · p. 4, Figure 1 · p. 4

rPPG Temporal Similarity

• PSDs from small windows within a short rPPG clip should be similar because heart rate varies little over short intervals.The example compares two 5-second windows from a 10-second signal; the short-term condition is examined in sensitivity analysis.Source: rPPG Temporal Similarity · p. 4, Figure 2 · p. 5

Cross-video rPPG Dissimilarity

• Different videos are expected to have different PSDs, whose main observed cross-video difference is the heart-rate peak.The account notes that respiration rate and heart-rate variability can also contribute to differences, even when heart rates are similar.Source: Cross-video rPPG Dissimilarity · p. 5, Figure 3 · p. 5

HR Range Constraint

• The method focuses on PSD frequencies from 0.66 Hz to 4.16 Hz, corresponding to the stated 40–250 bpm heart-rate range.Source: HR Range Constraint · pp. 5–6

4.4 Contrastive Loss Function

• The loss pulls together PSDs from the same video and pushes apart PSDs from different videos, combining positive and negative terms.The method uses PSDs within 0.66–4.16 Hz, corresponding to its stated heart-rate range constraint.Source: Figure 4 · p. 6, §4.4 · p. 8, Negative Loss Term · p. 8

4.1 Preprocessing

• The model crops and resizes faces to 128 × 128 using facial landmarks to locate a fixed-size bounding box across frames.The bounding-box size is 1.2 times the first frame’s vertical landmark range.Source: §4.1 · p. 6

4.2 Spatiotemporal rPPG (ST-rPPG) Block Representation

• A modified PhysNet 3DCNN uses adaptive average pooling to output an ST-rPPG block of shape T × S × S, with each location’s receptive field covering part of the face.The paper states that all spatial positions can include rPPG information.Source: §4.2 · pp. 6–7

4.3 rPPG Spatiotemporal Sampling

• Training samples K clips with randomly chosen start times at every spatial position, yielding S · S · K rPPG clips per ST-rPPG block.At test time, the method averages the ST-rPPG block across spatial dimensions to obtain an rPPG signal.Source: §4.3 · pp. 7–8

4.4 Contrastive Loss Function

• The positive loss uses mean squared error between PSDs sampled from the same video, normalized by the number of positive pairs.This implements the within-video similarity constraint.Source: Positive Loss Term · p. 8
• The negative loss pushes apart PSDs from different videos; the paper says its four rPPG observations help exclude noise that violates spatial, temporal, or heart-rate constraints.Examples include local periodic eye blinking, non-constant-frequency head motion or facial expressions, and light flickering beyond the heart-rate range.Source: Negative Loss Term · p. 8, Why Our Method Works · p. 9

5.1 Experimental Setup and Metrics

• Experiments cover five RGB and NIR datasets, using intra-dataset testing on PURE, UBFC-rPPG, OBF, and MR-NIRP and cross-dataset testing on MMSE-HR.Accuracy is evaluated with MAE, RMSE, and Pearson correlation for HR, and with STD, RMSE, and correlation for HRV features.Source: Datasets · p. 9, Evaluation Metrics · p. 10

5.2 Intra-dataset Testing

• In UBFC-rPPG HRV testing, Contrast-Phys significantly outperforms traditional methods and the unsupervised baseline, with only a marginal difference from supervised methods.The intra-dataset results also include HR comparisons.Source: HRV Estimation · p. 10, Table 2 · p. 11, Table 1 · p. 11

5.3 Cross-dataset Testing

• On MMSE-HR cross-dataset testing, Contrast-Phys outperforms Gideon2021 and is close to supervised methods.The model is trained on UBFC-rPPG and tested on MMSE-HR.Source: §5.3 · p. 11, Table 3 · p. 12

5.4 Running Speed

• 802.45 fps for Contrast-Phys versus 387.87 fps for Gideon2021 during training; Contrast-Phys also reaches a lower IPR sooner.On OBF, it reaches its lowest IPR after about 2.5 hours versus about 5 hours for Gideon2021, with lowest IPRs of about 0.36 and 0.66, respectively.Source: §5.4 · pp. 11–12, Figure 7 · p. 12

5.5 Saliency Maps

• With injected periodic noise, Contrast-Phys remains unaffected while Gideon2021 fails; during head motion, its saliency maps still focus on skin areas.The authors attribute noise robustness to the patch being confined to one region, violating rPPG spatial similarity.Source: §5.5 · p. 13, Figure 8 · p. 13, Table 4 · p. 13

5.6 Sensitivity Analysis

• On UBFC-rPPG, a 2×2 spatial resolution performs well and 10s is the best tested temporal length.The 1×1 setting, which omits spatial similarity, performs worse; 4×4 and 8×8 yield noisier samples, while 5s gives coarse PSD estimates and 30s slightly violates the short-term signal condition.Source: §5.6 · p. 14, Table 5 · p. 14

• Contrast-Phys significantly outperforms the previous unsupervised baseline and is on par with state-of-the-art supervised rPPG methods.Source: §6 · p. 14
• Contrast-Phys can be trained without ground-truth physiological signals and uses spatiotemporal contrast based on four observations about rPPG.Source: §6 · p. 14
• The authors propose combining supervised and unsupervised rPPG methods in future work to further improve performance.Source: §6 · p. 14