PaperlayerView exact revision on arXiv

Source-linked AI summary

PPG-Bench: A Unified Benchmark for PPG Foundation Models

Arvasu Kulkarni, Mahir Jain, Aditya Ray Mishra, Sandeep Singh, Siddharth Panwar

arXiv:2610.13435v1cs.LG

TL;DR

PPG foundation models are difficult to compare, and prior comparisons do not establish whether their representations improve on physiological features or generic time-series pretraining. PPG-Bench evaluates public models across datasets and tasks under standardized regimes and controls. The results show that current models often lack a meaningful advantage over simpler alternatives.

  • Problem

    PPG-FM evaluations vary in datasets, task definitions, preprocessing, downstream heads, finetuning, hyperparameter optimization, and baselines, leaving comparisons unable to establish value over physiological features or generic time-series pretraining.

    Source: §1 · p. 1

  • Method

    PPG-Bench evaluates five public PPG FMs across nine datasets and 26 tasks under four downstream regimes, using model-specific optimization, subject-disjoint evaluation, and random/constant, PRV, and MOMENT controls.

    Source: §1 · p. 2, §4 · p. 3

  • Results

    Current PPG FMs often lack an advantage over simpler controls: 24 (35.3%) of 68 task–regime combinations show no meaningful improvement over random chance, and 57 (83.8%) show no PPG-specific FM outperforming both PRV and MOMENT.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    Current PPG FMs are not yet strong general-purpose representations, so their value should be judged against simpler alternatives as well as other FMs.

    Source: §6 · p. 8

    The results concern released models; a causal conclusion about PPG-specific pretraining would require controlled retraining of the same architecture on PPG and generic time-series corpora.

    Source: §5 · p. 5

Abstract

from arXiv · show

Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream tasks. These models are difficult to compare: different evaluations use different datasets, task definitions, preprocessing, downstream heads, hyperparameter tuning, and baselines. We present PPG-Bench, a unifying open-source benchmark evaluating five public PPG FMs across nine open-access datasets and 26 tasks spanning cardiovascular state estimation, activity, sleep, screening, and identification. We evaluate models under four downstream regimes of increasing adaptability (linear probing, random forests, frozen MLP probes, and end-to-end finetuning), with model-specific hyperparameter optimization and subject-disjoint cross-validation. We also evaluate against three controls: random/constant baseline, a deterministic pulse-rate-variability (PRV) feature set, and a generic time-series FM (MOMENT). Across all 68 task-regime combinations, 35.3% show no meaningful FM advantage over the random/constant baseline, and in 83.8% no PPG-specific FM outperforms both PRV and MOMENT. We also find that added downstream head capacity generally improves performance, but can cause significant degradation if the model is performing at chance/constant level. Current PPG FMs therefore cannot yet claim to provide generalised and transferrable representations of PPG signals, and future evaluation in this field should include checks against simple, non-learned controls. Code for the benchmark and instructions to evaluate more models can be found here - https://github.com/mannas-ai/ppg-bench.

1 Introduction

PPG foundation models are hard to compare because evaluation choices vary, and existing comparisons do not establish whether their representations improve on physiological features or generic time-series pretraining. PPG-Bench standardizes evaluation and finds that PPG-specific models often lack an advantage over simpler controls.

  • Evaluation differences in datasets, task definitions, preprocessing, downstream heads, finetuning, hyperparameter optimization, and baselines make current PPG-FM comparisons difficult.Source: §1 · p. 1
  • PPG-Bench evaluates five public PPG FMs across nine open-access datasets and 26 tasks using four downstream regimes and three controls.The controls are a random/constant baseline, deterministic PRV features, and the generic time-series FM MOMENT.Source: §1 · p. 2
  • 24 (35.3%) of 68 task–regime combinations show no meaningful PPG-FM improvement over the random chance baseline, and 57 (83.8%) show no PPG-specific FM outperforming both PRV and MOMENT.Source: §1 · p. 2

2 Related Work

PPG-Bench addresses variation in model-specific evaluation protocols and complements existing biosignal benchmarks by comparing multiple public PPG models across multiple datasets. Its common, extensible framework also supports standardized evaluation of future models and datasets.

  • PPG FM studies define their own protocols, so dataset selection, task formulation, preprocessing, downstream heads, finetuning, hyperparameter optimization, and comparators can differ.Source: §2 · p. 2
  • Earlier PPG evaluations either compare a limited set of models across many datasets or multiple models on one dataset; PPG-Bench covers multiple models and independently collected datasets.The paper characterizes it, to the authors’ knowledge, as the first independent PPG FM benchmark to cover both.Source: §2 · p. 2
  • Other biosignal efforts include EEG-FM-Bench for EEG models (Xiong et al., 2025) and ECG comparisons of specialist and general-purpose models (Al-Masud et al., 2026).Source: §2 · p. 2
  • PPG-Bench evaluates released checkpoints under a common protocol and provides an extensible framework for adding models and datasets.Source: §2 · p. 2

3 Datasets, Tasks, and Preprocessing

PPG-Bench spans diverse tasks and recording conditions, while its task audit considers whether targets are informative and labels are well-defined. Preprocessing largely follows each pretrained model’s native pipeline.

  • Nine open-access datasets cover 17 primary and nine secondary tasks spanning cardiovascular estimation, activity, affect, sleep, screening, and identification.The datasets vary in recording environment, hardware, and sensor location.Source: §3 · pp. 2–3
  • Preprocessing is minimal and preserves each model’s native pipeline as faithfully as possible to avoid out-of-distribution inputs to released checkpoints.Source data are segmented and resampled as required; model-specific normalization, filtering, cropping, padding, or feature construction follows the pretrained model’s preprocessing.Source: §3 · p. 3
  • The task audit retains some externally available or low-incremental-value targets for comparability but does not treat them as strong evidence of FM capability.Examples include pregnancy stage, age, and BMI; physiological-age interpretation would require sufficiently accurate prediction and supporting evidence.Source: §3 · p. 3
  • For ambiguous continuous-score labels, PPG-Bench uses the established SDS threshold of 50 for depression and keeps WESAD valence and arousal as regression targets.The choices avoid a cohort-specific median threshold for TMD and information loss from binarizing WESAD’s 1-to-9 scores at 5.Source: §3 · p. 3

4 Methodology

PPG-Bench tests frozen representations and end-to-end adaptation under four increasingly flexible regimes, with controls that measure value beyond chance, physiological features, and generic time-series pretraining. Model-specific hyperparameter optimization and subject-disjoint evaluation support the comparisons.

  • Linear probing, random forests, frozen one-layer MLP probes, and end-to-end finetuning progressively increase downstream flexibility, with finetuning also adapting the representation.Source: §4 · p. 3
  • Three controls test whether PPG is predictive, whether FM representations improve on PRV features, and whether PPG-specific pretraining adds value over generic time-series learning.The controls are a random/constant baseline, PRV features, and MOMENT-1.Source: §4 · pp. 3–4
  • PPG-Bench lets each model select an optimization suited to it through hyperparameter optimization rather than imposing one downstream recipe.Linear probes and random forests use exhaustive grid search; frozen MLP probes and finetuning use Optuna with Hyperband.Source: §4 · p. 4
  • All evaluations use subject-disjoint five-fold outer cross-validation, with full inner cross-validation for linear probes and random forests and fixed validation splits for MLP probes and finetuning.The latter split is used because fully nested evaluation is substantially more expensive for those regimes.Source: §4 · p. 4

5 Results and Analysis

Across the benchmark, PPG foundation models often fail to beat simple controls, and greater downstream flexibility helps only when frozen representations already contain useful information. Subject-level aggregation improves evaluation results compared with treating windows as independent examples.

Overall Results

  • 35.3% of 68 method–task combinations show no meaningful improvement over the random chance baseline, and 83.8% show no PPG-specific FM outperforming both PRV and MOMENT.The authors conclude that current PPG FMs are not consistently strong general-purpose representations.Source: §5 · pp. 4–5
  • No model meaningfully beats the random/constant baseline on 4 of 17 tasks under linear probing, 6 each under Random Forests and frozen MLP probes, and 8 under finetuning.The authors note potential limits from single-channel signals, filtering, and sparse PPG-BP recordings.Source: §5 · p. 4
  • PPG-specific FMs outperform both controls on only 4/17 tasks under linear probing, 4/17 under Random Forests, 2/17 under frozen MLP probes, and 1/17 under finetuning.PRV uses classical physiological features, while MOMENT is a generic time-series FM; matching either control leaves no clear downstream advantage for the PPG-specific model.Source: §5 · pp. 4–5
  • When linear performance is no better than the random control, moving to MLP produces a mean change of −17.5%, and moving to finetuning produces −29.9%.Thus, added capacity can worsen results when the frozen representation is not already useful.Source: §5 · p. 5
  • AnyPPG performs best overall among the PPG-specific FMs, except for age estimation and biometric identification; PulsePPG is particularly strong for identification.MOMENT matches or exceeds PPG-specific models on several tasks.Source: §5 · p. 5
  • Among model–task pairs above the random control under linear probing, frozen MLP improves 58.5% with a mean gain of 5.9%, while finetuning improves 78.0% with a mean gain of 9.3%.Table 2 summarizes changes between evaluation methods; Figure 1 shows performance changes relative to linear probing for tasks where FMs outperform both controls.Source: §5 · p. 5, Table 2 · p. 6, Figure 1 · p. 6

Subject-level Evaluation

  • Subject-level aggregation improves every eligible comparison over window-level evaluation, supporting evaluation at the level where ground-truth labels are defined.Aggregation also reduces sensitivity to window-level noise, particularly for regression targets.Source: Table 4 · p. 7, §5 · p. 7
  • For subject-level regression, only mean aggregation is used; classification can select mean, median, or top-k mean, with selections fairly evenly spread across evaluation methods.Top-k mean averages the k = ⌈0.1n⌉ largest and smallest scores per class and keeps the more confident average.Source: §5 · pp. 6–7, Table 3 · p. 7

6 Conclusion

PPG-Bench standardizes evaluation across datasets, tasks, controls, and downstream adaptation regimes. Its results show that current PPG FMs often lack an advantage over simpler alternatives, and added model capacity does not reliably rescue weak representations.

6 Conclusion

  • PPG-Bench standardizes evaluation of PPG FMs across diverse datasets, tasks, controls, and downstream adaptation regimes.Source: §6 · p. 7
  • Across many tasks, current PPG FMs do not meaningfully outperform random/constant baselines, simple physiological features, or a generic time-series FM.Increasing downstream model capacity helps when useful task information is present, but does not reliably rescue weak representations.Source: §6 · p. 8

A.1 Compute details

The appendix documents benchmark compute costs, hyperparameter search procedures, post-hoc search-bound diagnostics, and model sizes.

A.1 Compute details

  • Finetuning used 360 GPU-hours, frozen MLP probes 160 GPU-hours, and Random Forests 4 GPU-hours.Evaluations ran on eight NVIDIA H200 GPUs except linear probes, which ran on CPU.Source: §A.1 · pp. 10–11

A.2 Hyperparameter Search Space

  • Linear probes and Random Forests use exhaustive grid search; frozen MLP probes and finetuning use TPE sampling with Hyperband pruning.The search-space table marks single-value parameters as fixed rather than searched; MLP probes use 48 trials and finetuning uses 48 trials.Source: §A.2 · p. 11, Table 5 · p. 11

A.3 Edge Pressure.

  • The post-hoc edge-pressure diagnostic counts selections at each numerical search axis’s minimum or maximum; values above 50% are highlighted.The authors report these diagnostics to identify search axes that could be expanded in future reruns; they did not change the finalized benchmark results.Source: §A.3 · pp. 11–12, Table 6 · p. 12

A.4 Model Sizes

  • Model sizes range from approximately 4M parameters for AnyPPG to 35.3M for MOMENT-1-small, while Random Chance and PRV are parameter-free.Source: §A.4 · p. 12, Table 7 · p. 12

B Results, Dataset and Tasks: Additional Details

The benchmark’s dataset-and-task table covers both primary and secondary tasks.

B Results, Dataset and Tasks: Additional Details

B.1 Extended Results

This section extends the reported primary-task linear-probe results to other evaluation regimes.

B.1 Extended Results

  • The section extends primary-task results beyond linear probes to random forests, MLP probes, and finetuning.Source: §B.1 · p. 12

B.2 Secondary Tasks

Most secondary-task PPG-FM results offer little advantage over the random baseline, with a few exceptions and trends that continue patterns from the primary benchmark.

B.2 Secondary Tasks

  • Nine additional tasks were included for completeness because they overlap with primary targets or are less commonly evaluated, with preliminary linear-probe and Random Forest results reported in Table 10.Source: §B.2 · p. 12
  • Most secondary-task PPG-FM results do not meaningfully exceed the Random Chance Baseline, with clearer exceptions for WESAD valence and arousal estimation and PPG-BP binary hypertension classification.Source: §B.2 · pp. 12–13
  • Random Forests generally provide no advantage over linear probing; AnyPPG is typically the strongest PPG-specific model and is frequently matched by MOMENT.Source: §B.2 · p. 13

Related papers

Loading 2610.13435v1…