Source-linked AI summary
PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography
Rencheng Song, Huan Chen, Juan Cheng, Chang Li, Yu Liu, Xun Chen
TL;DR
Inaccurate rPPG pulse signals limit reliable cardiac-feature estimation beyond average HR, motivating methods that recover realistic waveforms from facial videos. PulseGAN denoises rough CHROM signals with a conditional GAN using adversarial, waveform, and spectrum losses, and improves HR, HRV, and IBI accuracy in within- and cross-database tests. Its cross-database AVNN and SDNN errors improve over DAE and CHROM, while the paper notes that the evaluated HRV characteristics remain preliminary.
Problem
Inaccurate rPPG pulse signals limit applications requiring cardiac features beyond average HR, including HRV and IBI.
Method
PulseGAN denoises rough CHROM pulses with a conditional GAN trained using adversarial, time-domain waveform, and frequency-domain spectrum losses against reference PPG.
Results
PulseGAN improves HR, HRV, and IBI accuracy, with cross-database AVNNmae improving 20.85% over DAE and 41.19% over CHROM.
Takeaways & Limitations
The framework can be integrated with existing deep learning rPPG methods and may expand rPPG applications requiring waveform-derived cardiac features.
Takeaways & Limitations
The HRV characteristics evaluated in this paper are relatively preliminary.
Abstract
from arXiv · showhide
Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitation of inaccurate pulse signals. In this paper, a new framework based on generative adversarial network, called PulseGAN, is introduced to generate realistic rPPG pulse signals through denoising the chrominance signals. Considering that the cardiac signal is quasi-periodic and has apparent time-frequency characteristics, the error losses defined in time and spectrum domains are both employed with the adversarial loss to enforce the model generating accurate pulse waveforms as its reference. The proposed framework is tested on the public UBFC-RPPG database in both within-database and cross-database configurations. The results show that the PulseGAN framework can effectively improve the waveform quality, thereby enhancing the accuracy of HR, the heart rate variability (HRV) and the interbeat interval (IBI). The proposed method achieves the best performance compared to the denoising autoencoder (DAE) and CHROM, with the mean absolute error of AVNN (the average of all normal-to-normal intervals) improving 20.85% and 41.19%, and the mean absolute error of SDNN (the standard deviation of all NN intervals) improving 20.28% and 37.53%, respectively, in the cross-database test. This framework can be easily extended to other existing deep learning based rPPG methods, which is expected to expand the application scope of rPPG techniques.
I. INTRODUCTION
rPPG offers non-contact cardiac monitoring, but inaccurate pulse extraction limits reliable cardiac-feature estimation beyond average HR. PulseGAN addresses this by denoising rough chrominance-derived signals with conditional adversarial, time-domain, and spectrum-domain losses.
- rPPG records heartbeat-related facial-skin color changes using consumer-level cameras without contact sensors.
- Conventional rPPG assumptions are often unmet in realistic situations, limiting pulse-signal accuracy and reliable HR-feature calculation.
- Accurate rPPG waveforms are needed to calculate diverse features for stress detection, emotional classification, and health monitoring.
- PulseGAN uses a conditional GAN to transform rough CHROM pulses toward synchronously recorded reference PPG signals.
- Time-domain waveform and frequency-domain spectrum losses complement adversarial loss to match generated pulses with reference waveforms.
- PulseGAN is tested in within- and cross-database settings and improves HR, HRV, and IBI accuracy.
II. RELATED WORK
Prior rPPG work includes conventional BSS and skin-reflection-model methods, alongside deep models targeting HR or waveform generation. PulseGAN extends this landscape with a cGAN framework intended to improve generated pulse-waveform quality and integrate with existing methods.
- BSS-based rPPG methods assume statistical properties of linearly mixed pulse and noise signals.
- CHROM uses skin optical reflection modeling, projection, and alpha tuning to reduce interference and obtain a pulse signal.
- Deep rPPG methods include end-to-end and feature-decoder systems using CNNs, attention mechanisms, and latent physiological information.
- Some deep models directly generate or enhance pulse waveforms, including LSTM-based filters for noisy rPPG signals.
- PulseGAN introduces a cGAN framework to generate pulse waveforms and improve performance over a generator using waveform loss alone.
A. Acquisition of rough rPPG pulses
PulseGAN starts from a rough CHROM pulse and uses a conditional generator–discriminator arrangement to produce a waveform closer to reference PPG. The discriminator receives the rough signal as a conditioning input while judging generated versus reference signals.
- CHROM is selected to extract the rough rPPG pulse because it is fast and stable against motion artifacts.
- CHROM projects standardized RGB signals into chrominance signals S1 and S2, then applies alpha tuning to calculate the rough pulse X.
- The generator maps rough CHROM signal X to target rPPG signal G(X), while the discriminator distinguishes G(X) from reference PPG Xc.
- PulseGAN uses a conditional GAN in which X conditions discriminator inputs composed of either (G(X), X) or (Xc, X).
- The generator resembles a denoising autoencoder with skip connections, six convolutional encoder layers, and six deconvolutional decoder layers.
- The discriminator uses stacked one-dimensional convolutions and judges generated and reference waveforms with CHROM signal X as a condition.
C. Loss function
PulseGAN learns generated pulse waveforms using adversarial, time-domain waveform, and frequency-domain spectrum losses. Its evaluation compares waveform quality across HR, HRV, and IBI in within-database and cross-database settings.
- Loss design: PulseGAN trains a generator to produce waveform G(X) close to reference signal Xc from paired data.The input X is a rough pulse signal, and the reference Xc is used to guide training.
- Loss design: The generator loss combines adversarial, waveform, and spectrum losses to learn pulse characteristics in time and frequency domains.The spectrum loss uses 1024-point FFT representations, while λ and β weight the waveform and spectrum terms.
- Loss design: The discriminator distinguishes generated signals from reference signals, while the generator minimizes losses that improve generated waveform quality.The discriminator loss remains the LSGAN loss, and the generator is trained to learn time-frequency characteristics.
- Evaluation: The evaluation compares PulseGAN with conventional methods and a denoising autoencoder using averaged HR, HRV, and IBI quality measures.Tests use both within-database and cross-database configurations.
- Evaluation: The experiments use UBFC-RPPG for testing, with PURE and BSIPL-RPPG combined as the cross-database training set.A 10-second sliding window is used in both scenarios; the sliding steps differ between configurations.
B. Databases
The study uses UBFC-RPPG, PURE, and BSIPL-RPPG databases containing facial videos and synchronously acquired reference PPG signals. Their recording setups and HR distributions support within-database and cross-database evaluation.
- UBFC-RPPG: UBFC-RPPG contains 42 realistic-condition videos recorded at 640 × 480 pixels and 30 fps.Subjects played a time-sensitive mathematical game, while reference PPG was sampled at 60 Hz.
- Database characteristics: Figure 4 presents reference-PPG HR distributions for BSIPL-RPPG, PURE, and UBFC-RPPG.The distributions provide a database-level view of the HR ranges represented in the experiments.
- Database characteristics: Figure 5 presents the BSIPL-RPPG database setup.The setup includes a camera and a finger-mounted Contec CMS50E pulse oximeter connected to a computer for synchronous transfer.
- PURE: PURE contains 60 videos from 10 subjects performing six types of head motion during 640 × 480-pixel, 30-fps recordings.Reference PPG was collected with a Contec CMS50E pulse oximeter.
- BSIPL-RPPG: BSIPL-RPPG contains 37 healthy student subjects recorded with synchronized camera and pulse-oximeter measurements.Subjects remained still initially and performed apparent head movements later; videos and PPG signals lasted about 4.5 minutes.
C. Metrics
The paper evaluates generated pulse-waveform quality through HR, HRV, and IBI metrics derived from interbeat intervals.
- IBI sequences are calculated separately for generated and reference pulse signals before deriving cardiac features.
- Average HR is calculated from IBI, while AVNN and SDNN quantify HRV from normal-to-normal intervals.AVNN is the average of all NN intervals, and SDNN is their standard deviation.
- HR evaluation uses HRmae, HRrmse, HRmer, and Pearson correlation coefficient r.
- AVNNmae and SDNNmae compare PulseGAN-derived HRV features with reference PPG features across time windows.
- IBImae evaluates padded IBI vectors after expanding each vector to the same length as the PPG signal.Padding assigns each sample within an RR interval the value of that interval.
D. Experimental results
The experiments are organized into within-database and cross-database configurations.
- Experimental results are presented in a sequence of within-database and cross-database configurations.
Within-database:
Within-database experiments on UBFC-RPPG show that PulseGAN improves average HR consistency and HRV and IBI-related waveform measures over the compared methods.
- 4234 UBFC-RPPG samples are split into 3192 training samples from 30 subjects and 1042 testing samples from 12 subjects.
- Within-database:: PulseGAN achieves the best average HR performance, reducing large DAE errors and producing the smallest standard deviation.The reported result indicates the most accurate and stable average-HR estimation in this setting.
- Within-database:: PulseGAN improves both AVNNmae and SDNNmae over DAE, with AVNN and SDNN errors more concentrated around zero.
- Within-database:: PulseGAN slightly improves IBImae over DAE in the within-database configuration.
- Within-database:: In an example, IBIae is 112.44 ms for CHROM, 42.50 ms for DAE, and 24.67 ms for PulseGAN.
Cross-database:
In the cross-database setting, PulseGAN achieves the best reported HR, HRV, and IBI performance against the compared methods. Its waveform and interval estimates show improved agreement with reference signals.
- 13484 training samples came from PURE and BSIPL-RPPG, while 1470 UBFC-RPPG samples formed the testing set.
- PulseGAN achieves the best average HR results and outperforms CHROM and DAE in cross-database evaluation.Bland–Altman plots compare predicted and reference HR for CHROM, DAE, and PulseGAN.
- 20.85% and 41.19% are the AVNNmae improvements versus DAE and CHROM, respectively, while SDNNmae improves 20.28% and 37.53%.These results are reported for the cross-database test.
- PulseGAN produces a remarkable improvement over DAE in the cross-database IBI error distribution.
- In the illustrated cross-database example, IBIae errors are 65.01 ms for CHROM, 27.44 ms for DAE, and 23.11 ms for PulseGAN.The associated IBI vector and pulse waveform from PulseGAN are closer to their ground truths than those from DAE and CHROM.
CHROM DAE
Across within-database and cross-database cases, PulseGAN consistently outperforms DAE, while its cross-database results indicate strong generalization.
- CHROM DAE: Figure 10 compares Bland–Altman plots for CHROM versus DAE and DAE versus PulseGAN.
- CHROM DAE: PulseGAN’s cross-database results indicate strong generalization capability.
- CHROM DAE: PulseGAN consistently outperforms DAE in both within-database and cross-database cases.
- CHROM DAE: Figure 11 compares IBI sequences and rPPG pulse signals for a cross-database case.
V. CONCLUSION
The paper concludes that PulseGAN extracts high-quality rPPG pulse waveforms and consistently outperforms DAE and conventional methods in both evaluation settings. The authors also describe its integration potential and characterize the HRV analysis as preliminary.
- PulseGAN uses a GAN architecture with time-domain and spectrum-domain error losses to extract high-quality pulse waveforms from rough CHROM signals.
- PulseGAN consistently outperforms DAE and other conventional methods in both within-database and cross-database cases.
- The framework is easy to integrate with existing deep learning-based rPPG methods and may further improve their performance.
- The HRV characteristics calculated in this paper are relatively preliminary, although the attempts may expand rPPG application scope.