PaperlayerView exact revision on arXiv

Source-linked AI summary

Pixel-wise Exposure for Highly Robust In-Vehicle Remote-PPG

Jieying Wang, Xinqi Cai, Caifeng Shan, Wenjin Wang

arXiv:2609.36607v1cs.CV

TL;DR

A single global exposure cannot keep all facial regions within the sensor’s usable intensity range when illumination exceeds its dynamic range, limiting suitable rPPG observations. PixExpo instead captures a cyclic sequence of exposures and fuses observations pixel by pixel. On MEX-Drive, it reduced HR MAE from 13.94 to 6.73 bpm and increased SR from 25.95% to 63.24% relative to manufacturer-default auto-exposure.

  • Problem

    When facial illumination exceeds the sensor’s usable dynamic range, a single global exposure cannot place all facial locations within range, reducing suitable rPPG observations.

    Source: A. Theoretical Foundation · p. 3

  • Method

    PixExpo sequentially captures frames under a cyclic exposure schedule and selects each pixel’s observation closest to a target intensity, a proxy for avoiding saturation and severe underexposure.

    Source: §I · p. 2, §III · p. 3, 2) Pixel-Wise Multi-Exposure Fusion · p. 4

  • Results

    HR MAE fell from 13.94 to 6.73 bpm and SR rose from 25.95% to 63.24% relative to manufacturer-default auto-exposure on MEX-Drive.

    Source: §VI · p. 11

  • Takeaways & Limitations

    Under tested conditions, PixExpo had better mean performance than global adaptive exposure, and one-hot selection outperformed Gaussian-decay and inverse weighting.

    Source: §VI · p. 11

    The cohort covered only MST levels 4–6, and the fixed six-channel configuration can be suboptimal in low illumination because it limits exposure time.

    Source: 4) Limitations and Future Work · p. 11

Abstract

from arXiv · show

Remote photoplethysmography (rPPG) offers a promising non-contact solution for heart rate monitoring, yet its real-world robustness is fundamentally limited by an inherent hardware limitation: existing camera exposure control paradigms, whether fixed or auto-exposure, impose a uniform exposure time across all pixels within a frame. In high-dynamic-range scenes such as automotive cabins with strong directional sunlight, this spatially invariant exposure constraint inevitably leads to localized facial overexposure or underexposure, irreversibly corrupting the subtle pulsatile signals essential for rPPG at the point of capture, a physical degradation that no downstream algorithm can recover. To overcome this bottleneck, we propose PixExpo (Pixel-wise Exposure), a "temporal-for-spatial" framework that sequentially captures frames under a predefined cyclic exposure schedule and performs non-iterative pixel-wise fusion. At each pixel location, PixExpo selects the observation closest to an rPPG-motivated target intensity. This criterion seeks to reduce local saturation and severe underexposure rather than optimize perceptual appearance. PixExpo requires no sensor modification but assumes programmable frame-level exposure control. We validate the proposed PixExpo framework using our newly introduced MEX-Drive dataset, comprising 48 participants under real-world driving conditions. Experimental results demonstrate that PixExpo outperforms manufacture-default auto-exposure methods, reducing the mean absolute error (MAE) by 7.21 bpm (from 13.94 to 6.73 bpm) and increasing the success rate by 37.29 percentage points (from 25.95% to 63.24%) across challenging driving scenarios.

I. INTRODUCTION

In-vehicle illumination changes expose a hardware-level weakness in rPPG: global exposure cannot keep every facial region suitably exposed, and lost pulse information cannot be recovered downstream. PixExpo replaces one exposure per frame with cyclic multi-exposure capture and pixel-wise fusion; the paper also introduces MEX-Drive.

I. INTRODUCTION

  • Global exposure can leave some facial pixels overexposed and others underexposed in high-contrast driving scenes, irreversibly corrupting rPPG signals.The passage identifies this as a hardware-level limitation that downstream extraction or enhancement cannot repair.Source: §I · p. 1
  • Existing exposure adjustment can add latency, while perception-oriented exposure and fusion criteria can disrupt the subtle inter-frame chrominance variations needed for rPPG.The paper identifies iterative fitting and feedback control as latency sources during rapid lighting transitions.Source: §I · p. 1
  • PixExpo uses a predefined cyclic exposure schedule and non-iterative pixel-wise fusion to reduce localized saturation and severe underexposure without sensor modification.It selects the most suitable RGB observation at each pixel from frames captured at different global exposure settings.Source: §I · pp. 1–2
  • MEX-Drive provides synchronized six-level multi-exposure facial videos and clinical-grade ECG data from 48 participants.The dataset is introduced to facilitate further research.Source: §I · p. 2

II. RELATED WORK

Prior exposure strategies address global brightness, visual quality, or rPPG fidelity, but leave a gap in spatially robust exposure control for physiological monitoring. PixExpo combines scheduled multi-exposure capture with rPPG-motivated pixel-wise fusion.

II. RELATED WORK

  • The review organizes exposure-control strategies by spatial robustness, real-time adaptability, and whether their objective targets physiological measurement or perceptual image quality.It frames robust rPPG deployment under challenging real-world illumination as an open problem.Source: §II · p. 2

A. Standard Auto-Exposure Algorithms

  • Conventional auto-exposure methods use image statistics or optimization objectives but still produce one exposure setting per camera, limiting correction of bright and shadowed facial regions.Their responsiveness varies: iterative optimization or feedback may lag abrupt lighting changes, while low-latency implementations are possible.Source: A. Standard Auto-Exposure Algorithms · p. 2

B. Multi-Exposure Image and Video Processing

  • Traditional multi-exposure fusion combines differently exposed observations, but image and video methods primarily target visual quality rather than preserving pulsation-induced temporal color changes.The approaches include spatial-domain, transform-domain, and learning-based image fusion, as well as video enhancement using exposure diversity.Source: B. Multi-Exposure Image and Video… · p. 2

C. rPPG-Specific Global Auto-Exposure

  • rPPG-specific exposure controllers target signal fidelity but remain spatially global, so they cannot independently correct locally sunlit and shadowed facial regions.They optimize a single exposure for average facial-ROI conditions, even when the intra-scene dynamic range exceeds the sensor’s native capacity.Source: C. rPPG-Specific Global Auto-Exposure · p. 3

D. Summary and Positioning of PixExpo

  • PixExpo combines a predefined cyclic exposure schedule with rPPG-motivated pixel-wise fusion, addressing local exposure variation without iterative exposure search.Unlike traditional MEF, it prioritizes physiological integrity of skin pixels over visual aesthetics.Source: D. Summary and Positioning of PixExpo · p. 3

III. METHODOLOGY

PixExpo approximates spatial exposure adaptation by acquiring a sequence of globally exposed frames and selecting or weighting observations at each pixel. The theoretical foundation explains why a single global exposure cannot keep all facial pixels within a usable intensity range.

III. METHODOLOGY

  • PixExpo approximates a spatial exposure map by sequentially acquiring globally exposed frames and selecting or weighting observations at each pixel.The framework is designed for spatially nonuniform in-vehicle illumination.Source: §III · p. 3

A. Theoretical Foundation

  • Pulse-related intensity variation is lost through clipping or can become comparable to sensor and quantization noise at very low exposure.An ideal spatial exposure map would bring each location’s DC intensity closest to a target, which PixExpo approximates through sequential global exposures and computational pixel-wise fusion.Source: A. Theoretical Foundation · p. 3
  • When facial illumination spans more than the sensor’s usable intensity range, no single exposure can make every facial location photometrically suitable for rPPG.A global setting therefore reduces the number of skin pixels providing suitable observations.Source: A. Theoretical Foundation · p. 3

B. The Proposed PixExpo Framework

PixExpo pairs SDK-based cyclic multi-exposure acquisition with pixel-wise fusion. It ranks each pixel’s exposure observations by distance from a target intensity and combines them into a fused frame.

1) SDK-Based Multi-Exposure Acquisition:

  • Each acquisition cycle captures six consecutive frames at increasing exposure times, repeating a predefined schedule without iterative estimation or feedback.Sequential capture samples a range of facial brightness levels; timing and spatial misalignment are considered in later analysis.Source: 1) SDK-Based Multi-Exposure Acquisition · p. 4

2) Pixel-Wise Multi-Exposure Fusion:

  • The target-intensity deviation is a photometric proxy that favors observations less prone to saturation or severe underexposure, not a direct measure of physiological signal quality.The target is fixed at 140 on the processed 8-bit intensity scale.Source: 2) Pixel-Wise Multi-Exposure Fusion · p. 4
  • At each pixel, PixExpo ranks the exposure candidates by deviation from the target and uses the ranked observations to compute a fused RGB pixel.The highest-ranked candidate is the observation closest to the target intensity.Source: 2) Pixel-Wise Multi-Exposure Fusion · p. 4

1) One-hot weighting:

The default one-hot strategy keeps only the exposure candidate closest to the target, while alternative weighting strategies blend candidates.

1) One-hot weighting:

  • One-hot weighting retains only the highest-ranked candidate and copies its complete RGB vector into the fused frame.This is PixExpo’s default because it avoids contributions from suboptimal exposures, unlike strategies that blend candidates for spatial smoothness.Source: 1) One-hot weighting · p. 4, 3) Dynamic inverse weighting · p. 4

3) Dynamic inverse weighting:

  • Dynamic inverse weighting blends exposure candidates using weights based on photometric deviation, with epsilon preventing division by zero.For the Gaussian strategy, the index k refers to rank after sorting by deviation, not the original exposure channel.Source: 3) Dynamic inverse weighting · p. 4

C. Potential Artifacts of Discrete Exposure Selection

The paper examines artifacts from discrete exposure selection and describes MEX-Drive’s collection and acquisition setup. The dataset covers varied driving conditions, using synchronized video and ECG recordings.

C. Potential Artifacts of Discrete Exposure Selection

  • Exposure switching can create spatial discontinuities and temporal variation; band-pass filtering cannot remove switching components within the heart-rate frequency band.Temporal normalization can partly reduce regional DC differences, and spatial aggregation can attenuate localized discontinuities.Source: C. Potential Artifacts of Discrete… · p. 4

A. Dataset: MEX-Drive

  • MEX-Drive was collected to test multi-exposure acquisition and fusion under spatially nonuniform in-vehicle illumination, using a distinct protocol focused on high-dynamic-range driving scenes.It uses the same hardware configuration and participant pool as ExpDrive.Source: A. Dataset: MEX-Drive · p. 4, Figure 2 · p. 5

1) Experimental Design and Protocol:

  • The 48 licensed drivers were aged 20–65 and all Asian, with MST skin tones 4–6; the protocol included varied weather, roads, routes, and natural driver activities.Skin-tone variability was not a focus, and the study did not include a mechanism specifically addressing it; mean reference HR was 76.2 ± 10.8 bpm.Source: 1) Experimental Design and Protocol · pp. 4–5

2) Acquisition Hardware:

  • Two synchronized RGB cameras captured video while a clinical-grade monitor recorded chest-lead ECG at 500 Hz, with timestamps enabling temporal alignment.For the multi-exposure camera, exposure time varied between frames while automatic exposure and gain controls were disabled.Source: 2) Acquisition Hardware · p. 5

3) Acquisition Configurations:

The acquisition configurations compare PixExpo with fixed, full-frame auto-exposure, perception-oriented fusion, and global adaptive exposure. Evaluation uses ECG-referenced heart-rate accuracy and rPPG signal quality.

3) Acquisition Configurations:

  • All strategies produced 15-fps outputs, but their acquisition rates and nominal exposure limits differed: 15 fps and about 66.6 ms for full-frame AE, 45 fps and about 22.2 ms for triplet fitting, and 90 fps and about 11.1 ms for the six-stage sequence.The six-stage sequence supplied recordings for fixed exposure, Mertens fusion, and PixExpo.Source: 3) Acquisition Configurations · p. 5
  • The baseline comparison tested PixExpo against full-frame AE, fixed exposure, and Mertens fusion, with Mertens inputs registered and PixExpo operating on the unregistered sequence.A separate concurrent-camera experiment compared PixExpo with face-ROI and skin-ROI AE controllers in seven additional participants.Source: 3) Acquisition Configurations · p. 5
  • A second scenario compared PixExpo with triplet-frame global adaptive exposure under the same driving conditions.One camera implemented the method proposed in [35] while the other acquired the six-stage cyclic sequence.Source: 3) Acquisition Configurations · p. 5

B. Evaluation Metrics

  • Evaluation uses ECG-referenced HR accuracy and rPPG signal quality rather than conventional HDR image-quality metrics; POS is the default estimator.The metrics include MAE, success rate, and SNR; additional pretrained rPPG models were evaluated without MEX-Drive training or fine-tuning.Source: B. Evaluation Metrics · p. 6

V. RESULTS AND DISCUSSION

PixExpo outperformed conventional exposure strategies on participant-level rPPG measures and cross-dataset tests, and led ROI-based auto-exposure across POS, CHROM, and ICA in an additional experiment.

1) Comparison with Conventional Exposure Strategies:

2) Additional Comparison with ROI-Based AE:

B. Scenario II: Comparison with Fixed and Adaptive Global Exposure

In Scenario II, PixExpo improved overall results over fixed and global adaptive exposure, with stronger gains in sunny conditions; in rainy or overcast conditions, global adaptive exposure performed slightly better.

1) Overall Performance:

  • PixExpo achieved MAE 6.30 ± 3.39 bpm versus 7.79 ± 3.69 bpm for global adaptive exposure and 9.75 ± 4.61 bpm for fixed exposure; its SR was 65.24 ± 18.04%.Source: 1) Overall Performance · p. 7, Figure 6 · p. 8
  • PixExpo improved MAE and SR for most participants versus global adaptive exposure, but several participants had higher MAE or lower SR with PixExpo.Source: 1) Overall Performance · pp. 7–8

2) Performance under Different Illumination Conditions:

3) Representative Case Studies:

Representative cases show PixExpo improving HR estimates under several driving conditions, while low-light results expose a tradeoff in the fixed six-exposure configuration. A controlled low-light experiment found stronger mean performance with two exposure channels, but its seven participants do not establish a universal optimum.

3) Representative Case Studies:

  • In direct sunset, PixExpo achieved an MAE of 2.44 bpm and SR of 90.2%, versus 9.84 bpm and 52.9% for fixed exposure and 6.79 bpm and 70.6% for global adaptive exposure.PixExpo selected shorter-exposure observations in strongly illuminated regions, reducing local clipping.Source: 3) Representative Case Studies · p. 9
  • With head rotation in sunny conditions, PixExpo achieved an MAE of 3.56 bpm and SR of 78.7%, versus 12.5 bpm and 27.1% for fixed exposure and 9.65 bpm and 35.7% for global adaptive exposure.The fused frame showed exposure-selection boundaries, but these artifacts did not dominate the recovered HR trace in this example.Source: 3) Representative Case Studies · p. 9
  • Under stable overcast conditions, PixExpo had the lowest MAE (1.59 bpm) and highest SR (93.6%) among the three strategies in this example.Fixed exposure had an MAE of 6.80 bpm and SR of 71.7%; global adaptive exposure had an MAE of 3.25 bpm and SR of 81.6%.Source: 3) Representative Case Studies · p. 9
  • In low-light rain, PixExpo’s MAE was 4.89 bpm and SR 74.3%, versus 3.53 bpm and 77.7% for fixed exposure and 3.22 bpm and 78.7% for global adaptive exposure.At 90 fps, the nominal maximum exposure time was 11.1 ms, limiting photon accumulation.Source: 3) Representative Case Studies · p. 9

4) Effect of Exposure Channel Count under Low Light:

  • With N = 2, mean MAE was 3.75 bpm, mean SNR 4.92 dB, and mean SR 90.15%; with N = 6, the respective values were 6.99 bpm, 1.19 dB, and 67.76%.The authors relate the six-channel configuration’s low-light disadvantage to the shorter integration time imposed by its higher total frame rate.Source: Table III · p. 9, 4) Effect of Exposure Channel Count… · p. 9
  • The channel-count comparison was a controlled experiment with seven participants and does not establish N = 2 as universally optimal for low-light conditions.Each channel-count configuration was newly acquired rather than generated by subsampling six-channel recordings.Source: 4) Effect of Exposure Channel Count… · p. 9

C. Ablation Study on Fusion Weighting Strategies

One-hot weighting produced the strongest reported fusion metrics, while PixExpo maintained a dominant spectral component near reference HR during transient disturbances. Registration reduced ghosting in conventional fusion but showed no measurable rPPG benefit for PixExpo under the tested conditions.

C. Ablation Study on Fusion Weighting Strategies

  • One-hot weighting achieved the lowest MAE (6.73 ± 3.56 bpm) and highest SR (63.24 ± 18.95%) and SNR (0.60 ± 3.27 dB) across 48 participants.It outperformed Gaussian-decay and inverse weighting for all three metrics in corrected post-hoc tests.Source: Table IV · p. 9, C. Ablation Study on Fusion Weighting… · p. 9

1) Robustness to Motion and Illumination Changes:

  • During head motion, vehicle vibration, and rapid illumination change, PixExpo maintained a dominant spectral component close to reference HR, unlike fixed exposure and camera auto-exposure.The six exposure levels were acquired at 90 fps, yielding a complete cycle of about 66.7 ms; spatial averaging over the skin ROI may attenuate small local misalignments.Source: Figure 10 · p. 10, 1) Robustness to Motion and… · p. 10

2) Effect of Inter-frame Registration:

  • For PixExpo, registration yielded no significant differences in MAE, SR, or SNR in the 14-participant subset; mean HR MAE was 7.26 ± 3.83 bpm without and 7.41 ± 3.83 bpm with registration.Registration visibly reduced ghosting in Mertens fusion, while PixExpo outputs remained similar before and after registration.Source: Figure 11 · p. 10, 2) Effect of Inter-frame Registration · p. 10

3) Temporal Characteristics of Exposure Selection:

Exposure switching was more frequent near facial boundaries and high-gradient regions, but its descriptors lacked the pronounced narrowband peak seen in the recovered rPPG signal. The paper also identifies low-light and skin-tone scope limits, while concluding that PixExpo improves in-vehicle rPPG when frame-level exposure control and a sufficiently high acquisition rate are available.

3) Temporal Characteristics of Exposure Selection:

  • The PSR map showed more frequent exposure switching near facial boundaries and high-gradient regions, while FSR and mean selection index lacked a pronounced narrowband peak comparable to the recovered rPPG signal’s 1.52 Hz peak.The recovered signal peak was close to the mean reference HR of 1.56 Hz; the observations suggest switching was not a dominant source of pulse-like artifacts in this recording.Source: 3) Temporal Characteristics of… · pp. 10–11, Figure 12 · p. 11

4) Limitations and Future Work:

  • The fixed six-level stack requires 90-fps acquisition for 15-fps output, limiting maximum integration time to about 11.1 ms and potentially reducing photon accumulation and temporal SNR in dark scenes.Future work will investigate illumination-aware adjustment of the number and distribution of exposure levels.Source: 4) Limitations and Future Work · p. 11
  • The cohort covered only MST levels 4–6, so PixExpo’s performance across other skin-tone groups remains to be established.The paper calls for broader skin-tone representation and skin-tone-stratified evaluation.Source: 4) Limitations and Future Work · p. 11

VI. CONCLUSIONS

  • On MEX-Drive’s 48 participants, PixExpo reduced HR MAE from 13.94 to 6.73 bpm and increased SR from 25.95% to 63.24% relative to manufacturer-default auto-exposure.Additional experiments reported better mean performance than global adaptive exposure under tested conditions and favored one-hot over Gaussian-decay and inverse weighting.Source: §VI · p. 11
  • The conclusion identifies frame-level exposure control and a sufficiently high acquisition rate as conditions for PixExpo’s practical in-vehicle use.Future validation is proposed across additional cameras and more severe motion conditions.Source: §VI · p. 11
Loading 2609.36607v1…