Robust tracking of respiratory rate in high-dynamic range scenes using mobile thermal imaging

Motivation

• Conventional chest belts and oronasal probes require contact, can be uncomfortable, and may produce incorrect readings through motion artifacts.Contact can also be unacceptable for some medical and chronic conditions.Source: §1 · pp. 1–2
• The authors target thermal respiration tracking for mobile settings, which prior work had not investigated despite thermal cameras becoming small, lightweight, and smartphone-compatible.Thermal imaging is passive and does not require lighting sources.Source: §1 · p. 2

Research gap

• Prior thermal respiration methods track nostril temperature changes, but motion artifacts challenge ROI tracking and existing trackers lack robustness to environmental thermal dynamics.The cited prior work had focused on stationary indoor settings.Source: §1 · pp. 2–3

Contributions and evaluation

• The approach combines adaptive thermal quantization, enhanced nostril tracking, and Thermal Voxel signals, and the authors report outperforming existing approaches in three laboratory and real-world experiments.Experiments covered controlled breathing and motion, sedentary activity, and fully mobile indoor and outdoor activity.Source: §1 · p. 3

2. Background and key challenges

• Low-cost, small-sized thermographic systems make mobile respiration monitoring plausible, but general environments pose challenges beyond controlled indoor laboratories.The paper frames reliable monitoring during everyday activity as a key goal.Source: §2 · p. 3

Challenge 1: High thermal dynamic range scenes

• Facial temperature changes with ambient conditions, so a fixed temperature range can lose facial morphology when temperatures fall outside that range.The cited example uses a fixed 28°C to 38°C range that cannot adapt when outdoor facial temperatures fall below 28°C.Source: Challenge 1: High thermal dynamic… · p. 3, Figure 1 · p. 4
• Expanding the color-mapped temperature range captures more values but reduces digital resolution, contrast, and sensitivity to subtle signals.The passage motivates continuously adaptive color mapping for dynamic thermal changes.Source: Challenge 1: High thermal dynamic… · p. 4

Challenge 2: Combined motion artifact and air exchanges in the nostril

• Unconstrained motion and breathing deform nostril shape, while prior ROI tracking evaluations largely used constrained indoor motion.The authors identify a need for tracking robust to ROI deformation.Source: Challenge 2: Combined motion artifact… · pp. 4–5
• Average nostril temperature can miss weak, localized breathing changes and is sensitive to ROI shifts, wind, viewing direction, and low spatial resolution.The paper argues for a respiratory feature robust to these factors.Source: Challenge 2: Combined motion artifact… · p. 5

3. Methods: robust respiration tracking algorithm

• The system chains Optimal Quantization, Thermal Gradient Flow, and Thermal Voxel-based Respiratory Rate Estimation to map temperatures, track nostrils, and extract respiratory signals.The procedure adapts color mapping to temperature distributions, uses gradient magnitude and point tracking, then integrates thermal voxels inside the nostril.Source: §3 · p. 5, Figure 2 · p. 5

3.1 Optimal Quantization for high thermal dynamic range scenes

• For each frame, Optimal Quantization removes temperatures beyond 1.96 standard deviations, then iteratively selects a threshold for mapping facial temperatures across changing thermal ranges.The thresholding process adapts to time-varying histograms and does not require background pixels in all four image corners.Source: §3.1 · p. 6

3.2 Thermal Gradient Flow: Nostril-region tracking

• Thermal Gradient Flow tracks nostril features on gradient-magnitude maps with Median Flow and KLT, using gradient-based cross-correlation to recover the ROI when features are lost.The gradient map sharpens the nostril–ala boundary, while the ROI can be reset when tracked-point counts fall below a threshold.Source: §3.2 · pp. 7–8

3.3 Respiratory rate estimation through Thermal Voxel integration

• Thermal Voxel estimation integrates voxels within a concave volume to capture breathing-induced nostril heat changes rather than relying on spatial average temperature.The volume boundary uses a temporal moving average of spatial mean temperatures to stabilize it and account for global thermal changes.Source: §3.3 · p. 8
• Respiratory rate is estimated by filtering the voxel feature to 0.1–0.85 Hz and selecting the frequency with maximum power spectral density from short-time autocorrelation.A sudden change in thermal-distribution skewness can indicate a misplaced ROI whose value should be rejected.Source: §3.3 · pp. 8–10

• The protocols were designed to span different levels of thermal dynamics and motion artifacts for an in-depth evaluation of the proposed methods.Source: §4 · p. 10
• The datasets comprise thermal imaging of respiration collected under protocols with differing levels of complexity.Source: §4 · p. 10
• Fully tracked nostril-ROI sequences from the collected datasets are open to the research community.Source: §4 · p. 10

• The FLIR One for Android recorded thermal images at 160x120 spatial resolution and less than 9 frames per second across an 8–14 µm spectral range.The camera’s emissivity setting was 0.98, the stated value for human skin.Source: §4.1 · p. 10
• A chest-belt sensor recorded reference respiration waveforms at 256Hz, and Thermal Voxel sequences were spline-interpolated to 256Hz for direct comparison.Source: §4.1 · p. 10
• The proposed methods were implemented in MATLAB 2015b and validated on a Windows 7 desktop with a Core i3-4160T processor and 8GB RAM.Source: §4.1 · p. 10

4.2 Procedure and dataset

• The three protocols were independently designed for distinct purposes, and each dataset came from a new participant group.Source: §4.2 · p. 10
• The first protocol included five participants; the other two used slightly larger groups because greater variability in breathing patterns was expected, with only healthy participants and mild physical activity permitted.Source: §4.2 · p. 11
• The experiments received ethics approval, and the fully tracked ROI sequences were made publicly available.Source: §4.2 · p. 11

4.2.1. Dataset 1: controlled respiration in environments with non-constant temperature

• The first protocol used five healthy adults to evaluate guided breathing in four winter locations, while participants remained as still as possible.Each participant followed 10, 15, and 30 breaths/min patterns; recordings totaled approximately 80 minutes.Source: §4.2.1 · p. 11, Figure 5 · p. 11

4.2 Procedure and dataset

• The second protocol recorded spontaneous breathing during three unconstrained two-minute desk activities in a quiet laboratory room in summer.Ten participants generated 60 minutes of recordings with natural movement and changing ambient temperature, including head rotations that could temporarily hide the nostrils.Source: §4.2.1 · p. 12, Figure 6 · p. 12

4.2.3. Dataset 3: unconstrained respiration in fully mobile context and varying thermal dynamic range scenes

• The third protocol captured fully mobile breathing during indoor and outdoor physical activities using a headset-shaped camera rig.Eight participants completed walking, movement, and stair tasks indoors and walking at different paces outdoors; the dataset totaled approximately 96 minutes.Source: §4.2.3 · pp. 12–13, Figure 6 · p. 12

4.3.1. Evaluating ROI-tracking performances

• Tracking evaluation compared Optimal Quantization-based Thermal Gradient Flow with mean-shift, Sparse Representation, and Median Flow, using fully tracked frames as the performance measure.Static-quantization baselines used a fixed range of ±5°C from the first-frame average face temperature; repeated-measures ANOVA and Bonferroni-adjusted post-hoc tests compared trackers.Source: §4.3.1 · p. 13

4.3.2. Evaluating respiration rate tracking performances

• Respiration estimates from Thermal Voxel were compared with average nostril-ROI temperature and synchronized to chest-belt references using maximum-amplitude cross-correlation.The evaluation used automatically tracked ROI sequences.Source: §4.3.2 · p. 13
• Agreement between respiration-rate measurements was assessed with Bland–Altman plots and RMSE using 20-second windows with 15 seconds of overlap.Figure 7 presents the synchronization procedure and respiration-related goodness probability as the rSQI.Source: §4.3.2 · p. 14, Figure 7 · p. 14

4.4 Respiratory signal quality index (rSQI)

• Pr is the respiration-related goodness probability used as the respiratory Signal Quality Index (rSQI) to identify poor-quality extracted signals.The paper adapts a goodness metric based on physiological signal power in a frequency range of interest.Source: §4.4 · p. 14
• The method divides energy in the expected breathing-rate band by total energy within half the sampling frequency to satisfy a statistical probability condition.The expected frequency boundaries are exemplified as 0.1 Hz and 0.85 Hz; the sampling frequency is exemplified as 256 Hz.Source: §4.4 · p. 14
• In Figure 7(b), the area between dotted lines represents the numerator of Eq. (12).Source: §4.4 · p. 15

Dataset 1: Overall results

• 100.00% tracking success (SD=0.0) for Thermal Gradient Flow in Dataset 1 Places B–D exceeded Mode Seeking, Sparse Representation, and Median Flow in high-dynamic outdoor scenes.The methods had mean success rates of 47.25%, 71.31%, and 76.35%, respectively.Source: Figure 8 · p. 15, Dataset 1: Overall results · p. 16

5.1 Nostril-region tracking performance

• Motion artifacts were greater during sedentary and physical activity than in controlled conditions, with head-direction changes and rig oscillation producing different movement patterns.Figure 9 quantifies motion artifacts as Euclidean distance, in pixels, from the first-frame nostril-ROI origin.Source: Figure 9 · p. 15, §5.1 · p. 15

Dataset 1: Overall results

• In Dataset 1, environment, algorithm, and their interaction significantly affected tracking success, but post-hoc comparisons within each variable found no significant differences.For outdoor Places B–D, Thermal Gradient Flow significantly exceeded Mode-Seeking and Sparse Representation; indoor algorithm differences were not significant.Source: Dataset 1: Overall results · p. 16, Table 1 · p. 16, Table 2 · p. 16

5.2 Respiratory rate estimation performance

• Thermal Voxel signals had higher respiration-related goodness than averaging in Datasets 2 and 3, while Dataset 1 distributions were similar.For Dataset 3, Thermal Voxel had Pr M=0.9640 and the belt ground truth M=0.9170; the ground truth was less clear except during the mostly stationary segment A.Source: §5.2 · pp. 17, 19, Figure 10 · p. 18
• 0.459 bpm versus 0.993 bpm RMSE in Dataset 1, 1.03 bpm versus 2.11 bpm in Dataset 2, and 2.45 bpm versus 2.59 bpm in Dataset 3 favored Thermal Voxel integration over averaging.Dataset 2 and 3 agreement analyses used samples with rSQI at least 0.9825 because the belt sensor was not reliable as ground truth.Source: Figure 11 · p. 19, Dataset 1: Bland-Altman and RMSE… · p. 19, Dataset 2 and 3: Bland-Altman and RMSE… · p. 20

Dataset 1: Bland-Altman and RMSE analysis

• Dataset 1 respiratory-rate estimates had RMSE 0.459 bpm with Thermal Voxel integration and 0.993 bpm with averaging.Thermal Voxel integration also had narrower 95% limits of agreement (-0.7956 to 0.9721 bpm) than averaging (-1.8670 to 2.0179 bpm).Source: Figure 11 · p. 19, Dataset 1: Bland-Altman and RMSE… · p. 19

Dataset 2 and 3: Bland-Altman and RMSE analysis

• Dataset 2 RMSE fell from 2.11 bpm with averaging to 1.03 bpm with Thermal Voxel integration, while Dataset 3 RMSEs were 2.59 bpm and 2.45 bpm, respectively.Because belt-sensor ground-truth quality was poorer in Datasets 2 and 3, the agreement analysis selected measurements with Pr ≥ 0.9825.Source: Dataset 2 and 3: Bland-Altman and RMSE… · p. 20, Figure 13 · p. 21, Figure 14 · p. 21

6. Discussion

• The proposed Optimal Quantization and Thermal Gradient Flow methods address high thermal dynamic ranges and motion-amplified tracking challenges beyond controlled laboratory settings.The discussion frames this as extending earlier stationary and indoor thermography work to mobile respiration tracking.Source: §6 · pp. 21–22
• During physical activity with high thermal dynamic range and motion artifacts, Thermal Gradient Flow significantly outperformed all other nostril-tracking methods.In still, high-dynamic-range scenes it also outperformed the alternatives; in sedentary motion, the three newer methods performed significantly better than Mode Seeking, without significant differences among them.Source: §6 · p. 22
• The experiments do not cover sudden ambient-temperature transitions or varying humidity, such as leaving a heated building in winter or being in a pool or sauna.Source: §6 · p. 22
• Thermal Voxel integration generally outperformed temperature averaging, including in controlled guided-breathing tests; under motion artifacts, both methods had relatively low agreement with the chest-belt sensor.For comparisons in the motion-artifact datasets, the authors used rSQI to compare each method with ground truth.Source: §6 · pp. 22–23
• The mobile camera has relatively low spatial and temporal resolution, unsteady sampling that may affect extremely fast breathing, and occasional lens-related temperature outliers.The authors state that outlier reduction can minimize the effects of extreme values, while higher resolution may further enhance tracking.Source: §6 · p. 23

7. Conclusion

• The approach combines Optimal Quantization, Thermal Gradient Flow, and Thermal Voxel integration; the authors report significantly better nostril tracking than three state-of-the-art algorithms in high-dynamic-range scenes.They report that respiratory-rate tracking achieved highly robust accuracy, suggesting potential for real-world situations.Source: §7 · p. 23

Funding

• The work acknowledges support from the University College London Overseas Research Scholarship (UCL-ORS).Source: Funding · p. 23