Source-linked AI summary
NeoTriFuse: Reliability-Aware Multimodal Fusion under Missingness Heterogeneity for Neonatal Mortality Risk Prediction
Jiyuan Tian, Qincheng Shen, Ye Lin, Yu Gao, Haohui Lu
TL;DR
Neonatal mortality prediction is difficult because severe outcomes are rare, measurements are noisy and incomplete, and risk patterns vary across infants. NeoTriFuse uses missingness-aware multimodal fusion of static, temporal, and summary evidence, with auxiliary length-of-stay training, and shows strong performance with reasonably stable nearby hyperparameter settings.
Problem
Neonatal mortality prediction remains difficult because severe outcomes are rare, physiologic measurements are noisy and incomplete, and identical vital-sign patterns can differ across infants.
Method
NeoTriFuse combines static perinatal risk, temporal vital dynamics, and patient statistical summaries while using missingness and coverage as reliability signals to guide fusion and an auxiliary length-of-stay objective.
Results
Removing the global transformer causes the largest F1 degradation, the summary branch produces a clear F1 drop, and mean F1 ranges from 0.6556 to 0.6736 across nine configurations.
Takeaways & Limitations
Missingness- and coverage-gated fusion preserves observation reliability as an explicit signal while combining physiologic trajectories with stable patient-level descriptors.
Takeaways & Limitations
The study is limited by its retrospective full-record benchmark setting, lack of matched external reimplementation, and limited independent cohorts with the same multimodal inputs and fold definitions.
Abstract
from arXiv · showhide
Neonatal mortality risk prediction from bedside monitoring data remains challenging due to extreme class imbalance, heterogeneous clinical risk factors, multi-scale temporal dynamics, and substantial missingness. We propose NeoTriFuse, a reliability-aware multimodal fusion framework for missingness-heterogeneous neonatal monitoring data. Unlike conventional multimodal approaches that treat missingness primarily as a preprocessing issue, NeoTriFuse models missingness as an explicit reliability signal that dynamically modulates modality contributions during fusion. The framework integrates static perinatal variables, local-global temporal encoders, and patient-level statistical summaries through reliability-guided gating mechanisms, while jointly optimizing mortality prediction and an auxiliary length-of-stay objective. NeoTriFuse achieves competitive performance, with an F1 score of 0.6736 +/- 0.0216 and an AUROC of 0.9454 +/- 0.0056. Ablation studies indicate that the local-global temporal architecture and patient-level summary branch contribute most substantially to predictive performance, while reliability-aware gating provides additional improvements on threshold-dependent metrics under heterogeneous observation completeness. Sensitivity analyses further suggest stable performance across nearby hyperparameter settings. Overall, the findings support reliability-aware multimodal fusion as a practical approach for neonatal mortality prediction under realistic clinical missingness conditions.
1 Introduction
Neonatal mortality prediction must combine heterogeneous clinical context with short- and long-term monitoring patterns while addressing rare outcomes and incomplete observations. NeoTriFuse tests reliability-aware multimodal fusion, using missingness and coverage to guide contributions from static, temporal, and summary representations.
- Neonatal risk assessment is difficult because severe outcomes are rare, measurements are noisy and incomplete, and identical vital-sign patterns vary by infant context.
- Short-term instability and slower multi-day changes require models that capture multiple temporal scales alongside stable perinatal context.
- NeoTriFuse combines static perinatal variables, temporal vital dynamics, and patient-level statistical summaries to address neonatal mortality prediction under missingness heterogeneity.
- Missingness and observation coverage are modeled as reliability signals that dynamically modulate static, temporal, and summary representations during fusion.
- The local–global temporal architecture combines convolutional layers for short-term physiological variation with transformer encoding for longer temporal dependencies.
- A dedicated summary branch preserves means, variability, extrema, last observation, sequence length, and coverage as clinically informative aggregate patterns.
2 Related Work
Related neonatal risk-prediction work combines auditable static variables, continuous physiologic monitoring, learned sequence representations, and carefully engineered summaries. Evidence from the PAS 2024 challenge and missingness research motivates explicit summaries, rigorous validation, and reliability-aware fusion.
- Traditional neonatal risk models use gestational age, birth weight, and early clinical condition, while reviews emphasize cohort definition, validation, missing-data handling, and interpretability.
- Heart-rate characteristics and HR/SpO2 dynamics provide complementary information for neonatal infection, mortality, respiratory complications, and related ICU forecasting tasks.
- PAS 2024 challenge results indicate that logistic regression, random forests, boosted trees, and neural networks can all remain competitive when features and thresholds are selected carefully.
- Clinical monitoring missingness may reflect workflow, monitoring duration, device availability, or clinical stability, motivating its use as an observation-reliability signal rather than simple noise.
3 Method
NeoTriFuse combines static perinatal variables, local-global temporal representations, and patient-level summaries, while using missingness-derived reliability signals to modulate branch contributions before mortality prediction. An auxiliary length-of-stay objective provides additional training supervision.
- Input representation: NeoTriFuse predicts mortality from temporal HR/SpO2 sequences, static perinatal variables, and patient-level summary statistics.The temporal input includes monitoring features and explicit missingness indicators; static variables include demographic and perinatal covariates.
- Static and Summary Branches: The dedicated summary branch combines feature-wise statistics and coverage descriptors with learned temporal patterns.Summary features include mean, standard deviation, minimum, maximum, last observed value, sequence length, and coverage ratio.
- Local–Global Temporal Encoding: Local convolutions capture short-term changes, while a transformer encoder models longer-range temporal dependencies.The two temporal representations are concatenated and projected into a shared temporal representation.
- Reliability-Aware Fusion: The reliability vector combines temporal coverage with static and summary completeness to generate branch-wise gates.These gates modulate temporal, static, and summary representations, reducing reliance on sparse or unreliable information.
- Prediction Heads and Objective: The model jointly trains mortality prediction and auxiliary length-of-stay regression using focal binary cross-entropy and SmoothL1 losses.The LOS objective supplies dense supervision and regularizes the shared representation, while its weight controls the tradeoff with mortality classification.
4 Experiments
The evaluation compares NeoTriFuse with tabular, sequence, and missingness-aware multimodal baselines using fixed official folds and repeated random seeds. It also tests architectural contributions and stability under changes to hidden dimension and LOS weighting.
- Research Questions: The experiments address baseline performance, component contributions, and stability under moderate hyperparameter changes.The research questions cover standard tabular and sequence comparisons, ablations of major components, and changes to hidden dimension and LOS loss weighting.
- Dataset and Evaluation Setting: The study uses the PAS 2024 NICU Mortality Prediction Challenge dataset with patient-level mortality evaluation over all available monitoring windows.Official fold assignments define the data split, and the cohort contains demographic information, outcome labels, and sequential HR/SpO2 summary windows.
- Evaluation Protocol: Performance is reported with F1, precision, recall, and AUROC across official folds and 10 random seeds.Seeds affect stochastic training operations, while fold assignments remain fixed across runs.
- Evaluation Metrics: Validation-selected thresholds are used for F1, precision, and recall, while AUROC separately measures threshold-independent ranking performance.This distinguishes ranking quality from classification balance at a selected operating point in the rare-mortality setting.
- Baselines: Comparisons include traditional machine-learning, neural sequence, missingness-aware, and multiscale temporal fusion baselines.The baseline set includes Logistic Regression, Random Forest, XGBoost, LightGBM, CatBoost, GRU, CNN, Transformer, GRU-D, TNformer-MP, and TIM-MSFL.
- Ablation Study: Ablations remove local and global temporal encoders, the tabular summary branch, reliability-aware fusion, the reliability vector, and the LOS auxiliary objective.The No Reliability Vector setting retains gating and fusion but replaces the missingness-derived reliability embedding with zeros.
5 Results
NeoTriFuse achieved the strongest overall classification profile among evaluated baselines, while ablations and sensitivity analyses identified the main contributing components and showed stability across nearby settings.
- Baseline comparison: NeoTriFuse achieved the highest F1, recall, and AUROC among traditional, neural sequence, and missingness-aware baselines.
- Baseline comparison: NeoTriFuse improved precision-recall balance in addition to ranking quality, whereas several baselines had competitive AUROC or precision but weaker recall-balanced F1.
- Ablation analysis: Removing the global transformer caused the largest F1 degradation, followed by removing the tabular summary branch.
- Ablation analysis: Removing the reliability vector preserved high AUROC but reduced F1 and recall, indicating its main contribution was to threshold-dependent decision balance.
- Sensitivity analysis: Across nine sensitivity configurations, mean F1 ranged from 0.6556 to 0.6736 and mean AUROC from 0.9382 to 0.9457, indicating reasonable stability near tested settings.The highest mean F1 occurred at hidden dimension 96 and LOS loss weight 0.3, but nearby settings remained close.
6 Conclusion
NeoTriFuse combines temporal, static, and patient-summary representations by using missingness and coverage as explicit reliability signals. The study is limited by retrospective benchmark evaluation and lacks matched external reimplementation and independent cohorts with equivalent inputs and fold definitions.
- Missingness and coverage gate temporal, static, and summary representations while preserving observation reliability as an explicit fusion signal.
- The study is limited by retrospective full-record benchmarking, absent matched external reimplementation, and limited independent cohorts with equivalent multimodal inputs and fold definitions.