PaperlayerView exact revision on arXiv

Source-linked AI summary

A Leakage-Aware Multimodal Evaluation Framework for Early Intraoperative Acute Kidney Injury Prediction

Quang Minh Nguyen, Duc Minh Le, Ho Nhat Minh Nguyen, Thuy Quynh Nguyen, Trong Nghia Nguyen

arXiv:2609.26848v1cs.LGcs.AI

TL;DR

Early intraoperative AKI prediction is challenging, and evidence under strict leakage control remains incomplete. This study evaluates waveform-only SynerT and multimodal, leakage-safe extensions using information from the first 60 minutes. SynerT-Stack achieved the strongest overall discrimination, while recalibration improved absolute-risk behavior of the multimodal models.

  • Problem

    The literature incompletely characterizes early waveform trajectories versus structured renal-risk context under strict leakage-control settings.

    Source: §2 · p. 3

  • Method

    The study evaluates waveform-only SynerT, a multimodal extension with structured context, and leakage-safe stacking within a strict early-prediction framework.

    Source: §1 · p. 2, §3.5 · p. 7, §3.5.2 · p. 8

  • Results

    SynerT-Stack achieved the highest AUROC, AUPRC, and F1-max across models, while cross-fitted Platt recalibration reduced ECE from 0.350 to 0.005 for SynerT-MM and from 0.316 to 0.027 for SynerT-Stack.

    Source: §4.1 · p. 9, §4.4 · p. 11

  • Takeaways & Limitations

    The findings support evaluating multimodal context, calibration, and transport robustness rather than relying on waveform-model complexity alone.

    Source: §5 · p. 14

    The outcome was creatinine-only, without urine-output ascertainment, and the external cohort was small and unmatched, so larger perioperative validation remains necessary.

    Source: §4.2 · p. 10, §5 · p. 14

Abstract

from arXiv · show

Postoperative acute kidney injury (AKI) after major non-cardiac surgery carries substantial morbidity, yet early intraoperative risk stratification remains difficult. In this retrospective cohort study, we propose SynerT, a waveform-only hybrid temporal backbone that combines a causal dilated TCN with a hierarchy of dilated recurrent layers to encode early intraoperative physiologic trajectories for AKI risk prediction. Building on SynerT, we further design two model variants that extend the backbone with structured clinical context: SynerT-MM, a late-fusion multimodal extension that integrates hemodynamic burden summaries and preoperative covariates, and SynerTStack, a leakage-safe stacked ensemble that combines cross-validated predictions from SynerT-MM with strong tabular baselines at the meta-learning stage. All models are evaluated under a strict leakage-aware framework on VitalDB, a high-fidelity perioperative database, with prediction restricted to information available within the first 60 intraoperative minutes. Among 2,413 waveform-usable cases (180 AKI-positive; 7.46% prevalence), SynerT fell well below strong structured-data baselines, demonstrating that waveform-only temporal modeling is insufficient under strict early constraints. SynerTMM recovered discrimination by incorporating hemodynamic burden summaries and preoperative covariates, and SynerT-Stack achieved the best overall performance across AUROC, AUPRC, and F1-max. Cross-fitted Platt recalibration substantially corrected calibration defects in both multimodal variants, and decision-curve analysis confirmed the recalibrated stacked model delivered the strongest net clinical benefit across low-to-intermediate thresholds.

1 Introduction

The study addresses gaps in strict leakage control and comparisons of waveform-only versus multimodal early AKI prediction. It evaluates a leakage-aware framework centered on SynerT and tests whether structured context and stacking improve prediction.

1 Introduction

  • The study presents a disciplined evaluation framework rather than claiming a fundamentally novel architecture, with SynerT as its waveform-only temporal backbone.The motivation is the uneven evidence base for early AKI prediction, including limited strict leakage control and rigorous comparison of waveform-only and multimodal designs.Source: §1 · p. 2
  • The study establishes a leakage-aware framework restricting dynamic inputs to the first 60 intraoperative minutes, using fold-specific preprocessing, and building stacked predictions only from out-of-fold outputs.Its evaluation includes calibration, decision-curve analysis, and a cross-setting transport stress test.Source: §1 · p. 2
  • Waveform-only SynerT underperforms strong structured-data baselines, while SynerT-MM and especially SynerT-Stack yield stronger discrimination and clinical utility after cross-fitted Platt recalibration.The study hypothesizes that integrating structured perioperative context via leakage-safe ensembling is needed for superior discrimination and calibrated risk estimation.Source: §1 · p. 2

2 Related Work

Prior work supports perioperative AKI prediction and multimodal information, but strong tabular models remain difficult to outperform. The paper addresses the incomplete characterization of waveform-only signal versus structured renal-risk context under strict leakage control.

2 Related Work

  • The study operationalizes postoperative AKI using creatinine criteria because urine-output data were unavailable.It notes KDIGO as the standard basis for AKI definition and cites interactions among baseline susceptibility, operative stress, and postoperative assessment as motivation for combining risk context with physiologic trajectories.Source: §2 · p. 3
  • Peng et al., Park et al., and Lee et al. describe informative temporal signals, complementary preoperative and physiologic information, and competitive boosting-based tabular models, respectively.The present work asks how much performance can be recovered by adding structured context and leakage-safe ensembling to waveform-first modeling.Source: §2 · p. 3
  • Prior literature notes limited deployment, subpopulation-robustness, and clinically meaningful evaluation evidence for multimodal prediction, while emphasizing calibration and decision-curve analysis.The discussion draws on multimodal precision-health literature, perioperative AKI concerns, and prediction-evaluation work.Source: §2 · p. 3
  • Prior studies leave unclear how much early predictive signal comes from waveform trajectories versus structured renal-risk context under strict leakage-control settings.This gap motivates evaluating waveform-only modeling, multimodal fusion, leakage-safe stacking, calibration, and transport robustness together.Source: §2 · p. 3

3 Methodology

The methodology frames SynerT as the waveform-only temporal branch and extends it with structured context and leakage-safe ensembling. The cohort and outcome are defined using VitalDB cases and creatinine-based postoperative AKI.

3 Methodology

  • SynerT is the waveform-only temporal branch, while SynerT-MM adds structured context and SynerT-Stack adds leakage-safe ensembling.The pipeline overview includes temporal waveform preprocessing and encoding, structured feature embedding, late fusion, and a meta-learner ensemble.Source: Figure 1 · p. 4

3.1 Notation

  • The notation distinguishes case-level temporal sequences and structured static covariates used in the prediction setup.It also defines MAP as the intraoperative mean arterial pressure series and specifies hypotension thresholds of 65 and 60.Source: §3.1 · p. 4

3.2 Data Source, Cohort, and Outcome Definition

  • The primary creatinine-labeled cohort had 3,242 cases; 2,413 passed waveform availability and quality screening, including 180 AKI-positive cases (7.46%).The study used VitalDB, which contains synchronized waveform, numeric, clinical, and laboratory records.Source: §3.2 · p. 4
  • The outcome was creatinine-based postoperative AKI, using the most recent preoperative creatinine within 30 days and postoperative creatinine within 7 days.The target was aligned with the creatinine component of KDIGO.Source: §3.2 · pp. 4–5

3.3 Input Representation

Dynamic inputs are minute-one intraoperative physiologic signals represented with explicit observation indicators. Structured context adds preoperative covariates and MAP burden summaries at two hypotension thresholds.

3.3.1 Dynamic Physiologic Signals

  • The dynamic input uses seven physiologic channels from the first 60 intraoperative minutes, resampled to 1 Hz, with at most 3,600 time steps per case.Each time step concatenates resampled values with matched observation indicators; preprocessing preserves missingness and uses training-fold-only scaling.Source: §3.3.1 · p. 5

3.3.2 Structured Context and Hemodynamic Burden

  • Structured context combines preoperative demographic, laboratory, and comorbidity covariates with MAP burden summaries at 65 and 60 mmHg thresholds.Burden features include area under the MAP-deficit curve, episode counts, longest hypotensive run, and short-term MAP variability; they are treated as exposure descriptors, not causal threshold claims.Source: §3.3.2 · pp. 5–6

3.4 SynerT: Waveform-Only Temporal Backbone

SynerT uses only the early waveform value-indicator sequence, with no static covariates or hemodynamic burden summaries. Its temporal backbone combines channel projection, causal dilated convolutions, a dilated recurrent hierarchy, and masked pooling to produce an AKI risk probability.

3.4 SynerT: Waveform-Only Temporal Backbone

  • SynerT takes the waveform value-indicator sequence as its sole input and excludes static clinical covariates and hemodynamic burden summaries.The backbone is organized into four stages.Source: §3.4 · p. 6

3.4.1 Stage 1: Channel Projection

  • A pointwise 1 × 1 convolution projects the input into a uniform latent space, mixing information across channels before temporal modeling.The projected representation has dimension d_h.Source: §3.4.1 · p. 6

3.4.2 Stage 2: Causal Dilated TCN

  • Residual TCN blocks apply causal convolutions with exponentially increasing dilation, and left-padding plus trimming prevents access to future-facing outputs.Doubling dilation captures multi-scale patterns within the 60-minute observation window.Source: §3.4.2 · pp. 6–7

3.4.3 Stage 3: Dilated RNN Hierarchy and Fusion

  • A hierarchy of dilated recurrent layers processes temporally subsampled inputs, and learned gating fuses its output with the TCN representation.The fused representation combines the TCN’s local multi-scale patterns with longer-range recurrent dependencies.Source: §3.4.3 · p. 7

3.4.4 Stage 4: Masked Temporal Pooling and Classification Head

  • Masked mean pooling over valid time steps forms a fixed-length case embedding, which dropout and a linear projection convert into an AKI risk probability.Dividing by valid sequence length rather than padded batch length avoids penalizing shorter observation windows.Source: §3.4.4 · p. 7

3.5 Model Families and Variants

The paper evaluates a waveform-only backbone, a late-fusion multimodal extension, and a leakage-safe stacked ensemble. The variants add structured context and combine out-of-fold model predictions at the meta-learning stage.

3.5.1 SynerT-MM

  • SynerT-MM separately encodes the waveform sequence and structured static features, then concatenates their representations for classification.The static branch uses a learned projection, while the temporal branch uses the SynerT backbone.Source: §3.5.1 · p. 8

3.5.2 SynerT-Stack

  • SynerT-Stack’s logistic meta-learner combines out-of-fold probabilities from SynerT-MM, random forest, and logistic regression with leakage-safe static features.Training the meta-learner only on out-of-fold predictions avoids meta-level information leakage by construction.Source: §3.5.2 · p. 8

3.5 Model Families and Variants

  • The three model families distinguish waveform-only SynerT from variants that add structured context or combine base-model predictions.Table 1 summarizes the information sources available to the main model families.Source: §3.5 · pp. 7–8, Table 1 · p. 9

3.6 Evaluation Design and Leakage Control

Evaluation uses fold-specific preprocessing and out-of-fold predictions to control leakage. It assesses discrimination, operating-point performance, calibration, clinical utility, and transport without refitting on the stress-test cohort.

3.6 Evaluation Design and Leakage Control

  • Five-fold cross-validation uses training-fold-only scalers, and stacked predictions are built exclusively from out-of-fold base-model outputs to prevent leakage.The evaluation includes discrimination metrics AUROC and AUPRC, with particular emphasis on AUPRC given the 7.46% event prevalence.Source: §3.6 · p. 9
  • Evaluation includes operating-point metrics at 95% specificity, pooled out-of-fold calibration diagnostics, and decision-curve analysis across threshold probabilities from 0.02 to 0.25.Calibration diagnostics include Brier score, ECE, reliability plots, and calibration intercept and slope.Source: §3.6 · p. 9
  • SynerT-MM and SynerT-Stack receive leakage-safe cross-fitted Platt recalibration, while eICU-CRD Demo serves as a transport stress test scored without refitting.Source: §3.6 · p. 9

4 Experiments and Results

Across the main comparison, SynerT-Stack led overall, while waveform-only SynerT remained below strong structured-data baselines. Calibration, decision-curve, and operating-point analyses further favored recalibrated multimodal models, although the eICU transport result was only a proxy.

4.1 Main Model Comparison

  • SynerT-Stack achieved the highest AUROC, AUPRC, and F1-max across models; SynerT-MM improved discrimination over SynerT but remained directionally, not significantly, better than random forest.Secondary tabular baselines and waveform backbones did not surpass the multimodal variants.Source: §4.1 · pp. 9–10, Table 2 · p. 10

4.2 Cross-Setting Transport Stress Test on eICU-CRD Demo

  • On eICU-CRD Demo, SynerT-Stack retained AUROC 0.831 ± 0.026 and AUPRC 0.555 ± 0.177, but the small, unmatched cohort supports only a proxy transport result.Most standalone models degraded under setting shift.Source: §4.2 · p. 10, Figure 2 · p. 10

4.3 Statistical Testing of the Main Contribution

  • Random forest significantly outperformed SynerT, SynerT-MM’s point estimate exceeded random forest without statistical significance, and SynerT-Stack significantly improved over both.The paired AUPRC tests used fold-paired Wilcoxon tests and bootstrap deltas from aligned out-of-fold predictions.Source: §4.3 · p. 11, Table 3 · p. 11

4.4 Calibration Assessment

  • Cross-fitted Platt recalibration reduced ECE from 0.350 to 0.005 for SynerT-MM and from 0.316 to 0.027 for SynerT-Stack while leaving discrimination largely unchanged.Raw random forest calibrated best, while both multimodal variants were substantially miscalibrated.Source: §4.4 · p. 11, Table 4 · p. 12

4.5 Decision-Curve Analysis

  • Across most low-to-intermediate thresholds, recalibrated SynerT-Stack provided the highest net benefit; both recalibrated multimodal models remained positive from 0.02 to 0.25.Random forest fell below treat-none at approximately the 0.10 threshold.Source: §4.5 · p. 12, Figure 3 · p. 12

4.6 ROC, Precision-Recall, and Operating-Point Evaluation

  • At 95% specificity, SynerT-Stack achieved the highest sensitivity and positive predictive value, with SynerT-MM next among the neural variants.Figure 4 presents threshold-dependent ROC views for the principal model families.Source: §4.6 · p. 12, Table 5 · p. 13, Figure 4 · p. 13

5 Discussion

Under strict leakage-aware evaluation, waveform-only modeling was insufficient for early PO-AKI risk stratification, while multimodal context and leakage-safe stacking produced the strongest overall results. Calibration and decision-curve analysis strengthened the case for the stacked model, but creatinine-only labeling and a small unmatched external cohort limit the evidence.

5 Discussion

  • Leakage-safe stacking of physiologic trajectories, structured renal-risk context, and hemodynamic burden summaries produced the strongest overall results.The eICU-CRD Demo analysis suggested the stacked ensemble was less brittle under setting shift than standalone models.Source: §5 · p. 13
  • Cross-fitted Platt recalibration corrected important calibration defects in the multimodal models, after which the stacked model showed superior net benefit across low-to-intermediate thresholds.The study argues that improved discrimination alone was insufficient for deployment-ready risk estimation.Source: §5 · p. 13
  • The eICU-CRD Demo was a small, ICU-based, mismatched transport stress test rather than definitive external validation, motivating validation on larger perioperative datasets.Source: §5 · pp. 13–14
  • AKI labeling relied on creatinine without urine-output ascertainment, and the external cohort was small and unmatched.The authors identify these as limits and call for validation on larger perioperative datasets.Source: §5 · p. 14

6 Conclusion

The study concludes that early PO-AKI prediction benefits more from multimodal context, calibration, and robustness assessment than from temporal-model complexity alone. Its leakage-aware evaluation framework is presented as a transferable template for early intraoperative risk stratification.

6 Conclusion

  • Leakage-safe stacking of complementary learners achieved the strongest supported result, while waveform-only temporal modeling was insufficient for postoperative AKI discrimination.Source: §6 · p. 14
  • Cross-fitted Platt recalibration improved absolute-risk behavior for both multimodal models, and the recalibrated stacked model had the most favorable clinical-utility profile across low-to-intermediate thresholds.Source: §6 · p. 14
  • The eICU-CRD Demo stress test suggested the stacked model was less brittle under cross-setting shift, while motivating matched surgical cohorts for external validation.Source: §6 · p. 14
  • The framework offers a transferable template for separating waveform, structured-context, and calibration-aware ensemble contributions under strict clinical deployment constraints.Source: §6 · p. 14
Loading 2609.26848v1…