PaperlayerView exact revision on arXiv

Source-linked AI summary

TRIAGE: Dialectical Reasoning for Explainable Risk Prediction on Irregularly Sampled Medical Time Series with LLMs

Hyeongwon Jang, Gyouk Chu, Changhun Kim, Joonhyung Park, Hangyul Yoon, Eunho Yang

arXiv:2606.09030v1cs.LGcs.AIcs.CL

TL;DR

Clinical early-warning systems need calibrated, comparable risk scores and verifiable rationales, but LLM reasoning can polarize graded risk into overconfident predictions. TRIAGE trains an LLM to reason dialectically over alternative outcomes, improving benchmark performance and calibration while producing grounded explanations.

  • Problem

    LLM-based clinical risk prediction on irregularly sampled medical time series struggles to provide continuous, cross-patient comparable risk scores together with clinically grounded rationales.

  • Method

    TRIAGE generates outcome-specific rationales, derives risk from implicit outcome probabilities conditioned on them, and uses dialectical reasoning supervision followed by self-refinement.

  • Results

    TRIAGE improves AUPRC by 3.3% and reduces calibration error by 81% over competitive baselines across three irregular medical time-series benchmarks.

  • Takeaways & Limitations

    TRIAGE supports pursuing predictive performance and grounded clinical explanation together within a single LLM-based decision-support system.

  • Takeaways & Limitations

    Evaluation is restricted to binary prediction, the pipeline is more expensive than lightweight baselines, and rationale quality was assessed by LLM judges rather than clinical experts.

Abstract

from arXiv · show

Clinical early warning systems built on electronic health records, in which clinical observations are recorded as irregularly sampled medical time series (ISMTS), must deliver both calibrated risk scores for patient triage and interpretable rationales that clinicians can verify. Large Language Models (LLMs) have been explored for this task, yet they collapse graded clinical risk into overconfident binary predictions. This risk polarization undermines both calibration and cross-patient comparability. To address this, we propose TRIAGE, a framework that trains an LLM to generate dialectical reasoning over competing clinical outcomes by eliciting outcome-specific rationales. This dialectical formulation mitigates risk polarization, enabling a single LLM to yield continuous risk scores grounded in explicit clinical reasoning. Evaluated on three ISMTS benchmarks, TRIAGE achieves an average AUPRC improvement of 3.3% and reduces calibration error by 81% compared to the competitive baselines. An LLM-as-a-judge assessment further shows that our rationales surpass post-hoc explanations from the baseline by 20% in clinical reasoning quality. The source code is available at https://github.com/HyeongWon-Jang/TRIAGE .

1 Introduction

Clinical early-warning systems need both calibrated, comparable risk scores and clinician-verifiable rationales for irregularly sampled medical records. TRIAGE addresses LLM risk polarization by eliciting dialectical, outcome-specific reasoning and deriving continuous risk from it.

  • EHR observations are irregularly sampled and missing, while clinical decision support requires both comparable risk scores and rationales grounded in clinical reasoning.
  • Existing approaches typically provide either continuous risk scoring without natural-language reasoning or reasoning with discrete predictions that are not cross-patient comparable.
  • LLM reasoning causes risk polarization because rationales pre-commit to one outcome and present one-sided confirmation evidence.
  • TRIAGE generates outcome-conditioned rationales, derives risk from the resulting implicit probability distribution, and trains with dialectical reasoning supervision followed by self-refinement.
  • 3.3% AUPRC improvement and 81% lower calibration error were achieved over strong baselines, alongside more clinically reasonable, patient-grounded rationales.

2 Related Work

Prior clinical time-series methods specialize in irregular sampling but often separate prediction from explanation. LLM approaches similarly divide between answer-token risk scoring, hard-label reasoning, and auxiliary rationale generation.

  • Prior ISMTS models use RNN, ODE, set-based, interpolation, attention, transformer, and graph architectures for irregular medical observations.
  • LLM clinical-time-series methods commonly extract risk from answer-token probabilities or generate rationales alongside discrete class labels.
  • Some approaches offload prediction to neural models while using LLMs for auxiliary rationales, summaries, or discrete predictions.

3 Problem Setup

The paper formulates ISMTS clinical risk prediction as supervised classification over irregular patient records and static attributes. Its primary output is a patient-specific class-probability vector rather than only a discrete label.

  • The dataset contains patient records, static descriptors, and outcome labels, with each irregular measurement represented by time, value, and variable index.
  • The predictor outputs a probability vector over clinical classes for each patient.
  • The discrete class prediction is the maximum-probability class, but the probability vector is the primary discriminative risk estimate.

4 Preliminary Study

The preliminary study shows that ordinary reasoning-then-prediction collapses continuous risk into extreme probabilities through pre-committed and one-sided rationales. TRIAGE therefore structures reasoning around alternative outcomes.

  • The study defines risk polarization as collapse of the score distribution into a degenerate extreme when an LLM reasons before predicting.
  • 86.4%±18.8% average predicted-class probability under answer-only scoring rose above 99.98% for every patient after introducing reasoning.
  • 71.7% of rationales ended with an explicit committed verdict immediately before the answer position.
  • One-sided rationales cite supportive evidence while disregarding countervailing signals, motivating dialectical deliberation over candidate outcomes.
  • The resulting design principles require preserving continuous risk and separately weighing evidence for each candidate outcome.

5 TRIAGE

TRIAGE combines outcome-specific dialectical reasoning with implicit-probability risk estimation and a two-stage training pipeline. It is designed to preserve graded, cross-patient-comparable risk while producing rationales that consider competing clinical outcomes.

  • 5.1 Dialectical Reasoning over Alternative Outcomes: TRIAGE generates a dedicated rationale for each candidate outcome before deriving risk from the model’s implicit outcome-token distribution.This replaces one-sided reasoning with explicit inspection of alternative outcomes.
  • 5.1 Dialectical Reasoning over Alternative Outcomes: TRIAGE serializes observed measurements with time-invariant attributes and a structured task definition rather than representing the full observation matrix.The patient prompt concatenates P, textualized attributes tzi, and temporally ordered observations tsi.
  • 5.1 Dialectical Reasoning over Alternative Outcomes: For binary tasks, the model separately examines positive and negative outcomes, surfacing supporting signals that one-sided reasoning would discard.The rationale order is interchangeable, and both orderings are used as data augmentation.
  • 5.1 Dialectical Reasoning over Alternative Outcomes: Dialectical rationales present evidence for both outcomes without committing before the final answer token, preserving continuous and cross-patient-comparable risk estimates.The final decision follows a dedicated header and directly precedes the outcome token.
  • 5.2 Training Pipeline: TRIAGE uses supervised fine-tuning on synthesized outcome-conditioned rationales followed by reinforcement learning on self-generated samples.Stage 2 applies cross-entropy to the final decision token and GRPO to preceding reasoning tokens.
  • 5.2 Training Pipeline: Batch-level rewards compare each sample’s score with the average score of opposite-class samples to encourage cross-patient comparability.This batch-level design enhances separation beyond intra-sample rewards.

6 Experiments

TRIAGE is evaluated on three ISMTS benchmarks using discrimination, calibration, and robustness tests. After reinforcement learning, it achieves leading predictive and calibration performance while remaining robust when variables are masked.

  • Experimental setup: Experiments cover P12, P19, and MIMIC-III, measuring AUROC, AUPRC, ECE, and Brier score.P19 predicts sepsis onset, whereas P12 and MIMIC-III predict in-hospital mortality; AUPRC is emphasized because of severe class imbalance.
  • Predictive performance: TRIAGE achieves the best average rank of 1.58 after reinforcement learning, placing first or second on every predictive metric.SFT alone reaches an average rank of 4.25, while zero-shot GPT-5.1 and gpt-oss-120b rank 10.50 and 11.67.
  • Predictive performance: Against GRU-D, TRIAGE improves mean AUROC by 0.8% and mean AUPRC by 3.3%.
  • Calibration: Reinforcement learning reduces mean ECE by 80% and mean Brier score by 49%, producing the best calibration scores on every benchmark.SFT alone reaches parity with ISMTS baselines but remains slightly behind zero-shot GPT-5.1.
  • Robustness under limited information: With 10% to 50% of variables masked, TRIAGE remains robust across P12 and MIMIC-III.On P12 it matches the strongest baseline in AUPRC but trails slightly in AUROC; on MIMIC-III it leads AUPRC at every masking ratio and AUROC at four of five.

7 Further Analysis

Further analyses show that dialectical reasoning and batch-level rewards improve both risk prediction and explanation quality. TRIAGE’s direct rationales outperform a post-hoc explanation pipeline on clinical reasoning assessment.

  • Reasoning structure: Answer-only SFT improves zero-shot inference without explanations, while one-sided rationale SFT inherits risk saturation and underperforms answer-only SFT.One-sided reasoning drives answer-token probability toward 1.0, even when predictions are averaged over 10 sampled outputs at 10× inference cost.
  • Reasoning structure: TRIAGE’s dialectical reasoning reaches 86.9% AUROC and 56.4% AUPRC while retaining clinically grounded explanations.
  • RL reward design: Batch-level rewards improve both discrimination and calibration relative to sample-level rewards.The reported improvements include higher AUPRC and lower ECE and Brier scores, reflecting an inter-sample learning signal.
  • Clinical reasoning quality: TRIAGE achieves a higher total IDEA score than the post-hoc baseline, 7.744 versus 6.474.The largest gain is in interpretive summary (+0.902), with an additional improvement in alternative diagnosis (+0.288).
  • Clinical reasoning quality: Qualitative analyses further report more patient-specific and clinically grounded reasoning traces for TRIAGE.

8 Conclusion

TRIAGE grounds clinical risk estimation in reasoning over alternative outcomes, combining graded risk prediction with clinician-oriented rationales. Its results support pursuing predictive performance and grounded explanation together in LLM-based clinical decision support.

  • TRIAGE generates rationales for alternative outcomes before deriving risk from the LLM’s implicit probability.This design addresses risk polarization associated with reasoning that commits to a single trajectory.
  • With dialectical reasoning supervision and self-refinement, TRIAGE outperforms competitive baselines while producing rationales aligned with clinicians’ evaluation of patient evidence.

Limitations

The study is limited to binary prediction, incurs substantially higher computational cost than lightweight baselines, and evaluates reasoning with LLM judges rather than clinicians. Generated rationales may also contain inaccuracies or biases, so TRIAGE is not a clinical tool.

  • The evaluation covers only binary prediction tasks, leaving multi-class and multi-label clinical settings for future work.
  • The LLM-based pipeline is considerably more expensive than lightweight baselines such as GRU-D during training and inference.Multi-step rationale generation creates overhead that may be prohibitive under strict low-latency requirements.
  • Reasoning quality is assessed with LLM-as-a-judge using IDEA rather than clinical experts.The authors aggregate across diverse judges and ground analyses in medical literature, but identify expert evaluation as an important next step.
  • Generated rationales may contain inaccuracies or biases, and TRIAGE is presented as a research prototype rather than a clinical tool.Its outputs should not substitute for qualified clinical judgment.

Ethical Considerations

MIMIC-III access is governed by the PhysioNet credentialed Data Use Agreement. Accordingly, the full MIMIC-III pipeline uses locally hosted open-weight models rather than third-party LLM services.

  • All MIMIC-III preprocessing, rationale generation, training, and evaluation are performed locally with self-hosted, open-weight models.This includes all zero-shot LLM baselines.

A.1 Implementation Details

The implementation compares probability channels, examines reasoning behavior, and specifies TRIAGE’s inference and training procedures across three binary ISMTS benchmarks. TRIAGE uses implicit probabilities for its main risk score and averages two rationale orderings at inference.

  • Probability selection: The preliminary comparison defines verbalized probability as response-text output and implicit probability as next-token probability for the answer tokens.The comparison uses MIMIC-III with reasoning disabled on gpt-oss-120b and Qwen3-8B.
  • Probability selection: Implicit probability improves AUPRC while reducing ECE and Brier score by roughly 4×, whereas verbalized outputs concentrate on a few anchor values.Seven values cover 75.3% of gpt-oss-120b estimates and three cover 63.5% of Qwen3-8B estimates.
  • Probability selection: The implicit probability varies continuously over [0, 1] and is differentiable, while verbalized output is discrete and nondifferentiable for gradient-based alignment.TRIAGE therefore adopts the implicit probability as its risk score throughout.
  • Zero-shot evaluation: For zero-shot reasoning benchmarks, TRIAGE compares a multi-sample outcome-based estimator with a single-response verbalized probability.The outcome-based estimator exceeds the verbalized score only around 30 sampled responses, so verbalized probability is used for efficiency.
  • Inference: TRIAGE inference samples outcome-specific rationales, reads answer-token logits, and computes the positive-outcome probability.The algorithm receives a policy, patient features, observations, a prompt template, and a binary outcome set.
  • Reasoning analysis: The preliminary reasoning study finds that traces typically confirm one outcome rather than weigh evidence on both sides.Two traces for the same patient cite exclusively deterioration or stabilization signals while remaining internally coherent.
  • Datasets: The evaluation uses P12, P19, and MIMIC-III, all binary classification benchmarks with severe class imbalance.P12 and P19 use five random 8:1:1 splits, while MIMIC-III uses a fixed 7:1.5:1.5 split and five random seeds.
  • Training: TRIAGE training includes dialectical reasoning supervision followed by self-refinement with GRPO and a cross-entropy loss on the decision token.The stated objective combines L_GRPO(θ) and λ L_CE(θ), with GRPO applied to reasoning tokens.

E.1 Low-Resource Training Experiments

The low-resource experiments test whether TRIAGE benefits from pretrained priors when labeled data are scarce. Its strongest gains occur at 1% training data, while performance becomes comparable with GRU-D at 10%.

  • Motivation and setup: TRIAGE builds on a pretrained LLM and can leverage medical and reasoning priors without extensive task-specific supervision.The comparison is against ISMTS baselines trained from scratch.
  • Motivation and setup: The low-resource study fine-tunes TRIAGE with rsLoRA and omits reinforcement learning to isolate supervised reasoning data efficiency.Adapters are applied across attention and MLP projection layers.
  • Motivation and setup: Baseline tuning additionally sweeps smaller batch sizes and broader learning rates to reduce under-tuning as an explanation for observed differences.
  • Results: At P12-1%, TRIAGE outperforms GRU-D by 4.4% AUROC and 11.1% AUPRC.Figure 5 compares TRIAGE with ISMTS baselines across 1%, 5%, and 10% training fractions.
  • Results: The improvement over baselines diminishes as labeled data increase, with TRIAGE and GRU-D performing comparably at P12-10%.The authors report this pattern as consistent with pretrained knowledge contributing most when supervision is scarce.
  • Results: TRIAGE retains a consistent advantage over corresponding baselines across Qwen3-1.7B, Qwen3-8B, Qwen3-4B, and Llama 3.2 3B backbones.The result suggests reasoning supervision generalizes beyond a single model.
  • Results: Bidirectional averaging and single-direction inference perform comparably on P12 under supervised fine-tuning.A single direction can therefore serve as a lower-cost alternative, although bidirectional averaging provides a small additional benefit.

F Qualitative Results

TRIAGE’s qualitative evaluation compares its clinical reasoning with STraTS-based post-hoc explanations and examines hallucinations in generated traces. Case studies show stronger temporal and clinically grounded reasoning, while highly fluctuating trajectories remain challenging.

  • Case studies: TRIAGE captured progressive recovery over time as a clinically meaningful indicator of improving renal perfusion.The baseline instead emphasized isolated or contradictory attribution signals and did not capture the temporal trajectory as comprehensively.
  • Limitations: Highly fluctuating temporal trajectories remain difficult: neither the baseline nor TRIAGE fully captured the patient’s non-monotonic temperature pattern.The case involved initial hypothermic risk, partial recovery, and subsequent decline.
  • Hallucination analysis: 1.5% of 200 P12 reasoning samples contained severe hallucinations, according to the LLM-as-a-judge assessment.The analysis defines severe hallucinations as references to features or values absent from the patient record.
  • Hallucination analysis: Explicit blank fields when no supporting evidence exists may help explain why dialectical reasoning rarely fabricates opposing evidence when one side is clear-cut.This safeguard is attributed to the data-construction process.
  • Case studies: TRIAGE scored 8.0 versus 5.0 for STraTS+XAI in the survived-patient case study.The comparison uses an LLM-as-a-judge assessment of clinical reasoning quality.
  • Case studies: 8.4 versus 6.1: TRIAGE received the higher clinical-reasoning score in the mortality case study than STraTS+XAI.TRIAGE explicitly incorporated severe hyperkalemia, elevated BUN, and other clinically critical indicators into its reasoning chain.
Loading 2606.09030v1…