PaperlayerView exact revision on arXiv

Source-linked AI summary

A Two-Level Clinical Novelty Prioritization Framework for Disease Trajectories Discovered from Longitudinal Electronic Health Records

Jin-Jia Zheng, Vincent S. Tseng

arXiv:2610.20512v1cs.LG

TL;DR

Structural support from graph-based EHR analyses does not establish whether discovered disease trajectories are semantically novel. This study applies literature-calibrated path- and node-level scoring to a frozen motif catalog and separately adjusts clinical priority for specificity and graph reliability. In downstream evaluation, timing associations did not increase monotonically with semantic score, and branch-enhanced forecasting did not show preferential effectiveness in the high-semantic subgroup.

  • Problem

    Graph-based EHR analyses produce disease-trajectory catalogs, but structural support alone does not establish semantic novelty or distinguish uncommon relations from unusual combinations.

    Source: §1 · p. 1

  • Method

    The framework scores path- and node-level semantic sparsity in a frozen catalog of previously discovered motifs, then separately adjusts priority for specificity and graph reliability.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Timing improvements did not increase monotonically with semantic score, and branch-enhanced forecasting was not preferentially effective in the high-semantic subgroup.

    Source: §5 · p. 12

  • Takeaways & Limitations

    The findings support separating pure literature-relative semantic evidence from reliability-adjusted prioritization, with recombination treated as a diagnostic contrast rather than synergistic novelty.

    Source: §5 · p. 11, §7 · p. 13

    Evaluation used a single-hospital MIMIC-IV setting, and the last recorded discharge cannot guarantee endpoint-free follow-up elsewhere.

    Source: §6 · p. 12

Abstract

from arXiv · show

Graph-based analysis of electronic health records (EHRs) yields disease-trajectory catalogs, but structural support does not establish semantic novelty. We present a two-level framework to prioritize 99,527 branching motifs previously discovered by an upstream patient-disjoint, train-only MIMIC-IV v3.1 pipeline and frozen after validation-based catalog selection. The framework separates path-level and node-to-endpoint semantic sparsity from reliability-adjusted clinical priority. Three deterministic textualizations per motif or node-target pair were embedded with BAAI/bge-small-en-v1.5 and calibrated against 166,972 pre-2023 PubMed references using mean top-20 cosine distance. Positive recombination occurred in 8.63% of motifs and had negligible effect on global ranking, whereas specificity and support/GNN reliability terms changed clinical-priority ordering. Robustness analyses across neighbor counts, templates, encoders, generic-label taxonomies, score settings, PrimeKG, and corrected density-based RND showed that broad ordering was more stable than hard top-k membership. A semantic-dose analysis did not support a monotonic relation between score and forecasting benefit after endpoint and token-count adjustment (1.96 days per SD; 95% CI, -0.75 to 4.68). In the frozen catalog-q90 high-semantic subgroup, branch-enhanced forecasting reduced conditional event-time MAE by 20.1 days (95% CI, 13.3-27.0) and sMAPE by 2.37 percentage points (95% CI, 1.81-2.93), while discrimination and endpoint-ranking differences were inconclusive. Corrected RND showed low tail agreement with the primary score (Spearman rho = 0.430; q90 Jaccard = 0.125) despite a similar aggregate timing association. These findings support representation-aware, uncertainty-aware literature-relative prioritization of discovered trajectories rather than causal discovery or clinician-adjudicated novelty.

1. Introduction

The study addresses how to prioritize already-discovered EHR disease trajectories for semantic novelty, which structural support alone cannot establish. It separates path- and node-level literature-relative evidence from reliability-adjusted clinical priority.

1. Introduction

  • Graph-based studies had produced inspectable trajectory paths and branching motifs, but structural support does not establish their semantic novelty.The task is to identify candidates for literature review, model inspection, or expert assessment—not to discover new trajectories.Source: §1 · p. 1
  • A single full-path score cannot distinguish uncommon node-endpoint relations from unusual combinations of otherwise familiar relations.Broad or administrative labels can inflate semantic distance without increasing clinical informativeness.Source: §1 · p. 1
  • The framework scores complete motifs, node-to-endpoint relations, and positive path-minus-node recombination, then separately adjusts clinical priority for generic labels and graph reliability.The scores rank retrospective hypotheses, not probabilities of clinical novelty.Source: §1 · p. 1
  • The analysis uses a frozen MIMIC-IV v3.1 catalog whose upstream motifs were derived from train-only data and whose selection policies were set on validation data.The study also evaluates textualization uncertainty, robustness, and censor-aware downstream forecasting.Source: §1 · p. 2

2. Related work

Prior EHR research supports both inspectable trajectory mining and predictive sequence modeling, while literature-based novelty and graph methods offer complementary evidence. This work separates those signals to prioritize existing motifs for review.

2.1 Disease trajectories and sequential EHR modeling

  • Explicit trajectory mining yields inspectable sequences and branches, while neural sequence models use patient histories for prediction rather than assessing whether a progression pattern is established in biomedical knowledge.Frequent or predictive EHR patterns are not automatically novel, motivating prioritization when discovery produces many motifs.Source: §2.1 · p. 2
  • The framework ranks the already-discovered, frozen branching motifs by literature-relative uncommonness, specificity, and graph reliability while preserving explicit motifs for expert review.It does not generate new trajectories.Source: §2.1 · p. 2

2.2 Graph mining and reliability

  • Graph support and learned link scores measure discovery reliability, not novelty; a frequent pattern may be well known, while a rare one may be underexplored or noisy.Combining reliability and novelty too early obscures which signal drives a high score.Source: §2.2 · p. 2

2.3 Literature-based discovery and semantic novelty

  • Literature-based discovery, atypical-combination analysis, semantic predication, and embedding distance offer ways to find underexplored concepts, relations, or documents.For branching motifs, novelty may lie in node-to-endpoint relations, whole-path configuration, or both, and estimates depend on textualization and representation choices.Source: §2.3 · p. 3

2.4 Biomedical representation learning and prioritization

  • Different biomedical encoders and deterministic wording variants can change semantic rankings, so robustness across representations is stronger evidence than stability under one template or encoder.The framework reports pure semantic evidence separately from specificity- and graph-reliability-adjusted clinical priority.Source: §2.4 · pp. 3–4

3. Methods

The methods apply literature-calibrated semantic scoring to a frozen catalog of previously discovered branching motifs, then distinguish pure semantic evidence from reliability-adjusted priority. Downstream evaluation and robustness analyses use prespecified settings and censor-aware procedures.

3.1 Framework overview

  • The framework decomposes each frozen branching motif into node-target pairs and a full-path representation, then reports pure semantic scores separately from reliability-adjusted clinical priority.The primary analysis covers 99,527 previously discovered motifs; the explicit-path schematic illustrates the decomposition.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 Cohort, leakage control, and frozen motif catalog

  • The catalog comes from an upstream train-only graph pipeline, with validation-based selection frozen before novelty analysis; diagnoses were drawn from MIMIC-IV v3.1 for four endpoints.The endpoints are COPD, stroke, CKD, and dementia.Source: §3.1 · p. 4, §3.2 · pp. 4–5

3.3 Semantic artifact and deterministic textualization

  • The semantic artifact contains 99,527 motifs with three fixed-template textualizations each, embedded against 166,972 pre-2023 PubMed records using mean distance to 20 nearest references.Each of 16,667 unique node-target pairs also has three textualizations; templates are deterministic rather than generated online.Source: §3.3 · p. 5

3.4 Two-level semantic novelty

  • The semantic score combines full-path percentile, mean node-to-endpoint percentile, and positive path-minus-node recombination with prespecified weights of 0.50, 0.30, and 0.20.Recombination is a derived contrast, not an independent semantic source.Source: §3.4 · p. 5

3.5 Clinical-priority score and generic-label audit

  • Clinical priority subtracts a generic-label penalty and adds support and GNN reliability percentile ranks to the semantic score.The description-based generic-label rule is a transparent heuristic, not a clinical ontology or adjudicated specificity standard.Source: §3.5 · p. 6

3.6 Threshold systems and downstream subgroup

  • External path/node quadrants use P_i and N_i thresholds of 95, while downstream high-semantic membership uses the frozen global pure-semantic q90 threshold.For each patient-index-endpoint row, semantic evidence is the maximum score among up to 24 retrieved endpoint motifs; high-row membership need not equal 10%.Source: §3.6 · p. 6

3.7 Censor-aware downstream evaluation

  • Censor-aware evaluation removes prevalent endpoints and 30-day events, masks unknown outcomes with insufficient follow-up, and evaluates forecasting on eligible at-risk rows.The final test set includes 15,676 index examples from 4,631 patients; most ranking examples have only one at-risk candidate, creating a ceiling effect.Source: §3.7 · p. 6

3.8 Statistical analysis

  • Paired primary-subgroup intervals use 5,000 patient-cluster bootstrap resamples, while textualization uncertainty resamples wording variants with the cohort and literature corpus fixed.The analysis uses different uncertainty procedures for distinct questions; Holm adjustment applies only within named comparison families.Source: §3.8 · p. 6

4. Results

The frozen catalog showed high external semantic percentiles, while sensitivity analyses found that recombination had little ranking effect and specificity and reliability settings could alter clinical-priority ordering.

4.1 Final catalog distribution and external quadrants

  • 92.16 and 96.29 were the mean path and node external percentiles across 99,527 motifs, respectively, with the node distribution especially concentrated.The authors attribute this pattern to domain/template mismatch and interpret external percentiles as semantic sparsity in the chosen embedding space, not probabilities of true clinical novelty.Source: §4.1 · p. 7, Table 2 · p. 7

4.2 Generic taxonomy, recombination, and component sensitivity

  • Changing the generic-label taxonomy produced rank correlations of 0.9598 and 0.9923, while increasing the penalty from 0.12 to 0.48 yielded rho = 0.7703 and mean absolute rank change 15,037.The 0.48 setting was post hoc, so the prespecified 0.12 remains primary.Source: §4.2 · p. 8, Table 3 · p. 8
  • 8.63% of motifs had positive recombination, and removing it barely changed global ranking (rho = 0.999861).The mean weighted contribution was 0.000144; the authors retain recombination as a diagnostic contrast, not evidence of synergistic clinical novelty.Source: §4.2 · p. 8, Table 3 · p. 8
  • Across 313 weight/component settings, 81.8% retained rho >= 0.90 and 62.6% retained top-10% Jaccard >= 0.70 against the primary clinical-priority ranking.The four-axis Pareto analysis identified 230 motifs (0.23%) on the non-dominated front.Source: §4.2 · p. 8, Table 3 · p. 8

4.3 Textualization-evidence uncertainty

Wording-resampling analyses found a stable high-score core despite only moderate stability in exact ranking. The downstream high-semantic subgroup was compositionally enriched for stroke and dementia, which qualifies performance comparisons.

4.3 Textualization-evidence uncertainty

  • At top-k 1,000, mean overlap was 0.679 (95% evidence interval [0.660, 0.699]), while q90 membership retained a stable core despite unstable exact ordering.Of 9,954 point-estimate q90 motifs, 6,782 remained high in at least 80% of resamples and 5,653 in at least 95%.Source: Table 4 · p. 8, §4.3 · p. 8

4.4 Frozen high-semantic subgroup

  • Among 58,328 endpoint rows with semantic evidence, 35,538 met the frozen q90 criterion, including 9,255 positive events among at-risk high-semantic rows.The high-semantic subgroup was enriched for stroke and dementia, while the non-high subgroup contained more COPD and CKD events.Source: §4.4 · p. 9
  • Performance comparisons between the high- and non-high-semantic groups require accounting for their different endpoint composition and difficulty.Table 5 reports five-seed downstream performance by frozen semantic subgroup; Table 6 reports paired patient-cluster bootstrap differences in the high-semantic subgroup.Source: §4.4 · p. 9, Table 5 · p. 9, Table 6 · p. 9

Metric

Composition-standardized analyses retained timing improvements in the high-semantic subgroup, but adjusted semantic-score dose analyses did not establish a monotonic forecasting benefit. Endpoint-specific discrimination and timing findings varied and were exploratory.

4.4.1 Semantic dose-response and missing-score analysis

  • A one-SD semantic-score increase had an adjusted +1.961-day branch-minus-history absolute-error difference (95% CI -0.754 to 4.677), establishing no monotonic dose benefit.The analysis adjusted for endpoint and retrieved-token count; the missing-score subgroup instead showed +162.328 days (95% CI 94.345-228.545), not a token-removal or causal effect.Source: §4.4.1 · p. 9

4.5 Endpoint-specific high-semantic results

  • In endpoint-specific analyses, COPD showed AUROC +0.01374, AUPRC +0.01013, and MAE reduction 11.16 days; these analyses were exploratory.CKD, dementia, and stroke discrimination intervals crossed zero; endpoint-metric multiplicity adjustment did not retain the stroke MAE comparison.Source: §4.5 · p. 9
  • 18.16, 18.66, and 18.50 days were the MAE reductions under the common natural mix, equal-endpoint meta-average, and endpoint-count matching, respectively.Corresponding sMAPE reductions were 2.21, 2.24, and 2.28 percentage points; sensitivity varied by representation, with the neutral-template count-matched MAE interval crossing zero.Source: §4.5 · p. 10

4.6 Encoder, textualization, and neighbor-count robustness

Robustness checks generally preserved broad motif ordering more than exact high-ranking membership, while subgroup definitions and timing associations varied across representations and comparators. The ontology baseline captured a different, coarse signal rather than serving as clinical validation.

4.6 Encoder, textualization, and neighbor-count robustness

  • Across representation reruns, broad motif ordering was moderately to strongly preserved, but exact extreme membership was representation-sensitive and patient-row agreement with the primary high-q90 definition was incomplete.Alternative templates also changed endpoint composition, limiting pooled subgroup interpretation.Source: §4.6 · p. 10, Table 7 · p. 10
  • Across k = 1, 5, 10, 20, 50, and 100, broad ordering remained correlated while hard top sets changed; high-semantic MAE reductions ranged from 19.08 to 22.84 days.For k = 1 versus k = 20, rho = 0.9387 and top-100 Jaccard = 0.3889; primary k = 20 was retained because it was prespecified.Source: §4.6 · p. 10

4.6.1 Corrected density comparator and representation dependence

  • Corrected RND had Spearman rho = 0.430072 and catalog-q90 Jaccard = 0.125445 versus the primary score, despite a -25.230-day branch-minus-history MAE difference.The corrected RND high-group AUPRC interval crossed zero, and the non-high group also showed timing improvement, so novelty-specific effect modification was not established.Source: §4.6.1 · p. 10
  • The progression-focused corpus had rho = 0.950851 and q90 Jaccard = 0.665300 versus the primary composite, whereas PubMedBERT had 0.576470 and 0.181311.External-95 saturation also differed by template family: structured templates reached 99.477% for path and 100% for node percentiles.Source: §4.6.1 · p. 10

4.7 External-threshold and ontology-path sensitivity

  • External-both95 covered 92.8% of semantically available rows, whereas catalog q90 classified 60.9% as high; the study retained frozen catalog q90 for the primary subgroup.External path95, node95, and either95 classified 93.3%, 99.6%, and 99.7%, respectively.Source: §4.7 · p. 11
  • PrimeKG mappings covered 118 of 522 catalog codes, and ontology distance correlated weakly with sentence-based semantic novelty (rho = 0.1557).PrimeKG was treated as a complementary robustness baseline, not an ontology gold standard or clinical validation.Source: §4.7 · p. 11

4.8 Blinded review and external-validation status

The planned blinded clinician review and external-EHR validation did not yield results. Anonymous model-behavior cases illustrate computational sensitivity, not independent clinical validation.

4.8 Blinded review and external-validation status

  • The endpoint-balanced 80-case review packet and locked analysis plan were complete, but clinician ratings had not been submitted, so no clinician agreement or validation result was reported.External-EHR validation was also not conducted because no approved compatible longitudinal EHR and diagnosis mapping were available.Source: §4.8 · p. 11
  • Without approved compatible longitudinal EHR data and diagnosis mapping, the reported validation scope does not extend to external EHRs.Source: §4.8 · p. 11

4.9 Anonymous model-behavior cases

  • Four anonymized seed-42 cases were selected by the largest risk-probability decrease after deleting the top motif, illustrating computational sensitivity rather than independent clinical validation.The CKD example was retained despite a large timing error; Table 8 is scoped to anonymous high-semantic model-behavior cases.Source: §4.9 · p. 11, Table 8 · p. 11

5. Discussion

In the frozen high-semantic subgroup, branch-enhanced forecasting improved conditional timing error, but discrimination and endpoint-ranking differences were inconclusive. Robustness analyses favored broad ordering over fixed top-k membership, while semantic-dose analyses did not establish a monotonic benefit.

5. Discussion

  • Positive recombination occurred in 8.63% of motifs and had negligible global-rank influence, whereas specificity and support/GNN reliability materially changed clinical-priority ordering.Recombination is best treated as a diagnostic contrast, while Ssemantic represents pure literature-relative evidence and Sclinical a reliability-adjusted prioritization score.Source: §5 · p. 11
  • Broad rankings were more stable than hard top-k membership, and timing improvement persisted across k = 1-100.Endpoint-specific inclusion frequencies, score components, rank intervals, and endpoint composition are more informative than a single cross-endpoint hard ranking.Source: §5 · pp. 11–12
  • Endpoint- and token-adjusted semantic-dose analyses did not support a monotonic relationship between score and forecasting benefit.The missing-score subgroup’s timing degradation is a conditional association, not evidence that removing semantic evidence causes harm.Source: §5 · p. 12
  • Corrected RND had low agreement with the primary score’s tail (rho 0.430; q90 Jaccard 0.125), despite similar aggregate conditional timing associations.Broad ordering could remain correlated while extreme motif identities, patient-row membership, and endpoint composition changed appreciably.Source: §5 · p. 12
  • External 95th-percentile rules labeled nearly all semantically available rows high, making their positive timing results closer to broad-cohort sensitivity than selective novelty validation.Catalog-relative q90 is the primary downstream semantic subgroup; external percentiles, density scores, alternative corpora, and ontology distance are complementary sensitivities, not ground truth.Source: §5 · p. 12
  • Within the frozen high-semantic subgroup, branch tokens reduced conditional event-time error, but discrimination and endpoint-ranking differences were inconclusive.Similar timing and discrimination improvements also appeared in the non-high subgroup, so results do not support preferential effectiveness for clinically novel trajectories.Source: §5 · p. 12

6. Limitations

The study’s scope is limited by its single-hospital data, heuristic diagnosis and generic-label representations, and sensitivity of novelty estimates to literature and modeling choices.

6. Limitations

  • MIMIC-IV represents a single hospital system, and CCS/CCSR aggregation can merge heterogeneous diagnoses.The generic-label taxonomy is a description-based heuristic rather than a clinician-adjudicated ontology; alternative taxonomies changed hard top-set membership.Source: §6 · p. 12
  • Semantic novelty depends on the literature universe, encoder, textualization, neighbor count, local-density definition, and endpoint composition.Sensitivity analyses preserved aggregate timing associations more consistently than exact high-ranked motif identities; external percentiles could saturate under some templates.Source: §6 · p. 13
  • No external-EHR validation, completed blinded clinician adjudication, or human semantic calibration is available.The exploratory dose family used 1,000 bootstrap resamples with pointwise rather than simultaneous percentile intervals; the method is intended for retrospective hypothesis prioritization, not diagnosis, treatment selection, causal inference, or automated discovery claims.Source: §6 · p. 13

7. Conclusion

The framework prioritizes a frozen catalog of previously discovered branching motifs using separate semantic and reliability-adjusted scores. Its findings support representation-aware retrospective prioritization, while limiting claims about clinical novelty, causality, clinician validation, and generalization.

7. Conclusion

  • The framework prioritizes a frozen catalog of 99,527 branching motifs using path- and node-level literature-relative evidence separated from reliability-adjusted clinical priority.Specificity and reliability terms affected global priority ordering, while recombination had little effect and semantic-dose analyses did not establish a monotonic forecasting relationship.Source: §7 · p. 13
  • Broad rankings and aggregate conditional timing associations were more robust than exact hard top-k identities, while corrected RND showed low tail overlap despite a similar timing association.In the frozen high-semantic subgroup, branch-enhanced forecasting reduced conditional event-time error, but discrimination and endpoint-ranking gains were inconclusive and similar benefits occurred outside the subgroup.Source: §7 · p. 13
  • The findings support uncertainty-aware, representation-aware retrospective prioritization of discovered trajectories, not claims of clinical novelty, causality, clinician validation, or cross-institution generalization.Source: §7 · p. 13

Related papers

Loading 2610.20512v1…