PaperlayerView exact revision on arXiv

Source-linked AI summary

LUNG-KGMM: Knowledge-Guided Multimodal Learning for Lung Cancer Incidence Prediction

Chunlei Yang, Shuyan Li, Zhong Cao

arXiv:2608.14657v1cs.LGcs.CV

TL;DR

Existing lung cancer risk prediction is limited by single-modality data and insufficient use of structured clinical knowledge. LUNG-KGMM integrates longitudinal EHR, sanitized radiology reports, chest-radiograph representations, and guideline-derived knowledge for 1-to-6-year prediction, achieving macro AUROC 0.885, AUPRC 0.507, and Brier score 0.034 in MIMIC while cross-cohort validation indicates a need for local adaptation.

  • Problem

    Routine-care lung cancer risk prediction remains limited by incomplete multimodal evidence, potential report leakage, and insufficiently structured clinical guidance.

  • Method

    LUNG-KGMM combines longitudinal EHR, leakage-sanitized radiology reports, multiview chest-radiograph representations, and report-triggered guideline relations for 1-to-6-year incident lung cancer prediction.

  • Results

    Macro AUROC 0.885, AUPRC 0.507, and Brier score 0.034 were achieved by the final LUNG-KGMM input stack in MIMIC.

  • Takeaways & Limitations

    The framework supports reproducible lung cancer incidence-risk research, while Xiamen validation shows that deployment requires local adaptation and prospective validation.

  • Takeaways & Limitations

    No-retraining cross-cohort performance decreased substantially, indicating that local terminology mapping, recalibration, and likely retraining are required.

Abstract

from arXiv · show

Early identification of lung cancer risk is critical for timely intervention, yet existing prediction models are limited by their reliance on single data modalities and their inability to leverage structured clinical knowledge. We propose LUNG-KGMM, a knowledge-guided multimodal framework that integrates longitudinal electronic health records, radiology reports, chest radiograph representations, and guideline-derived knowledge for 1-to-6-year incident lung cancer prediction. To address modality heterogeneity and potential data leakage, we develop a leakage-sanitized report processing pipeline and a horizon-masked cumulative training objective that handles incomplete follow-up. We further introduce a knowledge-graph representation of clinical guidance that encodes report-triggered finding-attribute-action relations as an auditable knowledge stream. We build a multimodal development cohort from the publicly available MIMIC databases and construct a real-world validation cohort from the Xiamen Medical Big Data Platform. Extensive experiments on the MIMIC cohort demonstrate that LUNG-KGMM achieves superior performance over state-of-the-art methods, and validation on the Xiamen cohort further characterizes its cross-cohort portability and the need for local adaptation. The MIMIC development cohort is publicly accessible; the Xiamen cohort is governed by local data privacy regulations.

1 Introduction

Lung cancer poses a substantial global and Chinese population-health challenge, while routine-care risk identification remains distinct from protocolized LDCT screening. LUNG-KGMM addresses this gap through knowledge-guided multimodal longitudinal prediction using EHR, sanitized reports, chest radiographs, and guideline relations.

  • Introduction: Lung cancer remains a leading cause of cancer mortality worldwide and a major population-health challenge in China, with broader economic burden and health-resource allocation implications.Respiratory comorbidity and smoking-related risk are common in China, as documented by the China Pulmonary Health study and national statistics.
  • Introduction: LDCT screening reduces lung cancer mortality in high-risk populations, while guidelines define eligibility, surveillance, implementation, and pulmonary-nodule follow-up.Risk-prediction and pulmonary-nodule malignancy models further refine CT-screening selection.
  • Introduction: Routine-care risk identification differs from protocolized screening because chest radiographs are not LDCT examinations and guideline features do not impose LDCT thresholds on radiograph pixels.Smoking exposure, nodule annotations, follow-up decisions, and imaging pixels may be incomplete, while reports contain both legitimate index-time findings and possible diagnostic-history information.
  • Introduction: The framework predicts 1-6-year incident lung cancer risk from index-time multimodal data using yearly cumulative labels and follow-up masks.It excludes prior lung cancer and combines Transformer encoders for EHR, reports, and guideline terms with multiview TorchXRayVision chest-radiograph features.
  • Introduction: LUNG-KGMM achieves the strongest point estimate among compared MIMIC models by integrating EHR, leakage-sanitized radiology reports, chest radiographs, and report-triggered guideline relations.The study also constructs a large-scale real-world Xiamen cohort with 1-6-year incident lung cancer labels for within-cohort and cross-cohort evaluation.

2 Related Work

Prior work uses longitudinal EHR data and chest radiograph representations for lung cancer risk prediction, while emphasizing rigorous cohort definition, modality alignment, follow-up handling, and external validation. Imaging research has progressed from single-task classifiers toward longitudinal and multimodal risk modeling.

  • EHR-based prediction: EHR models transform diagnoses, procedures, medications, laboratory values, and visit sequences into patient-level lung cancer risk representations using recurrent, attention-based, and Transformer-based architectures, including Doctor AI, RETAIN, and Med-BERT.
  • Methodological requirements: Lung cancer incidence prediction requires careful index-time definition, prior-cancer exclusion, modality alignment, follow-up masking, and external validation, including when using PyHealth.
  • Medical imaging AI: Chest radiograph research has advanced from single-task classifiers toward longitudinal risk prediction and multimodal clinical modeling, supported by reusable pretrained representations and scalable feature-learning encoders.

3 Proposed Method

LUNG-KGMM integrates pre-index EHR data, leakage-sanitized radiology reports, multiview chest-radiograph features, and report-triggered guideline knowledge. Its modality-specific architecture uses concat fusion and horizon-masked cumulative-risk training for incomplete follow-up.

  • Radiology reports: Report processing removes direct cancer diagnosis, prior cancer, and treatment-history terms while retaining index-time findings and recommendations, with tokens capped at 256.MIMIC and Xiamen reports use different extracted term types, reflecting cohort-specific report content.
  • Image representation: The image branch uses 1024-dimensional TorchXRayVision DenseNet121 features, averaging frontal PA/AP and lateral embeddings into a study-level multiview representation.The default image baseline instead uses a CNN with grayscale 224-pixel chest-radiograph inputs.
  • Guideline knowledge: The final knowledge route expands documented report triggers into finding-attribute-action relation tokens using guideline-derived knowledge-graph structure.Relations encode concepts such as nodule size and density, persistence assessment, suspicious morphology, and diagnostic-workup recommendations; the KG re-encodes report evidence rather than independently observing or recommending care.
  • Model architecture: LUNG-KGMM combines EHR tokens, leakage-sanitized report tokens, multiview XRV image features, and knowledge-graph guideline terms through concat fusion.The prediction head outputs six cumulative risk logits for 1- through 6-year lung cancer incidence; concat fusion achieved the highest macro 1/3/6-year AUROC among evaluated fusion choices.
  • Training objective: Horizon-masked cumulative binary cross-entropy weights each of six logits by its follow-up mask and normalizes over observable horizons, leaving unobserved horizons excluded.Training uses Adam with learning rate 1e-3, weight decay 1e-5, gradient-norm limit 5.0, dropout 0.2, and early stopping patience 4.

4 Experiments

On MIMIC, LUNG-KGMM outperformed classic and multimodal baselines, achieving macro AUROC 0.885, macro AUPRC 0.507, and Brier score 0.034. Ablations showed the importance of sanitized reports, knowledge-graph guideline terms, and complementary multimodal inputs, while Xiamen supported real-world validation under privacy restrictions.

  • Cohorts: The study used MIMIC for multimodal development and method comparisons, while the Xiamen cohort provided real-world EHR and radiology-report validation data governed by privacy restrictions.MIMIC was assembled from MIMIC-IV 3.1, MIMIC-IV Note, MIMIC-CXR images, and MIMIC-CXR reports.
  • Method comparison: LUNG-KGMM achieved macro AUROC 0.885, macro AUPRC 0.507, and Brier score 0.034, exceeding MLP 0.870, MedFuse-style 0.825, and DrFuse-style 0.876 on MIMIC.These comparisons used the same MIMIC test split and 1/3/6-year incidence endpoints; the gains are modest point-estimate improvements and do not establish clinical benefit.
  • Clinical operating characteristics: At the top 10% risk threshold, positive predictive value was 0.135, 0.271, and 0.510 for 1-, 3-, and 6-year risk, with corresponding sensitivities of 0.764, 0.718, and 0.541.At specificity >=0.95, sensitivity was 0.709, 0.679, and 0.541 for 1-, 3-, and 6-year risk, respectively.
  • Single-modality: Report-only macro AUROC rose from 0.854 with original reports to 0.873 after leakage-sensitive sanitization, while EHR-only, CNN, and multiview XRV models reached 0.743, 0.620, and 0.787.Reports captured semantic risk information, including nodules, mass-like lesions, ground-glass components, suspicious morphology, and follow-up recommendations; sanitization removed explicit cancer and treatment-history language.
  • Guideline representation: Knowledge-graph guideline terms achieved macro AUROC 0.885 and AUPRC 0.507 versus 0.860 without guideline terms, outperforming structured terms, retrieved text, and executable paths.The relation-aware route linked findings, risk modifiers, and management concepts in an auditable representation, but remained partially redundant with the report rather than an independent modality.

1/3/6 AURO · 1/3/6 AUPRC

Across the reported 1/3/6 AURO and AUPRC evaluations, knowledge-graph terms achieved the strongest listed result, while multimodal configurations and alternative guideline representations produced the accompanying comparisons. Cross-cohort transfer without retraining showed that the task schema and architecture were portable, but fixed learned weights required local adaptation.

  • 1/3/6 AUPRC: EHR plus report achieved 0.845, 0.391, and 0.039, whereas EHR plus multiview XRV achieved 0.820, 0.357, and 0.041.The complete input combines EHR, leakage-sanitized report tokens, multiview XRV features, and a specified guideline representation.
  • 1/3/6 AUPRC: The guideline representations produced 0.876, 0.475, and 0.035 for structured terms plus score; 0.873, 0.499, and 0.034 for retrieved guideline text; and 0.868, 0.497, and 0.035 for executable path plus score.These are the three alternative guideline representations reported in the evaluation.
  • 1/3/6 AUPRC: The report-containing rows used leakage-sanitized report tokens unless explicitly labeled as original report, and complete input included EHR, report, multiview XRV features, and the specified guideline representation.This qualification defines the input construction for the modality and guideline comparisons.
  • 4.4 Real-world Validation in Xiamen E-cohort: Table 4 evaluated two no-retraining transfer settings: a primary no-image setting with EHR, leakage-sanitized report, and KG terms, and a negative sensitivity setting that added XRV features.The image-enabled setting used real MIMIC CXR embeddings but report-conditioned synthetic XRV features in Xiamen, and was weaker because of image-domain mismatch.
  • 4.4 Real-world Validation in Xiamen E-cohort: The transfer results showed that the task schema and architecture could be executed across cohorts, but learned weights were not directly transportable across English and Chinese reports or real and synthetic image representations.Deployment would require terminology harmonization, feature alignment, recalibration, and likely site-specific retraining because of differences in language, reporting, healthcare systems, and image availability.

5 Limitations

The study is limited by using chest radiographs rather than LDCT and by incorporating CT-oriented guidance into knowledge structures without establishing applicability to CXR pixels or guideline-adherent decisions. The sanitized report-only model also approached the complete model’s performance, indicating a limitation in attributing gains to multimodal integration.

  • 5 Limitations: MIMIC-CXR contains chest radiographs rather than LDCT, limiting the direct clinical scope of the multimodal framework.The knowledge graph documents radiology-report concepts but includes relations derived from CT-oriented guidance.
  • 5 Limitations: The knowledge graph must not be interpreted as applying Lung-RADS or Fleischner management criteria to CXR pixels or demonstrating guideline-adherent clinical decisions.
  • 5 Limitations: Macro AUROC was 0.873 for the sanitized report-only model versus 0.885 for the complete model, showing closely matched performance.

6 Conclusion

LUNG-KGMM provides a reproducible framework for 1-to-6-year lung cancer incidence prediction, integrating multimodal clinical data with leakage control, follow-up masking, and guideline-derived relations. Its task schema is also evaluated in a data-governed real-world clinical setting.

  • 6 Conclusion: LUNG-KGMM provides a reproducible framework for 1-to-6-year lung cancer incidence risk prediction.
  • 6 Conclusion: In MIMIC, the framework integrates longitudinal EHR, leakage-sanitized radiology reports, pretrained chest-radiograph representations, and report-triggered guideline relations.It preserves explicit cohort construction, follow-up masking, and modality ablation.
  • 6 Conclusion: Xiamen real-world validation demonstrates that the task schema can be evaluated in a data-governed clinical setting.

Supplementary Appendix for PRCV 2026 LUNG-KGMM … Xiamen Report-conditioned Synthetic-image Negative-control Analysis

The supplementary appendix defines leakage-aware, index-time multimodal experiments across public MIMIC and hospital-based Xiamen cohorts, then reports Xiamen validation and synthetic-image sensitivity analyses. LUNG-KGMM improved Xiamen performance when EHR, sanitized reports, and guideline knowledge were combined, while synthetic images provided limited incremental value and were not treated as real-imaging evidence.

  • Datasets and Experiment Setting: Experiments used index-time histories, report tokens, cumulative 1-6-year incidence labels, horizon-specific follow-up masks, and macro 1/3/6-year AUROC for model selection.Macro AUPRC and Brier score were secondary metrics, and the public development cohort combined MIMIC-IV, MIMIC-IV Note, MIMIC-CXR radiographs, and radiology reports while Xiamen data required local approval and security review.
  • Within-cohort, Synthetic-image and Cross-cohort Experiments: Within-cohort analyses used patient-level train/validation/test splits, while cross-cohort no-retraining tests assessed portability using no-image KG transfer and a negative image-domain sensitivity setting.MIMIC included real CXR-derived XRV features, whereas primary Xiamen experiments used EHR, sanitized reports, and KG terms; Xiamen’s synthetic subset used report-conditioned image features.
  • Radiology report processing and leakage-sensitive sanitization: Radiology reports were processed as index-time clinical interpretations with language-specific term extraction and a maximum length of 256 tokens, while main report-containing experiments removed direct diagnosis and treatment-history language.Sanitization preserved legitimate index-time risk findings while excluding cancer diagnosis, prior cancer, staging, treatment, and related terms.
  • Literature Baselines: MedFuse-style and DrFuse-style baselines followed the main experiments’ splits, masks, sanitized reports, available CXR inputs, optimization, and metrics but generally lacked LUNG-KGMM-specific KG and multiview XRV additions.This setup evaluates method families rather than whether every comparator benefits from the final model’s domain-specific components.
  • Supplementary Results: In the MIMIC test set’s top 10% predicted-risk group, PPV increased from 0.135 at 1 year to 0.510 at 6 years, indicating stronger enrichment of future lung-cancer events at longer horizons.The supplementary operating-point analysis also examined thresholds maintaining specificity at or above 0.95.
  • Xiamen Within-cohort Validation: Adding KG guideline terms to Xiamen EHR-plus-report modeling improved macro AUROC from 0.912 to 0.919, AUPRC to 0.683, and yielded a Brier score of 0.038.Table S2 reports EHR-only and report-only macro AUROCs of 0.873 and 0.870, respectively; the primary Xiamen analysis used no original imaging pixels because of privacy and data-access restrictions.
  • Xiamen Report-conditioned Synthetic-image Negative-control Analysis: The report-conditioned synthetic-image branch was an exploratory missing-image sensitivity analysis, with EHR plus synthetic-image features reaching macro AUROC 0.810 and KG-augmented multimodal modeling reaching 0.809 and AUPRC 0.219.Synthetic image features alone showed limited discrimination, and the findings do not establish that generated images replace real imaging pixels or add clear value beyond the original report text.
Loading 2608.14657v1…