Source-linked AI summary
Hierarchical MoE for Multi-Modal ILD Diagnosis
Alec K. Peltekian, Gorkem Durak, Halil Ertugrul Aktas, Carrie Lynn Richardson, Mary Carns, Kathleen Aren, GR Scott Budinger, Anthony J. Esposito, Alexander Misharin, Alok Nidhi Choudhary, Ankit Agrawal, Ulas Bagci
TL;DR
The paper tackles ILD classification when regional imaging patterns and clinical information vary in diagnostic value across patients. It introduces a hierarchical multimodal MoE with a frozen imaging expert, grouped EHR sub-experts, and two-stage gating. Under patient-level cross-validation, the selective model achieved the highest mean AUC among evaluated methods, although its improvement over imaging-only REN was not statistically significant.
Problem
Existing ILD models often process the lung globally or use fixed multimodal fusion, while imaging-only structured MoE models omit clinically important EHR information.
Method
A hierarchical multimodal MoE combines five lobe-specific imaging experts with clinically grouped EHR sub-experts through EHR and modality-level gating.
Results
0.8750±0.0443 mean AUC was achieved by the selective hierarchical MoE, versus 0.8646±0.0467 for imaging-only REN.
Takeaways & Limitations
The framework provides case-specific summaries of anatomical, modality, and EHR-group utilization while achieving the highest mean AUC among evaluated methods.
Takeaways & Limitations
The single-system, scleroderma-associated ILD cohort may limit generalizability, and learned gate activations do not establish causal clinical importance.
Abstract
from arXiv · showhide
Mixture-of-experts (MoE) models combine specialized predictors under learned routing, offering a principled mechanism for leveraging heterogeneity in medical data. We present a hierarchical multimodal MoE for interstitial lung disease (ILD) classification that integrates a frozen, pre-trained imaging expert with structured electronic health records (EHR) via two-stage gating. A modality-level gate assigns patient-specific weights to imaging and EHR predictions, while a sub-gating module decomposes the EHR branch into clinically defined feature groups with learned, group-specific contributions. This design preserves stable imaging representations while enabling input-dependent clinical weighting and explicit EHR specialization. Under strict patient-level cross-validation, the model achieved the highest mean AUC among the evaluated methods (0.8750 +- 0.0443), compared with 0.8646 for imaging-only REN and 0.7685 for SwinUNETR. The framework extends interpretability across anatomical regions, imaging--EHR utilization, and clinically defined EHR feature groups.
1 Introduction
The paper addresses the limits of global imaging and fixed multimodal fusion for heterogeneous, regionally distributed ILD by introducing structured, patient-specific routing across imaging and EHR information.
- ILD has heterogeneous, regionally distributed involvement, but most deep-learning approaches process the lung globally, diluting regional signals and limiting interpretability.
- Existing anatomically structured MoE models improve region-aware ILD classification but remain imaging-only and omit structured clinical information central to diagnosis.
- Patient-specific modality relevance makes fixed fusion potentially biased because imaging and structured EHR provide complementary information whose diagnostic value varies across patients.
- Routing mechanisms allocate representational capacity across modality- and domain-specialized experts, combining feature fusion with anatomical and clinical structure.
- The proposed model uses two-level gating: a modality gate weights imaging and EHR, while an EHR sub-gate learns contributions from clinically defined feature groups.
2 Methods
The method combines anatomically structured CT experts, clinically grouped EHR sub-experts, and adaptive modality fusion, trained with a frozen imaging branch under patient-level cross-validation.
- Dataset and Preprocessing: 597 patients with 1,898 longitudinal chest CT scans formed the retrospective cohort, including 365 ILD-positive patients.
- Dataset and Preprocessing: CT volumes were resampled, segmented into five anatomical lobes, intensity-windowed, and resized to 96×96×96 voxels.
- Radiomics-Guided Lobe Importance: 107 radiomics features per lobe were used to train fold-specific XGBoost classifiers whose validation AUCs determined fixed lobe-importance weights.
- Lobe-Aware Imaging Expert: Five lobe-specific 3D SwinUNETR experts process masked regions, aggregate with radiomics-derived weights, and produce a 48-dimensional imaging representation.
- Hierarchical EHR Expert: EHR variables were partitioned into clinically coherent groups, with full and selective hierarchies using seven groups and 69 variables versus three groups and 12 variables.
- Hierarchical EHR Expert: Group-specific EHR sub-experts use input-dependent routing weights, while the modality gate adaptively balances imaging and EHR representations before ILD classification.
- Training Protocol: Training proceeds from lobe-importance estimation to imaging training and multimodal optimization with the imaging expert frozen.
3 Results
The selective hierarchical MoE achieved the strongest mean AUC among evaluated methods, while analyses linked selective clinical routing to balanced modality use and calibration-related behavior. Results also assessed temporal linkage, longitudinal stability, fusion baselines, and interpretability limits.
- Performance: 0.8750±0.0443 mean AUC was achieved by the selective hierarchical MoE, versus 0.8646±0.0467 for imaging-only REN, but the paired difference was not statistically significant.The selective model significantly outperformed SwinUNETR, while the numerical improvement over REN requires confirmation in a larger evaluation.
- Hierarchical EHR grouping: 0.8750 versus 0.8496 mean AUC favored the selective over the full hierarchy, which distributed routing across 69 variables and seven groups.The selective configuration concentrated routing on pulmonary function, disease phenotype and serology, and key biomarkers; this interpretation does not establish that excluded variables are uninformative.
- Robustness and longitudinal stability: 0.0812±0.0964 was the mean within-patient standard deviation of predicted ILD probability among patients with more than one CT scan.Temporal EHR–CT grace-window changes from 0 to 90 days produced negligible AUC changes, while variability was slightly lower in ILD-positive than ILD-negative patients.
- Fusion baselines: 0.8750±0.0443 AUC was highest among evaluated fusion strategies, ahead of REN, concatenation-logistic regression, concatenation-MLP, and EHR-only logistic regression.These comparisons do not independently isolate the contributions of the modality gate and EHR sub-gate.
- Interpretability: Gate activations represent routing behavior rather than causal feature importance, so higher expert utilization does not establish independent clinical importance.The interpretability analysis examined modality-level and EHR sub-expert gates, including ILD-stratified activations, feature-group weights, confidence relationships, and selective-versus-full configurations.
- Calibration and error analysis: 0.131±0.038 ECE and 0.135±0.037 Brier score indicated reasonable calibration across five folds.Among 960 test samples, 17.4% were misclassified and 42.5% of errors occurred at confidence ≥0.8; high-confidence errors were associated with imaging-dominant routing.
4 Discussion and Concluding Remarks
The hierarchical multimodal MoE achieved the highest mean AUC among evaluated methods and provided interpretable, case-specific routing summaries. However, its generalizability and component-level contributions remain constrained by cohort scope, preprocessing assumptions, and unevaluated gating components.
- The framework provided case-specific summaries of anatomical, modality, and EHR-group utilization.Its routing spans imaging, EHR, and clinically defined feature groups.
- The numerical improvement over imaging-only REN was not statistically significant.The authors therefore motivate further evaluation in larger and external cohorts.
- Single-system sampling and focus on scleroderma-associated ILD may limit generalizability to other etiologies.The study also identifies potential error propagation from automated segmentation and limitations from preprocessing missing EHR values.
- The current comparisons do not independently isolate the contributions of the modality gate and EHR sub-gate.Future work includes component-wise gating ablations and multi-institutional external validation.