Source-linked AI summary
Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging
Todd C. Hollon, Cheng Jiang, Asadur Chowdury, Mustafa Nasir-Moin, Akhil Kondepudi, Alexander Aabedi, Arjun Adapa, Wajd Al-Holou, Jason Heth, Oren Sagher, Pedro Lowenstein, Maria Castro, Lisa Irina Wadiura, Georg Widhalm, Volker Neuschmelting, David Reinecke, Niklas von Spreckelsen, Mitchel S. Berger, Shawn L. Hervey-Jumper, John G. Golfinos, Matija Snuderl, Sandra Camelo-Piragua, Christian Freudiger, Honglak Lee, Daniel A. Orringer
TL;DR
Timely molecular testing for diffuse gliomas is limited by uneven access, lengthy turnaround times, and challenging interpretation. DeepGlioma uses rapid SRH imaging and AI to screen key molecular alterations, accurately predicting the mutations defining adult-type diffuse glioma classification and potentially augmenting conventional diagnostic methods.
Problem
Molecular subgrouping of diffuse gliomas depends on laboratory testing that is not uniformly available, often takes days to weeks, and complicates clinical care and trial enrollment.
Method
DeepGlioma combines deep neural networks with stimulated Raman histology to screen molecular alterations from fresh glioma specimens using image features.
Results
DeepGlioma predicted IDH mutations, 1p19q-codeletion, and ATRX mutations from SRH images without fluorescence in-situ hybridization or genetic sequencing.
Takeaways & Limitations
DeepGlioma can provide rapid molecular screening that focuses confirmatory testing and preserves tumor specimens for definitive molecular testing.
Takeaways & Limitations
External testing was restricted to the US and Europe, so validation in diverse global demographics remains needed.
Abstract
from arXiv · showhide
Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment. However, timely molecular diagnostic testing for patients with brain tumors is limited, complicating surgical and adjuvant treatment and obstructing clinical trial enrollment. In this study, we developed DeepGlioma, a rapid ($< 90$ seconds), artificial-intelligence-based diagnostic screening system to streamline the molecular diagnosis of diffuse gliomas. DeepGlioma is trained using a multimodal dataset that includes stimulated Raman histology (SRH); a rapid, label-free, non-consumptive, optical imaging method; and large-scale, public genomic data. In a prospective, multicenter, international testing cohort of patients with diffuse glioma ($n=153$) who underwent real-time SRH imaging, we demonstrate that DeepGlioma can predict the molecular alterations used by the World Health Organization to define the adult-type diffuse glioma taxonomy (IDH mutation, 1p19q co-deletion and ATRX mutation), achieving a mean molecular classification accuracy of $93.3\pm 1.6\%$. Our results represent how artificial intelligence and optical histology can be used to provide a rapid and scalable adjunct to wet lab methods for the molecular screening of patients with diffuse glioma.
1 Introduction
Diffuse glioma molecular diagnosis is important but often inaccessible or slow because it depends on specialized laboratory testing and expert interpretation. DeepGlioma is proposed as a rapid, automated approach using optical imaging of fresh surgical specimens.
- 1 Introduction: Molecular markers increasingly define diffuse glioma diagnosis and treatment, but testing requires laboratory techniques that are not uniformly available.These techniques include immunohistochemistry, cytogenetic testing, and often next-generation sequencing.
- 1 Introduction: Long molecular-testing turnaround times can complicate prognostic prediction, surgical decisions, adjuvant therapy selection, and clinical-trial enrollment.The reported turnaround is days to weeks even in well-resourced settings.
- 1 Introduction: DeepGlioma addresses this gap through automated analysis of rapid optical imaging from fresh, unprocessed surgical specimens.The proposed system is intended to simplify molecular classification of diffuse gliomas.
2 Results
DeepGlioma combines rapid stimulated Raman histology with learned visual and genetic representations to screen key diffuse-glioma alterations and subgroups. Prospective and cross-validation results show strong classification performance, while ATRX prediction remains comparatively challenging but improves with additional training data.
- 2 Results: SRH images from 373 adult diffuse glioma patients train a convolutional neural network visual encoder using two Raman-shift image channels.The channels correspond to 2,845 cm^-1 and 2,930 cm^-1, and patches are sampled from whole-slide images.
- 2 Results: Public glioma genomic data train a genetic embedding model that represents mutation co-occurrence relationships associated with molecular subgroups.The learned genetic representation is intended to improve SRH-based classification.
- 2 Results: A transformer integrates pretrained SRH and genetic encoders during training, then predicts each gene’s mutational status from SRH patch embeddings alone during inference.Masked-gene prediction supplies the training objective while inference uses only optical-image information.
- 2 Results: 153 patients were evaluated prospectively across four tertiary medical centers in the United States and Europe to test generalization across populations, care settings, and imaging systems.The cohort comprised consecutive adults undergoing biopsy or resection for primary, non-recurrent diffuse glioma.
- 2 Results: ATRX prediction was consistently more challenging across leave-institution-out experiments, while its performance improved by at least +2% across all evaluation metrics versus prospective testing.The authors indicate that additional training data can address this challenge.
- 2 Results: 94.2% balanced accuracy was achieved for IDH status versus 91.4% for IDH1-R132H immunohistochemistry in the prospective cohort.For patients aged 55 years or younger, DeepGlioma reached 97.0% balanced accuracy versus 90.0% for IHC, and it correctly classified all non-canonical mutations in the cohort.
- 2 Results: 91.5% accuracy was achieved for three-way WHO CNS5 molecular subgroup classification, a +4.6% increase over the benchmark model.The gains were attributed to greater sensitivity for IDH-mutant gliomas and explicit modeling of mutation co-occurrences; accuracy reached 94.4% in patients aged 55 years or younger.
3 Discussion
DeepGlioma is presented as a rapid SRH-based screening adjunct for molecular diagnosis of diffuse gliomas, addressing uneven access and the delays of conventional testing. The authors propose that timely subgrouping could support surgical planning, clinical-trial enrollment, and broader diagnostic access, while noting demographic and interpretability limitations.
- Clinical contribution: DeepGlioma predicted IDH, 1p19q-codeletion, and ATRX mutations from SRH images without fluorescence in-situ hybridization or genetic sequencing.The system also automated molecular subtyping according to the WHO classification scheme.
- Clinical contribution: Rapid molecular screening could let clinicians confirm the most likely diagnostic mutations rather than use a diagnostic shotgun approach.Because SRH is non-consumptive, it preserves scant specimens for definitive molecular testing.
- Surgical care: DeepGlioma could support surgical goals by differentiating diffuse glioma subgroups for a better-calibrated risk-benefit analysis.The discussion links molecular subgrouping with tailoring surgical care.
- Clinical trials: Rapid molecular screening could initiate clinical-trial enrollment at the earliest stages of patient care.The discussion notes that trial eligibility is often defined by genetic subgroups and that fewer than 10% of glioma patients are enrolled.
- Limitations: The external testing cohort was restricted to the US and Europe, and the model is not directly interpretable.The authors identify validation in a diverse global demographic and discovery of predictive optical features as future needs.
- Conclusion: The study concludes that AI-based screening has potential to augment conventional diagnostics and improve the access and speed of molecular diagnosis.This conclusion is stated within the scope of brain tumor care.
4 Methods
The methods combine clinical SRH imaging, deep visual representation learning, and genomic information to build a rapid molecular-classification workflow. The pipeline processes fresh surgical specimens into image patches and trains multi-label models using contrastive visual features and genetic embeddings.
- Study objectives: The study aimed to develop a rapid SRH-based deep-learning tool for WHO CNS5 classification and test it prospectively in a large multicenter cohort.The objectives also included testing whether diagnostic mutations produce learnable spectroscopic, cytologic, and histoarchitectural changes.
- SRH acquisition: Fresh viable tumor specimens were imaged with fiber-laser-based stimulated Raman scattering microscopy during surgery.The operating surgeon provided grossly lesional-appearing tissue for SRH imaging.
- SRH preprocessing: SRH images were acquired as raw 16-bit channels, registered across sequential acquisitions, and split into non-overlapping 300x300-pixel patches.A subtraction of the registered channels generated a red channel highlighting nuclei and protein-rich structures.
- Visual representation learning: A ResNet50 visual encoder learned image representations, while larger models did not provide better performance.The visual encoder was used for SRH feature extraction before multi-label classification training.
- Visual representation learning: The supervised contrastive objective used a distinct projection layer for each label to accommodate multi-label classification.Paired positive examples were sampled from the same label, and the final loss was label-weighted.
- Optimization: Models were trained for 50 epochs with Adam, an initial learning rate of 0.001, cosine annealing, and a batch size of 256.Training also used random cropping, Gaussian blur, flipping, and random erasing.
- Visual representation learning: The patchcon strategy retained the visual encoder after contrastive pretraining for subsequent multi-label classification training.The method is described as weakly supervised, patch-based contrastive representation learning.
4.6 Diffuse glioma genetic embedding
The method learns a genetic embedding from large-scale glioma genomic data and integrates it with visual features for multi-label molecular classification. Masked-label transformer training explicitly models relationships among genetic alterations.
- Genomic embedding: 2777 diffuse glioma patients contributed public genomic data for pretraining the genetic embedding model.Sources included TCGA, CGGA, ICGC, Rembrandt, MSK Data Catalog and the Mayo Glioblastoma Xenograft National Resource.
- Genomic embedding: The genetic embedding represents gene relationships using a pairwise co-occurrence matrix of mutations observed in the same tumors.Global vector embeddings are weighted to avoid overweighting the most common co-occurrence pairs.
- Multi-label classification: Two multi-label strategies were tested: independent linear binary relevance classifiers and a transformer that models genetic-label dependencies.The transformer uses attention to model mutation co-occurrence in the context of pretrained visual features.
- Multi-label classification: Masked-label training randomly hides genetic mutations and trains the model to predict the masked genes.The approach provides semantically informative supervision for multi-label classification.
- Transformer design: The transformer output latent space is constrained to match the pretrained genetic embedding space rather than using a conventional classification token or feedforward head.The embedding-layer weights remain fixed during transformer encoder training.
4.8 Whole slide segmentation, patient inference, and molecular subgrouping
Whole-slide inference aggregates patch-level tumor segmentation and molecular predictions into patient-level genetic outputs. An algorithm then maps these predictions to mutually exclusive adult-type diffuse glioma subgroups.
- Whole slide segmentation: Whole-slide SRH images are patched, and a segmentation model classifies each patch as tumor, normal brain, or nondiagnostic tissue.Patch predictions are aggregated for patient-level diagnosis, with tumor content considered during inference.
- Patient inference: Patient-level molecular inference combines DeepGlioma outputs across patches while weighting predictions by the tumor segmentation model.The inference function uses tumor-classified patches to aggregate molecular predictions.
- Molecular subgrouping: Algorithm 1 returns glioblastoma, IDH-wildtype when the patient-level IDH prediction is below the threshold τ = 0.5.The rule is applied before the IDH-mutant subgroup branches.
- Ablation experiments: Three ablations tested visual learning loss, multi-label classification strategy, and masked-label supervision using balanced hold-out validation sets of 20 patients.Each validation set contained 10 IDH-mutant and 10 IDH-wildtype tumors.
4.10 Molecular heatmap generation
DeepGlioma generates interpretable molecular heatmaps by pooling overlapping patch predictions and retaining regions that satisfy subgroup-specific molecular conditions. These maps also support evaluation across molecular settings and inspection of classification errors.
- Heatmap generation: Dense patch sampling at 100-pixel steps enables local probability pooling from overlapping SRH predictions.Tumor-segmented regions are retained while normal and nondiagnostic regions are converted to corresponding molecular heatmap values.
- Molecular subgroup heatmaps: Oligodendroglioma heatmaps use a conditional mask requiring IDH and 1p19q-codeletion probabilities to exceed their thresholds.The mask identifies spatial regions satisfying the molecular subgroup criteria.
- Molecular subgroup heatmaps: Astrocytoma heatmaps use a conditional mask requiring IDH positivity together with either low 1p19q-codeletion or high ATRX probability.The thresholds τ, φ and π were all set to 0.5 without hyperparameter tuning.
- Evaluation and errors: Molecular heatmaps enabled evaluation in different molecular settings and identification of potential error sources such as low tumor infiltration and image quality.They also supported correct IDH-wildtype prediction in patients with recurrent CDKNA1 and TERT promoter mutations.
- Prospective testing: Prospective testing used a balanced classification accuracy endpoint for diffuse glioma IDH mutational status.The control was conventional laboratory screening, while the experimental arm used DeepGlioma predictions.
5 Data availability
Public genomic training data are available through the cited repositories, but raw patient imaging and genetic data from the clinical study are not publicly available. Sharing requires institutional review and a material transfer agreement.
- Public genomic data: The genomic training data are publicly available through the listed repositories and are provided in Extended Data Table 2.Institutional Review Board approval covered SRH imaging and data collection.
- Restricted patient data: Raw patient imaging and genetic data are restricted and not publicly available because they were used under institutional permission.Requests are evaluated for intellectual-property and patient-privacy obligations and limited to non-commercial academic purposes.
6 Code availability
The paper’s code and experiment-reproduction scripts are publicly available on GitHub.
- The code was implemented in Python 3.8 using PyTorch 1.9.0.
- All code and scripts for reproducing the experiments are available at MLNeurosurg/deepglioma on GitHub.
A Extended Data Figures
The extended-data figures document DeepGlioma’s workflow, datasets, representation learning, model evaluation, subgroup analyses, heatmaps, non-canonical cases, and error analysis.
- Extended Data Figure 1 presents the overall workflow of intraoperative SRH and DeepGlioma.
- Extended Data Figures 2–6 cover the training dataset, multi-label contrastive learning, SRH representation comparisons, genetic embeddings, and ablation studies with cross-validation.
- Extended Data Figures 7–8 report patient demographic and molecular subgroup analyses of DeepGlioma performance.
- Extended Data Figures 9–10 show molecular heatmaps and evaluation on non-canonical diffuse gliomas.
- Supplementary Figures 1–2 provide qualitative heatmap analysis of molecular glioblastomas and error analysis.
B.1 Supplementary Data Tables (Available Online)
The supplementary data tables cover DeepGlioma’s training, genomic aggregation, and prospective testing datasets, including multiclass predictions.
- The supplementary tables include the DeepGlioma training dataset.
- They include an aggregated public diffuse glioma genomic dataset.
- They include a prospective multicenter testing dataset with DeepGlioma multi-label predictions.
- They include a prospective multicenter testing dataset with multiclass model predictions.
B.2 Supplementary Figures
The supplementary figures detail DeepGlioma’s imaging workflow, training data, representation-learning methods, evaluation results, subgroup behavior, molecular heatmaps, and classification errors.
- Workflow: The workflow loads a fresh surgical specimen for SRH imaging and uses DeepGlioma to screen molecular alterations from optical images.The system targets rapid molecular screening without tissue processing or human interpretation.
- Training dataset: The training dataset contains 373 adult diffuse glioma patients collected from November 2015 through November 2021.The dataset includes IDH-wildtype, IDH-mutant/1p19q-codeleted, and IDH-mutant/1p19q-intact groups.
- Representation learning: DeepGlioma uses multi-label contrastive learning to form visual representations from SRH image patches.Label-specific projectors compute contrastive losses that are summed and backpropagated through a shared image encoder.
- Representation learning: PatchCon produces more uniformly distributed SRH embeddings than cross-entropy training and organizes representations along IDH and 1p19q-codeletion axes.The comparison includes randomly initialized, cross-entropy-trained, and PatchCon encoders.
- Genetic embedding: The genetic embedding model uses public genomic data and clusters mutations that co-occur within diffuse glioma molecular subgroups.Both mutant and wildtype statuses are included in the learned vector space.
- Subgroup analyses: Demographic analyses show high performance across patient age, race, and sex, while molecular heatmaps illustrate subgroup-specific prediction patterns.IDH classification remains high across age groups, although older patients have greater class imbalance and proportional performance drops from false negatives.
- Special cases and errors: Supplementary cases evaluate non-canonical diffuse gliomas and molecular glioblastomas, while error analysis identifies low tumor infiltration and dense atypical features as challenges.DeepGlioma correctly identifies some molecular alterations despite negative IDH1-R132H immunohistochemistry in an IDH2-mutant case.