Source-linked AI summary
Deep Learning Enables Automatic Detection and Segmentation of Brain Metastases on Multi-Sequence MRI
Endre Grøvik, Darvin Yi, Michael Iv, Elisabeth Tong, Daniel L. Rubin, Greg Zaharchuk
TL;DR
Brain metastasis detection and segmentation is laborious, particularly in multi-sequence 3D MRI, while few machine-learning studies had addressed this setting. The study applies a fully convolutional CNN to multi-sequence MRI and reports high detection and segmentation performance, supporting its potential clinical use.
Problem
Manual detection and segmentation of brain metastases is tedious and time-consuming, while few ML studies had applied such approaches to metastases.
Method
A modified 2.5D GoogLeNet fully convolutional CNN detected and segmented metastases from multi-sequence MRI using manually delineated ground truth.
Results
AUC was 0.98 overall, while precision, recall, and Dice-score were 0.79, 0.53, and 0.79, respectively, at the average optimal threshold.
Takeaways & Limitations
Deep learning using multi-sequence MRI can aid detection and segmentation of brain metastases in a clinically relevant setting.
Takeaways & Limitations
The study is limited by its small sample size and single-center retrospective design, and multi-site testing remains necessary to assess clinical value.
Abstract
from arXiv · showhide
Detecting and segmenting brain metastases is a tedious and time-consuming task for many radiologists, particularly with the growing use of multi-sequence 3D imaging. This study demonstrates automated detection and segmentation of brain metastases on multi-sequence MRI using a deep learning approach based on a fully convolution neural network (CNN). In this retrospective study, a total of 156 patients with brain metastases from several primary cancers were included. Pre-therapy MR images (1.5T and 3T) included pre- and post-gadolinium T1-weighted 3D fast spin echo, post-gadolinium T1-weighted 3D axial IR-prepped FSPGR, and 3D fluid attenuated inversion recovery. The ground truth was established by manual delineation by two experienced neuroradiologists. CNN training/development was performed using 100 and 5 patients, respectively, with a 2.5D network based on a GoogLeNet architecture. The results were evaluated in 51 patients, equally separated into those with few (1-3), multiple (4-10), and many (>10) lesions. Network performance was evaluated using precision, recall, Dice/F1 score, and ROC-curve statistics. For an optimal probability threshold, detection and segmentation performance was assessed on a per metastasis basis. The area under the ROC-curve (AUC), averaged across all patients, was 0.98. The AUC in the subgroups was 0.99, 0.97, and 0.97 for patients having 1-3, 4-10, and >10 metastases, respectively. Using an average optimal probability threshold determined by the development set, precision, recall, and Dice-score were 0.79, 0.53, and 0.79, respectively. At the same probability threshold, the network showed an average false positive rate of 8.3/patient (no lesion-size limit) and 3.4/patient (10 mm3 lesion size limit). In conclusion, a deep learning approach using multi-sequence MRI can aid in the detection and segmentation of brain metastases.
A PREPRINT
The preprint lists the study authors and their institutional affiliations. It identifies co-first authorship and equal contributions.
- Endre Grøvik, Darvin Yi, Michael Iv, Elisabeth Tong, Daniel L. Rubin, and Greg Zaharchuk are listed as authors.
- Grøvik and Yi share co-first authorship, while Iv, Tong, Rubin, and Zaharchuk contributed equally.
- The authors are affiliated with radiology, diagnostic physics, and biomedical data science departments.
INTRODUCTION
Brain metastases are a common complication of metastatic cancer, and contrast-enhanced MRI is central to their diagnosis and follow-up. Manual detection and segmentation remain demanding, motivating deep learning approaches for multi-sequence MRI.
- Contrast-enhanced MRI is the key imaging technique for diagnosing brain metastases and assessing treatment response longitudinally.
- 12.1% of patients with metastatic disease had brain metastases at diagnosis in a survey of more than 26,000 patients.
- Manual metastasis detection and segmentation are tedious and time-consuming with multi-sequence 3D imaging, despite requiring high accuracy.
- Volumetric analysis can add cost and complexity and is unavailable at all centers when performed manually by radiologists.
- A fully convolutional CNN is used to automatically detect and segment brain metastases from multi-sequence MRI because few ML studies had addressed this setting.
Patient Population and Imaging Parameters
The retrospective study included patients with known or possible metastatic disease and complete required MRI sequences. Imaging was acquired across 1.5T and 3T clinical scanners from major vendors.
- The retrospective inclusion criteria required known or possible metastatic disease, no prior surgery or radiation therapy, and all required MR sequences.
- Only patients with at least one metastatic lesion were included, and mild patient motion was not an exclusion criterion.
- The protocol covered examinations performed at both 1.5T and 3T across all major clinical scanner vendors.
- Imaging included pre- and post-gadolinium T1-weighted 3D CUBE, post-gadolinium 3D axial IR-prepped FSPGR, and 3D CUBE FLAIR sequences.
Image Segmentation and Co-registration
Two experienced neuroradiologists manually delineated enhancing lesions to establish ground truth. The multimodal images were co-registered to IR-FSPGR space, and brain masks were propagated across sequences.
- Two experienced neuroradiologists manually delineated and cross-checked each enhancing lesion to establish ground-truth segmentations.
- Lesions were outlined slice by slice on post-gadolinium 3D T1-weighted IR-FSPGR images, with guidance from 3D FLAIR and post-gadolinium 3D T1 spin-echo data.
- Pre- and post-contrast T1 CUBE and FLAIR images were co-registered to IR-FSPGR space using normalized mutual information.
- Brain masks were generated from pre-contrast T1-weighted 3D CUBE images using BET and propagated to the other sequences.
Convolutional Neural Network Details
The study used a modified 2.5D fully convolutional GoogLeNet to process multi-sequence MRI and produce voxel-level metastasis probability maps. Training used lesion-enriched sampling and a stratified test-set design.
- Architecture: The network used a 2.5D fully convolutional architecture based on modified GoogLeNet.The architecture reduced down-sampling and replaced fully connected layers with a convolutional transpose layer.
- Output: The network output a voxel-wise metastasis probability map ranging from 0 to 1.The output colormap was overlaid on a post-contrast BRAVO image.
- Input representation: Seven slices from each of four MRI sequences formed a 28-channel input.The slices comprised one center slice plus three slices above and below for each sequence.
- Training: Lesion-enriched sampling placed lesion-containing frames in at least half of each training batch.Sixteen of 32 images per batch came from frames containing at least some ground-truth lesion.
- Evaluation design: The test set was stratified into patients with 1-3, 4-10, and >10 metastases, with 17 cases selected from each group.The split contained separate training, development, and test cases.
Statistical Analysis
Detection was evaluated with voxel-level ROC analysis and subgroup comparisons, while thresholded predictions were assessed for detection and segmentation accuracy using complementary metrics.
- ROC analysis: Voxel-level detection was evaluated using patient-specific ROC curves and area under the curve within the brain mask.Sensitivity and specificity were derived from the cut-off maximizing Youden’s index.
- Thresholded evaluation: A development-set probability threshold was used to evaluate precision, recall, and Dice/F1 segmentation accuracy.Lesion-level false positives were also counted with and without a 10 mm3 lesion-size criterion.
- Statistical comparison: Subgroup detection and segmentation metrics were compared using the Wilcoxon rank sum test with a 5% significance level.The subgroups were defined by the number of metastases per patient.
RESULTS
The network was evaluated on 51 patients containing 856 lesions and showed strong voxel-level discrimination across metastasis-count subgroups. Thresholded performance included moderate recall, a Dice score of 0.79, and fewer false positives when small objects were excluded.
- RESULTS: 51 test patients contained 856 lesions, following training on 100 cases and development on 5 cases.The cohort included multiple primary malignancies and a mean age of 63±12 years.
- RESULTS: AUC was 0.98±0.04 overall, with subgroup AUCs of 0.99±0.01, 0.97±0.05, and 0.97±0.03 for 1-3, 4-10, and >10 metastases.At the optimal cut-off, sensitivity and specificity were 94% and 97%, respectively.
- RESULTS: 0.79±0.20 precision, 0.53±0.22 recall, and 0.79±0.12 Dice-score were obtained using the average development-set threshold of 0.93.These metrics were calculated after selecting a common probability threshold for including a voxel as metastasis.
- RESULTS: 8.3 false-positive lesions per case without a size limit decreased to 3.4 per case when only lesions ≥10 mm3 were counted.The corresponding average sensitivity was 83%.
- RESULTS: Patients with few metastases had the highest sensitivity and lowest numbers of false positives.Subgroup differences were reported for detection and segmentation metrics.
DISCUSSION
The modified 2.5D GoogLeNet CNN showed high-accuracy brain-metastasis detection and segmentation on diverse multi-sequence MRI, including patients across lesion-count subgroups. Its potential clinical utility is balanced by limitations involving study scope, false positives, input-sequence requirements, and unresolved architecture comparisons.
- Main findings: The modified 2.5D GoogLeNet CNN detected and segmented brain metastases on multi-sequence MRI with high accuracy and demonstrated subgroup performance.The study reports subgroup analysis across patients with few, moderate, and extensive metastatic disease.
- Generalizability: The study included diverse imaging conditions, using both 1.5T and 3T multi-vendor scanners and patients with extensive disease, rather than restricting evaluation to SRS cases.The authors identify this diversity as a distinguishing feature from prior studies and as a more challenging test of overall performance.
- Clinical relevance: The network combines visualization, quantification, and segmentation into one step, producing outputs applicable to radiotherapy planning with minimal user interaction.This addresses a workflow in which detection and segmentation are time-consuming and subject to interobserver variation.
- Limitations: The findings remain limited by a small single-center retrospective sample, false positives near skull-base vascular structures or cortex, and dependence on four MRI contrasts.The authors also state that systematic comparison of 2D, 2.5D, and 3D architectures and testing on multi-site data remain necessary.