Source-linked AI summary
Scale-Aware 3D Deep Learning for Robust Brain Metastasis Detection in Multimodal MRI
Sylvain Jaume, Hongming Wang, Simon K. Warfield
TL;DR
Brain metastasis detection is difficult because lesions vary in size and small lesions occupy little of a 3D input. The framework fuses whole-volume predictions from 96^3 and 64^3 3D U-Nets, improving precision, lesion-level F1, and false-positive burden on 97 development patients, while agreement filtering did not improve F1.
Problem
Brain metastases vary widely in size and location, creating spatial-scale imbalance that makes small-lesion detection difficult.
Method
The method applies independently trained 96^3 and 64^3 3D U-Nets to complete MRI volumes and combines their aligned probability maps through weighted late fusion.
Results
Cross-FOV fusion improved precision and lesion-level F1 while substantially reducing false positives relative to individual models, and exceeded same-FOV fusion in F1 and false-positive burden.
Takeaways & Limitations
Cross-FOV probability fusion is a simple, computationally practical way to use complementary spatial context, but it does not solve very-small-lesion detection.
Takeaways & Limitations
The same-FOV control used five rather than 50 training epochs, so the observed cross-FOV advantage is supportive rather than definitive evidence for field-of-view effects.
Abstract
from arXiv · showhide
Detecting brain metastases in magnetic resonance imaging (MRI) remains challenging because lesions vary widely in size and appearance, with very small metastases occupying only a minute fraction of a three-dimensional input. We investigate whether combining different spatial fields of view (FOVs) improves lesion detection in multimodal MRI and present a scale-aware 3D deep-learning framework. The method uses independently trained $96^3$ and $64^3$ 3D U-Nets whose whole-volume probability maps are combined by weighted late fusion. This design allows us to study the effect of spatial context separately from image resolution and modality choice. On a 97-patient development cohort, cross-FOV fusion improved lesion-level precision and F1 while substantially reducing false positives relative to the individual models. A same-FOV ensemble control showed that these gains were not explained solely by averaging independently trained networks, supporting a contribution from complementary spatial context. An exploratory cross-FOV agreement filter reduced false positives but did not improve overall F1. These results support cross-FOV probability fusion as a simple and computationally practical strategy for improving the precision-false-positive trade-off in 3D brain-metastasis detection.
1 Introduction
Brain metastases span sizes and locations, making small-lesion detection difficult for 3D networks. This study examines whether controlled differences in spatial field of view and probability fusion improve lesion-level detection.
- Small metastases occupy little of a 3D input, while larger fields of view provide context but reduce their relative representation.
- The study controls the backbone while testing sampling, spatial FOV, and post-hoc probability fusion for lesion-level detection.
- The framework compares compact 963 and 643 3D U-Net inputs, with sensitivity stratified by lesion volume and operating-point analyses.
2 Methods
Experiments use the multi-institutional BraTS-METS dataset with multimodal MRI and a fixed 97-patient internal development cohort. Evaluation preserves lesion-size strata and distinguishes whole-volume lesion results from patch-level validation evidence.
- Experiments use multi-institutional pre- and post-treatment multiparametric MRI from the BraTS-METS dataset without pretrained segmentation weights.
- The DEV97 cohort contains 97 patients and 667 connected-component ground-truth lesions, with whole-volume lesion results reported separately from patch validation.
- T1c and T2-FLAIR provide complementary tumor and surrounding-signal contrasts on a shared 1-mm isotropic grid, while no modality ablation was performed.
- Lesions are evaluated in six fixed physical-volume strata ranging from below 0.010 mL to at least 0.500 mL.
2.3 Baseline 3D Network
The baseline is a compact 3D U-Net that processes multimodal MRI through a shared architecture and restores predictions to the 1-mm input grid. The proposed workflow applies the same design at two input FOVs and fuses their whole-volume probability maps.
- The shared U-Net uses an encoder–bottleneck–decoder with skip connections and outputs a binary tumor logit map at the input resolution.
- The dual-FOV workflow applies independently trained 3D U-Nets to T1c and T2-FLAIR, then combines reconstructed probability maps by weighted late fusion.
- The two branches retain the same architecture and training procedure while differing in their 963 versus 643 input fields of view.
2.4 Patch-Based Training
Patch training uses lesion-centered positives and non-lesional negatives with a shared optimization setup across FOV models. A shorter seed-43 control tests whether ensemble gains could arise from averaging independently trained networks, but limits that comparison.
- Positive patches are lesion-centered and negatives sample non-lesional brain regions, with approximately balanced sampling and shared Dice–focal optimization.
- Both models train with Adam at 10^-4, batch size 2, and up to 50 epochs, retaining the best validation-Dice checkpoint.
- The baseline field of view is 963 voxels.
- The seed-43 643 control trained for five epochs rather than the primary 50-epoch schedule, so it may underestimate a fully optimized same-FOV ensemble.
2.5 Small-Lesion-Aware Sampling
The method increases small-lesion representation during patch sampling while keeping the network architecture and underlying lesion unchanged across FOVs.
- Positive patches were weighted 4, 3, 2, and 1 across decreasing lesion-volume strata, while negatives received unit weight.
- Both FOV models share the same compact 3D U-Net architecture and differ in input field of view.
- 55.6% of positive patches represented lesions < 0.025 mL, with 2,504 positive and 2,496 negative patches sampled.
- The 643 model uses center-cropped 963 patches, enlarging lesion occupancy while preserving the underlying training lesion.
2.7 Scale-Aware Multi-Field-of-View Inference
Scale-aware inference combines separately trained 963 and 643 networks only after reconstructing aligned whole-volume probability maps, enabling post-hoc FOV fusion without changing either backbone.
- Separately trained 963 and 643 networks interact only after whole-volume probability reconstruction, isolating field of view and post-hoc fusion effects.
- Each network uses overlapping sliding-window inference to produce spatially aligned whole-volume probability maps.
- The fused map is a weighted combination of the 963 and 643 probability maps, with α assigned to the large-context predictor.
- Fusion weights can be evaluated without retraining, but α may reflect both complementary errors and calibration differences because no calibration was applied.
2.8 Whole-Volume Lesion Extraction
Whole-volume predictions are thresholded and converted into individual lesion candidates through connected-component analysis and minimum-volume filtering.
- Thresholded probability maps become binary tumor predictions, and connected-component analysis identifies individual predicted lesions.
- Components below a configurable minimum physical volume are removed before lesion measurements are recorded.
- The primary fusion comparison used a 0.010 mL filter that may disadvantage predictions from the smallest reference-lesion stratum.
2.9 Lesion Matching and Evaluation
Lesion-level evaluation uses greedy one-to-one IoU matching and reports sensitivity, precision, F1, and false positives per patient, with fusion assessed at a common component-volume threshold.
- Predicted and reference components are greedily matched one-to-one at IoU ≥0.10, defining TP, FP, and FN for lesion-level metrics.
- Sensitivity, precision, F1, and FP/patient quantify detection performance after matching.
- The retained 643 checkpoint and challenge-container setup limit the official score as an external replication of the separately trained internal ensemble.
- Fusion-weight ablation reports the threshold with highest observed lesion-level F1 for each weight at a common 0.010 mL component volume.
3 Results
Cross-FOV probability fusion improved lesion-level precision and F1 while reducing false positives on the 97-patient development cohort. Benefits persisted across threshold sweeps and exceeded same-FOV ensemble performance, although very small lesions remained difficult and external validation was limited.
- 3.3 Multi-Field-of-View Fusion: At α = 0.60, fusion achieved sensitivity 0.582, precision 0.602, F1 0.592, and 2.639 FP/patient, improving F1 17.2% and reducing FP/patient 57.2% versus the 963 endpoint.Bootstrap analysis estimated a paired F1 difference of +0.087 (95% CI 0.024–0.150; p = 0.003).
- 3.4 Same-FOV Ensemble Control: Cross-FOV fusion outperformed same-FOV ensembling, improving F1 by 0.096 and precision by 0.158 while reducing FP/patient by 1.763.The sensitivity difference was not significant, supporting an additional precision/F1 and false-positive benefit from cross-FOV diversity.
- 3.5 Lesion-Size Performance of the Selected Fusion Model: Sensitivity increased from 0.057 below 0.010 mL to 0.891 at ≥0.500 mL, while fusion’s main benefit within size strata was precision and F1 through false-positive suppression.The size-stratified analysis used the selected α = 0.60 fusion with threshold 0.60 and a 0.010-mL component filter.
- 3.5 Lesion-Size Performance of the Selected Fusion Model: Removing component filtering increased sensitivity from 0.582 to 0.619 but raised FP/patient from 2.64 to 6.32 and reduced F1 from 0.592 to 0.488.The 0.010-mL filter therefore represented an empirical trade-off between very-small-lesion sensitivity and false-positive suppression.
- 3.6 Operating-Point Robustness: F1 reached 0.593 with cross-FOV fusion, versus 0.515 for the 963 model and 0.413 for the 643 model, across independently selected thresholds.Fusion achieved sensitivity 0.559 and 2.26 FP/patient at threshold 0.65, compared with 0.624 sensitivity and 5.49 FP/patient for the 963 model.
- 3.6 Operating-Point Robustness: Agreement filtering reduced FP/patient from 2.639 to 2.268 and increased precision from 0.602 to 0.630, but sensitivity fell from 0.582 to 0.561 and F1 was unchanged.The paired-bootstrap F1 difference was +0.001 with p = 0.786.
4 Discussion
Cross-FOV fusion improved lesion-level performance and false-positive burden relative to same-FOV ensembling, while residual errors and validation limitations constrain interpretation.
- Because the same-FOV control used five epochs instead of the primary models’ 50-epoch schedule, the cross-FOV advantage is supportive rather than definitive evidence for FOV-specific effects.A fully matched same-FOV control could potentially achieve stronger performance.
- Cross-FOV fusion reached F1 0.593 and 2.258 FP/patient, versus 0.496 and 4.021 for the same-FOV ensemble; sensitivity differences were not significant.F1 and false-positive differences were significant, supporting complementary spatial context beyond generic ensembling.
- Only 4/70 lesions below 0.010 mL were detected by the selected fusion, showing that very small-lesion detection remained a major limitation.Neither oversampling nor the 64^3 FOV alone improved the 96^3 baseline.
- The agreement filter reduced FP/patient from 2.64 to 2.27 without improving F1, indicating that false-positive suppression alone did not improve overall detection.The audit identified both lesion-like mimics and anatomical or boundary-associated errors.
5 Conclusion
The study combines independently trained predictions from two spatial fields of view to improve brain-metastasis detection. Cross-FOV fusion improved precision and F1 while reducing false positives, but very small lesions and external generalization remained difficult.
- Cross-FOV fusion improved F1 by 0.096 and reduced false positives by 1.76 per patient versus same-FOV ensembling, without a significant sensitivity difference.The comparison used paired bootstrap analysis on the 97-patient development cohort.
- The framework combines whole-volume probability maps from independently trained 96^3 and 64^3 models while keeping image resolution and modalities fixed.This design targets complementary spatial context through weighted late fusion.
- Performance increased with lesion size, and the agreement filter reduced false positives and increased precision but lowered sensitivity without significantly improving F1.These findings indicate that cross-FOV fusion improves precision and false-positive control but does not solve small-lesion detection.
- External validation remained substantially weaker, with an ET instance F1 of 0.385, underscoring the difficulty of robust generalization.The internal development results therefore do not fully establish external robustness.