Source-linked AI summary
Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation
Sol Lee, Hyunji Kim, Sungrae Hong, Donghee Han, Mun Yi
TL;DR
Missing MRI modalities complicate brain tumor segmentation, and existing calibration approaches do not account for combination-specific prediction difficulty. MMA-LTS calibrates voxel-wise confidence using modality availability and spatial difficulty; experiments report improved calibration while preserving segmentation accuracy across missing-modality scenarios.
Problem
Existing calibration methods are modality-agnostic or assume difficulty falls as modalities are added, although brain tumor prediction difficulty depends on which modalities are missing.
Source: §1 · p. 2Method
MMA-LTS is a plug-in post-hoc method that conditions voxel-wise calibration on modality availability and spatial difficulty without retraining the segmentation model.
Source: §1 · p. 3, §2 · p. 3, §2.1 · pp. 4–5Results
MMA-LTS achieves the lowest ECE across most tumor regions and 15 modality combinations while preserving DSC; WT/TC/ET ECE averages are 10.2/14.5/15.1 on BraTS and 9.5/12.0/11.6 on FeTS.
Source: §1 · p. 3, §3.2 · p. 6Takeaways & Limitations
Across diverse missing-modality scenarios and multiple backbones, MMA-LTS is a practical step toward trustworthy clinical deployment.
Source: §4 · p. 9Further validation is needed in other anatomical domains and imaging modalities, with reliability metrics beyond ECE, formal statistical testing, and real-world clinical validation.
Source: §4 · p. 9
Abstract
from arXiv · showhide
Multimodal brain tumor segmentation typically leverages multiple MRI modalities, yet incomplete modality acquisition is common in clinical practice due to protocol heterogeneity and scan failures. Although recent methods maintain segmentation accuracy under missing modality conditions, they frequently overlook prediction reliability, leading to miscalibrated confidence estimates that hinder clinical adoption. Existing calibration techniques are largely modality-agnostic or assume that prediction difficulty decreases monotonically as additional modalities become available. However, in brain tumor segmentation, prediction difficulty depends primarily on which modalities are absent rather than how many, leading to combination-specific and spatially heterogeneous calibration errors. To address this, we propose Missing Modality-Aware Local Temperature Scaling (MMA-LTS), a post-hoc voxel-wise confidence calibration method. It estimates a spatially adaptive temperature field conditioned on a modality-availability learnable token and a voxel-wise difficulty score. Experiments on BraTS 2020 and FeTS 2024 show that MMA-LTS improves calibration while preserving the segmentation accuracy of state-of-the-art models across diverse missing-modality scenarios, thereby enhancing trustworthiness toward clinical deployment.
1 Introduction
Incomplete MRI modalities make brain tumor segmentation necessary in clinical workflows, but accurate predictions can still have poorly calibrated confidence. MMA-LTS calibrates predictions according to modality availability and voxel-wise difficulty, improving calibration while preserving segmentation accuracy.
1 Introduction
- Earlier missing-modality approaches synthesized absent modalities or trained separate models for each combination, while newer methods embed inputs into a shared latent space.The shared-latent approach enables a unified architecture.Source: §1 · pp. 1–2
- DC-Seg achieves higher DSC yet has larger ECE than RobustSeg, illustrating that strong segmentation performance can coexist with poorly calibrated confidence.Reliable confidence matters for identifying unreliable tumor delineations and supporting expert intervention.Source: §1 · p. 2
- Prediction difficulty depends on which modalities are missing rather than how many, producing combination-specific and spatially heterogeneous calibration errors.Figure 1 compares cases with the same modality count but different combinations.Source: §1 · pp. 2–3, Figure 1 · p. 2
- MMA-LTS uses modality-aware conditioning and spatial difficulty gating for voxel-wise post-hoc calibration, improving calibration across BraTS 2020 and FeTS 2024 while preserving segmentation accuracy.It is a plug-in method that does not require retraining the segmentation model.Source: §1 · p. 3
2 Method
MMA-LTS trains a calibration module on validation data to produce spatially adaptive confidence estimates for a pretrained segmentation model under arbitrary modality availability. It combines spatial features, voxel-wise difficulty, and modality availability to estimate temperatures for voxel-wise calibration.
2 Method
- MMA-LTS trains a calibration module on Dval to calibrate a pretrained segmentation model on Dtest without retraining the model.The framework assumes disjoint Dtrain, Dval, and Dtest splits.Source: §2 · p. 3
2.1 Missing Modality-Aware Local Temperature Scaling
- A spatial encoder combines segmentation features and logits into an embedding used to estimate voxel-wise temperatures.Voxel difficulty is estimated from class-conditional and class-agnostic feature distributions using Mahalanobis-distance-based estimation.Source: §2.1 · pp. 3–4
- MMA-LTS uses a voxel-wise difficulty score and a learnable modality-availability token to condition temperature estimation on spatial difficulty and available modalities.The difficulty score reflects regions that are atypical within their class but unremarkable globally; the modality token summarizes available and missing states.Source: §2.1 · p. 4
- A temperature estimator uses spatial-difficulty gating and modality-conditioned FiLM modulation to produce a spatial temperature field that rescales logits into calibrated voxel-wise probabilities.The method trains with ModDrop and optimizes negative log-likelihood over the voxel grid.Source: §2.1 · pp. 4–5
3 Experiments
Experiments on BraTS 2020 and FeTS 2024 compare MMA-LTS with calibration baselines across missing-modality settings and evaluate its components, runtime, and sensitivity. MMA-LTS reduces ECE while preserving segmentation accuracy.
3.1 Implementation Details
- Experiments use BraTS 2020 and FeTS 2024, three segmentation backbones, and DSC/ECE evaluation, with post-hoc calibration methods trained on validation splits.The datasets contain four MRI modalities and three tumor subregions; ECE is computed over foreground voxels using 10 bins.Source: §3.1 · pp. 5–6
3.2 Experiment Results
- BraTS 2020 WT/TC/ET average ECE fell to 10.2/14.5/15.1 versus second-best LTS at 13.1/16.8/16.5; FeTS 2024 reached 9.5/12.0/11.6.The authors also report consistent ECE reduction across RobustSeg and mmFormer backbones.Source: Table 1 · p. 6, §3.2 · p. 6, Table 2 · p. 7
- Removing voxel-wise difficulty estimation or replacing it with sample-averaged difficulty raises ECE across tumor regions; removing or count-encoding modality availability also degrades ECE.The ablation study averages ECE over modality combinations.Source: §3.2 · p. 7, Figure 3 · p. 8
- For two cases with equal modality counts but different combinations, uncalibrated confidence patterns differ, while VDS maps highlight ambiguous regions and calibration maps indicate where stronger correction is needed.Residual overconfidence remains near tumor boundaries.Source: §3.2 · p. 8
- MMA-LTS achieves the lowest ECE with inference time comparable to temperature-scaling methods and lower training cost than retraining approaches; MC has higher inference cost.Fig. 4(a) compares ECE against inference and training time.Source: §3.2 · p. 8, Figure 4 · p. 8
- ECE is stable for Cf ≥128 and de ≥16, and performance saturates beyond 50% of Dval.Fig. 4(b) examines sensitivity to Cf, de, and validation-set size.Source: §3.2 · p. 8, Figure 4 · p. 8
4 Conclusion
MMA-LTS calibrates brain tumor segmentation confidence under missing modalities by conditioning on modality availability and voxel-wise difficulty. Experiments report ECE reductions across datasets and backbones while preserving segmentation accuracy, with broader validation left for future work.
4 Conclusion
- MMA-LTS is a plug-in post-hoc method for correcting modality-combination-specific and spatially heterogeneous miscalibration in brain tumor segmentation with missing modalities.It conditions on modality availability and voxel-wise difficulty without retraining the segmentation model or compromising accuracy.Source: §4 · pp. 8–9
- Experiments on BraTS 2020 and FeTS 2024 report consistent ECE reductions across multiple backbones and diverse missing-modality scenarios.The authors describe this as a practical step toward trustworthy clinical deployment.Source: §4 · p. 9
- Future work includes validation in other anatomical domains and imaging modalities, reliability metrics beyond ECE with formal statistical testing, and real-world clinical validation.These are stated as directions for extending the study.Source: §4 · p. 9