PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantifying Volumetric Risk: Class-Aware Asymmetric Weighted Conformal Prediction for 3D Medical Image Segmentation

Shadi Alijani, Fereshteh Aghaee Meibodi, Homayoun Najjaran

arXiv:2610.09392v1cs.CV

TL;DR

Clinical volumetric segmentation needs calibrated uncertainty, while the paper identifies accuracy-focused approaches and a need for calibration. CA-WCP introduces class-aware asymmetric weighted conformal prediction for volumetric uncertainty. On the evaluated benchmarks, coverage intervals contain the nominal level for every class and widths are smaller than the symmetric weighted baseline.

  • Problem

    The paper identifies a need for calibrated uncertainty in clinical volumetric segmentation, beyond approaches focused on segmentation accuracy.

    Source: §1 · p. 2

  • Method

    CA-WCP is a class-aware asymmetric weighted conformal prediction framework for volumetric uncertainty quantification.

    Source: §6 · p. 23

  • Results

    8–14% smaller average interval widths than the symmetric weighted baseline, while 95% Clopper–Pearson coverage intervals contain the nominal 90% level for every class.

    Source: §6 · p. 23

  • Takeaways & Limitations

    The reported benchmarks support classwise coverage at the nominal level alongside narrower intervals than the symmetric weighted baseline.

    Source: §6 · p. 23

    Evaluation covers one clinical cohort and one synthetic benchmark; validation on acquired multi-center abdominal CT remains outstanding.

    Source: §5.1 · p. 22

Abstract

from arXiv · show

Reliable volumetric segmentation is critical for clinical diagnostics, yet foundation models such as MedSAM remain deterministic and lack calibrated uncertainty under distribution shift. Existing conformal prediction methods offer statistical guarantees but are frequently applied in 2D and assume symmetric error distributions, so they do not capture the class-specific biases that arise in 3D multi-class segmentation. We propose Class-Aware Asymmetric Weighted Conformal Prediction (CA-WCP), which combines latent-space density-ratio weighting for covariate shift with directional quantiles for the lower and upper volume bounds, and scales each bound by a class-specific asymmetry factor derived from validation-set false-positive and false-negative rates. We prove that CA-WCP retains the weighted-exchangeability marginal coverage guarantee for every class, and we evaluate it on 3D brain tumor segmentation (BraTS 2020) and on a synthetic multi-organ CT benchmark constructed under covariate shift. On both benchmarks the 95\% Clopper--Pearson interval for the observed coverage of CA-WCP contains the nominal 90\% level for every semantic class, while interval width is reduced by 8--14\% relative to symmetric weighted conformal prediction. We further encode the calibrated intervals into structured prompts for a multimodal large language model to produce uncertainty-conditioned radiology reports, linking distribution-shift-aware uncertainty quantification to interpretable clinical communication.

1. Introduction

Reliable clinical volumetric segmentation requires calibrated uncertainty, but existing approaches often emphasize accuracy, assume symmetric errors, or do not address multi-class segmentation under distribution shift. The paper introduces a class-aware asymmetric weighted conformal framework and evaluates its coverage and uncertainty-reporting components.

Motivation

  • Existing volumetric conformal methods assume symmetric errors, although structures in multi-class segmentation exhibit different false-positive and false-negative patterns.The introduction also identifies distribution shift and a gap in applying conformal prediction to 3D multi-class volumetric segmentation under covariate shift.Source: §1 · p. 3

Main contributions

  • CA-WCP extends weighted conformal prediction to 3D multi-class segmentation by combining directional volume bounds with class-specific asymmetry factors.The factors use validation-set false-positive and false-negative rates to add a conservative margin on the side of the dominant error.Source: §1 · p. 4
  • The authors prove marginal coverage under weighted exchangeability with known likelihood ratios and state the approximation introduced when those ratios are estimated.The proof is identified as Proposition 1 in Appendix A.Source: §1 · p. 4
  • The evaluation covers BraTS 2020 and a controlled synthetic multi-organ CT benchmark, reporting exact k/n coverage and Clopper–Pearson intervals for every class.The synthetic benchmark tests whether directional error asymmetry reflects structure scale rather than brain-tumor morphology alone.Source: §1 · pp. 3–4
  • The paper encodes calibrated prediction intervals into structured prompts for uncertainty-conditioned multimodal LLM reports.It also describes a preliminary assessment by an independent radiologist blinded to prompt configuration.Source: §1 · p. 4

2. Background

The background reviews medical segmentation foundation models, uncertainty methods, conformal prediction under distribution shifts, and medical LLMs. It positions explicit class calibration and directional volumetric errors as the paper’s distinction from prior work.

2.1. Foundation Models for 2D and 3D Medical Segmentation

  • Medical SAM adaptations span 2D fine-tuning and native 3D or video-style prompt propagation, while these methods largely neglect uncertainty and assume stable settings.Examples include MedSAM, SAM-Med2D, and SAMed for 2D images, and SAM-Med3D, SAM3D, and MedSAM-2 for 3D capabilities.Source: §2.1 · p. 5

2.2. Uncertainty Quantification in Medical Segmentation

  • Probabilistic models, uncertainty-aware losses, adapters, and specialized fine-tuning offer confidence heuristics, but generally lack conformal prediction’s formal statistical guarantees.The section frames uncertainty quantification as important for robustness in medical segmentation.Source: §2.2 · p. 5

2.3. Conformal Prediction and Distribution Shifts

  • Many segmentation conformal methods operate in 2D or at region level, while class effects are often implicit; this framework calibrates class effects explicitly and couples them to directional volumetric error.The section also reviews conformal applications to 3D medical biomarkers and methods addressing distribution shifts.Source: §2.3 · p. 6

2.4. LLMs in Medical Imaging and Uncertainty Communication

  • Despite advances in medical LLMs, integration of statistically calibrated uncertainty quantification into LLM-based medical systems remains limited.The background cites medical question-answering and report-drafting models as well as image-grounded dialogue systems.Source: §2.4 · p. 6

3. Method

The method combines covariate-shift weighting with class-specific asymmetric volumetric intervals shaped by validation-set error patterns, with marginal coverage guarantees for each class under weighted exchangeability and true likelihood-ratio weights. It also converts calibrated intervals into class-wise confidence levels for uncertainty-conditioned radiology reporting using an LLM.

3.1. Problem Formulation

  • The method targets intervals containing the true structure volume with probability at least 1 − α for multi-class 3D segmentation.The paper sets α = 0.10 for 90% coverage.Source: §3.1 · p. 7

3.2. Multi-Head Volumetric Backbone

  • A shared-encoder, three-head volumetric backbone produces lower-bound, mean, and upper-bound segmentation estimates; CA-WCP is post-hoc and leaves masks and Dice accuracy unchanged.The heads use separate decoder branches, with different Tversky-loss settings.Source: §3.2 · pp. 7–8

3.3. Weighted Conformal Prediction

  • For covariate shift, weighted conformal prediction estimates calibration-sample density-ratio weights by classifying latent features as test or calibration data.The method also weights the test feature and uses normalized weights to compute weighted quantiles.Source: §3.3 · p. 8
  • Validation data show class-specific error biases: necrotic core has false-negative bias, edema large false-positive bias, and enhancing tumor moderate false-positive bias.The reported false-positive and false-negative rates are normalized by ground-truth volume, so values above 100% indicate severe over-segmentation.Source: §3.3 · p. 9

3.4. Class-Aware Asymmetric Weighted Conformal Prediction

  • CA-WCP integrates weighted conformal prediction, directional scores, and validation-derived class-specific asymmetry factors that couple interval geometry to each class’s FP/FN error structure.The paper distinguishes this integration from applying Mondrian calibration to a symmetric weighted score.Source: §3.4 · p. 12

Proposition 1 (Marginal Coverage). Fix a class c and let γ(c)

  • Under weighted exchangeability with weights proportional to the true likelihood ratio and directional quantiles at level 1 − α/2, conditional on γ(c), the guarantee is marginal separately for each class, not simultaneous at level 1 − α.Source: Proposition 1 (Marginal Coverage). Fix… · p. 13

3.5. Uncertainty-Conditioned Multimodal LLM Report Generation

  • The section introduces translating quantitative uncertainty estimates into clinically readable multimodal-LLM reports.The supplied passage gives this aim but no report-generation procedure.Source: §3.5 · p. 14
  • Relative interval width (RW) is normalized by mean-head volume using the same asymmetry-scaled endpoints as Eq. (6), then mapped to discrete confidence levels using thresholds set from validation-set RW quantiles, including High (RW < 35%).Source: §3.5 · p. 14
  • Volumes are converted to milliliters using image spacing before being passed to the LLM.Source: §3.5 · p. 14
  • GPT-4o at temperature 0 receives the MR image, generated segmentation mask, and structured conformal outputs including the derived confidence level.The prompt directs the model to use more cautious phrasing for low-confidence predictions.Source: §3.5 · p. 14
  • Generated reports follow radiology convention with a “Findings” section.Source: §3.5 · p. 14

4. Experiments

The experiments assess CA-WCP’s coverage and interval widths on BraTS and synthetic CT, and examine how calibrated intervals support uncertainty-conditioned report generation.

4.1. Implementation Details

  • The lower, mean, and upper MedSAM heads used precision-focused, symmetric, and recall-focused Tversky weights, with fixed thresholds for binarization.Latent features were L2-normalized 256-dimensional vectors, and logistic regression estimated calibration-to-test density ratios.Source: §4.1 · p. 15

4.2. Experimental Setup

  • BraTS 2020 used a four-way split with 73 test subjects from unseen institutions, while synthetic CT used a shifted test set and liver, kidneys, and spleen.BraTS calibration and test cohorts came from different institutions; CT test volumes used different conditioning prompts and structure-scale parameters.Source: §4.2 · pp. 15–16

4.3. Results

  • Directional WCP reduced mean width from 9.7 to 6.7 mL but fell below target coverage in all classes; Symmetric WCP under-covered edema and over-covered enhancing tumor.Directional WCP’s three class coverages were 87.7%, 84.9%, and 89.0%, although each Clopper–Pearson interval contained 90%.Source: §4.3 · p. 17
  • 90.4% (66/73) coverage in every BraTS class at mean width 8.5 mL was 12% narrower than Symmetric WCP and 27% wider than Directional WCP.The coverage difference between Directional WCP and CA-WCP was not statistically resolved for this split.Source: §4.3 · p. 17, Table 2 · p. 18
  • On synthetic CT, CA-WCP moved coverage for all three organs toward nominal while reducing mean width, separating scale-related asymmetry from acquisition-shift confounds.The benchmark induced conditioning shift and covered liver, kidneys, and spleen across large, medium, and small structure scales.Source: §4.3 · pp. 18–19, Table 3 · p. 19

4.4. Uncertainty-Conditioned Report Generation

  • Report evaluation compared image-and-segmentation prompts with prompts that also supplied calibrated intervals and confidence, using 10 test cases and radiologist-written references.A second radiologist, blinded to prompt configuration, rated factual accuracy and whether hedging matched interval width.Source: §4.4 · p. 19
  • BLEU-4 was 0.42 for reports generated from quantitative intervals and scored against templated references, which constrain lexical variance.The references were standardized clinical-template reports written by a board-certified radiologist from ground-truth masks.Source: §4.4 · pp. 19–20
  • Uncertainty concordance rose from 2.1 to 4.6 with calibrated intervals, and model confidence tracked conformal bounds rather than segmentation appearance.Source: §4.4 · p. 20

5. Discussion

Directional quantiles supply most of the width reduction, while validation-derived asymmetry factors add conservative class-specific margins; synthetic CT shows the same qualitative pattern for large and small structures. The discussion also flags uncertainty from single-split evaluation, limited report assessment, and synthetic data that do not establish real-CT deployment.

5. Discussion

  • Mean width falls by roughly 30% relative to Symmetric WCP because directional quantiles avoid widening both margins for a dominant error direction.The asymmetry factors then add a conservative, class-specific margin on the side where validation errors concentrate.Source: §5 · p. 20
  • Because γ ≥1, the class-specific margin can only widen intervals and cannot invalidate the guarantee of Proposition 1.The margin has a width cost, leaving the choice between the two operating points to the user.Source: §5 · pp. 20–21
  • Point coverage fell slightly below target, and a single split of 73 cannot establish whether the shortfall is systematic or a finite-sample effect.Repeated random splits are needed to assess whether weight-estimation error and CA-WCP’s coverage gap are within sampling uncertainty.Source: §5 · p. 20, §5.1 · p. 22
  • On synthetic CT, CA-WCP shows the same qualitative pattern: the over-covered large structure loses excess width while the under-covered small structure gains coverage.This suggests the effect depends on directional error structure rather than brain-tumor specificity.Source: §5 · p. 21

5.1. Limitations and Future Directions

  • The evaluation uses one clinical cohort and one synthetic benchmark; learned synthetic CT inherits anatomical priors, so acquired multi-center abdominal CT validation remains necessary before deployment conclusions.Test cohorts of n = 73 and n = 100 also limit coverage-estimation precision; all results come from a single calibration/test split.Source: §5.1 · p. 22
  • The generated-report assessment covers 10 cases and one rater, so it indicates changes in reporting language rather than clinical validation.The discussion proposes multi-reader evaluation on larger paired image-report cohorts.Source: §5.1 · p. 22

6. Conclusion

CA-WCP combines density-ratio weighting with class-aware asymmetric conformal bounds for volumetric segmentation, retaining a marginal coverage guarantee under weighted exchangeability. Across two benchmarks, its observed-coverage intervals contain the nominal 90% level for every class, with narrower intervals than the symmetric weighted baseline.

6. Conclusion

  • CA-WCP combines density-ratio weighting with asymmetric conformal prediction and retains a marginal coverage guarantee under weighted exchangeability.The proof also accounts for the approximation introduced by estimated weights.Source: §6 · p. 23
  • 8–14% smaller average interval widths than the symmetric weighted baseline, while 95% Clopper–Pearson intervals contain nominal 90% coverage for every class on both benchmarks.The benchmarks are BraTS 2020 and a controlled synthetic multi-organ CT benchmark.Source: §6 · p. 23
  • Most of the interval-width gain comes from directional quantiles, while asymmetry factors add a conservative class-specific margin.Source: §6 · p. 23
  • Calibrated intervals can be encoded in multimodal LLM prompts to produce uncertainty-conditioned reports whose hedging tracks interval width.CA-WCP is a post-hoc wrapper that adds shift-aware uncertainty without modifying the underlying model.Source: §6 · p. 23

Appendix A. Proof of Proposition 1

The proof establishes per-direction coverage and then shows that validation-based widening preserves the weighted-exchangeability argument. Its guarantee assumes known likelihood-ratio weights; estimated weights introduce an approximation.

Appendix A. Proof of Proposition 1

  • Validation-derived factors are computed from Dval, disjoint from calibration and test data; conditioning on them does not disturb weighted exchangeability.Source: Appendix A · p. 23
  • Per-direction scores are deterministic functions of the data points, so weighted exchangeability carries over and the weighted conformal guarantee applies.Source: Appendix A · p. 24
  • The proof uses true likelihood-ratio weights for the weighted conformal argument and separately accounts for approximation from estimated weights.Source: Appendix A · p. 24
  • Monotone widening adds non-negative increments on the FP-driven branches, preserving the per-direction bounds after scaling.Source: Appendix A · p. 24

1. Since the directional scores are non-negative and q(c)

The proof combines the directional bounds to establish the stated coverage claim.

1. Since the directional scores are non-negative and q(c)

  • The union-bound step combines the lower- and upper-endpoint failure events to obtain the coverage claim.Source: §1 · p. 24

Related papers

Loading 2610.09392v1…