A Contrast-Adaptive Method for Simultaneous Whole-Brain and Lesion Segmentation in Multiple Sclerosis

1. Introduction

• MRI can detect MS lesions and characterize atrophy, which is faster in MS than in healthy controls and has been associated with disability in deep gray matter and cortex.Lesion prevalence and dynamics are used clinically for diagnosis, disease staging, and assessing therapeutic efficacy.Source: §1 · p. 1
• Automated analysis is needed because manual lesion labeling is cumbersome and variable, while manually labeling normal-appearing structures for atrophy is impractical.Automated tools could characterize lesions and structures from MRI and support disease tracking and clinical-trial testing.Source: §1 · p. 1
• Existing tools may generalize poorly across scanners and protocols, are rarely publicly available, and often lack dedicated structure-level atrophy assessment.Some methods report only aggregate volumes or require pre-segmented lesions and lesion filling.Source: §1 · p. 2
• The proposed open-source tool jointly segments white matter lesions and 41 neuroanatomical structures, avoiding separate lesion-segmentation and atrophy-analysis phases.The figure illustrates the output on T1w-FLAIR input.Source: §1 · p. 2, Figure 1 · p. 2
• Prior joint methods were limited in the number of structures evaluated, and one supervised approach is constrained when applied to data unlike its training scans.The cited methods evaluated whole-brain segmentation on 10 and 7 structures, respectively; the latter method's applicability to substantially different data is limited.Source: §1 · p. 2
• Compared with the preliminary version, this article uses more advanced lesion shape and appearance models and provides broader validation.The validation includes evaluation of whole-brain segmentation and comparisons with human inter-rater variability.Source: §1 · p. 2

2. Contrast-adaptive whole-brain segmentation

• SAMSEG models segmentation as inference under a generative process and extends to 41 structures without preprocessing or assumptions about scanner platform or pulse-sequence count.Labels are sampled from a segmentation prior, and MRI data are sampled from a likelihood conditioned on those labels.Source: §2 · pp. 2–3

2.1. Segmentation prior

• A deformable probabilistic atlas represents structure locations with a tetrahedral mesh and uses a topology-preserving deformation prior.The atlas is learned from training segmentations using mesh simplification, groupwise registration, and EM-based label-probability estimation.Source: §2.1 · p. 3

2.2. Likelihood function

• The likelihood uses Gaussian appearance models for structures and represents smooth bias fields as spatial basis functions added to log-transformed intensities.The likelihood parameters have a flat prior.Source: §2.2 · p. 3

2.3. Segmentation

• Segmentation fits the model parameters to each scan, then assigns each voxel its highest-probability label, enabling adaptation to scan-specific intensity characteristics.The class means and variances are estimated from each target scan individually.Source: §2.3 · pp. 3–4

3. Modeling lesions

• SAMSEG is augmented with a binary lesion map and latent shape code, adding lesion modeling to whole-brain segmentation.The joint model retains the anatomical segmentation prior and introduces lesion-specific appearance parameters; Fig. 2 distinguishes the existing SAMSEG components from the additions.Source: §3 · p. 4, Figure 2 · p. 4

3.1.2. Modeling the spatial location of lesions

• The lesion prior combines shape constraints from the latent code with voxel location probabilities in the deformed anatomical atlas.Atlas-mesh lesion probabilities are estimated from 54 MS subjects with manually annotated lesions.Source: §3.1 · p. 4, §3.1.2 · p. 5

3.1.1. Modeling lesion shapes

• Lesion masks are modeled with a variational autoencoder: a low-dimensional Gaussian code is decoded into lesion probabilities across voxels.The encoder approximates the latent-code posterior, and training uses stochastic gradient variational Bayes because integrating over codes is intractable; the shape model was trained on 212 subjects’ manual masks, augmented to 1,484 segmentations.Source: §3.1.1 · pp. 4–5

3.2. Likelihood function

• Lesion voxels use a separate Gaussian intensity model, whose parameters are conditionally constrained by a normal-inverse-Wishart prior linked to white matter.The authors use ν = 500 pseudo-voxels and κ = 50 for 1 mm³ scans, scaling ν inversely with voxel size; these values were selected in pilot experiments on other datasets.Source: §3.2 · pp. 5–6

3.3. Segmentation

• After estimating parameters jointly in a simplified model without lesion-shape constraints, the method estimates lesion probabilities and assigns remaining voxels to anatomical labels.Lesion probabilities are approximated with Monte Carlo sampling and compared with a user-specified threshold; candidate lesions must exceed the gray-matter mean in FLAIR and/or T2 when available.Source: §3.3 · pp. 6–7

4.1. Datasets

• Four datasets acquired with different scanners, field strengths, protocols, and resolutions are used; required preprocessing is limited to co-registration and resampling to a reference scan.The evaluation includes public challenge data and cohorts such as MSSeg, Trio, and Achieva; the original MSSeg challenge test set of 38 subjects was unavailable at the time of writing.Source: §4.1 · p. 7

4.2. Benchmark methods for lesion segmentation

• The lesion benchmarks are LST-lga and NicMsLesions, and both target T1w-FLAIR input, unlike the proposed method, which is not tuned to a particular modality combination.LST-lga grows a thresholded lesion-belief map along FLAIR-hyperintense voxels; NicMsLesions uses a cascade of two 3D CNNs and has a publicly available implementation.Source: §4.2 · p. 8

4.3. Metrics and measures

• Evaluation uses segmentation volumes, Pearson correlations with reference volumes, and Dice scores; probabilistic lesion maps are thresholded using leave-one-out tuning within each dataset.For ISBI, thresholds were instead tuned on the five challenge training subjects.Source: §4.3 · p. 8

5.1. Illustration of the method

• The lesion shape model and intensity constraints reduced false positives and produced more realistic lesion shapes, especially in the low-load example.The comparison examined T1w-FLAIR scans from one subject with low lesion load and one with high lesion load.Source: §5.1 · p. 9
• With low lesion load, the prior kept the lesion Gaussian variance wide and its mean near white matter, supporting estimation from limited lesion data.At high lesion load, the prior had little influence and left the lesion Gaussian free to fit the data.Source: §5.1 · p. 9
• The lesion shape prior improved segmentations for small and medium lesion loads, while performance was unchanged at high lesion loads.The comparison removed the shape prior and retuned the lesion threshold for fairness.Source: §5.1 · p. 9
• On Achieva T1w-FLAIR scans, detected lesion volumes averaged 8.95 ± 9.18 ml in MS subjects and 0.98 ± 0.77 ml in controls.Control detections were attributed to white-matter hyperintensities in some subjects and judged acceptable by the authors.Source: §5.1 · p. 9

5.2. Scanner and contrast adaptive segmentations

• The method adapted to different scanners and input-contrast combinations; visually, T1w improved whole-brain segmentation, while FLAIR alone degraded cortical white-gray boundaries.The examples covered MSSeg, Trio, and Achieva scans.Source: §5.2 · p. 9
• Across MSSeg, Trio, and Achieva, lesion Dice was best with FLAIR included, and adding contrasts did not hurt performance.The evaluation compared seven input combinations; T1w-FLAIR was judged well-suited when whole-brain and lesion segmentation were considered together.Source: §5.2 · p. 9
• Considering lesion and whole-brain segmentation together, the authors judged T1w-FLAIR well-suited, while the method also accepts other or additional contrasts.Source: §5.2 · p. 9

5.3. Lesion segmentation

• The benchmark comparison used T1w-FLAIR because that input combination was required by the benchmark methods; the section also planned an ISBI evaluation.Source: §5.3 · p. 9

• On Achieva, all three methods performed similarly; on MSSeg and Trio, NicMSLesions trailed the other two, and the proposed method outperformed LST-lga only on MSSeg.No statistically significant differences between the proposed method and LST-lga were reported on the other datasets.Source: Comparison with benchmark lesion… · p. 11
• The MSSeg comparison may have favored NicMSLesions because its test subjects were included in its training data.The authors note this could potentially bias the results in NicMSLesions’ favor.Source: Comparison with benchmark lesion… · p. 11

• The method achieved mean Dice 0.58 on ISBI, compared with MSSeg 0.65, Trio 0.58, and Achieva 0.54.Source: Results on the ISBI data · p. 11
• The ISBI overall score was 87.87, around halfway in the original challenge ranking, though the website listed methods with much higher scores.The score combines Dice overlap, volume correlation, surface distance, and other metrics.Source: Results on the ISBI data · p. 11
• NicMSLesions suffered strongly from ISBI domain shift, while its partially retrained network performed comparably to the proposed model, which required no retraining.Preprocessing of the ISBI data prevented the authors from obtaining LST-lga results.Source: Results on the ISBI data · p. 11

Inter-rater variability

• The method’s mean Dice overlap with seven MSSeg raters was 0.57, below the human-rater mean range of 0.59–0.69.The comparison used average Dice between rater pairs and between each rater and the method.Source: Inter-rater variability · p. 11, Table 3 · p. 13
• The supplied figure captions describe lesion probabilities and Dice scores across input contrasts and datasets, but do not state comparative outcomes.Source: Figure 7 · p. 12, Figure 8 · p. 13

5.4. Whole-brain segmentation

• Whole-brain segmentation was validated indirectly because no ground-truth segmentations were available, focusing on lesion-filling alternatives and replication of known MS atrophy patterns.The evaluation covered 25 neuroanatomical regions from T1w-FLAIR scans; lesion labels remained separate, leaving holes in white matter so results reflected normal-appearing tissue.Source: §5.4 · p. 13

Comparison with lesion filling

• Silver standards were generated by masking lesion voxels during SAMSEG fitting because replacing lesion intensities did not work well in FLAIR experiments.Lesions were ignored during model fitting rather than explicitly intensity-replaced.Source: Comparison with lesion filling · p. 13
• Volume correlations were 0.988 for the method versus an inter-rater range of [0.988, 0.992], while Dice scores were 0.971 versus [0.978, 0.980].The correlation result was narrowly within the inter-rater range; Dice overlap slightly underperformed it.Source: Comparison with lesion filling · p. 13
• The comparison used seven raters’ manual lesion annotations in turn to measure discrepancies against human inter-rater variability.The study compared the proposed method’s segmentation with each rater-derived silver standard.Source: Comparison with lesion filling · p. 13

Detecting atrophy patterns in MS

• The section’s figures compare lesion Dice overlap across MSSeg, Trio, Achieva, and ISBI datasets, while Tables 4 and 5 summarize per-structure inter-rater consistency.The figures include LST-lga and NicMsLesions comparisons; the ISBI figure also includes partial retraining of NicMsLesions.Source: Figure 9 · p. 14, Figure 10 · p. 15, Table 4 · p. 15

6. Discussion and conclusion

• The method combines a lesion model with generative whole-brain segmentation, adapts appearance models across scanners and protocols without retraining, and was validated on four datasets.The conclusion also reports application to healthy controls and availability in FreeSurfer.Source: §6 · p. 16
• Simultaneous lesion and whole-brain segmentation targets the two-stage lesion-filling workflow and may support large cohort studies of disease-related morphology.The discussion also suggests multi-contrast tools may reduce the human-time burden of evaluating high-resolution scans, potentially increasing measure accuracy.Source: §6 · p. 16
• Automatic lesion segmentation produced neuroanatomical volumes within seven-rater variability, but its spatial Dice overlap did not fully reach human lesion-annotation performance.This supports potential use in large volumetric atrophy studies instead of lesion filling with manual lesion segmentations.Source: §6 · p. 16
• Users can tune the lesion-probability threshold to trade off false positives and false negatives; advanced research sequences are proposed as possible extensions but were not extensively tested.The threshold may be selected by visual inspection or cross-validation; MP2RAGE, DIR, FLAIR2, and T2* are named as prospective inputs.Source: §6 · p. 16

Detecting atrophy patterns in MS

• On Achieva, normalized volumes were lower in MS for cerebral white matter, cortex, thalamus, and caudate, and higher for lateral ventricles.The analysis included 23 controls and 50 MS subjects; Fig. 12 presents the group comparisons.Source: Detecting atrophy patterns in MS · p. 16, Figure 12 · p. 17

Appendix A. Parameter optimization in SAMSEG

• SAMSEG uses coordinate ascent, interleaving limited-memory BFGS optimization of θ_l with generalized EM optimization of θ_d.The GEM procedure constructs a tight lower bound using soft label assignments, then improves that bound with analytical updates.Source: Appendix A · p. 17

Appendix B. Parameter optimization

• In the augmented model, lesions can be treated as an extra class with index K+1, allowing the original SAMSEG optimization strategy to be reused.This reformulation applies with decoder outputs clamped to 1 and lesion probabilities represented in the atlas interpolation model.Source: Appendix B · p. 18
• Because the prior ties lesion and white-matter means and variances, modified parameter updates are used to ensure each GEM iteration improves the lower bound and objective.The appendix gives altered equations, including Ψ_les, for these updates.Source: Appendix B · p. 18

Appendix C. Estimating lesion probabilities

• A blocked Gibbs sampler estimates lesion probabilities by iteratively updating variables conditioned on the others, starting from an initial lesion segmentation.The initial segmentation comes from the parameter-estimation procedure described in Appendix B.Source: Appendix C · p. 18
• The conditional lesion probability combines lesion-image likelihood, decoder output, and the anatomical prior.The passage gives this conditional probability in terms of the lesion appearance model, f_i(h), and ρ_i(θ̂_l).Source: Appendix C · p. 18
• The implementation averages 50 samples after discarding the first 50 sampler sweeps as burn-in.The procedure repeatedly invokes the lesion-shape encoder and decoder, resampling between training and subject spaces with linear interpolation.Source: Appendix C · p. 18