Source-linked AI summary
Multi-task learning for the automatic grading of enlarged perivascular space burden using MRI
Jesse Phitidis, William N. Whiteley, Joanna M. Wardlaw, Miguel O. Bernabeu, Yajun Cheng, Xiaodi Liu, Junfang Zhang, Una Clancy, Stephen Makin, Roberto Duarte Coello, Susana Muñoz Maniega, Mark E. Bastin, Simon R. Cox, Maria del C. Valdés Hernández
TL;DR
Prior automated approaches had not addressed non-dichotomized PVS grading in both the BG and CSO while using segmentation masks in scoring-model development. The authors compared three silver-mask-informed approaches; multi-task learning was most successful and localized individual PVS.
Problem
Automated PVS scoring had not combined non-dichotomized BG-and-CSO grading with use of PVS segmentations in model development.
Source: §1 · p. 4Method
The study compared three approaches using semi-automatically generated silver-standard PVS masks alongside radiological scores to predict modified Potters/Wardlaw BG and CSO grades.
Source: §1 · p. 4, §2.1 · p. 5Results
Multi-task learning was the most successful mask-integration approach, localized individual PVS, and produced scores with sensible confidence patterns and clinical-marker associations.
Source: §5 · pp. 22–23Takeaways & Limitations
Automated scoring can support examination of neurovascular-health associations while visual scoring remains the current clinical standard.
Source: §5 · p. 23Observer-model inter-rater agreement was below agreement between trained observers, and the dataset limited the power of this comparison.
Source: §4.3 · p. 22
Abstract
from arXiv · showhide
Enlarged perivascular spaces (PVS) visible in brain magnetic resonance imaging (MRI) are increasingly thought to be linked to poor brain health. PVS are elongated structures of less than 3 mm in diameter and can be numerous. To reflect the incidence of PVS, radiologists visually score their burden following a clinical grading scale - a task that would benefit from automation to accelerate analyses and overcome the influence of inter-observer differences. We developed and evaluated methods for training machine learning models to score PVS incidence in the basal ganglia (BG) and centrum semiovale (CSO) leveraging the Potters/Wardlaw scale. The novelty in our work lies in the use of imperfect, semi-automatically generated "silver-standard" PVS segmentation masks during training, in addition to PVS radiological scores. We comparatively evaluated a conditional convolutional neural network (CNN) which accepts PVS masks as an extra input channel, a multi-task CNN which performs both PVS segmentation and scoring, and a logistic regression model which utilises features derived from PVS masks to predict PVS scores. Multi-task learning was the most effective method, achieving a mean average precision of 64.08% compared to 60.22% for the conditional CNN, 52.11% for a baseline CNN trained only to predict PVS scores, and 49.32% for the logistic regression model. The multi-task model showed an ability to localise individual PVS not shown by the other CNNs, and behaved in a probabilistically sensible way, predicting with lower confidence on inherently harder classes. Age, sex, hypertension status, white matter hyperintensity volume, and ischaemic stroke lesion status were shown to be associated with the multi-task model's PVS score predictions and the ground truth in a similar way.
1. Introduction
Prior automated approaches had not addressed non-dichotomized Potters/Wardlaw scoring in both the BG and CSO or used PVS segmentations to develop scoring models. This work compares mask-informed approaches, with multi-task learning incorporating segmentation supervision and better localizing individual PVS.
1. Introduction
- The Potters/Wardlaw scale grades BG and CSO by maximum single-hemisphere PVS counts across axial slices, while enlarged PVS are related to vascular risk factors and small vessel disease.For BG and CSO, the scale assigns grades from 0 to 4 based on PVS counts.Source: §1 · p. 2
- Earlier automated scoring used dichotomized scales or isolated regions; no prior work automated non-dichotomized Potters/Wardlaw scoring in both the BG and CSO.The introduction also identifies a gap in using PVS segmentations to develop automated scoring models.Source: §1 · pp. 3–4
- Semi-automatically generated silver-standard masks provide a lower-level signal and can support training on additional images without ground-truth PVS scores.The masks are imperfect, but cheaper to generate than manual segmentations or visual scores at scale.Source: §1 · p. 4
- Multi-task learning incorporated segmentation supervision into PVS-score models and yielded better localization of individual PVS.The introduction contrasts this localization ability with the other scoring approaches.Source: §1 · p. 4
2. Data
The study combines five MRI datasets with observer-rated BG and CSO scores and silver-standard masks, then prepares region masks and compares mask-derived PVS counts with observer scores.
2.1. Datasets and ground truth
- Five datasets provided T1w, T2w, and FLAIR MRI; four had trained-observer BG/CSO scores, while semi-automated masks were available for some subjects.The masks were called silver-standard because most were not fully manually derived or meticulously expert-scrutinized; VALDO contributed six manually generated masks.Source: §2.1 · pp. 4–5
- MSS1–3 were stroke cohorts, LBC1936 a community elderly cohort, and VALDO supplied six manually segmented whole-volume scans from SABRE.The three MSS cohorts differed in recruitment period and scanner; LBC1936 scans came from Wave 2.Source: §2.1.1 · p. 5, §2.1.2 · p. 5, §2.1.3 · p. 5
2.2.1. General
- Scans were rigidly registered and resampled to 1 mm isotropic resolution, then brain-cropped, zero-padded, and intensity-normalized to [0,1].Normalization saturated intensities at the 2nd and 98th percentiles.Source: §2.2.1 · p. 6
2.2.2. Generating region of interest masks
- SynthSeg labels seeded BG and CSO masks; BG was dilated 6 mm with artifact exclusions, while CSO was white matter minus exclusions and BG.The exclusion template covered midline and ventricular-adjacent areas where artifacts frequently mimic PVS.Source: §2.2.2 · pp. 6–7
2.3. Agreement between PVS masks and PVS scores
- They counted 2D connected components per slice and hemisphere, taking each ROI’s maximum, but treated observer score as ground truth because ratings reflect perceived severity and other factors.Those factors can include other small-vessel-disease markers and scan quality.Source: §2.3 · pp. 8–9
- Silver-standard mask-derived counts disagreed strongly with observer scores in both regions, especially CSO, making direct algorithmic count-to-score conversion inadequate.The authors attribute the mismatch partly to imperfect masks and to observer scores reflecting perceived severity and other scan or disease factors.Source: Figure 3 · p. 8, §2.3 · pp. 8–9
2.4. Data splits and modified PVS score
- Table 2 catalogs subject counts and mask/score availability by dataset and split; the split was 70/10/20, with six unscored VALDO cases added to training.Those cases were used as additional samples for the segmentation U-Net and multi-task CNN.Source: Table 2 · p. 9, §2.4 · p. 9
- Because scores 0 and 4 were rare, they merged into 1 and 3; the authors hoped success on this modified scale could extend to the full scale with more data.Figure 4 shows the original and modified score frequencies.Source: §2.4 · pp. 9–10, Figure 4 · p. 10
3. Methods
The methods predict BG and CSO PVS scores on the Potters/Wardlaw scale using score-only, segmentation-informed, and feature-based approaches. They also describe model training, selection, and evaluation procedures.
3. Methods
- The methods compare score-only, segmentation-conditioned, multi-task, and logistic-regression approaches for predicting BG and CSO scores.The overview identifies the approaches and their use of segmentation predictions, joint mask prediction, or features derived from masks.Source: §3 · p. 10, Figure 5 · p. 11
3.1. Segmentation U-Net
- A 3D U-Net trained on scans with silver-standard masks predicts PVS probability maps from T1w, T2w, and FLAIR MRI for downstream scoring.Training combines Dice and cross-entropy losses.Source: §3.1 · pp. 10, 12
3.2. Baseline CNN
- The baseline 3D ResNet predicts BG and CSO score distributions from multimodal MRI and regional binary masks, without using available segmentation masks.It is trained with cross-entropy loss.Source: §3.2 · p. 12
3.3. Conditional CNN
- The conditional CNN follows the baseline setup but adds soft PVS predictions from a pre-trained segmentation U-Net as an input channel.The U-Net produces soft pseudo-labels for score-labelled subjects, including subjects whose scans were used to train that U-Net.Source: §3.3 · p. 12
3.4. Multi-task CNN
- The multi-task CNN shares a 3D ResNet encoder between score classification and a U-Net-like mask decoder, training both tasks simultaneously.Its loss weights segmentation and classification components; β normalises their baseline contributions across an epoch when subjects lack ground truth for both tasks.Source: §3.4 · pp. 12–13
3.5. Logistic regression
- Separate BG and CSO multinomial logistic regression models use validation-set feature and kernel selection, with selected features combining demographics and U-Net-derived PVS measures.The BG model uses age, BG PVS volume, and maximum hemispheric slice count; the CSO model uses sex, total PVS count, and maximum hemispheric slice volume.Source: §3.5 · p. 13
3.6. Random seeds and model checkpoints
- The CNNs were trained with three random initialisation seeds and ensembled at inference, while logistic regression used no ensemble.Checkpoints were selected by validation Dice for the U-Net and validation mAP for the score-prediction CNNs.Source: §3.6 · pp. 13–14
3.7. Evaluation metrics
- Segmentation is evaluated with Dice similarity coefficient, while score classification uses mAP, macro-average F1, and accuracy.Classification metrics use 10,000 test-set bootstrap samples for 95% confidence intervals and Bonferroni-corrected pairwise-difference tests (n = 6).Source: §3.7 · p. 14
4. Results and discussion
The multi-task CNN was the best or jointly best PVS scoring approach across reported metrics, and its validation examined segmentation, localization, probabilistic behavior, clinical associations, and rater agreement. Results also show limits: human observers likely remain more reliable between raters, and multi-task segmentation performance was below that of the dedicated U-Net.
4.1. Segmentation model validation
- 62.84% test-set DSC was achieved by the segmentation U-Net against silver-standard masks; mean count differences were below one, with wider confidence intervals for CSO.The wider CSO intervals reflect higher PVS counts from both masks.Source: §4.1 · pp. 15–16, Figure 7 · p. 16
- Segmentation U-Net masks had similar downstream utility to silver-standard masks in the conditional CNN and logistic regression scoring pipelines.The conditional CNN and logistic regression experiments are reported in Appendix B.Source: §4.1 · p. 16
- The multi-task CNN achieved 54.27% test-set DSC, below the dedicated U-Net, with lower segmentation performance attributed partly to its architecture and scoring-based checkpoint selection.The reported DSC values were 55.81%, 56.88%, and 54.27% for training, validation, and test sets; segmentation supervision was beneficial for scoring but not necessarily mutually beneficial.Source: §4.1 · pp. 16–17
4.2. Comparison of PVS scoring models
- The multi-task CNN was best or jointly best across all scoring metrics and was the only method with a significant mean F1 improvement over logistic regression (p = 0.03).Table 3 reports the comparative classification results; the significance result comes from paired difference bootstrap testing.Source: Table 3 · p. 17, §4.2 · p. 17
4.3. Multi-task model validation
- GradCAM showed several salient features on visible PVS, while baseline and conditional CNN maps did not appear to localize PVS in the first layer.The authors caution against over-interpreting these visualizations.Source: §4.3 · p. 19, Figure 9 · p. 19
- The model was less confident on intermediate score 2 than on scores 1 or 3, and high-confidence predictions generally corresponded with accuracy.Score 2 denotes 11–20 PVS, a bounded interval with a smaller margin for error; errors also tended to reflect confusion with score 2.Source: §4.3 · p. 20, Figure 10 · p. 21
- Clinical variables required similarly directed normalized beta coefficients for ground-truth and predicted scores, with WMH volume the most significant predictor across models.The analysis used a subset of 200 test-set cases with available clinical variables.Source: §4.3 · p. 20, Table 4 · p. 21
- Human observers likely remained more reliable than the CNN by inter-rater agreement, although this comparison was limited by the dataset size.The reported comparison used a subset rated independently by two trained observers.Source: §4.3 · pp. 20, 22, Figure 11 · p. 22
5. Conclusion
The conclusion presents multi-task learning as the most successful of the evaluated approaches for scoring PVS using silver-standard masks. It reports PVS localisation and clinically relevant prediction behaviour, while noting the modified scale and single data split as limitations.
5. Conclusion
- Multi-task learning emerged as the most successful approach for predicting PVS scores using silver-standard segmentation masks.The conclusion describes evaluation of three modelling methodologies on a modified Potters/Wardlaw scale.Source: §5 · p. 22
- GramCAM analysis showed that multi-task learning localised individual PVS, unlike the baseline and conditional CNNs.Source: §5 · p. 23
- The model’s predictions had similar associations with clinical markers to observer ground truth and lower confidence on harder classes.Observer agreement was 0.66 for BG scores and 0.36 for CSO scores.Source: §5 · p. 23
- The model uses a modified Potters/Wardlaw scale because few dataset subjects had scores of 0 or 4.The authors identify use of the full scale, including the binary midbrain score, as future work.Source: §5 · p. 23
- The authors position automated scoring as a practical tool for health-association studies while robust segmentation develops.They describe visual scoring as fast, practical, and widely applicable in current clinical practice.Source: §5 · p. 23
- The study’s single data split is a limitation despite bootstrapping test results and ensembling independent training runs.Source: §5 · p. 23
Appendix A.1. Common data augmentation and preprocessing
The appendix documents preprocessing and training procedures for the segmentation, scoring, and logistic-regression models. It also describes a comparison of original silver-standard masks with pseudo-labels for logistic regression and conditional CNN models.
Appendix A.1. Common data augmentation and preprocessing
- Augmentation used affine transforms, noise, blur, brightness and contrast changes, flips, and simulated low resolution; patches were renormalised to [0, 1].Intensities were normalised using the brain mask and saturated at the 2nd and 98th percentiles.Source: Appendix A.1 · p. 26
Appendix A.2. Segmentation U-Net
- The segmentation U-Net used nnU-Net practices, a six-level DynUNet, Dice plus Cross-Entropy loss, SGD, and 1,000 training epochs.The effective batch size was 6 through gradient accumulation.Source: Appendix A.2 · p. 26
Appendix A.3. Baseline and conditional CNNs
- Baseline and conditional CNNs used a 3D ResNet; the conditional CNN additionally received segmentation U-Net pseudo-labels as an input channel.The models also received MRI sequences and BG and CSO masks.Source: Appendix A.3 · pp. 26–27
Appendix A.4. Multitask CNN
- The multitask CNN added a lightweight segmentation decoder to the shared 3D ResNet and dynamically balanced segmentation and classification losses.Its decoder used upsampling and encoder skip connections; the base task weighting was α = 0.5.Source: Appendix A.4 · p. 27
Appendix A.5. Logistic regression
- Logistic regression used age, sex, and PVS volume and count features, with non-sex features standardised using training-set statistics.The model used LBFGS with an L2 penalty.Source: Appendix A.5 · p. 27
Appendix B. Silver-standard masks vs pseudo-labels
- The mask-source comparison restricted data splits to subjects with silver-standard masks and tested pseudo-labels for logistic regression and conditional CNN training.Source: Appendix B · p. 28
Appendix B.1. Logistic regression
For logistic regression, the appendix reports no statistically significant metric differences between models using original silver-standard masks and binarised pseudo-labels on the test subset. It also gives the selected polynomial degrees and feature sets for BG and CSO models.
Appendix B.1. Logistic regression
- The selected BG models used degree-2 polynomials with four original-mask features versus two pseudo-label features.Both feature sets included BG PVS volume and the maximum single-hemisphere, single-slice BG PVS count.Source: Appendix B.1 · p. 28
- The selected CSO models used a degree-1 polynomial with five original-mask features and a degree-3 polynomial with five pseudo-label features.Both sets included sex, age, whole-scan PVS volume, and CSO PVS volume; their fifth feature differed.Source: Appendix B.1 · p. 28
- No test-subset metrics differed significantly between logistic regression models using original masks and binarised pseudo-labels.The comparison covered N=197 subjects with silver-standard masks.Source: Table B.5 · p. 28, Appendix B.1 · p. 28
Appendix B.2. Conditional CNN
For the conditional CNN, the appendix reports no statistically significant metric differences between test-subset models supplied with original silver-standard masks or binarised pseudo-labels. This ablation trained one random seed per model.
Appendix B.2. Conditional CNN
- No test-subset metrics differed significantly between conditional CNNs using original masks and binarised pseudo-labels.The ablation trained one random seed for each model and evaluated N=197 test subjects with silver-standard masks.Source: Table B.6 · p. 29, Appendix B.2 · p. 29
Appendix C. GradCAM results
GradCAM maps show distinct first- and final-layer patterns across the three CNNs. Only the multi-task CNN’s first-layer maps visibly overlap PVS in both regions.
Appendix C.1. First layer features
- Baseline and conditional CNN first-layer maps show no noticeable PVS-related activations for BG or CSO score-3 targets.The figures show three training runs for each model.Source: Figure C.12 · p. 30, Figure C.13 · p. 31
- The multi-task CNN’s first-layer maps overlap PVS in both BG and CSO for either region’s score-3 target, suggesting both regions’ PVS may inform both scores.The maps show this pattern across three training runs.Source: Figure C.14 · p. 32
Appendix C.2. Final layer features
- Final-layer maps show strong BG activation for BG targets in all conditional and multi-task runs, while CSO targets activate more outside BG than BG targets in those models.The baseline maps are described as not very interpretable; their middle column shows strong BG activation for both targets.Source: Figure C.15 · p. 33, Figure C.16 · p. 34, Figure C.17 · p. 35
Appendix D. Pairwise statistical testing results
Pairwise model comparisons used paired difference bootstrapping with Bonferroni correction.
Appendix D. Pairwise statistical testing results
- Pairwise model differences were tested with 10,000 bootstrap iterations and Bonferroni correction across six comparisons.The correction was applied to control for multiple comparisons between models.Source: Appendix D · p. 36
Appendix D.1. Main method comparison
The BG mAP pairwise results report ties between logistic regression and both the baseline and conditional CNNs.
Appendix D.1. Main method comparison
- BG mAP comparisons label logistic regression as tied with the baseline CNN and conditional CNN.The reported differences are 0.001 for logistic regression versus baseline and -0.067 versus conditional CNN; both are marked Tie/None.Source: Appendix D.1 · p. 37
multitask_CNN
Pairwise results favor the multi-task CNN over logistic regression and the baseline CNN on mAP in both regions. The conditional CNN also outperforms those models on CSO mAP, with no reported difference from the multi-task CNN.
multitask_CNN
- For BG mAP, the multi-task CNN is favored over logistic regression and the baseline CNN in the reported pairwise results.The reported differences are -0.117 versus logistic regression and -0.118 versus baseline CNN.Source: multitask_CNN · p. 37
- All supplied pairwise BG F1 comparisons are labeled Tie/None, including comparisons between the multi-task CNN and the other models.The comparison of conditional and multi-task CNNs is also labeled Tie/None.Source: multitask_CNN · p. 37
- For CSO mAP, both the conditional and multi-task CNNs are favored over logistic regression and the baseline CNN, while their comparison is labeled Tie/None.The reported differences are -0.151 and -0.178 versus logistic regression, and -0.094 and -0.121 versus baseline, respectively.Source: multitask_CNN · p. 37