Source-linked AI summary
Deep Multi-scale Location-aware 3D Convolutional Neural Networks for Automated Detection of Lacunes of Presumed Vascular Origin
Mohsen Ghafoorian, Nico Karssemeijer, Tom Heskes, Mayra Bergkamp, Joost Wissink, Jiri Obels, Karlijn Keizer, Frank-Erik de Leeuw, Bram van Ginneken, Elena Marchiori, Bram Platel
TL;DR
Lacune annotation is difficult and subjective because lesions vary in appearance and can resemble perivascular spaces. The paper uses a two-stage CNN system with multi-scale and location context; on a 111-case test set, it detected 97.4% of observer-agreed lacunes at 0.13 false positives per slice.
Problem
Lacunes are difficult to annotate consistently because their appearance varies and they can resemble perivascular spaces.
Source: §1 · p. 1Method
A fully convolutional network detects candidates, followed by a 3D CNN that uses multi-scale analysis and explicit location features to reduce false positives.
Source: §1 · p. 2, §3.2.2 · p. 4Results
97.4% of lacunes agreed on by a majority of four trained observers were detected, with 0.13 false positives per slice.
Source: §6 · p. 9Takeaways & Limitations
A feasibility study found that a trained observer potentially improves when using the CAD system.
Source: §6 · p. 9Objective comparison with other methods was not possible because their implementations and datasets were unavailable.
Source: §5.4 · p. 8
Abstract
from arXiv · showhide
Lacunes of presumed vascular origin (lacunes) are associated with an increased risk of stroke, gait impairment, and dementia and are a primary imaging feature of the small vessel disease. Quantification of lacunes may be of great importance to elucidate the mechanisms behind neuro-degenerative disorders and is recommended as part of study standards for small vessel disease research. However, due to the different appearance of lacunes in various brain regions and the existence of other similar-looking structures, such as perivascular spaces, manual annotation is a difficult, elaborative and subjective task, which can potentially be greatly improved by reliable and consistent computer-aided detection (CAD) routines. In this paper, we propose an automated two-stage method using deep convolutional neural networks (CNN). We show that this method has good performance and can considerably benefit readers. We first use a fully convolutional neural network to detect initial candidates. In the second step, we employ a 3D CNN as a false positive reduction tool. As the location information is important to the analysis of candidate structures, we further equip the network with contextual information using multi-scale analysis and integration of explicit location features. We trained, validated and tested our networks on a large dataset of 1075 cases obtained from two different studies. Subsequently, we conducted an observer study with four trained observers and compared our method with them using a free-response operating characteristic analysis. Shown on a test set of 111 cases, the resulting CAD system exhibits performance similar to the trained human observers and achieves a sensitivity of 0.974 with 0.13 false positives per slice. A feasibility study also showed that a trained human observer would considerably benefit once aided by the CAD system.
1. Introduction
Lacunes are clinically relevant small-vessel-disease imaging features, but their varied appearance and resemblance to perivascular spaces make annotation subjective. The paper proposes a two-stage CNN approach and evaluates it against trained observers.
1. Introduction
- Lacunes are associated with increased risk of stroke, gait impairment, and dementia, and are imaging biomarkers of small vessel disease.Their number and location are recommended for analysis in small-vessel-disease neuroimaging studies.Source: §1 · p. 1
- Differentiating lacunes from perivascular spaces is difficult because both contain cerebrospinal fluid and perivascular spaces can enlarge or acquire a surrounding rim.Lacunes vary in appearance across MRI sequences, and the rim may be absent.Source: §1 · p. 1
- Earlier automated approaches used thresholding, handcrafted features, classifiers, and template matching, and one study reported improved radiologist performance with CAD detections.The cited methods include separate approaches for isolated and ventricular-adjacent lacunes.Source: §1 · p. 2
- The proposed method combines a fully convolutional network for candidate detection with a 3D CNN for false-positive reduction, adding multi-scale analysis and explicit location information.The authors evaluate it against trained observers in a 111-case test set.Source: §1 · p. 2
2. Materials
The study uses MRI data from two cohorts, with 1075 scans divided into training, validation, and test sets. It describes the imaging, annotations, and preprocessing used to prepare the data.
2. Materials
- The dataset comprised 654 RUNDMC scans and 421 FUTURE scans, for 1075 scans total.RUNDMC concerns SVD in adults aged 50–85 without dementia; FUTURE is a cohort of younger patients with cerebrovascular events.Source: §2 · p. 2
2.1. Magnetic Resonance Imaging
- Each subject had a 3D T1 MPRAGE scan at 1.0×1.0×1.0 mm and a FLAIR scan at 1.0×1.2×3.0 mm with a 0.5 mm slice gap.Source: §2.1 · p. 2
2.2. Training, Validation and Test Sets
- The 1075 cases were randomly split into 868 training, 96 validation, and 111 test scans.Source: §2.2 · p. 2
2.3. Reference Annotations
- Two trained raters delineated lacunes in training and validation images slice by slice, following the SVD neuroimaging study standards.Source: §2.3 · p. 2
2.4. Preprocessing
- Preprocessing was performed before the images were supplied to the networks.Source: §2.4 · p. 2
2.4.1. Image Registration
- T1 images were rigidly registered to FLAIR, and all subjects were non-linearly registered to the ICBM152 atlas.The T1-to-FLAIR registration optimized mutual information with trilinear interpolation; FSL-FLIRT and FSL-FNIRT were used, respectively.Source: §2.4.1 · pp. 2–3
2.4.2. Brain Extraction
- FSL-BET extracted brain masks from T1 images, which were transformed and applied to FLAIR images.Source: §2.4.2 · p. 3
2.4.3. Bias Field Correction
- FSL-FAST was applied to correct spatial intensity variations caused by RF inhomogeneities.It uses a hidden Markov random field and an associated expectation-maximization algorithm.Source: §2.4.3 · p. 3
3. Methods
The CAD scheme uses convolutional neural networks in two phases: candidate detection followed by false-positive reduction.
3. Methods
- The CAD scheme has two phases: a candidate detector and a false-positive reducer, both using CNNs.Source: §3 · p. 3
3.1. Candidate Detection
Candidate detection is framed as image segmentation: a CNN produces dense likelihood predictions, which are converted into candidate locations. The approach uses balanced patch sampling and full-image convolution to support efficient detection.
3.1. Candidate Detection
- The candidate detector frames detection as segmentation, using a CNN intended to provide fast, sensitive predictions while keeping candidate counts relatively low.The fully convolutional formulation supports dense image predictions in a few seconds for typical brain images.Source: §3.1 · p. 3
3.1.1. Sampling
- Training used 320,000 patches of size 51×51 from FLAIR and T1 images, with twice as many randomly sampled negative patches as positive patches.Positive patches came from lacune-mask voxels and were augmented by horizontal flipping.Source: §3.1.1 · p. 3
3.1.2. Network Architecture and Training Procedure
- The seven-layer CNN produced two-class likelihoods with a softmax output and used a single pooling layer to avoid unwanted shift invariance in segmentation.Its four convolutional layers used 20, 40, 80, and 110 filters; training also used batch normalization, dropout, L2 regularization, and validation-based early stopping.Source: §3.1.2 · p. 3
3.1.3. Fully Convolutional Segmentation and Candidate Extraction
- 0.93 sensitivity with 4.8 candidates per slice was achieved on validation data by thresholding local maxima at 0.1.The fully convolutional reformulation and shift-and-stitch method produce full-size dense predictions; local maxima help separate candidates that may be attached in coarser segmentations.Source: §3.1.3 · p. 4, Figure 2 · p. 4
3.2. False Positive Reduction
A 3D CNN reduces false positives by combining candidate appearance at multiple scales with explicit location features. The method also uses augmented training patches and averages predictions across test-time variants.
3.2. False Positive Reduction
- The false-positive reducer classifies each candidate as a lacune or false positive using multi-scale analysis and explicit location features.Location may help distinguish lacunes from enlarged perivascular spaces, which prominently occur in the basal ganglia.Source: §3.2 · p. 4
3.2.1. Sampling
- Three patch scales from FLAIR and T1 are used, with larger patches down-sampled to provide context while smaller patches retain detail.Positive patches were augmented by cropping and horizontal flipping; negative samples were also randomly cropped to avoid a positional sampling cue.Source: §3.2.1 · p. 4
3.2.2. Network Architecture and Training Procedure
- The late-fusion network processes each scale through a six-layer convolutional stream, concatenates the resulting features, and adds seven location features before classification.The location features include atlas coordinates and distances to the ventricles, cortex, and midsagittal brain surface.Source: §3.2.2 · pp. 4–5, Figure 3 · p. 5
3.2.3. Test-time Augmentation
- At test time, predictions are averaged over 242 cropping-and-flipping variants for each sample.The variants use the cropping and flipping operations described for patch sampling.Source: §3.2.3 · p. 5
3.3. Observer Study
Four trained observers independently marked lacunes in the test set after a training procedure covering lesion definitions, appearances across modalities, and look-alike structures. They also discussed detections and misses on a shared sample before independent rating.
3.3. Observer Study
- Four trained observers independently marked lacunes by selecting one representative point per lacune on each slice.Training included a session on definitions, FLAIR and T1 appearances, and similar-looking structures, followed by individual rating of 20 training subjects and a consensus discussion.Source: §3.3 · p. 5
3.4. Experimental Setup
The study compared CAD detections with trained observers using FROC analysis and majority-vote reference standards. It also evaluated whether observers benefited from CAD marks and measured contributions from contextual features and test-time augmentation.
3.4.1. FROC
- FROC evaluation compared CAD and each observer against a reference formed by majority vote from the other three observers.A detection was counted as a true positive when within 3 mm of a reference lacune marker; confidence intervals used 100 bootstrap samples.Source: §3.4.1 · pp. 5–6
- Table 1 reports candidate-detector sensitivity across observer-agreement definitions and reference standards formed by excluding one observer at a time.The candidate detector produced an average of 4.6 candidates per slice, or 213 per scan.Source: Table 1 · p. 6
3.4.2. Experiments
- The feasibility study showed CAD marks to an observer at a threshold with 0.88 sensitivity and 0.07 false positives per slice, then asked the observer to check suggestions and add missed lacunes.The study also compared the CAD system with four trained observers using FROC analysis.Source: §3.4.2 · p. 6
- The component analysis evaluated multi-scale analysis, location-feature integration, and test-time augmentation using average sensitivity below 0.4 false positives per slice.It used reference standards requiring agreement from at least two or at least three of the four observers and reported empirical p-values from 100 bootstraps.Source: §3.4.2 · p. 6
4. Results
The results report substantial variation in lacune counts across observers, candidate-detector sensitivity across agreement sets, and comparisons of CAD with observers. The paper also presents component analyses and examples of true and false detections.
4. Results
- Observers detected 64, 38, 142, and 106 lacune locations, respectively, and Table 1 reports candidate-detector sensitivity across agreement sets.The table also summarizes lacunes in agreement under different observer-agreement definitions.Source: §4 · p. 6
- The candidate detector produced an average of 4.6 candidates per slice, or 213 per scan; Figure 4 compares CAD and observer FROC curves.Figure 5 compares observer 2’s performance with and without CAD marks.Source: §4 · p. 6
- Figure 6 evaluates CAD against a reference requiring at least three of four observers and indicates the contributions of method components.Table 2 summarizes component comparisons using average sensitivity below 0.4 false positives per slice and reports p-values and scores.Source: §4 · p. 6
- Figure 7 shows candidate appearances for typical true positives, false positives, and false negatives in FLAIR and T1 slices.Source: §4 · p. 6
5. Discussion
The two-stage design limits expensive classification to a smaller candidate set, while contextual inputs and test-time augmentation improve the method. The study reports strong detection against observer consensus, but cross-method comparisons are not fair across different populations, protocols, and reference standards.
5.1. Two-stage Approach
- The candidate detector reduces the search to 4.6 candidates per slice before the more expensive 3D, multi-scale, location-aware network, also addressing training imbalance among hard negatives.A single-stage approach would require sampling sparsely from the large negative pool, limiting samples of informative voxels such as perivascular spaces.Source: §5.1 · pp. 6–7
5.2. Contribution of Method Ingredients
- Multi-scale analysis and explicit location features significantly improve performance, while test-time augmentation is also effective.Table 2 reports average sensitivity over operating points below 0.4 false positives per slice; Figure 6 depicts component contributions.Source: §5.2 · p. 7, Table 2 · p. 8, Figure 6 · p. 8
5.3. Feasibility Study on Improvement of Human Observers Using CAD
- A trained observer can considerably improve when shown the CAD detections while rating scans.The proposed explanation is that CAD markers ease the exhaustive search and reduce the probability of missing a lacune.Source: §5.3 · p. 8, Figure 5 · p. 8
5.4. Comparison to Other Methods
- At 0.13 false positives per slice, the method detects 97.4% of lacunes agreed on by at least three of four observers; cross-method results are not fair comparisons.Differences in test populations, MR protocols, and reference standards can influence results; Figure 4 compares the CAD system with trained observers on observer-specific consensus standards.Source: Figure 4 · p. 7, §5.4 · p. 8
6. Conclusion
The proposed deep-learning method detected 97.4% of lacunes agreed on by most of four trained observers, with 0.13 false positives per slice. Contextual information and test-time augmentation were effective components, and a feasibility study found that a trained observer potentially improves with CAD assistance.
6. Conclusion
- The method detected 97.4% of lacunes agreed on by most of four trained observers, with 0.13 false positives per slice; contextual information and test-time augmentation were effective components.A feasibility study also found that a trained observer potentially improves when using the CAD system.Source: §6 · p. 9