PaperlayerView exact revision on arXiv

Source-linked AI summary

Retinal vessel segmentation based on Fully Convolutional Neural Networks

Américo Oliveira, Sérgio Pereira, Carlos A. Silva

arXiv:1812.07110v2eess.IVcs.LGstat.ML

TL;DR

Retinal vessel segmentation is important for quantitative vascular assessment, but manual segmentation is error-prone and time-consuming. The paper combines SWT with a multiscale FCN and uses rotations for joint augmentation and prediction. It reports leading accuracy rankings on DRIVE and CHASE_DB1, leading AUC on all evaluated databases, and a trade-off between sensitivity and false detections.

  • Problem

    Manual retinal vessel segmentation is error-prone and time-consuming, while vessel attributes and image quality vary substantially.

  • Method

    The method combines SWT-based decomposition with a multiscale FCN and uses rotation operations jointly for data augmentation and prediction.

  • Results

    The method ranks first in accuracy on DRIVE and CHASE_DB1, ranks second on STARE, and leads in AUC on all three databases.

  • Takeaways & Limitations

    The approach improves Acc and AUC while reducing the combination of false positives and false negatives.

  • Takeaways & Limitations

    The reported comparison involves a trade-off between sensitivity and false detections.

Abstract

from arXiv · show

The retinal vascular condition is a reliable biomarker of several ophthalmologic and cardiovascular diseases, so automatic vessel segmentation may be crucial to diagnose and monitor them. In this paper, we propose a novel method that combines the multiscale analysis provided by the Stationary Wavelet Transform with a multiscale Fully Convolutional Neural Network to cope with the varying width and direction of the vessel structure in the retina. Our proposal uses rotation operations as the basis of a joint strategy for both data augmentation and prediction, which allows us to explore the information learned during training to refine the segmentation. The method was evaluated on three publicly available databases, achieving an average accuracy of 0.9576, 0.9694, and 0.9653, and average area under the ROC curve of 0.9821, 0.9905, and 0.9855 on the DRIVE, STARE, and CHASE_DB1 databases, respectively. It also appears to be robust to the training set and to the inter-rater variability, which shows its potential for real-world applications.

1. Introduction

Retinal vessel segmentation supports quantitative vascular assessment, but manual annotation is time-consuming and error-prone while vessel appearance and image quality vary substantially. The paper proposes combining SWT, multiscale FCN processing, and rotation-based augmentation and prediction to address these challenges.

  • Motivation: Retinal vascular morphology can assist diagnosis and monitoring of ophthalmologic and cardiovascular disorders.Relevant characteristics include branching patterns, length, width, tortuosity, and vessel angles.
  • Motivation: Manual vessel segmentation is error-prone and time-consuming even for experienced physicians.The paper therefore motivates automatic and accurate segmentation.
  • Motivation: Segmentation is difficult because vessels vary abruptly in size, shape, intensity, branching, and crossing, while retinal images may be low quality or pathological.These factors complicate reliable extraction of the vascular tree.
  • Related work: Existing approaches include filtering, morphology, tracking, multiscale, model-based, supervised, unsupervised, and deep-learning methods, each with reported constraints.Examples include missed or unseeded segments, difficult multiscale parameter selection, difficulty encompassing normal and abnormal vasculatures, and dependence on feature engineering.
  • Contributions: The proposed method combines SWT with a multiscale FCN to exploit the multiscale nature of retinal vessels.The SWT is used to add input channels, while the multiscale architecture is intended to handle vessels at different scales.
  • Contributions: Rotation operations support a joint strategy for data augmentation and prediction that uses learned vessel arrangement and orientation information to refine segmentation.The paper also introduces a multiple-prediction scheme motivated by this strategy and studies inter-physician annotations.

2. Method

The proposed method combines SWT-based multiscale input construction with a multiscale FCN that extracts and reconstructs features for retinal vessel segmentation. The pipeline also uses patch-based processing and CNN design choices intended to preserve spatial information and improve computational efficiency.

  • The pipeline has four stages: SWT-based input building, patch extraction, FCN classification, and prediction refinement.
  • Stationary Wavelet Transform: SWT is used instead of DWT because it avoids downsampling, preserves the input pixel count, and enables additional input channels.The resulting signals retain the original dimensions, are translation invariant, and contain multiresolution information.
  • Stationary Wavelet Transform: The two-dimensional SWT produces vertical, horizontal, and diagonal detail images at each decomposition level.The method uses the Haar wavelet and illustrates detail channels such as dV1, dH1, and dD1.
  • Patch extraction: Training uses extracted patches, overlapping patches during training, nonoverlapping output patches during testing, and zero padding to fit image dimensions.The method extracted 2750, 3250, and 3750 patches per image from DRIVE, STARE, and CHASE DB1, respectively.
  • Network design: The network applies max-pooling, spatial dropout, Xavier initialization, ReLU activations, and nearest-neighbor upsampling as core implementation choices.Spatial dropout removes entire feature maps, while max-pooling discards potentially redundant features and promotes invariance to small details.
  • Fully Convolutional Neural Network: The FCN uses an encoder-decoder architecture: pooling summarizes neighboring features, while decoding combines reconstructed maps with encoder features through skip connections.Unlike FC-CNNs with fully connected layers, FCNs accept images of arbitrary size and output probability maps with matching dimensions.

2.4. Data Augmentation and Multiple Prediction

The method uses random 90° rotations for training augmentation and reverses those rotations during prediction so class-probability maps can be aligned and averaged. This design addresses FCN-specific pixel correspondence and ordering constraints.

  • Design rationale: The rotation strategy is presented as a way to use rotational information in FCNs while avoiding the alignment problems of FC-CNN-oriented approaches.The paper restricts augmentation to rotations because they were particularly useful and focuses on their use in FCNs.
  • Data augmentation: Rotated patches are randomly placed during training because consecutive ordering can harm FCN performance.FCNs preserve correspondence between input pixels and feature-map pixels, so internal rotation and addition would lose alignment.
  • Data augmentation: Each patch is augmented with three rotated copies at 90°, 180°, and 270°, increasing the patch set fourfold.Only multiples of 90° are used to avoid interpolation.
  • Multiple prediction: During prediction, each rotated patch is segmented separately, then its probability map is counter-rotated to align corresponding pixels.The aligned maps are averaged to produce one final segmentation.

3. Experimental Setup

The experiments use three public retinal-image databases, database-specific evaluation protocols, and standard segmentation metrics. Cross-validation is used where fixed train/test divisions are unavailable or unsuitable for comparison.

  • Datasets: The proposed method is evaluated on the DRIVE, STARE, and CHASE DB1 databases.The datasets contain 40, 20, and 28 images, respectively.
  • Evaluation protocol: The folds preserve dataset-specific composition, including pathological individuals in STARE and eye distribution in CHASE DB1.STARE folds contain half images from pathological individuals; CHASE DB1 folds contain three images of one eye and four of the other.
  • Annotations: Ground truth is selected from manual observers, with observer-based performance evaluation used to examine inter-rater variability.The first observer is used for DRIVE and CHASE DB1, while Hoover’s annotations are used for STARE.
  • Metrics: Performance is measured with sensitivity, specificity, accuracy, and area under the ROC curve.These metrics are selected to support comparison with other state-of-the-art works.

4. Results and Discussion

The study validates individual Base System components, compares the best model with state-of-the-art methods, and assesses clinical applicability and inter-rater variability. Statistical tests use paired Wilcoxon signed-rank tests at a 0.05 significance level.

  • Experimental strategy: The experiments remove or replace individual components to validate the Base System through ablation studies.The study begins by validating key components and performs ablations in Sections 4.1 and 4.2.
  • Model comparison: The best model is obtained by adding channels produced by the Stationary Wavelet Transform.This model is then compared with other state-of-the-art works.
  • Robustness evaluation: Clinical applicability is assessed with a cross-training strategy, and model behavior is analyzed under inter-rater variability.These evaluations extend beyond component-level validation.
  • Statistical analysis: Statistical significance is tested with the two-sided paired Wilcoxon signed-rank test at significance level 0.05.The choice reflects the population size and uncertainty about data normality.

4.1. Validation of the Base System

Base System ablations show that rotated-patch information, random presentation, multiple prediction, and spatial dropout contribute to segmentation performance on DRIVE. Removing or altering these components worsens results or vessel detection patterns.

  • Data Augmentation: Reducing the number of patches by four strongly deteriorates accuracy and AUC relative to the Base System.Keeping the original sample count with original or elastic patches yields results close to the reference.
  • Data Augmentation: No augmentation, oversampling, and elastic samples produce simultaneous false positives and false negatives, with elastic samples increasing detected-vessel tortuosity.The results suggest the network benefits most from information encoded by rotations.
  • Data Augmentation: Consecutive Rotations are detrimental to FCN performance, making vessel-segment detection more difficult than randomly placed rotations.The comparison supports random placement of rotated patches during training.
  • Multiple Prediction: Multiple prediction outperforms Simple Prediction on accuracy and AUC and makes the model less prone to false positives.The method averages outputs from the original patch and its rotations.
  • Regularization: Standard Dropout causes a statistically significant decrease in accuracy and AUC and leads to detection of fewer vessel segments.The Base System uses spatial dropout as its regularization strategy.

4.2. Validation of the Stationary Wavelet Transform

Adding SWT detail-coefficient channels improved the Base System on DRIVE, with second-level coefficients outperforming first-level coefficients and reducing false detections.

  • The tested SWT inputs concatenated detail coefficients at the first level, second level, or both levels with the green-channel Base System input.The best model was BS + d2, subsequently called Proposed.
  • All SWT variants improved the Base System in Acc and AUC.
  • The BS + d2 configuration was more effective than BS + d1 for SWT-enhanced vessel segmentation.The first level represents higher-frequency spectral information and induced more false positives.
  • Second-level SWT coefficients produced statistically significant Acc and AUC differences from the Base System.They also reduced the combined false positives and false negatives visible in probabilistic predictions.

4.3. Vessel Segmentation

The Proposed model was evaluated across DRIVE, STARE, and CHASE DB1, with performance reported for segmentation quality, computational cost, and best- versus worst-case examples.

  • Mean Acc exceeded the second observer in all databases, while mean Sp also exceeded that observer in all databases.Mean Sn exceeded the second observer in DRIVE and CHASE DB1 but not STARE.
  • 0.9667 Acc and 0.9903 AUC were achieved in the best DRIVE case, compared with 0.9497 Acc and 0.9786 AUC in the worst case.The corresponding best and worst Sn values were 0.9119 and 0.7628.
  • 0.9837 Acc and 0.9964 AUC were achieved in the best STARE case, compared with 0.9503 Acc and 0.9791 AUC in the worst case.The corresponding best and worst Sn values were 0.8527 and 0.7231.
  • 0.9744 Acc and 0.9909 AUC were achieved in the best CHASE DB1 case, compared with 0.9574 Acc in the worst case.The corresponding best and worst Sn values were 0.8541 and 0.8065.
  • Training took about 4 hours, and fully segmenting each retinal image took approximately 2 seconds.

4.4. Comparison with the State-of-the-art

Compared with state-of-the-art methods, the Proposed approach achieved strong sensitivity, accuracy, and AUC across the three databases, while exhibiting a sensitivity-specificity trade-off.

  • The method achieved the best joint Sn, Sp, and Acc performance on DRIVE and CHASE DB1.It ranked first in all metrics on CHASE DB1 and second in all metrics on STARE.
  • The method outperformed all other works in DRIVE for Sn and Acc, while ranking sixth for Sp.The authors describe a trade-off between Sn and Sp, with a better balance overall.
  • In a worst STARE case, the method failed to reject small haemorrhagic blobs, which may have affected Sp.The authors relate this to using 5-fold cross-validation, which reduced pathological images relative to leave-one-out cross-validation.
  • The method achieved much higher Sn across all databases and much higher Acc in DRIVE and CHASE DB1 than other deep learning methods.Its simpler encoder/decoder outperformed a more elaborate multiscale FCN combined with a CRF in the cited comparison.
  • The method achieved the highest AUC in all databases and reported AUC values above 0.98 for all three.The AUC comparison is independent of the threshold used for final segmentations.

4.5. Cross-Training

Cross-training between DRIVE and STARE tested robustness to the training set, showing generally strong absolute rankings but measurable performance decreases after database transfer.

  • Robustness to the training set was treated as crucial for practical use because new images may come from different acquisition devices.
  • 0.9505 and 0.9597 Acc were obtained after cross-training on DRIVE and STARE, down from 0.9576 and 0.9694.The decreases were about 0.7% and 1.0%, respectively.
  • 0.9748 and 0.9846 AUC were obtained after cross-training on DRIVE and STARE, down from 0.9821 and 0.9905.The reductions were approximately 0.7% and 0.6%, respectively.
  • The method ranked first in all cross-training metrics except Sn for STARE-to-DRIVE and Sp for DRIVE-to-STARE.
  • Training on STARE and testing on DRIVE reduced Sn because the network detected fewer thin vessels.In the reverse direction, Sn rose considerably because DRIVE typically has more annotated thin vessels than STARE.

4.6. Robustness to the Inter-Rater Variability

The model was evaluated against alternative human annotations to assess inter-rater variability, and its performance sometimes matched or exceeded the first observer. Changing the reference observer altered accuracy, revealing both observer disagreement and model sensitivity to the chosen ground truth.

  • Using the second observer as gold standard, the model outperformed the first observer in specificity and accuracy on STARE and CHASE DB1.Higher specificity indicates fewer false positives than the first observer relative to the second.
  • On DRIVE, the model exceeded the first observer in sensitivity, specificity, and accuracy against the second observer.The authors interpret this as fewer false positives and false negatives, including vessel segments identified only by the second observer.
  • Differences between independent human observers appeared larger than differences between the model and a new reference observer.The paper associates human inconsistency with day-to-day variation, fatigue, and stress.
  • When the reference changed from the first to the second observer, average accuracy fell from 0.9694 to 0.9365 on STARE and from 0.9653 to 0.9600 on CHASE DB1.These corresponded to approximate decreases of 3.4% and 0.5%, respectively.
  • On DRIVE, changing the reference increased average accuracy from 0.9576 to 0.9639, an approximate increase of 0.7%.The authors report that the model was closer to the second observer despite training according to the first.
  • Except on STARE, the two observers showed no clear difference in marking patterns, with each identifying vessels the other did not.The authors consider this discordance a limiting factor for further improvements because current results are already highly accurate.

5. Conclusions

The proposed FCN-based method combines rotation-based processing with SWT-derived channels to improve retinal vessel segmentation. It was competitive across three databases, robust to training-set and inter-rater variation, and fast enough to segment an image in about 2 seconds.

  • The paper proposes a novel FCN-based method for retinal vessel segmentation.
  • Rotation operations support data augmentation, while SWT decomposition supplies additional channels to the FCN.
  • The method ranked first in accuracy on DRIVE and CHASE DB1, ranked second on STARE, and led all three databases in AUC.
  • The method appeared robust to the training set and inter-rater variability, supporting its potential for screening and diagnostic systems.
  • Approximately 2 seconds were required to fully segment a retinal image using an efficient GPU implementation.
  • The added features improved accuracy and AUC while reducing the combined false-positive and false-negative errors.The authors state that these results suggest domain knowledge can still benefit deep-learning methods.
  • The amount of training data may limit CNN capacity, motivating future architectures as access to medical data improves.
Loading 1812.07110v2…