Source-linked AI summary
IterNet: Retinal Image Segmentation Utilizing Structural Redundancy in Vessel Networks
Liangzhi Li, Manisha Verma, Yuta Nakashima, Hajime Nagahara, Ryo Kawasaki
TL;DR
Retinal vessel segmentation is important for retinal-disease diagnosis but must recover vessels from scarce labeled data and degraded images. IterNet uses a UNet followed by weight-shared mini-UNets to refine vessel maps, achieving state-of-the-art performance across three commonly used datasets. The experiments use small, specified training and test splits without validation images.
Problem
Retinal vessel segmentation must handle obscured vessel details and structural disconnections despite publicly available datasets containing no more than 20 training images.
Method
IterNet applies a UNet to raw retinal images and iterated, weight-shared mini-UNets to refine the resulting vessel maps using skip connections.
Results
IterNet achieves state-of-the-art performance over three commonly used retinal vessel-segmentation datasets.
Takeaways & Limitations
IterNet provides an iterative UNet-based vessel-segmentation model that uses structural information in intermediate vessel maps and reports top performance across mainstream datasets.
Takeaways & Limitations
Experiments use fixed small training and test splits for DRIVE, CHASE-DB1, and STARE, with no validation images.
Abstract
from arXiv · showhide
Retinal vessel segmentation is of great interest for diagnosis of retinal vascular diseases. To further improve the performance of vessel segmentation, we propose IterNet, a new model based on UNet, with the ability to find obscured details of the vessel from the segmented vessel image itself, rather than the raw input image. IterNet consists of multiple iterations of a mini-UNet, which can be 4$\times$ deeper than the common UNet. IterNet also adopts the weight-sharing and skip-connection features to facilitate training; therefore, even with such a large architecture, IterNet can still learn from merely 10$\sim$20 labeled images, without pre-training or any prior knowledge. IterNet achieves AUCs of 0.9816, 0.9851, and 0.9881 on three mainstream datasets, namely DRIVE, CHASE-DB1, and STARE, respectively, which currently are the best scores in the literature. The source code is available.
1. Introduction
Retinal vessel segmentation supports diagnosis but is constrained by scarce labeled data, imaging degradation, and vessels that blend into background. IterNet addresses these challenges by refining vessel maps through iterative mini-UNets that exploit structural redundancy while sharing weights.
- Retinal examination supports diagnosis of retinal and systemic diseases, while vessel segmentation forms an essential basis for downstream analysis.
- Microvessels can resemble background, and poor illumination or sensor noise may erase information needed for complete, accurate segmentation.
- Public retinal datasets provide no more than 20 training images, limiting models’ ability to learn vessel-network structure from labeled examples.
- UNet and Deform UNet can produce optic-disk boundary errors and disconnected vessel segments in scarce-data settings.
- Human-like structural redundancy can help infer obscured vessel connections, motivating IterNet’s refinement of segmentation results rather than direct raw-image mapping.
- IterNet combines a UNet base module with iterated mini-UNets, weight sharing, and skip connections, and reports top performance across mainstream datasets.
2. Related Work
Prior retinal segmentation methods use local image information, iterative processing, or modified UNet architectures. IterNet instead separates raw-image feature extraction from repeated vessel-map refinement and focuses on exploiting extracted features to infer missing vessel pieces.
- Fully convolutional segmentation methods encode raw images into feature space and decode them into segmented images end to end.
- Traditional vessel segmentation uses local intensity or handcrafted features, including thresholding and masking, with some methods applying iterative processing.
- UNet-based related models include iterative reuse of outputs, dense blocks, and deformable architectures, but differ in how raw images and vessel results are processed.
- Unlike other UNet-based models, IterNet treats UNet feature extraction as sufficient and uses the extracted features to infer missing vessel segments.
3. IterNet
IterNet uses a UNet for initial vessel prediction and repeated mini-UNets for refinement, with shared weights and skip connections supporting training on limited data. Iterative outputs progressively improve segmentation, including connectivity-related errors.
- IterNet is motivated by staged human annotation, using resulting vessel maps to refine an initial segmentation with raw-image information.
- The base UNet outputs vessel probabilities, while mini-UNets refine preceding feature maps using a lighter architecture with fewer parameters.
- Shared weights make refinery modules equivalent to repeatedly applying one module, exposing them to changing segmentation failures and enabling training with only 20 samples.
- 0.9793, 0.9812, and 0.9815 are the AUCs for IterNet outputs Out 1, Out 2, and Out 3, respectively.
- Iterative application can refine missing vessel-network parts, while IterNet generally produces fewer segments in visualization results.
- Skip connections provide upper layers with access to near-input vessel features and connect the base UNet, each mini-UNet, and the mini-UNets with one another.
- Each output is trained with sigmoid cross-entropy, with output losses combined using weights set to 1.
4. Implementation Details
IterNet uses augmentation and image-patch strategies to address scarce retinal training data, with different trade-offs between prediction refinement and inference time.
- Data augmentation: Training samples are randomly modified in color, shape, brightness, and position to reduce overfitting and accommodate varied imaging conditions.A training sample generator produces these variations consistently during training.
- Image patches: Image patches increase the number of available training samples and can be used during training, prediction, or both.IterNet does not require fixed input-image sizes, supporting these alternatives.
- Image patches: Using patches for both training and testing produced the most refined predictions but required substantially longer inference because many patches had to be processed.The resulting patches are concatenated into the final result.
- Dataset examples: Figure 4 presents raw retinal images and dataset masks for DRIVE, CHASE-DB1, and STARE.The datasets are shown in panels (a), (b), and (c), respectively.
5. Experiments
Experiments evaluate IterNet across three retinal datasets against established models using pixel-level performance and vessel-network connectivity. IterNet generally performs best across datasets and achieves the highest connectivity, while architectural ablations reveal overfitting and performance costs.
- Experimental setup: Experiments use DRIVE, CHASE-DB1, and STARE, with 20, 20, and 16 training images respectively, and no validation images.The remaining images form the test sets for CHASE-DB1 and STARE; DRIVE uses a 20/20 split.
- Pixel-level comparisons: IterNet generally outperformed UNet, DenseBlock-UNet, and Deform-UNet in ROC comparisons across the three datasets.All compared state-of-the-art models already achieved AUC > 0.97, so the performance boost was small.
- Pixel-level comparisons: On DRIVE, both IterNet variants achieved superior AUCs to all other compared models, and were the only tested models with AUCs higher than 0.98.Patch-based prediction improved performance but required longer running time.
- Pixel-level comparisons: IterNet had the best performance in most metrics on both CHASE-DB1 and STARE under comparisons using consistent settings.The datasets lacked officially specified training/test splits and FoV masks, so reproduced models were compared under the same settings.
- Connectivity evaluation: Connectivity evaluates vessel-network structure because pixel-level metrics do not reflect whether segmented vessels remain connected for clinical analysis.The metric uses the area under the threshold-versus-connectivity curve, and IterNet achieved the highest connectivity on all three datasets.
- Qualitative results: Visualization results showed IterNet performing best across all three datasets, including connecting vessel segments that appear visually disconnected in raw images.The authors attribute this to IterNet’s iterative architecture and its understanding of vessel networks.
- Ablation studies: Without weight-sharing, IterNet encountered serious overfitting when N ≥ 2, while removing skip connections reduced AUCs to 0.9799, 0.9770, and 0.9808 on three datasets.With N = 2, one mini-UNet run produced an AUC of 0.9795 on DRIVE.
6. Conclusion
IterNet addresses errors in retinal vessel segmentation by refining an initial UNet prediction with iterative mini-UNets. Weight-sharing and skip-connections help it detect and correct defects, achieving state-of-the-art performance across three commonly used datasets.
- IterNet uses a standard UNet to produce an initial vessel-network prediction from raw retinal images.
- Iterative mini-UNets refine the initial segmentation to remove inconsistent vessels and missing pixels.
- Weight-sharing and skip-connections help IterNet identify and correct defects in intermediate segmentation results.
- IterNet achieves state-of-the-art performance across three commonly used retinal vessel-segmentation datasets.
(Supplemental Materials)
The supplemental results evaluate connectivity, architectural variants, inference cost, and segment counts. IterNet generally outperforms comparison methods, while simplified variants lose performance and smaller patch strides improve refinement at higher computational cost.
- IterNet almost always outperforms three other methods on connectivity-area measurements across DRIVE, CHASE-DB1, and STARE.Connectivity is measured as the area under curves plotted over different thresholds.
- Both evaluated IterNet variants suffer performance drops across all three datasets.The variants remove selected skip connections or replace mini-UNets with full-size UNets.
- Smaller patch strides improve refinement but substantially increase inference time.The experiments use 128 × 128 image patches with strides of 3 or 8 pixels.
- IterNet almost consistently produces fewer segments in visualized prediction results.