Source-linked AI summary
Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M. Taha, Vijayan K. Asari
TL;DR
Medical image segmentation needs efficient models that handle the parameter demands of encoding and decoding and operate across challenging medical-imaging datasets. The paper extends U-Net with recurrent and recurrent-residual convolutional designs, evaluating RU-Net and R2U-Net across three segmentation applications. The proposed models report better performance than U-Net and residual U-Net with the same number of network parameters.
Problem
Medical image segmentation requires efficient architectures because encoding and decoding increase parameter demands, while medical datasets can suffer from scarcity and class imbalance.
Method
The paper extends U-Net with recurrent convolutional layers and recurrent residual convolutional layers, producing RU-Net and R2U-Net.
Results
The proposed models show better segmentation performance than U-Net and residual U-Net with the same number of network parameters across retina, skin-lesion, and lung datasets.
Takeaways & Limitations
RU-Net and R2U-Net provide U-Net-based alternatives evaluated across patch-based retinal and end-to-end skin and lung segmentation settings.
Abstract
from arXiv · showhide
Deep learning (DL) based semantic segmentation methods have been providing state-of-the-art performance in the last few years. More specifically, these techniques have been successfully applied to medical image classification, segmentation, and detection tasks. One deep learning technique, U-Net, has become one of the most popular for these applications. In this paper, we propose a Recurrent Convolutional Neural Network (RCNN) based on U-Net as well as a Recurrent Residual Convolutional Neural Network (RRCNN) based on U-Net models, which are named RU-Net and R2U-Net respectively. The proposed models utilize the power of U-Net, Residual Network, as well as RCNN. There are several advantages of these proposed architectures for segmentation tasks. First, a residual unit helps when training deep architecture. Second, feature accumulation with recurrent residual convolutional layers ensures better feature representation for segmentation tasks. Third, it allows us to design better U-Net architecture with same number of network parameters with better performance for medical image segmentation. The proposed models are tested on three benchmark datasets such as blood vessel segmentation in retina images, skin cancer segmentation, and lung lesion segmentation. The experimental results show superior performance on segmentation tasks compared to equivalent models including U-Net and residual U-Net (ResU-Net).
I. INTRODUCTION
Medical image segmentation requires efficient architectures because encoding and decoding increase parameter demands, while medical datasets face scarcity and class imbalance. The paper introduces RU-Net and R2U-Net and evaluates them across multiple medical-imaging modalities.
- Motivation: Manual segmentation is slow and tedious, motivating algorithms that segment medical images quickly and accurately without human interaction.The paper also identifies data scarcity and class imbalance as limitations of medical image segmentation.
- Motivation: Medical segmentation architectures use encoding and decoding units, increasing parameter requirements relative to classification models.Encoding produces lower-dimensional feature maps, while decoding restores the original image dimensionality.
- Contributions: The study introduces two modified segmentation models using recurrent convolutional and recurrent residual convolutional networks.These models are named RU-Net and R2U-Net, respectively.
- Contributions: The proposed models are evaluated on retina blood vessel, skin cancer lesion, and lung segmentation tasks.Experiments use patch-based evaluation for retinal vessels and end-to-end image-based evaluation for skin lesions and lungs.
- Contributions: The study compares the proposed models with equivalent state-of-the-art methods using the same number of network parameters.The reported comparison includes U-Net and residual U-Net models.
II. RELATED WORK
Related work established deep residual and U-Net architectures for semantic segmentation, while later variants addressed deeper networks and medical-imaging constraints. This paper extends U-Net with recurrent and recurrent-residual convolutional designs.
- Deep segmentation networks: Deep residual models use identity mappings to facilitate training of very deep networks affected by vanishing gradients.Modern activation functions such as ReLU and ELU are also described as solutions to training difficulties.
- Prior segmentation architectures: FCN, SegNet, DeepLab, and Bayesian SegNet are representative convolutional segmentation architectures discussed in prior work.SegNet uses an encoding network and a decoder that upsamples lower-resolution feature maps.
- U-Net: U-Net combines convolutional encoding and decoding with skip connections to use global location and context while producing segmentation maps end to end.The paper describes U-Net as effective with few training samples.
- U-Net variants: U-Net variants include designs for medical imaging and volumetric segmentation, including 3D U-Net and residual V-Net.These approaches address specialized medical-image settings and introduce components such as residual connections or dice loss.
- Proposed extension: This paper proposes RU-Net with recurrent convolutional layers and R2U-Net with recurrent residual convolutional layers as U-Net-based semantic segmentation models.The following architecture section provides details of both models.
III. RU-NET AND R2U-NET ARCHITECTURES
The paper introduces RU-Net and R2U-Net, U-Net variants that replace regular convolutional layers with recurrent and recurrent-residual units. These designs retain U-Net encoding and decoding while accumulating features recurrently and preserving residual connectivity.
- RCL outputs feed RU-Net down-sampling and up-sampling layers, while R2U-Net passes final RCNN outputs through a residual unit.
- The R2U-Net residual block computes x_l+1 = x_l + ℱ(x_l, w_l), with the result passed to the next sampling layer.
- RU-Net uses forward recurrent convolutional layers in U-Net, whereas R2U-Net adds residual connectivity to recurrent convolutional layers.
- The recurrent units are unfolded across discrete time steps, with Fig. 5 illustrating t = 2 and t = 3 configurations.
- Compared with U-Net, the proposed architectures use recurrent or recurrent-residual layers, feature accumulation, and concatenation instead of cropping and copying.
- RU-Net and R2U-Net are designed with the same number of network parameters as U-Net and ResU-Net while showing better segmentation performance.
IV. EXPERIMENTAL SETUP AND RESULTS
The models were evaluated on three medical-image segmentation settings: retinal blood vessels, skin lesions, and lung lesions. The implementation used Keras and TensorFlow on a single GPU machine.
- The evaluation covered retinal blood-vessel segmentation, skin-cancer lesion segmentation, and lung segmentation from 2D images.
- Retinal experiments used the DRIVE, STARE, and CHASE_DB1 datasets.
- The implementation used Keras and TensorFlow on a single GPU machine with 56G of RAM and an NVIDIA GEFORCE GTX-980 Ti.
A. Database Summary
The database summary covers retinal vessel, skin-lesion, and lung-segmentation datasets, with patch-based preparation used for retinal images. Dataset-specific splits and resizing define the evaluation inputs.
- Blood Vessel Segmentation: DRIVE contains 40 retinal images, split into 20 training and 20 testing samples, with 190,000 extracted 48×48 patches for training and validation.
- Blood Vessel Segmentation: STARE uses leave-one-out testing, while CHASE_DB1 contains 28 images divided into 20 training and 8 testing samples.
- Skin Cancer Segmentation: The skin-lesion dataset contains 2,000 samples: 1,250 training, 150 validation, and 600 testing images resized to 256×256.
- Lung Segmentation: The lung dataset contains 534 labeled 2D samples, with 70% used for training and 30% for testing after resizing from 512×512 to 256×256.
B. Quantitative Analysis Approaches
The paper evaluates segmentation performance with classification, overlap, and curve-based metrics, and compares training accuracy across RU-Net, R2U-Net, ResU-Net, and U-Net.
- Performance metrics: Accuracy, sensitivity, specificity, F1-score, Dice coefficient, and Jaccard similarity are used for quantitative evaluation.The analysis defines these metrics using true-positive, true-negative, false-positive, and false-negative variables.
- Performance metrics: Specificity is calculated as a separate evaluation measure using Eq. (6).
- Overlap metrics: The Dice coefficient measures agreement between the ground truth and segmentation result using Eq. (7).GT denotes ground truth, while SR denotes the segmentation result.
- Overlap metrics: Jaccard similarity is included as an overlap-based evaluation measure using Eq. (8).
- Curve-based evaluation: AUC and ROC analyses evaluate the proposed approaches against existing state-of-the-art techniques.
- Training analysis: Figure 9 compares the training accuracy of RU-Net and R2U-Net with ResU-Net and U-Net.
C. Results 1) Retina Blood Vessel Segmentation Using the DRIVE Dataset
Across retina, skin-lesion, and lung segmentation experiments, the proposed recurrent models generally outperform equivalent U-Net and ResU-Net baselines. Results include stronger accuracy and AUC, with qualitative outputs often closely matching ground truth.
- Retina blood vessel segmentation: R2U-Net and RU-Net provide better training and validation performance than U-Net and ResU-Net on the DRIVE and STARE retina datasets.
- Retina blood vessel segmentation: The proposed retina models show better performance in AUC and accuracy across the evaluated retina blood-vessel datasets.
- Skin cancer lesion segmentation: R2U-Net achieves 0.9424 testing accuracy and 0.9419 AUC for skin-lesion segmentation, with 0.8616 average DC and 0.9421 JSC at t=3.
- Skin cancer lesion segmentation: The skin-lesion outputs generally match ground-truth lesion shapes and can reject a brighter non-target spot in the illustrated cases.
- Lung segmentation: For lung segmentation, the proposed models outperform U-Net and ResU-Net, while R2U-Net with t=3 achieves the highest AUC.
- Experimental configuration: Using t=3 recurrent layers improves performance with only a small increase in network parameters across skin and lung experiments.
D. Evaluation
The evaluation compares recurrent U-Net variants with U-Net and ResU-Net across patch-based retina segmentation and end-to-end skin and lung segmentation. The proposed models outperform the equivalent baselines in reported AUC and accuracy.
- Comparative results: The proposed models outperform ResU-Net and U-Net in AUC and accuracy on all three evaluated datasets.
E. Computational time
The paper reports per-sample testing times for retina blood-vessel, skin-cancer, and lung segmentation, with the computational results summarized in Table V.
- Testing time: Table V summarizes computational time during testing for retinal blood-vessel, skin-cancer, and lung segmentation.
V. CONCLUSION AND FUTURE WORKS
The paper concludes that RU-Net and R2U-Net extend U-Net with recurrent and recurrent residual convolutional networks and achieve better segmentation performance than U-Net and ResU-Net across three medical-imaging datasets.
- Conclusion: RU-Net and R2U-Net are U-Net extensions using recurrent convolutional and recurrent residual convolutional neural networks.
- Conclusion: Across retina blood-vessel, skin-cancer lesion, and lung segmentation, the proposed models perform better than U-Net and ResU-Net with the same number of network parameters.
- Conclusion: The proposed models improve performance during both training and testing phases on all three datasets.
- Future work: Future work will explore a feature-fusion strategy between encoding and decoding units.