Source-linked AI summary
Semi-supervised Skin Lesion Segmentation via Transformation Consistent Self-ensembling Model
Xiaomeng Li, Lequan Yu, Hao Chen, Chi-Wing Fu, Pheng-Ann Heng
TL;DR
Pixel-wise skin-lesion annotations are costly and time-consuming, motivating semi-supervised segmentation that can use unlabeled images. The paper encourages transformation-consistent predictions, and reports a new ISIC 2017 challenge record with 300 labeled samples, surpassing fully supervised state-of-the-art methods trained with 2000 labeled data.
Problem
Fully supervised skin lesion segmentation requires costly, laborious pixel-wise annotations from experienced dermatologists, and ambiguous lesions can be difficult to label accurately.
Source: §1 · p. 2Method
The method combines supervised and unsupervised losses and uses self-ensembling with rotation- and flipping-consistent predictions to exploit labeled and unlabeled data.
Source: §1 · pp. 2–3, §3.1 · p. 4Results
With 300 labeled data, the method sets a new ISIC 2017 skin lesion segmentation challenge record, exceeding fully supervised state-of-the-art methods trained with 2000 labeled data.
Source: §1 · p. 3Takeaways & Limitations
The semi-supervised approach leverages unlabeled data and largely reduces annotation efforts for dermatologists.
Source: §1 · p. 3The consistency-based regularization relies on the smoothness assumption that nearby data points are likely to share the same label.
Source: §3.1 · p. 5
Abstract
from arXiv · showhide
Automatic skin lesion segmentation on dermoscopic images is an essential component in computer-aided diagnosis of melanoma. Recently, many fully supervised deep learning based methods have been proposed for automatic skin lesion segmentation. However, these approaches require massive pixel-wise annotation from experienced dermatologists, which is very costly and time-consuming. In this paper, we present a novel semi-supervised method for skin lesion segmentation by leveraging both labeled and unlabeled data. The network is optimized by the weighted combination of a common supervised loss for labeled inputs only and a regularization loss for both labeled and unlabeled data. In this paper, we present a novel semi-supervised method for skin lesion segmentation, where the network is optimized by the weighted combination of a common supervised loss for labeled inputs only and a regularization loss for both labeled and unlabeled data. Our method encourages a consistent prediction for unlabeled images using the outputs of the network-in-training under different regularizations, so that it can utilize the unlabeled data. To utilize the unlabeled data, our method encourages the consistent predictions of the network-in-training for the same input under different regularizations. Aiming for the semi-supervised segmentation problem, we enhance the effect of regularization for pixel-level predictions by introducing a transformation, including rotation and flipping, consistent scheme in our self-ensembling model. With only 300 labeled training samples, our method sets a new record on the benchmark of the International Skin Imaging Collaboration (ISIC) 2017 skin lesion segmentation challenge. Such a result clearly surpasses fully-supervised state-of-the-arts that are trained with 2000 labeled data.
1 Introduction
Pixel-wise annotation for skin lesion segmentation is costly and difficult, while melanoma segmentation also faces visual variability and artifacts. The paper proposes transformation-consistent semi-supervised learning and reports a new ISIC 2017 challenge record with 300 labeled samples.
1 Introduction
- Melanoma is highly consequential in skin cancer diagnosis, and automatic segmentation is sought to assist dermatologists.Source: §1 · pp. 1–2
- Melanoma segmentation is challenging because lesion size, location, shape, and color vary, and images may contain artifacts such as hairs and veins.Source: §1 · p. 2, Figure 1 · p. 2
- Fully supervised approaches depend on extensive dermatologist-provided pixel labels, which are laborious, scarce, and difficult to assign for ambiguous lesions.Source: §1 · p. 2
- The method combines supervised and unsupervised losses, using self-ensembling to encourage consistent predictions under different regularizations.Source: §1 · p. 2
- Rotation- and flipping-consistency regularizes predictions across transformed inputs and outputs, addressing segmentation's need for predictions to transform with the image.Source: §1 · p. 3
- With 300 labeled data, the method sets a new ISIC 2017 segmentation-challenge record, exceeding fully supervised state-of-the-art methods trained with 2000 labeled data.Source: §1 · p. 3
2 Related Work
Prior skin-lesion segmentation work includes supervised deep-learning approaches, while medical-image semi-supervised methods have used hand-crafted features or CNNs. The paper distinguishes its goal of using unlabeled data from equivariant representations designed to improve network generalization.
2 Related Work
- Deep-learning skin lesion methods include residual networks, cascaded fully convolutional networks, and end-to-end convolutional networks, but require extensive pixel-wise annotations.Source: §2 · p. 3
- Equivariance work encodes transformations in networks to improve generalization, whereas this paper targets better use of unlabeled data in semi-supervised learning.Source: §2 · p. 3
- Medical semi-supervised methods span GMMs, hand-crafted features, and CNNs, with cited approaches limited by feature representation or risk of local minima.Source: §2 · pp. 3–4
3 Method
The method combines supervised learning with consistency regularization on labeled and unlabeled images. It adds rotation- and flipping-consistent predictions to address the mismatch between segmentation masks and ordinary CNN behavior under transformations.
3.1 Overview
- Training combines cross-entropy on labeled inputs with consistency regularization on labeled and unlabeled inputs under distinct perturbations.The regularization encourages smooth predictions for the same data under perturbations such as Gaussian noise, dropout, and randomized transformations.Source: §3.1 · pp. 4–5
3.2 Transformation Consistent Self-ensembling Model
- The transformation-consistent scheme aligns transformed inputs with transformed prediction maps because segmentation masks should transform with inputs while CNNs are not generally equivariant.The method focuses on rotation and flipping transformations.Source: §3.2 · pp. 5–6, Figure 3 · p. 5
3.3 Training and Inference Procedures
- Training samples one of eight quarter-turn/flip transforms with DenseUNet-167; inference uses the untransformed image, 0.5 thresholding, and hole filling.Figure 4 places supervised-only validation results above proposed-method results and overlays predicted red contours against blue ground truth.Source: §3.3 · pp. 6–7, Figure 4 · p. 7
4 Experiments and Results
On ISIC 2017, the method improves skin-lesion segmentation using labeled and unlabeled data, including with limited labels. It outperforms competing methods in the challenge comparison.
4.1 Dataset and Evaluation Metrics
- The ISIC 2017 challenge dataset has 2,000 annotated training images, 150 validation images, and 600 test images; final challenge rank is determined by JA.The challenge evaluates JA, DI, AC, SE, and SP.Source: §4.1 · p. 7
4.2 Analysis of Our Method
- With 50 labeled images, the semi-supervised method improved JA by 2.46%, DI by 2.64%, and SE by 3.60% over supervised-only training.The comparison was on the validation dataset; Table 1 also includes supervised training with regularization.Source: Table 1 · p. 8, §4.2 · p. 8
- Across labeled/unlabeled data settings, semi-supervised training consistently outperformed supervised-only training, while the performance gap narrowed as labeled data increased.Each semi-supervised experiment used 2,000 images in total, including labeled and unlabeled images.Source: Figure 5 · p. 8, §4.2 · pp. 8–9
- Transformation-consistent regularization and Gaussian-noise/dropout regularization each contributed performance gains, and using both together further boosted performance.The ablation experiments used the same training data with 50 labels.Source: §4.2 · p. 8
4.3 Comparison with Other Methods
- 79.8% JA versus 76.5% for [28]: the semi-supervised method achieved the best performance on the ISIC 2017 challenge test benchmark.The reported gains on DI and SE were 2.5% and 5.4%, respectively.Source: Table 2 · p. 9, §4.3 · p. 10
- With 50 labeled and 1,950 unlabeled images, the method improved JA by 2.46%, compared with gains of 1.55% for and 0.87% for [12].The compared methods use different network backbones, which the authors note affects performance.Source: Table 3 · p. 9, §4.3 · p. 10
4.2 Analysis of Our Method
- With all 2,000 images labeled, JA rose from 79.60% for supervised training to 80.02% for the method using the unsupervised loss.Both experiments used the same data augmentation, and the unsupervised loss was applied to all input data.Source: §4.2 · p. 9
5 Conclusion
The paper concludes that its transformation-consistent self-ensembling method effectively uses unlabeled data for skin-lesion segmentation. The authors propose exploring additional regularization forms and ensembling techniques.
5 Conclusion
- The authors report that the method is robust on the ISIC 2017 challenge dataset and could extend to other semi-supervised medical-imaging problems.They identify further regularization forms and ensembling techniques as future directions for better leveraging unlabeled data.Source: §5 · p. 10