Source-linked AI summary
Star Shape Prior in Fully Convolutional Networks for Skin Lesion Segmentation
Zahra Mirikharaji, Ghassan Hamarneh
TL;DR
Pixelwise FCNs do not account for high-level label dependencies, while encoding structural priors in deep learning frameworks has been less studied. The paper adds a star shape regularizer to end-to-end FCN training, penalizing non-star-shaped predictions. The approach ranked first among 21 competing methods on the challenge Jaccard ranking metric.
Problem
Although structural priors have been used in segmentation, incorporating them into FCNs remains less studied, where pixelwise losses assign labels independently without high-level label dependencies.
Source: §1 · p. 2Method
Results
First among 21 competing methods on the challenge Jaccard ranking metric; star shape regularization improved Jaccard by more than 3% for U-Net and ResNet-DUC, with p < 0.05.
Source: §3 · p. 6Takeaways & Limitations
The framework produces more accurate and plausible skin lesion segmentations without a user-defined object center or computationally expensive inference-time optimization.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Semantic segmentation is an important preliminary step towards automatic medical image interpretation. Recently deep convolutional neural networks have become the first choice for the task of pixel-wise class prediction. While incorporating prior knowledge about the structure of target objects has proven effective in traditional energy-based segmentation approaches, there has not been a clear way for encoding prior knowledge into deep learning frameworks. In this work, we propose a new loss term that encodes the star shape prior into the loss function of an end-to-end trainable fully convolutional network (FCN) framework. We penalize non-star shape segments in FCN prediction maps to guarantee a global structure in segmentation results. Our experiments demonstrate the advantage of regularizing FCN parameters by the star shape prior and our results on the ISBI 2017 skin segmentation challenge data set achieve the first rank in the segmentation task among $21$ participating teams.
1 Introduction
Skin lesion segmentation is challenging, and although shape priors have helped traditional segmentation methods, incorporating them into end-to-end FCNs has been less studied. The paper proposes an FCN loss term that enforces a star shape prior without requiring a manually specified object center.
1 Introduction
- Skin lesion segmentation is difficult because healthy and unhealthy skin vary in color, texture, lesion shape, and size across images and classes.The paper frames semantic segmentation as a preliminary step toward automatic image understanding.Source: §1 · p. 1
- Prior knowledge about target structure has been used to regularize energy-based segmentation, but FCNs have received less study in this regard.The authors note that pixel-level FCN losses assign labels independently without accounting for high-level label dependencies.Source: §1 · p. 2
- FCN pooling and strided convolutions reduce image resolution and lose fine structures, while resolution-recovery techniques only partially restore spatial detail.The cited techniques include deconvolution, skip connections, and dilated convolution.Source: §1 · p. 2
- Earlier star shape methods improved segmentation across varied objects, but Veksler’s approach and variants assumed the foreground object center was known.Such a center could be supplied through user interaction.Source: §1 · p. 3
- The proposed loss encodes a star shape prior in end-to-end FCN training, penalizing non-star-shaped predictions without requiring manually set object centers.The approach also avoids the computationally intensive inference-time optimization associated with energy-minimizing approaches.Source: §1 · p. 3
2 Methodology
The method trains an FCN to produce plausible segmentation maps without requiring a supplied object center at inference. It augments pixel-wise classification with a star-shape loss that penalizes violations in prediction maps.
2 Methodology
- The FCN is trained to generate plausible segmentation maps without extra training, user interaction, or a supplied object center at inference.The star-object center is found automatically from binary ground-truth maps during training.Source: §2 · pp. 3, 5
- Binary cross entropy penalizes each pixel’s predicted label when it deviates from its ground-truth label.The FCN output is the predicted probability that a pixel belongs to a skin lesion.Source: §2 · p. 4
- An object is star-shaped with respect to center c when every point on the segment from an interior point to c remains inside the object.The supplied figure illustrates a star-shaped object and examples of constraint violations.Source: §2 · p. 4, Figure 1 · p. 4
- The objective combines binary cross entropy with a star-shape loss that penalizes inconsistent labels along segments connecting pixels to the object center.The star-shape term applies when the pixels share a ground-truth label and the prediction for p differs from its ground truth; the conditions allow boundary discontinuities and ignore already-correct labels.Source: §2 · pp. 4–5
- Implementation examines the m closest pixels along each segment, quantizes line directions, and finds the object center from ground-truth maps during training.At inference, the trained network predicts in a forward pass without a supplied center.Source: §2 · p. 5
3 Experiments
Experiments evaluate star-shape-regularized U-Net and ResNet-DUC on the ISBI 2017 skin-lesion challenge data. ResNet-DUC achieved the challenge’s top Jaccard-based rank, while both architectures showed statistically significant Jaccard improvements with the prior.
3 Experiments
- The dataset contained 2,000 training, 150 validation, and 600 test images; only 0.14% of mask pixels violated the star-shape definition overall.Violation rates were 0.05% for training, 0.3% for validation, and 0.38% for test pixels; Fig. 2 shows examples.Source: §3 · p. 5, Figure 2 · p. 6
- The prior was evaluated with U-Net and ResNet-DUC, the latter using a ResNet-152 encoder with dense upsampling convolution and dilated convolutions.The encoder was pretrained on ImageNet; dense upsampling reconstructs fine detail from coarse feature maps, while dilated convolutions provide multi-scale context.Source: §3 · pp. 5–6
- Training used mini-batches of 12, validation-tuned hyperparameters, and five initial epochs of binary cross entropy before fine-tuning with the proposed loss.Training took two days and testing took one second per image on the reported 12 GB GPU.Source: §3 · p. 6
- More than 3% Jaccard-index improvement was reported for both U-Net and ResNet-DUC with the star-shape loss, with differences significant at p < 0.05.Significance was assessed on test-set Jaccard indices using the Wilcoxon signed rank sum test.Source: §3 · p. 6
- ResNet-DUC with the star-shape loss ranked first among 21 challenge methods by Jaccard index, and its prediction maps were smoother with a single lesion component in most cases.The other top-ranked teams used additional approaches such as model averaging, multi-scale inputs, and pre- or post-processing.Source: §3 · p. 6
4 Conclusion
The conclusion presents an end-to-end FCN loss that encodes the star shape prior for skin-lesion segmentation and reports convergence to an improved output space. The framework does not require inference-time optimization or a user-defined object center.
4 Conclusion
- Experiments indicate that leveraging prior knowledge in FCNs yields convergence to an improved output space for skin-lesion segmentation.The conclusion describes the resulting segmentations as more accurate and plausible.Source: §4 · pp. 6–7
- The framework avoids computationally expensive inference-time optimization and a user-defined object center.Source: §4 · p. 7
- Future work will extend FCNs to other priors, including anatomically meaningful priors for other 2D and 3D medical-imaging applications.Source: §4 · p. 7