Source-linked AI summary
Cost-Effective Active Learning for Melanoma Segmentation
Marc Gorriz, Axel Carlier, Emmanuel Faure, Xavier Giro-i-Nieto
TL;DR
Medical-image CNNs require large labeled datasets, while expert annotation is costly. This work uses Monte Carlo Dropout-based pixel-wise uncertainty in a Cost-Effective Active Learning framework, reaching 74% Dice after nine iterations, though some uncertainty regions remain difficult to distinguish without ground truth.
Problem
CNNs require large labeled datasets, but medical-image annotations by highly qualified professionals are costly.
Source: §1 · p. 1Method
The framework uses test-time Monte Carlo Dropout to estimate pixel-wise uncertainty and guide active sample selection for CNN-based medical-image segmentation.
Source: §1 · p. 1, §3.1 · p. 2Results
74% Dice Coefficient was reached after 9 active learning iterations with 2 training epochs per run, with samples remaining in the central region that may limit model performance.
Source: §4 · p. 5Takeaways & Limitations
The selection strategy plans sequential proportions of samples from uncertainty regions to improve network performance gradually without over-fitting.
Source: §3.2 · p. 4In a real case, accuracy cannot be computed without ground truth, and samples in the third and fourth regions become shuffled and cannot be discriminated.
Source: §3.2 · p. 4
Abstract
from arXiv · showhide
We propose a novel Active Learning framework capable to train effectively a convolutional neural network for semantic segmentation of medical imaging, with a limited amount of training labeled data. Our contribution is a practical Cost-Effective Active Learning approach using dropout at test time as Monte Carlo sampling to model the pixel-wise uncertainty and to analyze the image information to improve the training performance. The source code of this project is available at https://marc-gorriz.github.io/CEAL-Medical-Image-Segmentation/ .
1 Motivation
Medical-image annotation is costly, while CNNs require large labeled datasets; this work explores pixel-wise prediction uncertainty to guide active learning for medical-image segmentation.
1 Motivation
- CNNs need large labeled datasets, but highly qualified professionals make medical-image annotations costly.Active learning iteratively selects informative unlabeled images for annotation and adds them to the training set.Source: §1 · p. 1
- The selection criterion explored is uncertainty in pixel-wise predictions.Source: §1 · p. 1
- The work designs a framework for medical-image semantic segmentation that combines CNN training with Cost-Effective Active Learning.Source: §1 · p. 1
- It develops Monte Carlo Dropout-based information interpreters to analyze the network's intrinsic distribution.Source: §1 · p. 1
2 Related work
The related work describes CEAL's complementary use of human-labeled and pseudo-labeled samples, then outlines U-Net as the segmentation architecture used in this work.
2.1 Cost-Effective Active Learning (CEAL) algorithm
- CEAL [Wang et al. 2016] selects low-confidence samples for oracle labeling and high-confidence samples for automatic pseudo-labeling.The two groups are selected to represent different confidence levels of the current model on unlabeled data.Source: §2.1 · p. 2
2.2 CNN’s for Image Segmentation: U-Net architecture
- The approach uses U-Net [Ronneberger et al. 2015], a 23-layer CNN architecture designed for biomedical image segmentation.Its contracting convolutional path is paired with an upsampling path that concatenates features from the contracting path.Source: §2.2 · p. 2
3 Proposed methodology
The method estimates pixel-wise uncertainty with Monte Carlo Dropout, converts it into image-level scores, and uses those scores to support complementary sample selection. It weights uncertainty by distance from predicted contours and plans sequential selection across uncertainty regions.
3.1 Image uncertainty estimation
- Pixel-wise label uncertainty is estimated from the variance across repeated predictions with Monte Carlo Dropout.Dropout is applied during testing; the stated precision depends on the number of steps T and dropout probability pd, with maximum precision at T →∞.Source: §3.1 · pp. 2–3
- Figure 1 presents a pixel-wise uncertainty map built from 10 prediction steps.Source: Figure 1 · p. 3
- Summing uncertainty-map pixel values assigns higher scores to more doubtful segmentations for complementary sample selection.Source: §3.1 · p. 3
- A distance map weights uncertainty by each pixel's distance to the nearest predicted contour, boosting pixels farther from the contour.The stated aim is to address prediction variability across contours.Source: §3.1 · p. 3
3.2 Complementary sample selection
- The uncertainty score is related to prediction accuracy to distinguish undetected melanomas, highly uncertain samples, certain samples, and uncertain, wrong predictions.Undetected melanomas are described as informative despite low uncertainty; certain samples are candidates for pseudo-labeling.Source: §3.2 · p. 3
- Figure 2 depicts regions of uncertainty and a histogram projection.Source: Figure 2 · p. 3
- In real use, accuracy cannot be computed without ground truth, and samples in the third and fourth regions become indistinguishable in the uncertainty histogram.The method plans sequential selection of proportions from each region to improve performance gradually without over-fitting.Source: §3.2 · p. 4
4 Results and Future work
The approach reached 74% Dice after nine active-learning iterations, although samples remained in the central region and may limit performance.
4 Results and Future work
- The ISIC 2017 experiments simulated active learning by starting with a small portion of annotations and adding segments according to the method’s selections.The dataset provides pixel-wise melanoma segmentations.Source: §4 · p. 4
- The dataset contains 2,000 expert-annotated RGB dermoscopy images, converted to grayscale and adjusted to fit the CNN input dimensions.Annotations trace lesion boundaries as binary masks.Source: §4 · p. 4
- Training samples were augmented to improve the model’s generalization.The augmentation approach follows Krizhevsky et al. [2012].Source: §4 · p. 4
- Starting with 600 labeled samples, each iteration selected 10 images without detected melanoma, 10 with highest uncertainty, and 15 randomly; high-confidence predictions became pseudo-labels.Selections came from a pool of 1,000 images; Figure 3 summarizes data balance and heuristic parameters.Source: §4 · p. 4, Figure 3 · p. 4
- 74% Dice was reached after 9 active-learning iterations, but samples remained in the central region and may limit model performance.Each run used 2 CNN training epochs.Source: §4 · p. 5