Source-linked AI summary
Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network
Dwarikanath Mahapatra, Behzad Bozorgtabar, Jean-Philippe Thiran, Mauricio Reyes
TL;DR
Limited expert availability and intensive curation make robust medical-image classification and segmentation difficult to train. The paper combines active learning, conditional generative image synthesis, and Bayesian informativeness scoring, achieving near-full-dataset performance with about 33–35% labeled data and reducing annotation effort.
Problem
Robust medical-image classification and segmentation are challenging because datasets contain limited images spanning disease types and severity, while expert labeling and curation are intensive.
Method
The framework generates realistic chest X-ray images from original or altered masks with a cGAN, then uses Bayesian neural-network uncertainty to select informative samples for classification or segmentation training.
Results
About 33–35% labeled samples achieves almost equal classification and segmentation performance to the full dataset.
Takeaways & Limitations
The framework can substantially reduce the images and pixels requiring expert annotation while preserving near-full-dataset performance.
Abstract
from arXiv · showhide
Training robust deep learning (DL) systems for medical image classification or segmentation is challenging due to limited images covering different disease types and severity. We propose an active learning (AL) framework to select most informative samples and add to the training data. We use conditional generative adversarial networks (cGANs) to generate realistic chest xray images with different disease characteristics by conditioning its generation on a real image sample. Informative samples to add to the training set are identified using a Bayesian neural network. Experiments show our proposed AL framework is able to achieve state of the art performance by using about 35% of the full dataset, thus saving significant time and effort over conventional methods.
1 Introduction
Medical image classification and segmentation need large labeled datasets, but expert scarcity and curation effort make them difficult to assemble. The proposed approach combines active learning with synthetic image generation to select informative samples and reduce training-data requirements.
- Motivation: Medical image classification and segmentation support computer-aided diagnosis, but robust deep learning requires large labeled datasets.Medical images are difficult to curate because experts are limited and manual curation is intensive.
- Active learning: Active learning incrementally selects informative unlabeled samples, queries their labels, and adds them to the labeled set.In deep-learning settings, current active-learning methods may produce too few labeled samples to train or fine-tune convolutional neural networks.
- Proposed approach: The framework trains a conditional generative adversarial network to generate realistic chest X-ray images from anatomy masks and tests active learning for classification and segmentation.Generated samples are intended to overcome small training datasets while reducing the number of training samples needed for accurate models.
2 Methods
The framework combines conditional image generation, model training, and Bayesian uncertainty to create and select informative chest X-ray samples for active learning.
- Framework overview: The active-learning framework has three components: sample generation, classification or segmentation modeling, and sample-informativeness calculation.An initial small labeled set fine-tunes a pretrained model before iterative sample selection.
- Conditional image generation: A cGAN maps an observed image and latent mask encoding to a realistic output image, using adversarial and L1-based objectives.The generator is conditioned on the test X-ray and an original or altered lung-mask encoding.
- Conditional image generation: The content loss combines NMI, VGG feature distance, and MSE so minimizing it encourages generated images to differ in appearance from the input.NMI is higher for similar images, whereas VGG and MSE are lower; the combined loss is therefore used as a similarity-based term during minimization.
- Synthetic sample generation: Mask boundary displacement and other mask modifications generate varied anatomical or disease characteristics for synthetic X-ray images.The method selects 25 boundary points, displaces them by ±[15] pixels, and fits a B-spline to alter shape.
- Synthetic sample generation: Up to 200 synthetic images are produced per test image, including normal and nodule cases with different severity levels.Figure 2 distinguishes original images from synthetic images generated by altering mask characteristics.
- Sample selection: A Bayesian neural network combines aleatoric and epistemic uncertainty to rank generated samples for addition to the labeled training set.The uncertainty procedure uses sampled outputs and predicted variance in its predictive-uncertainty calculation.
- Implementation details: The implementation begins with pretrained VGG16 or ResNet18 models and iteratively augments selected samples before further fine-tuning.The initial labeled set contains 16 images from each class, while the top 16 uncertain images from each class are added at each iteration.
3 Experiments
Experiments on chest X-ray datasets evaluated active learning for classification and lung segmentation. Both tasks reached fully supervised performance using roughly 30–35% of the labeled data, while the selected samples reduced annotation effort.
- Experimental Setup: The experiments used 247 SCR chest X-ray images for training and a separate NIH test set of 400 images.The SCR dataset contained 93 normal and 154 nodule images, while the NIH test set contained 200 normal and 200 nodule images.
- Classification Results: Active learning classification performance reached almost the fully supervised level at approximately 35% of the dataset for both VGG16 and ResNet18.Performance was reported using sensitivity, specificity, and area under the curve; adding more samples produced no significant gain.
- Segmentation Performance: Lung segmentation reached the fully supervised performance level using 30–35% of the full dataset.Performance was evaluated with Dice Metric and Hausdorff Distance, improving steadily until stabilizing at the 35% threshold.
- Segmentation Performance: When fewer than 10% of training samples were used, segmentation was poor in the most challenging cases.The segmentation results improved as more labeled samples were added.
- Savings in Annotation Effort: At optimum performance, the active-learning training set contained 33% of the annotated image pixels, suggesting nearly 67% savings in expert time and effort.The savings reflect fewer images and pixels requiring expert annotation.
4 Conclusion
The proposed method generates chest X-ray images for active-learning-based model training by modifying associated image masks. A Bayesian neural network evaluates informativeness so the most informative generated samples can be added to training.
- Conclusion: The method generates chest X-ray images for active-learning-based model training by modifying the masks of associated images.Mask modification provides the basis for generating additional training samples.
- Conclusion: A Bayesian neural network calculates each generated image’s informativeness, and the most informative samples are added to the training set.The selection and addition process is repeated until labeled samples provide no additional information.