Source-linked AI summary
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation
Rushi Jiao, Yichi Zhang, Le Ding, Rong Cai, Jicong Zhang
TL;DR
Medical image segmentation depends on costly expert annotations, creating a need to exploit more readily available unlabeled data. This survey organizes deep semi-supervised methods, reviews their empirical evidence, and discusses limitations and unresolved problems.
Problem
Large, high-quality medical segmentation annotations are difficult and expensive to obtain, while efficiently exploiting unlabeled data without expert- examined annotations remains challenging.
Method
The survey provides a taxonomy of deep semi-supervised medical image segmentation methods covering pseudo labels, unsupervised regularization, and knowledge priors, alongside reviews of technical novelties and empirical results.
Results
The review analyzes empirical results across public benchmark datasets and discusses limitations and unsolved problems of existing approaches.
Takeaways & Limitations
Semi-supervised learning uses unlabeled data with limited labeled data to ease expert-annotation burden in medical image segmentation.
Takeaways & Limitations
Existing semi-supervised approaches still require some well-annotated labeled data, and knowledge-prior methods can depend on accurate registration or fixed anatomical locations and shapes.
Abstract
from arXiv · showhide
Medical image segmentation is a fundamental and critical step in many image-guided clinical approaches. Recent success of deep learning-based segmentation methods usually relies on a large amount of labeled data, which is particularly difficult and costly to obtain especially in the medical imaging domain where only experts can provide reliable and accurate annotations. Semi-supervised learning has emerged as an appealing strategy and been widely applied to medical image segmentation tasks to train deep models with limited annotations. In this paper, we present a comprehensive review of recently proposed semi-supervised learning methods for medical image segmentation and summarized both the technical novelties and empirical results. Furthermore, we analyze and discuss the limitations and several unsolved problems of existing approaches. We hope this review could inspire the research community to explore solutions for this challenge and further promote the developments in medical image segmentation field.
1. Introduction
Medical image segmentation supports multiple clinical applications, but deep models require costly expert annotations, especially for 3D scans. This survey reviews semi-supervised solutions, empirical results, limitations, and open problems.
- Motivation: Medical image segmentation labels anatomical structures such as organs and tumors to support diagnosis, treatment planning, radiation therapy, and quantitative analysis.The paper identifies segmentation as a highly active topic in medical image analysis.
- Motivation: Deep segmentation methods still require large amounts of high-quality annotations that are difficult and expensive to obtain from experts.The burden increases for 3D CT and MRI because experts delineate structures slice by slice.
- Semi-supervised learning: Semi-supervised segmentation combines limited labeled data with more readily acquired unlabeled data to reduce manual labeling burden.The paper describes this approach as practical for real-world clinical applications, while efficient exploitation of unlabeled information remains challenging.
- Survey scope: The survey reviews deep semi-supervised medical image segmentation methods and summarizes their technical novelties and empirical results.It focuses on deep semi-supervised segmentation rather than broader annotation-efficient approaches.
- Survey scope: The review also analyzes limitations and unsolved problems to encourage further research in medical image segmentation.Its stated aim is to inspire solutions for the annotation challenge and promote development in the field.
2. Preliminaries
Semi-supervised learning trains segmentation models by combining a small labeled subset with a larger unlabeled set under assumptions about data structure. The preliminaries distinguish learning settings and state the assumptions used to support generalization.
- Basic formulation: Semi-supervised learning uses large amounts of unlabeled data together with labeled data to train higher-performing segmentation models.The formulation targets data-efficient models whose performance approaches that of models trained on fully labeled data.
- Basic formulation: The labeled dataset is a relatively small subset of the full dataset, with M ≪ N in the stated formulation.Labeled cases contain input images and corresponding ground truth, while the remaining cases form the unlabeled set.
- Learning settings: Transductive learning predicts the unlabeled training samples, whereas inductive learning applies the trained model to new unseen data.The distinction depends on whether the test data are wholly available during training.
- Assumptions: Semi-supervised learning requires assumptions about the data distribution to generalize from finite training data to unseen data.The paper introduces cluster, low-density separation, and manifold assumptions as basic examples.
- Assumptions: The cluster, low-density, and manifold assumptions encourage similar, nearby, or same-cluster samples to receive similar labels and place decision boundaries in low-density regions.Unlabeled data can therefore help adjust the decision boundary under these assumptions.
3. Related Work on Semi-Supervised Medical Image Segmentation
Semi-supervised medical image segmentation methods are organized into pseudo-labeling, unsupervised regularization, and knowledge-prior strategies. These approaches use unlabeled images through iterative label generation, joint supervised–unsupervised training, or priors such as target shape and position.
- The survey groups semi-supervised medical image segmentation into pseudo-label, unsupervised-regularization, and knowledge-prior strategies.
- Semi-supervised learning with pseudo labels: Pseudo-label methods train an initial model on limited labels, generate masks for unlabeled cases, merge datasets, and iteratively fine-tune the model.
- Semi-supervised learning with pseudo labels: Label propagation generates pseudo-labels through prototypes or nearest-neighbor matching, transferring semantics from labeled references using feature-space similarity.
- Semi-supervised learning with pseudo labels: Pseudo-label quality is the main constraint because confirmation bias can amplify incorrect predictions into confident but erroneous labels.
- Semi-supervised learning with unsupervised regularization: Unsupervised regularization combines supervised and unsupervised losses during training, while consistency learning enforces similar predictions under perturbed inputs.
- Semi-supervised learning with unsupervised regularization: Consistency methods can use random augmentations, temporal ensembling, or an EMA-weighted teacher, but their effectiveness depends on appropriate perturbation strength and noise robustness.
3.2.2. Unsupervised Regularization with Co-Training
Co-training uses multiple views or network architectures to generate complementary predictions and enforce consistency on unlabeled medical images. The review also describes mechanisms for reducing noise, balancing supervision, and incorporating anatomical or class-aware information.
- Co-training: Co-training trains separate segmentation models on distinct data views, then progressively adds their unlabeled predictions to the training set.The views are assumed to contain sufficient information for independent predictions, while model consistency is encouraged across views.
- Consistency design: Multi-task mean teacher architectures jointly use segmentation, reconstruction, and signed distance field prediction, enforcing both intertask and inter-model consistency.
- Co-training: CNN- and transformer-based backbones can be combined in co-training to encourage consistency and complementarity between networks.
- Noise control: Uncertainty-aware co-training, self-paced learning, confidence maps, and mix-up decay are used to reduce the influence of noisy pseudo-labels.These methods progressively emphasize reliable supervision or easier regions before harder ones.
- Regularization and limitations: Entropy minimization and hard pseudo-labels encourage low-entropy predictions, but high-capacity models can produce confident errors that make entropy minimization ineffective.
- Medical-specific constraints: Anatomical priors and class-aware strategies address medical segmentation structure, imbalance, and uncertainty, while registration- or augmentation-based approaches have setting-specific limitations.Atlas methods may be unsuitable for targets with large positional variance, and their performance depends strongly on accurate registration.
4. Analysis of Empirical Results for Semi-Supervised Medical Image Segmentation
The review describes common overlap- and boundary-based metrics and summarizes benchmark settings used to compare semi-supervised segmentation methods. These benchmarks span left-atrium MRI, pancreas CT, brain-tumor MRI, and cardiac MRI tasks.
- Evaluation metrics: Dice Similarity Coefficient measures region overlap between ground truth and segmentation, while IoU provides an alternative overlap metric.
- Evaluation metrics: Hausdorff Distance evaluates boundary distance error because region-based DSC may miss boundary errors or small mis-segmented regions.95HD uses the 95th percentile of boundary-point distances to reduce the influence of small outlier subsets.
- Benchmarking: Semi-supervised benchmarks are used to support fairer comparisons under shared task settings and public datasets.
- Benchmark datasets: The LA benchmark contains 100 3D gadolinium-enhanced MR scans, with a cited setting using 80 scans for training and 20 for testing.Within the 80 training scans, 16 scans are used as labeled data and the remainder as unlabeled data.
- Benchmark datasets: The Pancreas CT dataset contains 82 3D abdominal CT volumes with varying slice thicknesses and axial slice counts.
- Benchmark datasets: BraTS 2019 provides multi-institutional MRI from 335 glioma patients, while ACDC provides multi-slice 2D cine cardiac MRI from 100 patients for a multi-class task.
5. Existing Challenges and Future Directions
The review identifies distribution shift, class imbalance, unequal unlabeled-data quality, and annotation cost as major unresolved challenges. It points toward cross-center evaluation, selective weighting, and integration with other annotation-efficient supervision schemes.
- Misaligned distribution and class imbalance: Benchmarks with small same-domain datasets may overstate generalization, whereas clinical models must work across centers and scanner vendors.
- Misaligned distribution and class imbalance: Adding unlabeled data from a mismatched distribution can lower performance compared with using no additional unlabeled data.
- Methodological analysis: Most methods treat unlabeled examples with a single unsupervised-loss weight, despite incorrect estimates potentially harming overall performance.
- Methodological analysis: Future methods should assign different weights to unlabeled examples and improve perturbation quality, especially for challenging cases.
- Integration with other annotation-efficient approaches: Semi-supervised learning can be combined with partial, image-level, box-level, pixel-level, scribble, or noisy annotations to further reduce reliance on fully labeled data.
6. Conclusion
The survey organizes deep semi-supervised medical image segmentation methods into three categories and reviews their empirical results. It also identifies limitations and unresolved problems to guide future research.
- The survey classifies existing methods into pseudo labels, unsupervised regularization, and knowledge priors.
- It summarizes the technical novelties and empirical results of these approaches on several public benchmark datasets.
- The survey discusses limitations and unresolved problems in existing approaches and aims to inspire further research.