Source-linked AI summary
Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis
Veronika Cheplygina, Marleen de Bruijne, Josien P. W. Pluim
TL;DR
Medical imaging often lacks labeled data despite growing datasets, motivating learning methods that use unlabeled data, weak labels, or data from other domains. This survey reviews semi-supervised, multiple-instance, and transfer learning across medical-image diagnosis, detection, and segmentation, then examines their connections and research opportunities. It concludes that these approaches are useful but that important questions remain about how to use them and generalize findings across applications.
Problem
Medical imaging frequently lacks labeled data because annotation is difficult, while alternative data or labels may be available.
Method
The survey synthesizes semi-supervised, multiple-instance, and transfer learning in medical imaging, focusing on their assumptions, connections, gaps, and opportunities.
Results
The survey discusses over 140 medical-image-analysis papers and reports that transfer learning was the most popular of the reviewed scenarios in recent years.
Takeaways & Limitations
Future research may benefit from examining connections between learning scenarios and generalizing results between applications.
Takeaways & Limitations
Without additional labels, there is a limit on what these methods can achieve, and some semi-supervised assumptions can lead to worse performance than supervised learning when they do not hold.
Abstract
from arXiv · showhide
Machine learning (ML) algorithms have made a tremendous impact in the field of medical imaging. While medical imaging datasets have been growing in size, a challenge for supervised ML algorithms that is frequently mentioned is the lack of annotated data. As a result, various methods which can learn with less/other types of supervision, have been proposed. We review semi-supervised, multiple instance, and transfer learning in medical imaging, both in diagnosis/detection or segmentation tasks. We also discuss connections between these learning scenarios, and opportunities for future research.
1. Introduction
Medical image analysis increasingly relies on machine learning, but labeled data remain scarce because annotation is costly, time-consuming, and often unavailable beyond the original study. This survey reviews semi-supervised, multiple-instance, and transfer learning as related ways to use alternative data or labels, while identifying connections, gaps, and future research opportunities.
- Medical image analysis uses machine learning extensively for segmentation and computer-aided diagnosis.
- Labeled medical images are scarce because manual annotation is expensive or time-consuming and may not be needed in clinical practice.
- Labeled datasets may also be unavailable to other researchers even after collection.
- Semi-supervised, multiple-instance, and transfer learning incorporate other available data or labels beyond traditional supervised learning.
- The survey overviews these scenarios, examines their connections, identifies research gaps, and proposes future opportunities.
- The review emphasizes learning scenarios and their assumptions rather than providing a complete summary of all related papers or specific classifiers.
2. Overview of techniques
The paper distinguishes learning scenarios by what labels and domains are available, using semi-supervised, multiple-instance, and transfer learning as alternatives to fully supervised training. These scenarios support different prediction targets, from individual instances to bags, and address unlabeled data, weak labels, or domain differences.
- The overview introduces these scenarios through emphysema classification in chest CT and provides notation and acronym tables.
- Semi-supervised learning: Semi-supervised learning supplements labeled training data with an unlabeled dataset to improve predictions.
- Multiple-instance learning: Multiple-instance learning trains on labeled bags containing instances with unknown labels related to the bag label.
- Multiple-instance learning: MIL can classify unseen bags, instances, or both, corresponding respectively to global diagnosis and localizing abnormalities.
- Transfer learning: Transfer learning addresses cases where source and target data come from different domains, such as different scanning protocols.
3. Semi-supervised learning
Semi-supervised learning combines labeled and unlabeled samples to improve medical-image classifiers, relying on assumptions such as smoothness, clustering, or low-density class separation. The survey covers label propagation, graph-based regularization, and semi-supervised SVMs across segmentation and diagnosis, while noting that benefits diminish with more labeled data and assumptions can fail.
- Scenario: Semi-supervised learning uses labeled samples DS together with unlabeled samples U to improve a classifier constructed from DS.The unlabeled set can include unannotated scans or diagnostically ambiguous subjects.
- Assumptions and methods: Common methods assume that feature-space structure reflects the decision function, including smoothness, clustering, or low-density separation.These assumptions motivate label propagation, graph regularization, and margin-based approaches.
- Self-training and co-training: Self-training repeatedly classifies unlabeled samples and adds selected high-confidence samples to the training set, with co-training using additional classifiers.The surveyed papers differ in how they select samples and whether expert verification is required.
- Self-training and co-training: Self-training is used mainly for segmentation, whereas diagnosis and detection applications are less common and involve sample sizes in the thousands.The survey reports applications in brain, retina, heart, chest CT, and cell-nuclei analysis.
- Performance and limitations: Increasing the number of samples improves performance, but the advantages of semi-supervised methods decrease as more labeled data becomes available.When additional assumptions do not hold, semi-supervised learning can perform worse than supervised learning.
- Graph-based methods and regularization: Graph-based methods propagate labels through similarity graphs, while manifold regularization uses graph Laplacians to encourage smoothness and semi-supervised SVMs favor margins excluding unlabeled samples.These approaches are applied to segmentation and computer-aided diagnosis, including Alzheimer’s disease or mild cognitive impairment classification.
4. Multiple instance learning
Multiple instance learning (MIL) trains on labeled bags whose instances have unknown labels, making it useful when global medical-image labels are available but local annotations are costly or unavailable. The survey distinguishes bag and instance goals, reviews global/local detection and false-positive reduction, and highlights trade-offs between bag-level performance, instance predictions, and interpretability.
- Core formulation: MIL uses labeled bags of instances with hidden instance labels related to the bag label, commonly assuming a positive bag contains at least one positive instance.Alternative assumptions include collective relationships among instances.
- Core formulation: MIL classifiers either infer instance predictions and aggregate them into bag predictions or represent each bag directly with a single feature vector.Primarily bag-level methods can provide instance labels, whereas exclusively bag-level methods generally cannot.
- Global detection: Global detection is the most common MIL application, classifying whole images despite weak labels that may not apply to every scan region.Applications include diabetic-retinopathy detection in retinal images and cancer-region detection in histopathology images.
- Global and local detection: Local detection is often relevant but unevaluated because instance-level validation labels are unavailable, even when methods produce local predictions.Some studies address global and local detection together using instance-level or primarily bag-level classifiers.
- Global and local detection: The best bag classifier need not be the best instance classifier, and adding more labeled bags does not necessarily improve instance-level performance.One study found unstable instance predictions from the best bag classifier, while another reported that additional labeled bags did not necessarily increase instance performance.
- Related scenarios: MIL and weak supervision overlap rather than forming disjoint categories, while false-positive reduction uses bags as candidate lesions and instances as correlated viewpoints.In false-positive reduction, the bag label applies in principle to all viewpoints, and combining them can outperform a supervised classifier.
5. Transfer learning
Transfer learning addresses medical-imaging problems in which source and target domains or tasks differ, using instance or feature transfer. The survey organizes these scenarios, reviews applications, and finds that transfer benefits depend on source-data similarity, size, and diversity rather than similarity alone.
- 5. Transfer learning: Transfer learning differs from conventional learning when source and target domains, tasks, or both are different.Domains may differ in feature spaces or sample distributions, while tasks may differ in label spaces or labeling functions.
- 5. Transfer learning: The survey focuses on instance transfer, which reweights source data, and feature transfer, which incorporates source knowledge into target representations.Table 5 classifies applications by weighting or aligning samples, pretraining, or multi-task learning.
- 5.2. Same domain, different tasks: In same-domain, different-task settings, multi-task or representation learning shares features across related tasks and can increase effective data or regularize classifiers.Joint learning may be used instead of independent binary classifiers or may precede the target task through an unsupervised representation-learning objective.
- 5.2. Same domain, different tasks: Multi-task learning provides its largest advantage at low sample sizes in simultaneous heart-wall-segment classification, where regularization is most needed.A separate application increased effective sample size by jointly learning tasks with different label spaces and non-combinable datasets.
- 5. Transfer learning: Transfer from ImageNet alone outperformed transfer from KaggleDR or their union, indicating that source-data size is not the only relevant factor.The survey summarizes a trade-off involving source-data similarity, size, and perhaps diversity.
- 5. Transfer learning: Transfer-learning benefits are not universal: bladder cancer-treatment-response prediction showed no statistically significant differences among networks without transfer, natural-image pretraining, or bladder-ROI pretraining.The survey therefore reports contrasting outcomes across transfer-learning applications rather than a uniformly positive effect.
6. Discussion
The survey finds that transfer learning has become the most popular learning scenario, while application characteristics shape scenario use. It identifies connections and gaps across scenarios, opportunities to use available data more fully, and limits on generalization and label-free improvement.
- Trends: Transfer learning is the most popular scenario in recent years, possibly because external datasets and pretrained models are readily available.Semi-supervised and multiple instance learning require specific types of data or labels, whereas transfer learning can use an external dataset alongside target data.
- Trends: Brain is the most common application, followed by histology/microscopy and abdomen, while breast, heart, and retina have relatively few papers.Around 10% of papers address multiple applications.
- Trends: MIL is frequently used in histology/microscopy but is less common for brain tasks because unordered, variably sized patches fit histology better than anatomically corresponding brain data.Anatomical correspondences in brain images are informative, making the MIL scenario less applicable there.
- Connections and gaps: Relevant learning scenarios are sometimes overlooked, and comparing related scenarios could clarify the fit between problem characteristics, assumptions, and methods.The survey also points to positive-and-unlabeled learning, MIL variants, multi-label supervision, clinical auxiliary outputs, and unsupervised pretraining as opportunities.
- Connections and gaps: The survey highlights links among SSL, MIL, and TL, including MIL as a constrained special case of SSL and comparisons between SSL and TL when domain information differs.MIL can also replace transfer learning when same-domain bag labels are easier to acquire, while shared representations connect MIL with multi-task or self-transfer learning.
- Full potential of available data: Method constraints can prevent full use of labeled data, such as converting regression or multiclass problems into binary MIL tasks or removing difficult classes.Removing one of two disease classes was reported to result in lower performance, possibly because fewer samples remained for learning.
- Full potential of available data: Modeling individual expert labels during training can outperform averaging those labels into consensus labels beforehand.The survey presents this as an opportunity to exploit multiple labels when ground truth is determined by expert consensus.
- Acquiring additional labels: The surveyed approaches may improve classifier robustness, but the authors see a limit to what can be achieved without additional labels.Active learning is presented as a way to query ambiguous or especially useful labels within a fixed labeling budget.
7. Conclusion
The survey reviews more than 140 medical image-analysis papers on learning with limited or alternative supervision. Transfer learning is the most popular scenario in recent years, but important questions remain about how to use these methods best.
- Conclusion: The survey discusses over 140 papers on classification in semi-supervised, multi-instance, and transfer learning scenarios.The papers address medical image analysis where representative annotated data is often lacking.
- Conclusion: Transfer learning is the most popular of the surveyed scenarios in recent years.The authors state that individual papers demonstrate the usefulness of these approaches while broader methodological questions remain.
- Conclusion: Future research is expected to benefit from examining how these methods can best be used.The conclusion frames this as an unresolved issue after reviewing the existing literature.