Source-linked AI summary
Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation
Cheng Ouyang, Carlo Biffi, Chen Chen, Turkay Kart, Huaqi Qiu, Daniel Rueckert
TL;DR
Medical FSS is constrained by scarce expert annotations and by imbalance that can erase local background information. The paper proposes SSL-ALPNet, which learns from superpixel pseudo-labels and adaptive local prototypes, and reports stronger performance than annotated-training FSS methods across medical imaging tasks.
Problem
Existing medical FSS methods commonly require many annotated training classes, although expert-labeled medical images are scarce and costly to obtain.
Method
SSL-ALPNet trains an FSS model on unlabeled images using superpixel-based pseudo-labels and adaptive local prototype pooling.
Results
The framework outperforms peer FSS methods trained with manual annotations and generalizes across CT, MRI, segmentation classes, and pathological cases.
Takeaways & Limitations
Superpixel-based self-supervision provides an effective route to medical image representation learning without manual labeling for training.
Abstract
from arXiv · showhide
Few-shot semantic segmentation (FSS) has great potential for medical imaging applications. Most of the existing FSS techniques require abundant annotated semantic classes for training. However, these methods may not be applicable for medical images due to the lack of annotations. To address this problem we make several contributions: (1) A novel self-supervised FSS framework for medical images in order to eliminate the requirement for annotations during training. Additionally, superpixel-based pseudo-labels are generated to provide supervision; (2) An adaptive local prototype pooling module plugged into prototypical networks, to solve the common challenging foreground-background imbalance problem in medical image segmentation; (3) We demonstrate the general applicability of the proposed approach for medical images using three different tasks: abdominal organ segmentation for CT and MRI, as well as cardiac segmentation for MRI. Our results show that, for medical image segmentation, the proposed method outperforms conventional FSS methods which require manual annotations for training.
1 Introduction
The paper addresses annotation scarcity and foreground–background imbalance in medical few-shot segmentation with SSL-ALPNet, combining superpixel-based self-supervision and adaptive local prototype pooling. It evaluates the framework across imaging modalities, segmentation classes, and pathological cases.
- Motivation: Medical image segmentation needs expert annotations that are costly, time-consuming, and often scarce across devices and hospitals.The diversity of anatomical structures and lesions also makes training a separate model for every unseen class impractical.
- Motivation: Existing FSS methods generally require many annotated training classes, limiting their suitability for annotation-scarce medical imaging.The paper proposes self-supervised learning to bypass this requirement.
- Proposed framework: SSL-ALPNet generates superpixel-based pseudo-semantic labels and uses transformed support-query pairs to learn few-shot segmentation from unlabeled images.The self-supervision task segments one superpixel against the rest while random geometric and intensity transforms encourage representation invariance.
- Proposed framework: Adaptive local prototype pooling preserves local class information by extracting multiple background representations, addressing ambiguity caused by extreme foreground–background imbalance.The background is large and spatially inhomogeneous, while the foreground is small and homogeneous.
- Advantages: The framework requires neither annotated training data nor post-SSL fine-tuning and uses a simple, parameter-free ALP module.The authors position these properties as advantages over existing FSS and self-supervised segmentation approaches.
2 Related Work
Related work spans prototypical and support-guided FSS, medical FSS with reduced data, self-supervised segmentation, invariance learning, and unsupervised superpixel generation. The paper distinguishes its approach by removing annotation and fine-tuning requirements for medical FSS.
- Few-shot semantic segmentation: FSS methods either inject support examples as guidance or construct support-derived prototypes and representations for segmenting query images.Prototypical networks make similarity-based predictions or tune query representations using semantic-class prototypes.
- Few-shot semantic segmentation: PANet achieved state-of-the-art natural-image FSS performance using a generic convolutional network and alignment regularization.The passage presents PANet as a representative prototypical-network approach.
- Medical FSS: Most medical FSS work reduces training data but still requires retraining before applying to unseen classes, while SE-Net was designed specifically for medical images.SE-Net is identified as the medical FSS method compared experimentally with the proposed approach.
- Self-supervised segmentation: Self-supervised segmentation has used handcrafted tasks such as transform prediction, inpainting, patch reordering, colorization, and difference detection.Medical-image adaptations commonly require second-stage fine-tuning after weight initialization.
- Representation learning: The proposed SSL technique creates variants to encourage invariant image representations, focusing this strategy on few-shot medical image segmentation.The related methodology is described as sharing a spirit with prior visual clustering and contrastive-learning work.
- Superpixels: Superpixels are compact, piece-wise smooth image segments generated by clustering local pixels using low-level image features.This work uses an efficient unsupervised graph-cut algorithm whose superpixels are more diverse in shape than those from SLIC.
3 Method
The method combines a prototypical few-shot segmentation network with adaptive local prototype pooling and superpixel-based self-supervised learning. It extracts multiple support prototypes, compares them with query features, and trains using superpixel pseudolabels so the network can segment unseen classes without fine-tuning.
- Inference: After self-supervised training, ALPNet can segment unseen classes using a few human-labeled support slices without testing-phase fine-tuning.Support-query episodes provide references for unseen-class prediction while the learned network weights remain fixed.
- Network architecture: The network maps support and query images into feature space, extracts prototypes from support features and labels, and classifies query pixels by similarity.The feature extractor, adaptive local prototype pooling module, and similarity-based classifier form the main components.
- Adaptive local prototype pooling: Adaptive local prototype pooling averages support features within spatial windows and under class masks to produce local and class-level prototypes.A foreground threshold of 0.95 assigns pooled regions to classes, while class-level pooling ensures prototypes for objects smaller than the pooling window.
- Similarity-based segmentation: The resulting prototype ensemble preserves intra-class local distinctions by representing different support regions separately.Each prototype is matched to query features to compute local similarity maps, which are fused into class-wise similarities and normalized into final probabilities.
- Superpixel-based self-supervised learning: Superpixel-based self-supervised learning generates pseudolabels from unlabeled images and uses them to encourage clustered, image-invariant representations.The framework performs offline superpixel pseudolabel generation followed by online episode composition and segmentation-loss training.
4 Experiments
Experiments evaluate SSL-ALPNet across abdominal CT/MRI and cardiac MRI under few-shot settings, including testing classes excluded from training. The method consistently outperforms comparison FSS methods without manual annotations, while ablations support the roles of transforms, SSL, ALP, and superpixel granularity.
- Datasets and settings: Evaluations cover Abd-CT, Abd-MRI, and Card-MRI, using five-fold cross-validation and a shared label set for abdominal organs and cardiac structures.Experiments use 1-way 1-shot evaluation after standardized 2D preprocessing.
- Datasets and settings: Setting 2 removes every training image containing a testing class, including when that class is unlabeled, to assess generalization to completely unseen classes.This is stricter than setting 1, where testing classes may appear as background during training.
- Quantitative and qualitative results: SSL-ALPNet consistently outperforms vanilla PANet and SE-Net2 by an average Dice score of >25 without manual annotation.Results are reported for abdominal and cardiac segmentation comparisons, with generally higher performance on homogeneous Abd-MRI than Abd-CT.
- Quantitative and qualitative results: Table 1 demonstrates strong generalization to unseen classes, supporting the use of superpixel-based self-supervised learning to learn diverse representations from unlabeled images.Qualitative examples are close to ground truth, including Abd-CT and Abd-MRI results under setting 2.
- Ablation studies: Ablations indicate that both SSL and ALP contribute greatly, their combination produces significant gains, and applying both support-query transforms yields the highest overall results.Intensity transformation alone can hurt liver performance, while pseudolabel superpixels should be reasonably smaller than semantic labels.
5 Conclusion
The paper presents a self-supervised few-shot segmentation framework for medical imaging that avoids manual labeling during training. It outperforms state-of-the-art methods and generalizes to unseen semantic classes in the reported experiments.
- The proposed self-supervised few-shot segmentation framework targets medical imaging without requiring manual labeling for training.
- The method demonstrates strong generalization to unseen semantic classes in the experiments.
- Superpixel-based self-supervision provides an approach for image representation learning with potential applications in semi-supervised and unsupervised image segmentation.