Source-linked AI summary
Semi-Supervised Medical Image Segmentation via Learning Consistency under Transformations
Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, Marleen de Bruijne
TL;DR
Scarce, costly segmentation labels motivate semi-supervised learning that can exploit abundant unlabeled medical images. The paper combines supervised segmentation with transformation consistency in a Siamese network and reports consistent gains, including 87 ± 1.5% mIOU with 10 labeled training images and performance comparable to state of the art using fewer labels.
Problem
Medical image segmentation often has scarce, costly dense labels despite available unlabeled images from similar distributions.
Method
A shared-weight Siamese network learns segmentation consistency under transformations on labeled and unlabeled images alongside supervised segmentation.
Results
87 ± 1.5% mIOU was achieved by SemiTC with 10 labeled training images and 10 validation images, while outperforming MS-Net at 20% supervision.
Takeaways & Limitations
Unlabeled-data consistency learning mainly accounts for the superior performance over the supervised baseline, supporting comparable state-of-the-art performance with more than five times fewer labeled images.
Takeaways & Limitations
The method assumes transformations have computable inverses, and the experiments focus on elastic deformations.
Abstract
from arXiv · showhide
The scarcity of labeled data often limits the application of supervised deep learning techniques for medical image segmentation. This has motivated the development of semi-supervised techniques that learn from a mixture of labeled and unlabeled images. In this paper, we propose a novel semi-supervised method that, in addition to supervised learning on labeled training images, learns to predict segmentations consistent under a given class of transformations on both labeled and unlabeled images. More specifically, in this work we explore learning equivariance to elastic deformations. We implement this through: 1) a Siamese architecture with two identical branches, each of which receives a differently transformed image, and 2) a composite loss function with a supervised segmentation loss term and an unsupervised term that encourages segmentation consistency between the predictions of the two branches. We evaluate the method on a public dataset of chest radiographs with segmentations of anatomical structures using 5-fold cross-validation. The proposed method reaches significantly higher segmentation accuracy compared to supervised learning. This is due to learning transformation consistency on both labeled and unlabeled images, with the latter contributing the most. We achieve the performance comparable to state-of-the-art chest X-ray segmentation methods while using substantially fewer labeled images.
1 Introduction
Medical image segmentation is constrained by scarce, costly dense annotations despite abundant unlabeled data. The paper proposes learning transformation consistency alongside supervised segmentation on labeled and unlabeled images.
- Dense medical-image segmentations are costly to produce, while unlabeled images from similar distributions are often abundant.
- The proposed approach learns prediction consistency under transformations on both labeled and unlabeled data in addition to supervised learning.
- A Siamese network feeds differently transformed versions of each image into identical branches and compares aligned pixel-wise predictions.
- The method focuses experimentally on equivariance to elastic deformations and evaluates labeled-data regularization, unlabeled-data quantity, and labeled-set size.
2 Method
The method combines supervised segmentation with transformation-consistency learning in a shared-weight Siamese network. Differentiable alignment compares branch predictions while allowing consistency gradients to update both branches without prescribing unlabeled targets.
- The composite objective combines a supervised loss on labeled examples with an unsupervised consistency loss over labeled and unlabeled images.
- Mini-batches combine labeled and unlabeled examples, sampling two transformations for each example before optimizing the batch objective.
- For each image, two differently transformed inputs are processed by branches sharing weights θ, and one prediction is transformed for pixel-wise comparison with the other.
- A differentiable transformation layer lets gradients flow through both branches, allowing either prediction or both predictions to change to optimize consistency.
- The consistency mechanism encourages transformation-consistent predictions without encouraging specific predictions for unlabeled images.
- The framework can support transformations whose inverse can be computed; this work uses elastic deformations and corresponding forward and backward deformation fields.
3 Experiments
Experiments use JSRT chest radiographs with multiple labeled-set sizes and cross-validation, comparing supervised and semi-supervised transformation-consistent variants. The design measures how consistency learning and unlabeled data contribute across annotation regimes.
- The JSRT dataset contains 247 posterior-anterior chest radiographs with segmentations for both lungs, both clavicles, and the heart.
- Five cross-validation splits use labeled training subsets of 5, 10, 25, or 50 images, with remaining training images assigned to the unlabeled pool.
- The study compares a purely supervised baseline, supervised transformation consistency, SemiTC using unlabeled training images, and SemiTC+ additionally using validation and test images as unlabeled data.
- Training uses a U-Net-like Siamese architecture, equal labeled and unlabeled counts in semi-supervised batches, and elastic deformation fields for augmentation.
- IOU averaged over the five anatomical structures is used as the mIOU evaluation metric, while IOU averaged over six classes is used for supervised and consistency losses.
4 Results and Discussion
Consistency regularization improved segmentation reliably, with unlabeled data providing the main performance gain over supervised baselines. The method also matched or exceeded several chest X-ray segmentation comparisons while using far fewer labels.
- 24 of 25 experiments showed improved mIOU from consistency regularization, including settings where all training images were labeled.The improvement was described as modest but reliable.
- SemiTC substantially outperformed SupTC with 5 or 10 labeled training images, and remained consistently better with larger labeled sets.SemiTC was better in all five cross-validation splits for the smallest labeled-set sizes and in at least four of five splits for larger sizes.
- Unlabeled-image consistency learning was mainly responsible for the proposed method’s superior performance over the baseline.Adding unlabeled images improved SemiTC and SemiTC+ over SupTC more than adding consistency to the supervised baseline.
- 87 ± 1.5% mIOU was achieved by SemiTC with fewer than 20% supervision, using 10 labeled training images and 10 validation images.MS-Net achieved 67% and 81% mIOU with 20% and 100% strong supervision, respectively.
- Using 124(123) labeled images, the baseline and SemiTC+ outperformed Dai et al. and Novikov et al. across all anatomical structures.They performed similarly to Frid-Adar et al.; lung and heart segmentation reached human-level performance, while clavicle segmentation approached it.
- With 20 images, SemiTC+ reached human-level lung segmentation and was 2.6% lower than observers in heart IOU.Clavicle performance was 2.6% and 2.2% lower than two automatic methods trained on the fully labeled dataset.
5 Conclusion
The paper concludes that transformation-consistency regularization improves chest X-ray segmentation with labeled and unlabeled data. It achieves state-of-the-art-comparable performance while using more than five times fewer labeled images.
- Transformation-consistency regularization improved segmentation with fully labeled training data and with additional unlabeled data.
- The method achieved performance comparable to state-of-the-art chest X-ray segmentation while using more than five times fewer labeled images.