Source-linked AI summary
Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentation
Yunhao Bai, Duowen Chen, Qingli Li, Wei Shen, Yan Wang
TL;DR
Semi-supervised medical image segmentation faces an empirical distribution mismatch between scarce labeled data and abundant unlabeled data, while separate learning paradigms can discard labeled-data knowledge. BCP addresses this by bidirectionally copy-pasting labeled and unlabeled image crops within a Mean Teacher framework, and experiments report gains including over 21% Dice improvement on ACDC with 5% labeled data.
Problem
Scarce labeled data and abundant unlabeled data create an empirical distribution mismatch, while separate learning paradigms can discard knowledge learned from labeled data.
Method
BCP bidirectionally copy-pastes random crops between labeled and unlabeled images, trains a Student network on mixed images, and uses pseudo-labels with ground-truth supervision in a Mean Teacher framework.
Results
Over 21% Dice improvement is reported on ACDC with 5% labeled data, while experiments across LA, Pancreas-NIH, and ACDC outperform state-of-the-art methods.
Takeaways & Limitations
Bidirectional copy-pasting reduces the distribution gap between labeled and unlabeled data without introducing new parameters or computational cost compared with the backbone network.
Takeaways & Limitations
The loss formulation depends on a smoothing coefficient λ.
Abstract
from arXiv · showhide
In semi-supervised medical image segmentation, there exist empirical mismatch problems between labeled and unlabeled data distribution. The knowledge learned from the labeled data may be largely discarded if treating labeled and unlabeled data separately or in an inconsistent manner. We propose a straightforward method for alleviating the problem - copy-pasting labeled and unlabeled data bidirectionally, in a simple Mean Teacher architecture. The method encourages unlabeled data to learn comprehensive common semantics from the labeled data in both inward and outward directions. More importantly, the consistent learning procedure for labeled and unlabeled data can largely reduce the empirical distribution gap. In detail, we copy-paste a random crop from a labeled image (foreground) onto an unlabeled image (background) and an unlabeled image (foreground) onto a labeled image (background), respectively. The two mixed images are fed into a Student network and supervised by the mixed supervisory signals of pseudo-labels and ground-truth. We reveal that the simple mechanism of copy-pasting bidirectionally between labeled and unlabeled data is good enough and the experiments show solid gains (e.g., over 21% Dice improvement on ACDC dataset with 5% labeled data) compared with other state-of-the-arts on various semi-supervised medical image segmentation datasets. Code is available at https://github.com/DeepMed-Lab-ECNU/BCP}.
1. Introduction
Semi-supervised medical image segmentation faces an empirical distribution mismatch because labeled data are scarce, while separate learning paradigms can discard labeled-data knowledge. BCP addresses this by bidirectionally copy-pasting labeled and unlabeled data within a Mean Teacher framework, producing strong gains across datasets.
- Problem: Few labeled images make it difficult to estimate the whole dataset’s distribution, creating an empirical mismatch with abundant unlabeled data.Existing methods often process labeled and unlabeled data under separate learning paradigms, which can discard knowledge learned from labeled data.
- Approach: BCP copy-pastes random crops from labeled images onto unlabeled backgrounds and reversely from unlabeled images onto labeled backgrounds.The bidirectional design is instantiated in a Mean Teacher architecture.
- Approach: The Student network uses mixed images and supervisory signals formed by combining labeled ground-truth maps with Teacher-generated pseudo-labels.This symmetric procedure transfers common semantics between labeled and unlabeled data in both directions.
- Results: Over 21% Dice improvement is reported on ACDC with 5% labeled data, alongside gains over state-of-the-art methods on LA, Pancreas-NIH, and ACDC.The method introduces no new training parameters or computational cost compared with the backbone network.
2. Related Work
Related work spans supervised medical image segmentation, semi-supervised consistency and geometric-constraint methods, and copy-paste augmentation. These approaches provide context for BCP’s use of copy-paste to connect labeled and unlabeled data.
- Medical Image Segmentation: Medical image segmentation research includes new 2D/3D architectures and methods that incorporate medical prior knowledge into training.
- Semi-supervised Segmentation: Semi-supervised segmentation commonly uses entropy minimization, consistency regularization, and Mean Teacher extensions with geometric or uncertainty-based constraints.Examples include SASSNet, DTC, UA-MT, and related representation-consistency methods.
- Copy-Paste: Copy-paste transfers cropped regions between images and has been applied to instance segmentation, semantic segmentation, and object detection.Mixup mixes whole images, whereas CutMix mixes image crops.
- Copy-Paste: Prior copy-paste methods include context-aware foreground placement, pretraining with random crops, and labeled-to-unlabeled knowledge transfer.The cited approaches motivate copy-paste as a general data-processing strategy for dense prediction tasks.
3. Method
BCP trains a Mean Teacher Student network on two bidirectionally mixed labeled–unlabeled images, using correspondingly mixed ground-truth and pseudo-label supervision. The framework aligns the training treatment of labeled and unlabeled data while learning shared semantics.
- The training set combines N labeled and M unlabeled medical-image volumes, with N much smaller than M.
- BCP randomly selects two labeled and two unlabeled images, then pastes labeled crops onto unlabeled backgrounds and unlabeled crops onto labeled backgrounds.The resulting mixed images are denoted Xout and Xin, respectively.
- The two mixed images are fed to the Student network so unlabeled images learn common semantics from labeled images in inward and outward directions.
- The Student is optimized by stochastic gradient descent, while the Teacher is updated by exponential moving average; the method also pretrains on labeled data.
- Pseudo-labels use threshold 0.5 for binary segmentation or argmax for multi-class segmentation after retaining the largest connected component.The connected-component step is intended to remove outlier voxels.
- BCP constructs supervisory signals by bidirectionally copy-pasting Teacher pseudo-labels and labeled ground-truth maps, then applies them to Student predictions for Xin and Xout.
4. Experiments
Experiments evaluate BCP on LA, Pancreas-NIH, and ACDC datasets using standard segmentation metrics, comparisons, visualizations, and ablations. BCP consistently outperforms competing methods, while bidirectional copy-paste and zero-centered masking are supported by component studies.
- Datasets and metrics: BCP is evaluated on LA, Pancreas-NIH, and ACDC using Dice, Jaccard, 95HD, and ASD metrics.Dice and Jaccard measure overlap, while ASD and 95HD measure boundary distances.
- LA dataset: BCP achieves the best performance on all four LA evaluation metrics and improves fine-detail segmentation, especially at difficult organ edges.The reported shape-related gains occur without explicit boundary or shape constraints during training.
- Pancreas-NIH dataset: 3.24%, 4.28%, and 1.16 are BCP’s improvements over the second-best method on Dice, Jaccard, and 95HD, respectively, for Pancreas-NIH.The comparison uses a V-Net backbone and does not apply post-processing.
- ACDC dataset: 21.76% is BCP’s reported maximum Dice improvement on ACDC with 5% labeled data, where results average four-class segmentation performance.The authors associate the large gain with generating more labeled–unlabeled combinations from 2D slices.
- Copy-Paste Direction: Bidirectional copy-paste outperforms inward-only, outward-only, and within-set copy-paste variants because those alternatives lack either consistent training or cross-set semantic transfer.Inward copy-paste uses unlabeled foreground with labeled background, whereas outward copy-paste uses labeled foreground with unlabeled background.
- Ablation Studies: Zero-centered masking outperforms random and contact masks, while performance is best at β = 2/3 and drops when β = 5/6 or when the copied region becomes smaller.The studies attribute these differences to foreground integrity and interaction with the background.
5. Conclusion
BCP reduces the distribution gap between labeled and unlabeled data through bidirectional copy-paste. Across three datasets, it improves segmentation substantially without adding parameters or computational cost, while extremely low-contrast targets remain difficult.
- BCP extends copy-paste bidirectionally to reduce the distribution gap between labeled and unlabeled data.
- Over 21% Dice improvement is achieved on ACDC with 5% labeled data, while BCP outperforms competing methods on LA, NIH-Pancreas, and ACDC.
- BCP introduces no new parameters or computational cost compared with the backbone network.
- Limitations: Extremely low-contrast target parts remain difficult to segment because BCP does not specifically enhance local attribute learning.