PaperlayerView exact revision on arXiv

Source-linked AI summary

Mutual Consistency Learning for Semi-supervised Medical Image Segmentation

Yicheng Wu, Zongyuan Ge, Donghao Zhang, Minfeng Xu, Lei Zhang, Yong Xia, Jianfei Cai

arXiv:2109.09960v4cs.CVcs.AI

TL;DR

Limited annotations make ambiguous medical image segmentation regions uncertain and difficult to exploit from unlabeled data. MC-Net+ uses decoder diversity to estimate these regions and mutual consistency with soft pseudo labels to regularize training. Extension experiments report superior performance over five existing methods and a new state of the art, while the design remains constrained by predefined decoders and limited up-sampling choices.

  • Problem

    Limited annotations leave ambiguous medical image segmentation regions uncertain and easily misclassified, while existing methods do not fully exploit unlabeled challenging regions.

  • Method

    MC-Net+ uses a shared encoder with multiple diverse decoders to estimate uncertainty and mutual consistency between decoder probabilities and soft pseudo labels.

  • Results

    Extension experiments report superior performance over five existing methods and a new state of the art for semi-supervised medical image segmentation.

  • Takeaways & Limitations

    MC-Net+ provides an end-to-end approach for learning invariant, low-entropy predictions in hard regions using unlabeled data.

  • Takeaways & Limitations

    The model requires multiple predefined decoders, and available up-sampling strategies are limited for new tasks.

Abstract

from arXiv · show

In this paper, we propose a novel mutual consistency network (MC-Net+) to effectively exploit the unlabeled data for semi-supervised medical image segmentation. The MC-Net+ model is motivated by the observation that deep models trained with limited annotations are prone to output highly uncertain and easily mis-classified predictions in the ambiguous regions (e.g., adhesive edges or thin branches) for medical image segmentation. Leveraging these challenging samples can make the semi-supervised segmentation model training more effective. Therefore, our proposed MC-Net+ model consists of two new designs. First, the model contains one shared encoder and multiple slightly different decoders (i.e., using different up-sampling strategies). The statistical discrepancy of multiple decoders' outputs is computed to denote the model's uncertainty, which indicates the unlabeled hard regions. Second, we apply a novel mutual consistency constraint between one decoder's probability output and other decoders' soft pseudo labels. In this way, we minimize the discrepancy of multiple outputs (i.e., the model uncertainty) during training and force the model to generate invariant results in such challenging regions, aiming at regularizing the model training. We compared the segmentation results of our MC-Net+ model with five state-of-the-art semi-supervised approaches on three public medical datasets. Extension experiments with two standard semi-supervised settings demonstrate the superior performance of our model over other methods, which sets a new state of the art for semi-supervised medical image segmentation. Our code is released publicly at https://github.com/ycwu1997/MC-Net.

1. Introduction

MC-Net+ targets the difficulty of exploiting unlabeled ambiguous regions in medical image segmentation by estimating uncertainty with diverse decoders and enforcing mutual consistency. Experiments across three public datasets report superior performance over five recent methods.

  • Motivation: Limited annotations and costly pixel/voxel-level labeling contribute to over-fitting and sub-optimal medical image segmentation performance.The paper motivates semi-supervised learning as a way to exploit unlabeled medical data.
  • Motivation: Existing semi-supervised methods often do not fully use the learning difficulties of unlabeled data, especially challenging regions.The paper identifies this as a remaining problem across consistency- and entropy-based approaches.
  • Motivation: Uncertainty maps show that highly uncertain predictions concentrate in difficult regions such as thin branch joints, while simpler regions are more likely correctly segmented.With more labeled data, the V-Net mainly refines a few hard areas and produces less ambiguous results.
  • MC-Net+: MC-Net+ uses one shared encoder and multiple slightly different decoders to estimate pixel/voxel-level uncertainty from output discrepancies.Different up-sampling strategies create decoder diversity, and the discrepancy indicates hard regions.
  • MC-Net+: The model sharpens decoder probabilities into soft pseudo labels and enforces mutual consistency between each decoder’s probabilities and the others’ pseudo labels.This minimizes output discrepancy during training and aims to capture useful features from unlabeled challenging regions.
  • Experiments: Extensive experiments report that MC-Net+ outperforms five recent methods and sets a new state of the art for semi-supervised medical image segmentation.The extended work adds experiments on Pancreas-CT and ACDC and evaluates methods in the same environment for fair comparison.

2. Related Work

Related work includes consistency-based, entropy-minimization, pseudo-label, auxiliary-task, and uncertainty-estimation approaches for semi-supervised segmentation. MC-Net+ combines consistency and entropy minimization while using decoder diversity to model uncertainty without separately training multiple models.

  • Consistency-based methods: Consistency-based methods rely on the smoothness assumption that small input perturbations should not cause obvious output deviations.Prior work applies data, feature, sample, model-level, or adversarial perturbations to enforce consistency.
  • Entropy minimization and pseudo labels: Entropy-minimization and pseudo-label methods encourage low-entropy predictions by converting probability maps into pseudo labels.Prior approaches use entropy modules, sharpening functions, fixed thresholds, uncertainty thresholds, or meta-learning.
  • Medical image segmentation: Medical segmentation methods also use co-training, attention, shape constraints, multi-scale or multi-view consistency, auxiliary tasks, and self-supervised representations.These approaches address limited labeled data through ensemble learning, structural constraints, feature learning, or proxy tasks.
  • Research gap: Existing methods can still neglect or underestimate unlabeled challenging regions during training, motivating more effective modeling without corresponding labels.The paper positions MC-Net+ as a response to this gap.
  • Uncertainty-aware consistency: Unlike a related cycled-consistency model using identical architectures and input noise, MC-Net+ uses a shared encoder with multiple slightly different decoders.Different up-sampling strategies are reported to improve intra-model diversity.
  • Uncertainty estimation: Model uncertainty can be estimated using ensembles or MC-Dropout, but separately trained ensembles increase computational cost.MC-Net+ predefines multiple sub-models and estimates epistemic uncertainty in one forward pass.

3. Method

MC-Net+ uses shared-encoder, diverse-decoder predictions to estimate uncertainty and applies mutual consistency constraints to regularize ambiguous unlabeled regions without multiple forward passes.

  • Uncertainty estimation: Statistical discrepancies among decoder outputs represent pixel- or voxel-level uncertainty and identify challenging regions for training.This design approximates model uncertainty more efficiently than repeatedly sampling sub-models with MC-Dropout.
  • Model architecture: MC-Net+ combines one shared encoder with three slightly different decoders built using distinct up-sampling strategies.The decoders use transposed convolution, linear interpolation, and nearest interpolation to increase intra-model diversity.
  • Mutual consistency training: The model converts decoder probability maps into sharpened soft pseudo labels before mutual learning between each decoder and the others.The sharpening temperature T controls pseudo-label construction, while mutual learning reduces discrepancies among outputs.
  • Mutual consistency training: Mutual consistency encourages invariant predictions and low-entropy outputs in highly uncertain regions.The constraints are designed to exploit unlabeled data through consistency and entropy minimization.
  • Optimization: MC-Net+ is trained end-to-end with a weighted supervised Dice loss and mutual consistency loss applied to labeled and unlabeled sets.The mutual consistency loss uses a discrepancy measure described as mean squared error with paired inputs.

4. Experiment

The experiments evaluate MC-Net+ under standard semi-supervised settings across three public medical datasets, covering both 3D binary and 2D multi-class segmentation.

  • Datasets and tasks: MC-Net+ is evaluated on the LA, Pancreas-CT, and ACDC public datasets.The LA and Pancreas-CT tasks use 3D binary segmentation, while ACDC uses 2D multi-class segmentation.
  • Data splits: For LA, 80 samples are used for training and 20 for validation because the testing set lacks public annotations.Performance is reported on the same validation set as other methods for fair comparisons.
  • Datasets and tasks: The Pancreas-CT dataset contains 82 three-dimensional abdominal CT scans, with 62 samples used for training and 20 for reporting performance.The scans have 512 × 512 resolution and slice thicknesses from 1.5 to 2.5 mm.
  • Datasets and tasks: The ACDC extension segments the myocardium and left and right ventricles from 2D cardiac MR slices.The dataset contains imaging samples from 100 patients for training, with a fixed patient-level split of 70, 10, and 20 patients for training, validation, and testing.
  • Experimental settings: Experiments use two standard semi-supervised settings with 10% or 20% labeled data and the remainder unlabeled.All compared methods were re-implemented and tested in an identical environment.
  • Evaluation and implementation: Quantitative evaluation uses Dice, Jaccard, average surface distance, and 95% Hausdorff distance.The 3D implementation uses V-Net, while the 2D ACDC implementation uses U-Net.

5. Result

Across the LA, Pancreas-CT, and ACDC datasets, MC-Net+ outperforms competing semi-supervised methods and handles challenging structures without added inference cost. It also extends to both 2D and 3D segmentation settings.

  • LA dataset: MC-Net+ produces more complete left-atrium segmentations, preserving fine details and eliminating most isolated regions without morphological post-processing.The qualitative comparisons highlight challenging areas and report no use of largest-connected-component refinement.
  • LA dataset: MC-Net+ significantly outperforms other semi-supervised methods in two LA labeling settings without increasing inference costs over V-Net.With 10% labeled data, Dice improves from 55% to 70%; with 20%, it reaches 91.07% versus 91.62% for fully supervised V-Net trained with all labels.
  • Pancreas-CT dataset: MC-Net+ achieves the highest Dice and Jaccard among the compared methods for semi-supervised pancreas segmentation, except for the multi-scale consistency method.The multi-scale extension achieves the best performance in each Pancreas-CT setting and can be incorporated with other multi-scale methods.
  • Pancreas-CT dataset: On Pancreas-CT, the single-scale model obtains comparable surface-based performance and accurately segments challenging regions.The reported qualitative examples concern difficult pancreas structures indicated in Figure 4.
  • ACDC dataset: On ACDC, MC-Net+ obtains the highest Dice and Jaccard with comparable surface-related performance in each semi-supervised setting.The evaluation covers myocardium and left- and right-ventricle targets, and class-level gains are reported with 10% and 20% labeled data.
  • Overall comparison: Across three datasets, MC-Net+ outperforms other state-of-the-art methods, applies to 2D and 3D tasks without backbone-specific dependence, and adds no inference cost.The authors associate these properties with potential use in automatic computer-aided diagnosis systems.

6. Discussion

The discussion attributes MC-Net+ gains primarily to mutual consistency among diverse decoders, while examining decoder count, sharpening, loss weighting, and discrepancy choices. It also identifies architectural and perturbation limitations for future work.

  • Ablation studies: Forcing three decoders to generate similar results yields the largest ablation gains, with average Dice gains of 5.28% and 4.59%.The gains are reported under the two labeled-data settings evaluated in the ablation study.
  • Decoder design: Mutual consistency makes decoder outputs more invariant, reducing ambiguous predictions and model uncertainty while allowing only the first decoder during testing.This design reduces inference costs relative to retaining all decoder outputs.
  • Decoder design: Adding more decoders improves performance, but gains decrease because of confirmation bias; the paper therefore uses three decoders to balance effectiveness and efficiency.The authors suggest more diverse sub-models may perform better when labeled data is adequate.
  • Limitations and future work: The method remains limited by its need for pre-defined decoders and limited up-sampling strategies, and it evaluates model-level rather than data-level perturbations.The authors propose exploring more varied architectures and data-specific perturbation operations because generic ColorJitter may not suit medical data.

7. Conclusion

MC-Net+ targets challenging regions in semi-supervised medical image segmentation with multiple decoders and mutual consistency training. Experiments show superior performance over five existing models on three medical datasets, setting a new state of the art.

  • Conclusion: Effectively leveraging challenging regions is identified as important for semi-supervised segmentation.
  • Conclusion: MC-Net+ uses one encoder and three slightly different decoders to identify highly uncertain regions.Decoder differences provide a signal for locating challenging regions during semi-supervised segmentation.
  • Conclusion: A mutual consistency constraint links probability outputs with soft pseudo labels to produce invariant, low-entropy predictions in hard regions.The scheme provides an end-to-end training mechanism for reducing uncertainty in challenging areas.
  • Conclusion: Extension experiments demonstrate superior performance over five existing models on three medical datasets.
  • Conclusion: MC-Net+ sets a new state of the art for semi-supervised medical image segmentation.
Loading 2109.09960v4…