PaperlayerView exact revision on arXiv

Source-linked AI summary

Multiscale Switch for Semi-Supervised and Contrastive Learning in Medical Ultrasound Image Segmentation

Jingguo Qu, Xinyang Han, Yao Pu, Man-Lik Chui, Simon Takadiyi Gunda, Ziman Chen, Jing Qin, Ann Dorothy King, Winnie Chiu-Wing Chu, Jing Cai, Michael Tin-Cheung Ying

arXiv:2603.18655v1cs.CVcs.AI

TL;DR

Limited annotations and ultrasound artifacts challenge segmentation, while existing SSL methods have limited unlabeled-data utilization and weak feature representations. Switch combines multiscale and frequency-domain switching with contrastive learning in a teacher-student framework, achieving strong results across six datasets, especially in low-data regimes. The authors identify domain specificity and added computational burden as limitations.

  • Problem

    Ultrasound segmentation is limited by scarce labeled data, speckle noise, low-contrast boundaries, and insufficient use of unlabeled data and robust representations.

  • Method

    Switch integrates Multiscale Switch, Frequency Domain Switch, and contrastive learning within a teacher-student semi-supervised framework.

  • Results

    Switch consistently outperformed state-of-the-art methods across six ultrasound datasets, particularly with 1% and 5% labeled data.

  • Takeaways & Limitations

    The framework is presented as a label-efficient and reliable solution for ultrasound medical imaging that may reduce annotation costs in clinical deployment.

  • Takeaways & Limitations

    The framework is optimized for ultrasound-specific characteristics and empirically tuned parameters, which may limit direct transferability to CT and MRI.

Abstract

from arXiv · show

Medical ultrasound image segmentation faces significant challenges due to limited labeled data and characteristic imaging artifacts including speckle noise and low-contrast boundaries. While semi-supervised learning (SSL) approaches have emerged to address data scarcity, existing methods suffer from suboptimal unlabeled data utilization and lack robust feature representation mechanisms. In this paper, we propose Switch, a novel SSL framework with two key innovations: (1) Multiscale Switch (MSS) strategy that employs hierarchical patch mixing to achieve uniform spatial coverage; (2) Frequency Domain Switch (FDS) with contrastive learning that performs amplitude switching in Fourier space for robust feature representations. Our framework integrates these components within a teacher-student architecture to effectively leverage both labeled and unlabeled data. Comprehensive evaluation across six diverse ultrasound datasets (lymph nodes, breast lesions, thyroid nodules, and prostate) demonstrates consistent superiority over state-of-the-art methods. At 5\% labeling ratio, Switch achieves remarkable improvements: 80.04\% Dice on LN-INT, 85.52\% Dice on DDTI, and 83.48\% Dice on Prostate datasets, with our semi-supervised approach even exceeding fully supervised baselines. The method maintains parameter efficiency (1.8M parameters) while delivering superior performance, validating its effectiveness for resource-constrained medical imaging applications. The source code is publicly available at https://github.com/jinggqu/Switch

I. INTRODUCTION

Ultrasound segmentation is hindered by difficult image characteristics, variable regions of interest, and scarce annotations. Switch addresses these challenges with multiscale spatial switching, frequency-domain switching, and contrastive learning in a teacher-student framework evaluated across six datasets.

  • Ultrasound supports non-invasive, real-time, and cost-effective imaging of superficial organs and tissues.
  • Manual ROI identification is time-consuming and varies in accuracy and consistency with radiologist expertise.
  • Speckle noise, low-contrast boundaries, variable ROI geometry, and inconsistent imaging quality make ultrasound segmentation and annotation challenging.
  • SSL combines limited labeled data with abundant unlabeled data, but remains relatively limited in ultrasound image segmentation.
  • Switch combines multiscale switching, frequency-domain switching, and contrastive learning within a teacher-student framework for superficial ultrasound segmentation.
  • The framework was evaluated on six superficial ultrasound datasets, including lymph node, breast, thyroid, and prostate data.

A. Medical Image Segmentation

Medical image segmentation extracts regions of interest at the pixel level, while SSL and contrastive learning use labeled and unlabeled data to improve representations. Switch uses a Mean Teacher architecture whose student learns from mixed samples and whose EMA teacher supplies pseudo labels.

  • Medical image segmentation classifies 2D images or 3D volumes at the pixel level to extract regions of interest.
  • Consistency regularization enforces similar predictions across augmented views, whereas pseudo labeling adds generated labels for unlabeled data.
  • Contrastive learning maximizes similarity between positive pairs and minimizes similarity between negative pairs to learn representations.
  • Existing methods inadequately handle variable ultrasound ROI geometry and may neglect cohesion between labeled and unlabeled information.
  • Switch uses identical U-Net teacher and student networks to leverage labeled and unlabeled data.
  • The student is trained by gradient descent on mixed samples, while the teacher generates pseudo labels as an EMA of student parameters.

A. Multiscale Switch

Multiscale Switch (MSS) combines labeled and unlabeled ultrasound images through randomly placed coarse and fine patches, producing mixed samples for semi-supervised training. Teacher-generated pseudo-labels and region-specific Dice/cross-entropy supervision support these reassembled samples.

  • MSS addresses fixed-area patch limitations by incorporating partial unlabeled information into labeled samples for ultrasound images with variable ROI size and location.
  • Binary masks combine p coarse patches and q fine patches to provide multiscale spatial mixing.Coarse patches are 128×128 and fine patches are 32×32.
  • Patch locations are randomly sampled within image boundaries using uniform distributions over valid upper-left coordinates.
  • The operation forms ux and xu by combining complementary masked regions from labeled samples x1, x2 and unlabeled samples u1, u2.
  • The teacher generates pseudo-labels for unlabeled images, while mixed samples receive region-specific Dice and cross-entropy supervision.The mixed loss uses separate weights for base and patch regions.

B. Frequency Domain Switch

Frequency Domain Switch (FDS) exchanges controlled low-frequency amplitudes between labeled and unlabeled ultrasound images while preserving phase, then uses reconstructed pairs for contrastive learning. This design transfers texture information without changing pixel-level semantic alignment.

  • FDS performs cross-sample frequency mixing within the same ultrasound domain to strengthen relationships between labeled and unlabeled data.
  • Fourier decomposition separates each image into amplitude and phase components before a centralized low-frequency region is selected for exchange.
  • The frequency area ratio ρ is typically 0.0175, balancing diversity and realism by limiting exchanged information.Larger regions may introduce excessive high-frequency noise and degrade structural integrity.
  • Amplitude switching retains each image’s high-frequency content while transferring low-frequency style from its counterpart.
  • Inverse Fourier reconstruction preserves original phase information, maintaining pixel-level alignment with semantic or pseudo-labels.
  • Same-location feature pairs from original and reconstructed mixed samples are treated as positives, encouraging robust invariant representations.

C. Consistency Regularization

Consistency regularization makes the student produce similar predictions for original mixed images and their frequency-domain reconstructed counterparts. The constraint is intended to improve invariance to frequency-domain perturbations.

  • Consistency regularization enforces similar model outputs for original mixed images and their frequency-domain reconstructed counterparts.
  • The consistency loss uses mean squared error between pre-softmax logit outputs and encourages invariance to frequency-domain perturbations.

D. Augmentations

The framework uses weak and strong augmentations, with reduced intensity for fragile ultrasound images, and combines supervised, contrastive, and consistency losses. Contrastive learning uses spatially corresponding positives and non-corresponding negatives to promote robust representations.

  • Augmentation design: Weak augmentation includes resize/crop and flips, whereas strong augmentation includes contrast, blur, brightness, sharpness, posterize, and solarize operations.
  • Augmentation design: Augmentation intensity is reduced and strongly destructive operations such as histogram equalization are removed because ultrasound ROI size and location vary substantially.
  • Loss design: The method combines MSS, unsupervised contrastive, and consistency-regularization losses in its overall objective.
  • Contrastive objective: Contrastive learning maximizes similarity for positive feature pairs and minimizes it for negative pairs using spatial correspondence.
  • Contrastive objective: FDS-based positives share anatomy under different texture or noise realizations, complementing Dice and cross-entropy supervision with invariance and inter-class separation.

F. Training Strategy

Switch trains in two phases: labeled-only pre-training with MSS initializes the student, followed by self-training that uses labeled and unlabeled data with EMA teacher updates.

  • The pre-training phase uses only labeled data to train the student network with MSS.
  • Pre-training provides a strong initialization for subsequent self-training.
  • Self-training uses labeled and unlabeled data with the full loss, while the teacher starts from student weights and updates through EMA.

IV. EXPERIMENTS

The experiments evaluate Switch across six ultrasound datasets spanning lymph nodes, breast lesions, thyroid nodules, and prostate, using defined partitions, preprocessing, and segmentation metrics.

  • Datasets: The evaluation covers six ultrasound datasets, including lymph-node, breast-lesion, thyroid-nodule, and prostate segmentation tasks.The datasets include LN-INT, LN-EXT, BUSI, DDTI, TN3K, and Prostate.
  • Datasets: LN-EXT is reserved exclusively as an external test set and is excluded from training.
  • Partitions: Datasets are generally split into training, validation, and test sets at 8:1:1, while TN3K uses an 8:2 split for training and validation with its test set unchanged.
  • Preprocessing: Private lymph-node images undergo cropping, template-region removal, and morphological preprocessing before evaluation.
  • Preprocessing: All six datasets are resized to 256×256 pixels for training and testing.
  • Metrics: Performance is measured with Dice, IoU, HD95, and ASD, where overlap metrics are higher-is-better and boundary-distance metrics are lower-is-better.

D. Implementation Details

Switch is implemented and compared across labeling ratios and ultrasound datasets, with results indicating strong accuracy, cross-domain performance, and parameter efficiency, alongside a model-complexity caveat.

  • Experimental design: The experiments compare Switch with nine semi-supervised or supervised baselines across five labeling ratios from 5% to 50%.
  • Caveat: Model-complexity differences may bias comparisons because several competing methods use 3.6M–3.8M parameters versus Switch's 1.8M.
  • Low-data performance: Switch achieves 80.04% Dice on LN-INT, 85.52% on DDTI, and 83.48% on Prostate at 5% labeling.
  • Generalization: Switch maintains cross-domain advantages on external LN-EXT validation, with HD95 improvements exceeding 3.0 across labeling ratios.
  • Parameter efficiency: Switch uses 1.8M parameters and improves over BCP by 9.19% Dice on LN-EXT, 5.96% on BUSI, and 3.45% on DDTI at 5% labeling.
  • High-data performance: At 50% labeling, Switch records the highest IoU across all six datasets and exceeds fully supervised training on LN-INT and LN-EXT.The reported IoUs are 77.30% on LN-INT, 70.94% on LN-EXT, 61.57% on BUSI, 81.12% on DDTI, 66.82% on TN3K, and 80.30% on Prostate.
  • Qualitative analysis: The qualitative results show more complete low-contrast boundaries and fewer false positives in artifact-affected regions.The reported artifact cases include shadowing, speckle noise, and irregular anatomical boundaries.

B. Ablation Studies

Ablations show that MSS supplies the largest single improvement, while complete module integration, balanced patch coverage, mixed augmentation, and tuned loss weights produce the strongest results.

  • Module effectiveness: MSS alone raises Dice from 59.52% to 75.15%, while the complete framework reaches 80.04% Dice.The full model improves Dice by 20.52% over baseline, and HD95 decreases from 55.57 to 26.51.
  • Module effectiveness: Adding FDS with contrastive learning slightly lowers Dice to 74.17%, while consistency regularization and augmentation recover performance to 75.65% and 76.79%.
  • Patch configuration: The best MSS configuration uses p = 2 coarse patches and q = 2 fine patches, reaching 80.04% Dice and 71.98% IoU.It covers 17/32 (53%) of the image area.
  • Patch configuration: Increasing patch counts to p = 3 and q = 4 raises HD95 from 26.51 to 33.74, indicating degraded boundary precision.
  • Augmentation: Weak augmentation improves Dice by 1.92% and IoU by 1.81%, whereas strong augmentation alone reduces Dice by 0.24% and IoU by 1.21%.
  • Augmentation: Combining weak and strong augmentations achieves 80.04% Dice and 71.98% IoU.
  • Loss sensitivity: The contrastive and consistency loss weights perform best at λcont = λconsist = 0.1.A contrastive weight of 0.5 reduces performance to 79.54% Dice, while excessive consistency weighting also causes decline.

C. Extremely Low-labeled Scenario

Under 1% labeled data, Switch shows strong robustness across four evaluated ultrasound datasets, while MSS improves spatial coverage and gradient smoothness over BCP through hierarchical multiscale mixing.

  • Extremely Low-labeled Scenario: 13.52% and 18.56% Dice margins over ABD are reported on LN-INT and LN-EXT, respectively, under the 1% labeled-data setting.BUSI and DDTI are excluded from this scenario because they lack enough labeled samples for the required batch size.
  • Extremely Low-labeled Scenario: 82.44% Dice and 22.64 HD95 are achieved on Prostate, while ABD retains a slight advantage on TN3K.Switch nevertheless surpasses BCP and β-FFT consistently in the reported comparisons.
  • MSS Analysis: 15.41% improved standard deviation and 8.64% lower gradient variance are achieved by MSS (p = 2, q = 10) under a comparable maximum-area constraint.The comparison uses maximum area coverage of 20/32 against the BCP strategy.
  • MSS Analysis: MSS uses coarse 128 × 128 and fine 32 × 32 patches to address large structures, detailed boundaries, and irregular anatomical coverage.This coarse-to-fine design is described as reducing geometric bias and producing smoother transitions that preserve anatomical continuity.

E. From Label Efficiency to Clinical Impact

Switch’s strongest gains occur with 5–10% labeled data, where MSS, FDS, and contrastive learning jointly support low-data segmentation. The framework also reports practical label-efficiency benefits, alongside domain, computational, and multiclass limitations.

  • Label Efficiency: The largest performance gains occur at 5–10% labeling, attributed to complementary effects from MSS coverage, FDS structural preservation, and contrastive learning.FDS is described as reducing reliance on pseudo-label quality when teacher pre-training is limited.
  • Clinical Impact: 80%+ Dice with 5% labeled data is reported as a 95% reduction in annotation requirements while maintaining clinically acceptable segmentation quality.The paper connects this result to deployment across new clinical sites and multiple anatomical regions with limited expert annotation.
  • Limitations: The framework is optimized for ultrasound, with empirically tuned frequency ratios and patch sizes that may limit direct transfer to CT and MRI.Adaptive parameter selection and domain-agnostic frequency analysis are identified as future research directions.
  • Limitations: Dual-stage training and FFT operations for FDS introduce additional computational burden.Selective frequency-band processing and approximated FFT implementations are proposed to reduce overhead while maintaining performance gains.
  • Limitations: Extending the method from binary to multiclass ultrasound segmentation introduces added complexity in pseudo-label generation and consistency enforcement.Class-aware mixing, hierarchical pseudo-labeling, and adaptive loss weighting are proposed for multi-organ scenarios.
  • Clinical Impact: Switch is presented as a label-efficient and reliable ultrasound segmentation solution with cross-center generalization and boundary precision.The conclusion frames the method as a direction for reducing annotation costs in clinical deployment.
Loading 2603.18655v1…