PaperlayerView exact revision on arXiv

Source-linked AI summary

'Squeeze & Excite' Guided Few-Shot Segmentation of Volumetric Images

Abhijit Guha Roy, Shayan Siddiqui, Sebastian Pölsterl, Nassir Navab, Christian Wachinger

arXiv:1902.01314v2cs.CV

TL;DR

Few-shot volumetric medical-image segmentation must learn new organs from very limited annotations despite the lack of pre-trained medical models and the need to coordinate slices across volumes. The paper introduces a two-arm architecture with channel squeeze-and-spatial excitation interaction blocks and a support-query slice-pairing strategy. On contrast-enhanced CT scans from the Visceral dataset, it reports outperforming the compared baselines and existing approaches by a large margin, while noting practical limits on initialization and precision.

  • Problem

    Few-shot segmentation must produce dense predictions for new classes from few annotations, while medical scans lack pre-trained models and require volumetric support-query slice pairing.

  • Method

    The method combines conditioner and segmenter arms through channel squeeze and spatial excitation blocks, training from scratch and pairing a few annotated support slices with query-volume slices.

  • Results

    The framework outperformed feature-fusion, classifier-regression, combined, and other compared approaches by a large margin on contrast-enhanced CT scans from the Visceral dataset.

  • Takeaways & Limitations

    Few-shot volumetric segmentation can be evaluated without pre-trained models by combining strong cross-arm interaction with strategic pairing of sparse support annotations.

  • Takeaways & Limitations

    A new query volume requires manual indication of the target organ’s start and end slices, and extremely limited supervision prevents very precise segmentation.

Abstract

from arXiv · show

Deep neural networks enable highly accurate image segmentation, but require large amounts of manually annotated data for supervised training. Few-shot learning aims to address this shortcoming by learning a new class from a few annotated support examples. We introduce, a novel few-shot framework, for the segmentation of volumetric medical images with only a few annotated slices. Compared to other related works in computer vision, the major challenges are the absence of pre-trained networks and the volumetric nature of medical scans. We address these challenges by proposing a new architecture for few-shot segmentation that incorporates 'squeeze & excite' blocks. Our two-armed architecture consists of a conditioner arm, which processes the annotated support input and generates a task-specific representation. This representation is passed on to the segmenter arm that uses this information to segment the new query image. To facilitate efficient interaction between the conditioner and the segmenter arm, we propose to use 'channel squeeze & spatial excitation' blocks - a light-weight computational module - that enables heavy interaction between both the arms with negligible increase in model complexity. This contribution allows us to perform image segmentation without relying on a pre-trained model, which generally is unavailable for medical scans. Furthermore, we propose an efficient strategy for volumetric segmentation by optimally pairing a few slices of the support volume to all the slices of the query volume. We perform experiments for organ segmentation on whole-body contrast-enhanced CT scans from the Visceral Dataset. Our proposed model outperforms multiple baselines and existing approaches with respect to the segmentation accuracy by a significant margin. The source code is available at https://github.com/abhi4ssj/few-shot-segmentation.

1. Introduction

The paper targets few-shot segmentation of volumetric medical scans, where annotations are scarce, pre-trained models are unavailable, and support slices must guide query-volume segmentation. It proposes a two-arm architecture with strong squeeze-and-excitation-based interaction and strategic volumetric slice pairing.

  • Motivation: Fine-tuning complex pre-trained segmentation networks can require hundreds of annotated images and may overfit when only a few new-class examples exist.The framework instead addresses training without relying on pre-trained models.
  • Motivation: Few-shot learning seeks to generalize to unknown semantic classes from only a few labeled examples, a setting especially relevant to scarcely annotated medical imaging.Dense pixel-level prediction remains challenging in this low-data regime.
  • Framework: The framework uses a conditioner arm to fuse a support slice and binary annotation into task-specific feature maps, which guide a segmenter arm on a query image.The support set supplies the new class information, while the segmenter produces its query-image mask.
  • Architecture: Channel squeeze and spatial excitation blocks create strong interactions at multiple locations between conditioner and segmenter arms, supporting training from scratch with light-weight modules.The blocks communicate conditioner-derived spatial maps to segmenter features and are used between encoder, bottleneck, and decoder blocks.
  • Volumetric strategy: The volumetric strategy pairs a few annotated support slices with all query-volume slices to maintain inter-slice segmentation consistency.This addresses the burden of annotating organs throughout 3D scans.

2. Prior Work

Prior few-shot segmentation methods largely adapt or condition models for 2D computer-vision images and depend on pre-trained networks. This paper distinguishes its setting by using only a few annotated slices of volumetric medical scans without an established support-query slice-pairing strategy.

  • Few-shot learning: Few-shot learning methods include adapting a base classifier, predicting classifiers near a base classifier with two-branch networks, and metric learning.These groups differ in how they use prior knowledge or similarity to handle new classes.
  • Deep few-shot segmentation: Early deep few-shot segmentation work used fine-tuning, classifier regression, multi-class extensions, or sparse landmark supervision on 2D image benchmarks.The cited approaches include video segmentation and PASCAL VOC evaluations.
  • Medical imaging gap: Existing few-shot segmentation approaches depend on pre-trained models, whereas medical imaging generally lacks such models and therefore requires training both arms from scratch.This difference makes direct transfer from computer-vision settings difficult.
  • Medical imaging gap: Medical volumetric scans also lack an established strategy for selecting and pairing support slices with query volumes.The paper uses annotations from only a few slices, unlike work using a single fully annotated volume.

3. Method

The method uses a two-arm few-shot architecture in which a conditioner extracts task-specific information from annotated support data and a segmenter applies it to query images. sSE interaction blocks connect the arms at multiple locations, while paired support slices enable volumetric inference.

  • 3.2. Architecture: The conditioner and segmenter use symmetric encoder-decoder layouts to support interaction between matching feature resolutions.Each arm has four encoder blocks, four decoder blocks, and a bottleneck.
  • 3.2.2. Interaction Block using ‘Squeeze & Excitation’ modules: Unlike approaches using a single final interaction, this method places sSE blocks between encoder, bottleneck, and decoder blocks because training occurs without a pre-trained network.The design aims to strengthen interaction while adding only marginal complexity and computation.
  • 3.2.1. Conditioner Arm: The conditioner processes support images and annotations to generate task-specific feature maps for segmenting a new query class.The support input stacks the image and binary annotation as two channels.
  • 3.2.2. Interaction Block using ‘Squeeze & Excitation’ modules: sSE blocks perform channel squeeze on conditioner features and spatial excitation on segmenter features to convey support information.The conditioner-derived spatial gate rescales segmenter activations through sigmoid values in [0, 1].
  • 3.4. Volumetric Segmentation Strategy: For volumetric inference, the query and support volumes are divided into groups, pairing each annotated support center slice with all query slices in the corresponding group.The strategy addresses the challenge of pairing sparse annotated support slices with every query-volume slice.

4. Dataset and Experimental Setup

The experiments evaluate few-shot organ segmentation on resampled contrast-enhanced CT scans from the Visceral Dataset. The setup uses separate silver-corpus training and gold-corpus validation and test data, with four folds over six organs.

  • 4.1. Dataset: The Visceral Dataset contains 65 silver-corpus scans and 20 gold-corpus scans, all resampled to a voxel resolution of 2mm3.The experiments target organ segmentation in contrast-enhanced CT scans.
  • 4.2. Problem Formulation: Silver-corpus scans provide training data, while the gold corpus supplies one support volume, 14 validation volumes, and 5 test volumes.The support volume is identified as Volume ID 10000132 1 CTce ThAb.
  • 4.2. Problem Formulation: The experimental semantic classes comprise six organs, including left kidney, right psoas muscle, and left psoas muscle.The supplied list abbreviates left and right as L. and R. and psoas muscle as P.M.
  • 4.2. Problem Formulation: Four folds are used so that each organ is treated as an unknown semantic class once per fold.Training and testing labels are reported separately for each fold.
  • 4.3. Training Details: Training hyperparameters were selected by manually trying different combinations rather than using a hyperparameter optimization framework.The authors note that automated optimization could produce better results but would be time-consuming.

5. Experimental Results and Discussion

Experiments identify spatial excitation as the stronger interaction mechanism and place it throughout the network. Skip connections instead cause support-mask copying and reduce segmentation performance.

  • Interaction module and placement: sSE modules achieved more than 0.4 Dice, whereas cSE modules achieved less than 0.1 Dice in the evaluated configurations.The authors associate cSE underperformance with class information being lost through global average pooling.
  • Interaction module and placement: BL-7, using sSE between all encoder, bottleneck, and decoder blocks, achieved the highest mean Dice score of 0.567.BL-7 outperformed the other baselines across all folds by 0.1 to 0.8 Dice points.
  • Skip connections: Skip connections caused the network to copy the support binary mask into query predictions, despite differences in support and query shapes.This copy over effect appeared across all folds in both training and testing.
  • Skip connections: Adding skip connections reduced Dice performance by 3%, while adding them only to the conditioner reduced it by 6%.Adding skip connections only to the segmenter made training unstable.
  • Model design: The conditioner generates task representations from support examples, which interaction blocks pass to the segmenter for query segmentation.The segmenter arm used 64 convolution kernels while the conditioner-arm complexity was varied across experiments.

5.4. Effect of the number of Support Slice Budget

The experiments examine support-slice budgets and comparisons with adapted baselines and supervised training. Performance saturates with a limited number of support slices, while fully supervised training remains stronger when extensive annotations are available.

  • Support-slice budget: For liver segmentation, Dice increased from 0.678 with k = 1 to 0.701 with k = 20, saturating near 0.70 with 12 slices.Kidney performance saturated at 0.46 and psoas-muscle performance at 0.50 for k values of at least 10.
  • Support-slice budget: The overall mean Dice score saturated at 0.56 with k = 10 support slices.The authors therefore concluded that k = 10 was the maximum support-slice requirement for their application.
  • Comparison with existing approaches: The proposed approach achieved 21% higher Dice points and 10 mm lower average surface distance than the adapted feature-fusion comparison.The comparison modified existing computer-vision methods because they were not directly applicable to this medical-image setting.
  • Comparison with upper bound model: The fully supervised upper-bound model had 20-40% higher Dice points and 1-7 mm lower average surface distance than the few-shot framework.The upper bound used all slices from 65 fully annotated scans, whereas the few-shot method used 10 annotated slices from one volume.

5.7. Qualitative Results

Qualitative evaluations show that the framework can segment organs despite substantial support-query differences in shape, size, and contrast. The discussion also examines support-volume sensitivity and spatial alignment in sSE interactions.

  • Qualitative results: The qualitative results cover liver, spleen, kidney, and psoas-muscle folds using support images, query annotations, and query predictions.The spleen example shows slight undersegmentation despite different support-query shapes and contrast.
  • Qualitative results: The model produced acceptable liver segmentation despite differences in support-query shape and size using only a single support slice.The kidney example remained difficult because the organ appeared as a small dot in the support slice.
  • Support-volume sensitivity: Support-volume selection changed Dice performance by 4% overall across five selected support volumes.The authors conclude that the support volume should represent the whole query set.
  • Spatial SE interaction: Bottleneck-only sSE interaction achieved a decent score despite low spatial resolution, indicating a spatially invariant fusion setting can still segment adequately.The bottleneck feature map had spatial size 16 × 32 in the reported configuration.
  • Spatial SE interaction: The sSE gating function can leave segmenter feature maps unchanged while calibrating the bottleneck interaction.This behavior provides an interpretation of how bottleneck-only and multi-location interaction configurations can relate.

6. Conclusion

The paper introduced a volumetric few-shot segmentation framework that addresses absent pre-trained models and scan volumetricity through channel squeeze and spatial excitation blocks and strategic slice pairing. It outperformed previous approaches by a large margin, while remaining limited in precision and requiring manual organ-boundary indications for query volumes.

  • Conclusion: The framework targets volumetric medical scans without relying on pre-trained models, addressing both missing initialization resources and scan volumetricity.It uses channel squeeze and spatial excitation blocks to train the framework from scratch.
  • Conclusion: Strategic pairing of 2D support slices with query-volume slices provides the proposed volumetric segmentation strategy.The paper also examined how support-volume choice and the number of budget slices affect performance.
  • Conclusion: The proposed framework outperformed feature-fusion, classifier-regression, and combined existing approaches by a large margin.These comparisons used contrast-enhanced CT scans from the Visceral dataset.
  • Limitations: Query-volume segmentation requires indicating the target organ's start and end slices, which may require manual interaction.This is identified as the first stated limitation.
  • Limitations: Few-shot segmentation cannot achieve very precise segmentation under extremely limited supervision, so highly accurate applications are directed toward traditional supervised learning with more annotations.The paper presents this as a limitation of the proposed approach.
  • Conclusion: The approach is generic and can be extended to other few-shot segmentation applications.Its independence from pre-trained models is presented as useful beyond computer vision applications.

List of IDs in the Visceral Dataset

The dataset section reports the IDs used for the support set, validation query set, and testing query set.

  • List of IDs in the Visceral Dataset: The Visceral Dataset section lists IDs for the support set.The supplied passage introduces the dataset split-ID listing.
  • List of IDs in the Visceral Dataset: The Visceral Dataset section lists IDs for the validation query set.The supplied passage names the validation query split among the reported IDs.
  • List of IDs in the Visceral Dataset: The Visceral Dataset section lists IDs for the testing query set.The supplied passage names the testing query split among the reported IDs.

Support Set

The support-set entry is identified by the dataset record 10000132 1 CTce ThAb.

  • Support Set: The support-set listing contains record ID 10000132.The supplied entry begins with this identifier.
  • Support Set: The support-set entry includes the label CTce ThAb.This label appears in the supplied dataset record.
Loading 1902.01314v2…