Source-linked AI summary
Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks
Abhijit Guha Roy, Nassir Navab, Christian Wachinger
TL;DR
F-CNN segmentation has focused mainly on spatial encoding and network connectivity, while adaptive feature-map recalibration is less explored. The paper introduces channel-wise, spatial, and concurrent squeeze & excitation modules, integrates them into three F-CNNs, and reports consistent performance improvements across two segmentation applications with minimal added complexity.
Problem
Adaptive recalibration of feature maps is less explored in F-CNN image segmentation than spatial encoding or network connectivity improvements.
Method
The paper integrates cSE, sSE, and scSE modules after encoder and decoder blocks in DenseNet, SD-Net, and U-Net.
Results
Consistent improvement is reported across three F-CNN architectures and two segmentation applications, with scSE adding approximately 1.5% to U-Net parameters.
Takeaways & Limitations
Squeeze & excitation recalibration is presented as a generic F-CNN component for improving segmentation performance while minimally increasing model complexity.
Abstract
from arXiv · showhide
Fully convolutional neural networks (F-CNNs) have set the state-of-the-art in image segmentation for a plethora of applications. Architectural innovations within F-CNNs have mainly focused on improving spatial encoding or network connectivity to aid gradient flow. In this paper, we explore an alternate direction of recalibrating the feature maps adaptively, to boost meaningful features, while suppressing weak ones. We draw inspiration from the recently proposed squeeze & excitation (SE) module for channel recalibration of feature maps for image classification. Towards this end, we introduce three variants of SE modules for image segmentation, (i) squeezing spatially and exciting channel-wise (cSE), (ii) squeezing channel-wise and exciting spatially (sSE) and (iii) concurrent spatial and channel squeeze & excitation (scSE). We effectively incorporate these SE modules within three different state-of-the-art F-CNNs (DenseNet, SD-Net, U-Net) and observe consistent improvement of performance across all architectures, while minimally effecting model complexity. Evaluations are performed on two challenging applications: whole brain segmentation on MRI scans (Multi-Atlas Labelling Challenge Dataset) and organ segmentation on whole body contrast enhanced CT scans (Visceral Dataset).
1 Introduction
The paper extends squeeze & excitation from image classification to F-CNN image segmentation by recalibrating feature maps spatially, channel-wise, or concurrently. These modules are integrated into three F-CNNs and evaluated on whole-brain and whole-body segmentation.
- Motivation: SE blocks recalibrate feature maps to emphasize useful channels while suppressing weak ones.The original SE mechanism spatially squeezes feature maps through global average pooling and excites channels using learned reweighting.
- Contribution: The paper introduces sSE for spatial excitation and scSE for concurrent spatial and channel-wise recalibration alongside channel-wise cSE.sSE is motivated by the importance of pixel-wise spatial information in segmentation; scSE combines the two recalibration outputs.
- Evaluation: SE blocks are integrated into DenseNet, SD-Net, and U-Net to test whether they function as generic components for F-CNN segmentation.The models are evaluated on whole-brain MRI segmentation of 27 structures and whole-body CT segmentation of 10 visceral organs.
2 Methods
The method inserts SE blocks after F-CNN encoder and decoder blocks to adaptively recalibrate feature maps. It implements channel-wise, spatial, and concurrent recalibration while adding only a small number of parameters.
- SE integration: SE blocks transform each encoder or decoder output feature map U into a recalibrated map used by subsequent pooling or upsampling layers.They can be placed after every encoder and decoder block in an F-CNN.
- cSE: cSE globally averages each channel, learns channel dependencies, and rescales channels with sigmoid activations.The resulting channel weights are adaptively tuned to emphasize important channels and ignore less important ones.
- sSE: sSE projects all channels at each spatial location into a single map, then uses sigmoid weights to emphasize relevant locations.Each spatial weight represents the relative importance of one location in the feature map.
- scSE: scSE adds the channel-excited and spatially excited feature maps to recalibrate U concurrently along channels and space.A feature-map location receives higher activation when both channel and spatial recalibration assign it high importance.
- Model complexity: 1.5% approximate parameter increase: scSE adds 3.3 × 10^4 parameters to a U-Net with about 2.1 × 10^6 parameters.The paper describes this as a very small fraction of overall network complexity.
3 Experimental Results
Experiments integrate cSE, sSE, and scSE blocks into three F-CNN architectures and evaluate them on brain and whole-body segmentation datasets. Results show consistent benefits, with spatial variants performing especially strongly in DenseNet evaluations.
- Experimental setup: Experiments compare U-Net, SD-Net, and Fully Convolutional DenseNet with cSE, sSE, and scSE blocks against vanilla architectures.Blocks are inserted after every encoder and decoder block.
- Datasets and tasks: Evaluations cover 27 cortical and subcortical brain structures in MALC MRI scans and 10 visceral organs in whole-body contrast-enhanced CT scans.MALC uses 15 training and 15 testing scans and has limited data with severe class imbalance.
- Architecture comparison: DenseNets yield the best performance among the compared network architectures.This comparison is reported alongside the finding that SE blocks add only a small amount of complexity.
- DenseNet results: sSE and scSE consistently outperform the normal DenseNet model across all brain structures, while cSE declines on several small challenging structures.The cited examples include the 3rd/4th ventricles, amygdala, and ventral DC.
- DenseNet results: DenseNet results on whole-body segmentation show a similar pattern across structures when comparing vanilla, cSE, sSE, and scSE models.Figure 3 reports structure-wise Dice performance on the Visceral dataset.
- Qualitative results: scSE improves qualitative segmentation examples by correcting under-segmentation of the left putamen and over-segmentation of the spleen.The examples cover both whole-brain MRI and whole-body ceCT applications.
4 Conclusion
The paper integrates squeeze-and-excitation blocks into F-CNNs for image segmentation and introduces spatial and concurrent variants. Across three architectures and two applications, the blocks consistently improve segmentation with negligible added complexity.
- The paper integrates squeeze-and-excitation blocks into F-CNNs and introduces spatial squeeze-and-excitation for image segmentation.
- SE blocks consistently improve performance across three F-CNN architectures and two segmentation applications.
- The accuracy gains come with a negligible increase in model complexity.