Source-linked AI summary
The Importance of Skip Connections in Biomedical Image Segmentation
Michal Drozdzal, Eugene Vorontsov, Gabriel Chartrand, Samuel Kadoury, Chris Pal
TL;DR
Biomedical FCNs traditionally rely on long skip connections, but the paper asks how to train much deeper networks without vanishing gradients. It extends FCNs with residual-style short skips, analyzes gradient flow and convergence, and finds that very deep models achieve near-state-of-the-art EM segmentation without post-processing.
Problem
Standard FCNs use long skips to recover spatial information, while deep networks face vanishing gradients that limit effective training of many layers.
Method
The paper extends residual networks to FCN segmentation by adding an expanding path and short skip connections alongside long resolution-matching skips.
Results
A very deep FCN achieves near-state-of-the-art performance on EM data without post-processing, while long and short skips benefit convergence.
Takeaways & Limitations
Short skips increase convergence speed and enable training very deep FCNs, whereas long skips alone do not alleviate vanishing gradients in deep networks.
Abstract
from arXiv · showhide
In this paper, we study the influence of both long and short skip connections on Fully Convolutional Networks (FCN) for biomedical image segmentation. In standard FCNs, only long skip connections are used to skip features from the contracting path to the expanding path in order to recover spatial information lost during downsampling. We extend FCNs by adding short skip connections, that are similar to the ones introduced in residual networks, in order to build very deep FCNs (of hundreds of layers). A review of the gradient flow confirms that for a very deep FCN it is beneficial to have both long and short skip connections. Finally, we show that a very deep FCN can achieve near-to-state-of-the-art results on the EM dataset without any further post-processing.
1 Introduction
The paper extends FCNs with short skip connections to support very deep biomedical segmentation networks, while analyzing their interaction with standard long skip connections. It reports faster convergence and near-state-of-the-art EM performance without post-processing.
- FCNs recover spatial information by expanding and merging features skipped from multiple contracting-path resolution levels.
- Long skip connections restore full output resolution, whereas short skip connections create gradient-flow shortcuts around nonlinearities.
- The paper extends FCNs with short skip connections to build very deep networks and analyzes both skip-connection types on EM ISBI 2012 data.
- Short skip connections speed learning convergence, while long and short skip connections benefit convergence in very deep networks.
- A very deep network with relatively few parameters achieves performance comparable to the state of the art on EM data without post-processing.
2 Residual network for semantic image segmentation
The proposed segmentation architecture extends residual networks with an expanding path and combines long feature skips with residual-style short skips. It uses configurable block types and evaluates binary cross-entropy and Dice losses.
- The architecture adds an expanding upsampling path to a residual network while spatially reducing and then expanding feature resolution.
- Equal-resolution features bypass the contracting path through long skips and are summed with expanding-path features.
- Bottleneck, basic, and simple blocks each contain convolution and activation functions, with optional normalization, downsampling, upsampling, and dropout.
- The experiments compare binary cross-entropy and Dice loss functions for sigmoid outputs and binary labels.
- Table 1 specifies the experimental model architecture and reports how many times each block is repeated.
3 Experiments
Experiments on EM segmentation evaluate dataset performance and the effects of long and short skip connections. The analyses compare training behavior, parameter updates, and published results under different network configurations.
- 3.1 Segmenting EM data: The experiments use EM data with 30 training images, reserving 5 images for validation and augmenting full-resolution 512 × 512 inputs.The test set contains another 30 unlabeled images.
- 3.1 Segmenting EM data: Qualitative test-set predictions compare binary cross-entropy, Dice loss, and Dice loss with 0.2 dropout at test time.The figure presents the original image alongside predictions from these three settings.
- 3.1 Segmenting EM data: Dice-loss models with implicit test-time model averaging through dropout improve border continuity and validation and test metrics over binary cross-entropy models.The comparison concerns output samples drawn with dropout at test time.
- 3.1 Segmenting EM data: The evaluation uses maximal foreground-restricted Rand and information-theoretic scores after thinning, denoted Vrand and Vinfo.These are the two metrics used for the EM dataset.
- 3.1 Segmenting EM data: The results are comparable to published state-of-the-art EM entries without post-processing, matching UNet while using fewer parameters and no sophisticated class weighting.UNet predictions were averaged over seven input rotations, whereas the reported method did not use post-processing.
- 3.2 On the importance of skip connections: Parameter updates are well distributed with short skip connections, whereas removing them leaves deep central layers with few updates in deep models.Long skip connections preserve update paths mainly for shallow parts; batch normalization increases the maximal updatable depth, while removing it reduces stability and requires a lower learning rate.
- 3.2 On the importance of skip connections: Across tested model variants, the bulk of updates initially occurs near the classification layer, consistent with earlier findings about classifier training.The cited observation links this behavior to randomly initialized weights retaining substantial performance after classifier training.
4 Conclusions
The paper studies how long and short skip connections affect fully convolutional networks for biomedical image segmentation. It finds that short skip connections improve convergence and enable very deep networks, while the resulting model reaches near-state-of-the-art EM performance without post-processing.
- Very deep networks achieve near-state-of-the-art results on the EM dataset without post-processing.
- Long skip connections provide gradient-flow shortcuts in shallow layers but do not alleviate vanishing gradients in deep networks.
- Short skip connections increase convergence speed and allow fully convolutional networks to be trained at very great depth.