Source-linked AI summary
U-Net: Convolutional Networks for Biomedical Image Segmentation
Olaf Ronneberger, Philipp Fischer, Thomas Brox
TL;DR
Biomedical segmentation requires pixel-level localization, but annotated images are scarce and prior sliding-window methods are slow and trade context against localization accuracy. U-Net combines a contracting path with a symmetric expanding path, high-resolution feature connections, strong data augmentation, and weighted loss for separating touching objects. On PhC-U373, U-Net achieves an average IOU of 92%, significantly better than the second-best algorithm’s 83%.
Problem
Biomedical segmentation requires pixel-level localization, but annotated images are scarce and prior sliding-window methods are slow and trade context against localization accuracy.
Method
U-Net combines a contracting path with a symmetric expanding path, high-resolution feature connections, strong data augmentation, and weighted loss for separating touching objects.
Results
On PhC-U373, U-Net achieves an average IOU of 92%, significantly better than the second-best algorithm’s 83%.
Takeaways & Limitations
Data augmentation enables U-Net to use very few annotated images while delivering effective biomedical segmentation across substantially different imaging applications.
Takeaways & Limitations
Comparisons are complicated because algorithms on this dataset use highly dataset-specific post-processing methods.
Abstract
from arXiv · showhide
There is large consent that successful training of deep networks requires many thousand annotated training samples. In this paper, we present a network and training strategy that relies on the strong use of data augmentation to use the available annotated samples more efficiently. The architecture consists of a contracting path to capture context and a symmetric expanding path that enables precise localization. We show that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks. Using the same network trained on transmitted light microscopy images (phase contrast and DIC) we won the ISBI cell tracking challenge 2015 in these categories by a large margin. Moreover, the network is fast. Segmentation of a 512x512 image takes less than a second on a recent GPU. The full implementation (based on Caffe) and the trained networks are available at http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net .
1 Introduction
U-Net addresses biomedical pixelwise segmentation with limited annotated data by combining a symmetric contracting–expanding architecture with strong augmentation and specialized loss weighting. It outperformed prior methods on neuronal EM segmentation and won the two most challenging 2D transmitted-light cell-tracking datasets by a large margin.
- Biomedical segmentation requires pixel-level localization, but thousands of annotated training images are generally unavailable, motivating methods that use limited data efficiently.Classification networks typically produce one image-level label, whereas biomedical tasks require assigning class labels to individual pixels.
- Sliding-window segmentation localizes effectively and expands patch-level training data, but overlapping patches make it slow and create a context–localization trade-off.Larger patches provide more context but reduce localization accuracy through additional pooling, whereas smaller patches expose the network to less context.
- The proposed architecture extends fully convolutional networks [9] with a contracting path, symmetric expansive path, and high-resolution feature connections for more precise localization.Upsampling replaces pooling in the expanding path, while successive convolutions assemble precise outputs from propagated context and high-resolution features.
- Excessive elastic-deformation augmentation teaches invariance to realistic tissue deformations without requiring those transformations in the annotated image corpus.The approach is especially relevant because deformation is a common source of variation in biomedical images and can be simulated efficiently.
- A weighted loss assigns large weights to background labels separating touching cells, improving separation of adjacent objects from the same class.Touching-object separation is identified as a major challenge in cell segmentation.
2 Network Architecture
The U-Net architecture combines a contracting path for convolutional feature extraction with an expansive path for segmentation output, using repeated convolutions, ReLUs, and max pooling in the contracting path. Seamless output tiling requires input tiles whose dimensions remain compatible with the 2x2 pooling operations.
- 2 Network Architecture: The architecture consists of a contracting path and an expansive path, with the contracting path repeatedly applying two 3x3 convolutions, ReLUs, and 2x2 max pooling for downsampling.The number of feature channels doubles at each downsampling step.
- 2 Network Architecture: Input tile dimensions must ensure that every 2x2 max-pooling operation receives layers with even x- and y-sizes, enabling seamless tiling of the output segmentation map.
3 Training
Training uses pixel-wise soft-max cross-entropy with class- and border-aware weighting, careful initialization, single-image high-momentum optimization, and strong augmentation for robustness with few annotated images.
- 3 Training: Training uses large input tiles, a batch size of one, and momentum 0.99 within Caffe’s stochastic-gradient-descent implementation.Unpadded convolutions make the output smaller than the input by a constant border width.
- 3 Training: Weights are initialized to approximately preserve unit variance across feature maps, using a variance determined by each neuron’s number of incoming nodes [5].For a 3x3 convolution with 64 preceding feature channels, the incoming-node count is N = 576.
- 3 Training: Data augmentation targets shift and rotation invariance plus robustness to deformations and gray-value changes, with random elastic deformations especially important for few annotated images.Smooth deformations use random displacement vectors on a coarse 3 by 3 grid sampled from a Gaussian distribution with 10 pixels standard deviation.
4 Experiments
The u-net is evaluated on neuronal-structure segmentation in electron microscopy and cell segmentation in light microscopy, achieving state-of-the-art neuronal segmentation results and a 92% average IOU on the PhC-U373 cell dataset. Its neuronal result exceeds the prior sliding-window convolutional network without additional pre- or postprocessing.
- Experimental scope: The experiments demonstrate the u-net on three different segmentation tasks, including neuronal structures in electron microscopy and cells in light microscopy.The neuronal dataset comprises 30 images of size 512x512 pixels.
- Neuronal structures: 0.0003529 warping error and 0.0382 rand-error establish a new best neuronal-segmentation result on the EM challenge [14].The u-net averages predictions over seven rotated input versions and uses no further pre- or postprocessing.
- Neuronal structures: The u-net significantly outperforms Ciresan et al.’s sliding-window network, which reported warping error 0.000420 and rand error 0.0504.Other algorithms use highly dataset-specific post-processing applied to Ciresan et al.’s probability map.
- Cell segmentation: 92% average IOU is achieved on the PhC-U373 cell-segmentation dataset from the ISBI cell tracking challenge 2014 and 2015.The dataset contains 35 partially annotated training images of Glioblastoma-astrocytoma U373 cells recorded by phase-contrast microscopy.
5 Conclusion
The u-net architecture performs very well across diverse biomedical segmentation applications while requiring few annotated images and only 10 hours of training on an NVidia Titan GPU. The authors provide its Caffe[6]-based implementation and trained networks, enabling application to further tasks.
- u-net achieves very good performance across very different biomedical segmentation applications.
- Data augmentation with elastic deformations enables training from very few annotated images, with a training time of only 10 hours on an NVidia Titan GPU (6 GB).
- The authors provide the full Caffe[6]-based implementation and trained networks, and expect u-net to apply easily to many more tasks.