Source-linked AI summary
UNet++: A Nested U-Net Architecture for Medical Image Segmentation
Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, Jianming Liang
TL;DR
Medical image segmentation requires highly accurate recovery of fine details because even marginal errors can affect clinical credibility and subsequent diagnosis. UNet++ addresses this with nested dense skip pathways and deep supervision, achieving average IoU gains of 3.9 and 3.4 points over U-Net and wide U-Net.
Problem
Medical image segmentation requires accurate recovery of fine target details because marginal errors can undermine clinical credibility and alter subsequent computer-generated diagnoses.
Method
UNet++ uses redesigned nested dense skip pathways and deep supervision to connect encoder and decoder subnetworks while reducing their feature-map semantic gap.
Results
3.9 and 3.4 average IoU points are gained over U-Net and wide U-Net, respectively, across four medical imaging segmentation datasets.
Takeaways & Limitations
UNet++ improves segmentation accuracy across lung nodule, colon polyp, cell nuclei, and liver imaging tasks relative to both evaluated baselines.
Takeaways & Limitations
Results for using UNet++ as a Mask-RCNN backbone were not included because of limited space.
Abstract
from arXiv · showhide
In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in comparison with U-Net and wide U-Net architectures across multiple medical image segmentation tasks: nodule segmentation in the low-dose CT scans of chest, nuclei segmentation in the microscopy images, liver segmentation in abdominal CT scans, and polyp segmentation in colonoscopy videos. Our experiments demonstrate that UNet++ with deep supervision achieves an average IoU gain of 3.9 and 3.4 points over U-Net and wide U-Net, respectively.
1 Introduction
Medical image segmentation requires unusually high accuracy because small errors can undermine clinical credibility and downstream diagnosis. UNet++ addresses this need with nested, dense skip connections that gradually enrich encoder features before decoder fusion.
- Motivation: Encoder-decoder models such as U-Net and FCN use skip connections to combine semantic decoder features with fine-grained encoder features.These connections help recover fine-grained details.
- Motivation: Medical segmentation demands higher accuracy than natural-image segmentation because marginal errors can degrade clinical usability.Precise masks are especially important for lesions and abnormalities.
- Motivation: Small segmentation errors can exclude diagnostically meaningful nodule patterns or alter computer-generated diagnoses, such as Lung-RADS assignments.Erroneous longitudinal growth measurements can produce incorrect screening categories.
- UNet++: UNet++ introduces nested and dense skip connections that gradually enrich high-resolution encoder features before fusing them with semantically rich decoder features.The design aims to capture finer foreground details more effectively.
2 Related Work
UNet++ builds on encoder-decoder segmentation architectures that use skip connections, while distinguishing itself from GridNet through nested dense skip pathways designed to bridge encoder-decoder semantic gaps. It can also serve as a Mask-RCNN backbone by replacing plain skip connections, although corresponding results are not reported in the paper.
- Related architectures: FCN and U-Net introduced skip connections, which help recover full spatial resolution at the network output.FCN sums up-sampled decoder maps with encoder maps, whereas U-Net concatenates them and adds convolutions and nonlinearities after up-sampling.
- Related architectures: GridNet wires feature maps in a grid but lacks up-sampling layers between skip connections, so it does not represent UNet++.GridNet generalizes several classical segmentation architectures through its encoder-decoder grid design.
- UNet++ distinction: UNet++ connects its encoder and decoder through nested dense convolutional blocks that bridge semantic gaps before feature fusion.For example, the gap between (X0,0) and (X1,3) is bridged using a dense convolution block with three convolution layers.
- Mask-RCNN integration: UNet++ can replace plain Mask-RCNN skip connections with nested dense skip pathways, but the paper does not report results for this backbone configuration.The authors refer readers to supplementary material for further details.
3 Proposed Network Architecture: UNet++
UNet++ combines an encoder-decoder backbone with redesigned nested, dense skip pathways and deep supervision. These pathways add convolutional processing and dense connectivity, while deep supervision supports accurate or faster pruned operation.
- Architecture overview: UNet++ uses an encoder sub-network followed by a decoder, connected through redesigned skip pathways and deep supervision.These components distinguish it from the original U-Net.
- Redesigned skip pathways: Redesigned skip pathways replace U-Net’s direct encoder-to-decoder feature transfer with dense convolution blocks whose layer counts depend on pyramid level.For example, the pathway between X0,0 and X1,3 contains three convolution layers, each preceded by concatenation.
- Redesigned skip pathways: Nodes at level j = 0 receive one encoder input, level j = 1 receives two encoder inputs, and level j > 1 receives j + 1 inputs.For j > 1, j inputs come from preceding nodes in the same skip pathway.
- Deep supervision: Deep supervision enables accurate mode, which averages outputs from all segmentation branches, and fast mode, which selects one branch for pruning and speed gain.The selected branch determines the resulting architecture’s complexity.
- Architecture overview: UNet++ differs from U-Net through convolutional skip layers, dense skip connections, and deep supervision.The paper associates these mechanisms with bridging the encoder-decoder semantic gap, improving gradient flow, and enabling model pruning.
4 Experiments
The experiments evaluate UNet++ on four medical imaging datasets against U-Net and wide U-Net, reporting segmentation performance and the effects of deep supervision and model pruning. UNet++ L3 reduces inference time while largely preserving IoU.
- Datasets: Four medical imaging datasets evaluate lesions and organs across different imaging modalities.The datasets cover lung nodules, colon polyps, liver, and cell nuclei segmentation tasks.
- Baseline models: U-Net and wide U-Net provide comparison baselines, with wide U-Net designed to have a similar parameter count to UNet++.This controls whether performance gains arise simply from having more parameters.
- Implementation details: UNet++ uses deep supervision across multiple segmentation branches, which is essential for accurate segmentation of different-scale slices.The passage states that a multi-scale approach using all segmentation branches is essential for accurate segmentation.
- Model pruning: UNet++ L3 achieves an average 32.2% reduction in inference time while degrading IoU by only 0.6 points.More aggressive pruning further reduces inference time but causes significant accuracy degradation.
5 Conclusion
UNet++ combines redesigned skip pathways with deep supervision to improve medical image segmentation, addressing encoder–decoder semantic gaps and supporting multiscale lesion segmentation. Across four medical imaging datasets, it achieved average IoU gains over U-Net and wide U-Net.
- 5 Conclusion: UNet++ combines re-designed skip pathways and deep supervision for more accurate medical image segmentation.The skip pathways target the semantic gap between encoder and decoder feature maps, potentially simplifying optimization.
- 5 Conclusion: Deep supervision enables more accurate segmentation, particularly for lesions appearing at multiple scales such as colonoscopy polyps.
- 5 Conclusion: 4 datasets covered lung nodule, colon polyp, cell nuclei, and liver segmentation.
- 5 Conclusion: 3.9 points average IoU gain over U-Net and 3.4 points over wide U-Net.