Source-linked AI summary
UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation
Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen, Jian Wu
TL;DR
Medical image segmentation needs better use of multi-scale features because UNet++ still does not fully exploit information across scales. UNet 3+ combines full-scale skip connections, deep supervision, hybrid loss, and classification guidance, and reports improved segmentation with fewer parameters on liver and spleen datasets.
Problem
UNet++ does not sufficiently explore full-scale information, limiting multi-scale feature use for accurate medical image segmentation.
Method
UNet 3+ combines full-scale skip connections and deep supervision with a hybrid loss and classification-guided module.
Results
UNet 3+ surpasses previous state-of-the-art approaches on liver and spleen datasets, with scores of 0.9675 and 0.9620 versus second-best results of 0.9341 and 0.9324, respectively.
Takeaways & Limitations
The method provides position-aware and boundary-aware segmentation while using fewer parameters and reducing over-segmentation in non-organ images.
Abstract
from arXiv · showhide
Recently, a growing interest has been seen in deep learning-based semantic segmentation. UNet, which is one of deep learning networks with an encoder-decoder architecture, is widely used in medical image segmentation. Combining multi-scale features is one of important factors for accurate segmentation. UNet++ was developed as a modified Unet by designing an architecture with nested and dense skip connections. However, it does not explore sufficient information from full scales and there is still a large room for improvement. In this paper, we propose a novel UNet 3+, which takes advantage of full-scale skip connections and deep supervisions. The full-scale skip connections incorporate low-level details with high-level semantics from feature maps in different scales; while the deep supervision learns hierarchical representations from the full-scale aggregated feature maps. The proposed method is especially benefiting for organs that appear at varying scales. In addition to accuracy improvements, the proposed UNet 3+ can reduce the network parameters to improve the computation efficiency. We further propose a hybrid loss function and devise a classification-guided module to enhance the organ boundary and reduce the over-segmentation in a non-organ image, yielding more accurate segmentation results. The effectiveness of the proposed method is demonstrated on two datasets. The code is available at: github.com/ZJUGiveLab/UNet-Version
1. INTRODUCTION
UNet 3+ addresses the limited use of multi-scale information in UNet and UNet++ by combining full-scale features with deep supervision. Its classification-guided module and hybrid loss target more accurate boundaries and reduced over-segmentation, with consistent improvements on liver and spleen datasets.
- Motivation: UNet++ improves UNet's skip connections with nested and dense connections but still does not sufficiently exploit information across full scales.Different-scale features provide complementary spatial-detail and organ-location information.
- Method: UNet 3+ redesigns encoder-decoder and decoder-decoder connections to combine fine-grained details with coarse-grained semantics from full scales.The architecture is designed to support organs appearing at varying scales.
- Method: A classification-guided module predicts whether an input contains an organ to reduce over-segmentation in non-organ images.The classification task provides guidance to the segmentation task.
- Method: Deep supervision learns hierarchical representations from full-scale aggregated feature maps through side outputs supervised by a hybrid loss.The contributions identify pixel-, patch-, and map-level optimization as components of the hybrid loss.
- Results: UNet 3+ yields consistent improvements over multiple baselines on liver and spleen datasets.The reported experiments evaluate the method on both organ datasets.
2. METHODS
UNet 3+ redesigns skip connections to aggregate encoder and decoder features across all scales, then applies deep supervision, hybrid loss, and classification guidance for accurate and efficient segmentation.
- Full-scale Skip Connections: UNet 3+ redesigns skip connections to combine multi-scale features and produce more accurate, position-aware, boundary-enhanced segmentation with fewer parameters.Each decoder layer incorporates smaller- and same-scale encoder features together with larger-scale decoder features.
- Full-scale Skip Connections: Full-scale aggregation combines low-level encoder detail with high-level decoder semantics through down-sampling, direct transfer, up-sampling, concatenation, and feature aggregation.The aggregation mechanism uses convolution, batch normalization, and ReLU activation after concatenating five same-resolution feature maps.
- Full-scale Skip Connections: Each decoder feature map aggregates information from N scales after channel reduction, yielding 64 × N channels.The architecture uses convolutional transformations to unify channels before feature aggregation.
- Full-scale Deep Supervision: Deep supervision produces a side output from every decoder stage and supervises these outputs against the ground truth after convolution, up-sampling, and sigmoid activation.This design learns hierarchical representations from full-scale aggregated feature maps.
- Hybrid Loss Function: The hybrid loss combines focal, MS-SSIM, and IoU losses across pixel-, patch-, and map-levels to capture large-scale structures and fine boundaries.MS-SSIM assigns higher weights to fuzzy organ boundaries.
- Classification-guided Module: The classification-guided module predicts whether an input contains an organ and multiplies the resulting binary classification output with each segmentation side output.The classification task is optimized with binary cross-entropy to remedy over-segmentation in non-organ images.
3. EXPERIMENTS AND RESULTS
Experiments on liver and spleen CT datasets compare UNet 3+ with UNet, UNet++, and other state-of-the-art methods. UNet 3+ achieves higher segmentation performance, while its hybrid loss and classification guidance improve boundary quality and suppress over-segmentation.
- Datasets: The evaluation uses 131 LiTS 2017 liver CT volumes and 49 spleen CT volumes, with separate training and testing sets.The liver dataset has 103 training and 28 testing volumes; the spleen dataset has 40 training and 9 testing volumes.
- Comparison with UNet and UNet++: UNet 3+ without deep supervision improves average performance over UNet and UNet++ by 2.7 and 1.6 points across two backbones and two datasets.The comparison uses Vgg-16 and ResNet-101 backbones.
- Comparison with UNet and UNet++: 0.4 point further improvement is obtained when UNet 3+ uses full-scale deep supervision for organs appearing at varying scales.This result is reported across the liver and spleen datasets.
- Comparison with the State of the Art: The hybrid loss improves segmentation by jointly optimizing pixel-, patch-, and map-level objectives, with MS-SSIM emphasizing fuzzy boundaries.The reported design targets both large-scale and fine structures with clearer boundaries.
- Comparison with the State of the Art: UNet 3+ outperforms the second-best result on liver with Dice 0.9675 against 0.9341 and on spleen with Dice 0.9620 against 0.9324.The comparison is reported against five recent state-of-the-art methods.
4. CONCLUSIONS
The paper concludes that UNet 3+ combines full-scale feature use with deep supervision for accurate, efficient segmentation. Classification guidance and hybrid loss further support position-aware and boundary-aware outputs on liver and spleen datasets.
- Conclusions: UNet 3+ uses full-scale feature maps and deep supervision to achieve accurate segmentation with fewer parameters.The architecture is designed to improve efficiency while retaining multi-scale information.
- Conclusions: Classification guidance and hybrid loss produce more accurate position-aware and boundary-aware segmentation maps.The conclusion attributes these additions to improved localization and boundary quality.
- Conclusions: Experiments on liver and spleen show that UNet 3+ surpasses previous state-of-the-art approaches and produces coherent organ boundaries.The reported outputs highlight organs and maintain coherent boundaries.