PaperlayerView exact revision on arXiv

Source-linked AI summary

UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation

Huimin Huang, Lanfen Lin, Ruofeng Tong, Hongjie Hu, Qiaowei Zhang, Yutaro Iwamoto, Xianhua Han, Yen-Wei Chen, Jian Wu

arXiv:2004.08790v1eess.IVcs.CVcs.LG

TL;DR

Medical image segmentation needs better use of multi-scale features because UNet++ still does not fully exploit information across scales. UNet 3+ combines full-scale skip connections, deep supervision, hybrid loss, and classification guidance, and reports improved segmentation with fewer parameters on liver and spleen datasets.

  • Problem

    UNet++ does not sufficiently explore full-scale information, limiting multi-scale feature use for accurate medical image segmentation.

    Source: §1 · p. 1

  • Method

    UNet 3+ combines full-scale skip connections and deep supervision with a hybrid loss and classification-guided module.

    Source: §1 · p. 1, §2.2 · p. 3, §2.3 · p. 3

  • Results

    UNet 3+ surpasses previous state-of-the-art approaches on liver and spleen datasets, with scores of 0.9675 and 0.9620 versus second-best results of 0.9341 and 0.9324, respectively.

    Source: §4 · p. 4, §3.3 · p. 4

  • Takeaways & Limitations

    The method provides position-aware and boundary-aware segmentation while using fewer parameters and reducing over-segmentation in non-organ images.

    Source: §2.1 · p. 2, §4 · p. 4, §3.3 · p. 4

Abstract

from arXiv · show

Recently, a growing interest has been seen in deep learning-based semantic segmentation. UNet, which is one of deep learning networks with an encoder-decoder architecture, is widely used in medical image segmentation. Combining multi-scale features is one of important factors for accurate segmentation. UNet++ was developed as a modified Unet by designing an architecture with nested and dense skip connections. However, it does not explore sufficient information from full scales and there is still a large room for improvement. In this paper, we propose a novel UNet 3+, which takes advantage of full-scale skip connections and deep supervisions. The full-scale skip connections incorporate low-level details with high-level semantics from feature maps in different scales; while the deep supervision learns hierarchical representations from the full-scale aggregated feature maps. The proposed method is especially benefiting for organs that appear at varying scales. In addition to accuracy improvements, the proposed UNet 3+ can reduce the network parameters to improve the computation efficiency. We further propose a hybrid loss function and devise a classification-guided module to enhance the organ boundary and reduce the over-segmentation in a non-organ image, yielding more accurate segmentation results. The effectiveness of the proposed method is demonstrated on two datasets. The code is available at: github.com/ZJUGiveLab/UNet-Version

1. INTRODUCTION

UNet 3+ addresses the limited use of multi-scale information in UNet and UNet++ by combining full-scale features with deep supervision. Its classification-guided module and hybrid loss target more accurate boundaries and reduced over-segmentation, with consistent improvements on liver and spleen datasets.

Motivation

  • UNet++ improves UNet's skip connections with nested and dense connections but still does not sufficiently exploit information across full scales.Different-scale features provide complementary spatial-detail and organ-location information.Source: §1 · p. 1

Method

  • UNet 3+ redesigns encoder-decoder and decoder-decoder connections to combine fine-grained details with coarse-grained semantics from full scales.The architecture is designed to support organs appearing at varying scales.Source: §1 · p. 1
  • A classification-guided module predicts whether an input contains an organ to reduce over-segmentation in non-organ images.The classification task provides guidance to the segmentation task.Source: §1 · p. 1
  • Deep supervision learns hierarchical representations from full-scale aggregated feature maps through side outputs supervised by a hybrid loss.The contributions identify pixel-, patch-, and map-level optimization as components of the hybrid loss.Source: §1 · p. 1

Results

  • UNet 3+ yields consistent improvements over multiple baselines on liver and spleen datasets.The reported experiments evaluate the method on both organ datasets.Source: §1 · p. 2

2. METHODS

UNet 3+ redesigns skip connections to aggregate encoder and decoder features across all scales, then applies deep supervision, hybrid loss, and classification guidance for accurate and efficient segmentation.

Full-scale Skip Connections

  • UNet 3+ redesigns skip connections to combine multi-scale features and produce more accurate, position-aware, boundary-enhanced segmentation with fewer parameters.Each decoder layer incorporates smaller- and same-scale encoder features together with larger-scale decoder features.Source: §2 · p. 2, §2.1 · p. 2
  • Full-scale aggregation combines low-level encoder detail with high-level decoder semantics through down-sampling, direct transfer, up-sampling, concatenation, and feature aggregation.The aggregation mechanism uses convolution, batch normalization, and ReLU activation after concatenating five same-resolution feature maps.Source: §2.1 · p. 2
  • Each decoder feature map aggregates information from N scales after channel reduction, yielding 64 × N channels.The architecture uses convolutional transformations to unify channels before feature aggregation.Source: §2.1 · p. 2

Full-scale Deep Supervision

  • Deep supervision produces a side output from every decoder stage and supervises these outputs against the ground truth after convolution, up-sampling, and sigmoid activation.This design learns hierarchical representations from full-scale aggregated feature maps.Source: §2.2 · p. 3

Hybrid Loss Function

  • The hybrid loss combines focal, MS-SSIM, and IoU losses across pixel-, patch-, and map-levels to capture large-scale structures and fine boundaries.MS-SSIM assigns higher weights to fuzzy organ boundaries.Source: §2.2 · p. 3

Classification-guided Module

  • The classification-guided module predicts whether an input contains an organ and multiplies the resulting binary classification output with each segmentation side output.The classification task is optimized with binary cross-entropy to remedy over-segmentation in non-organ images.Source: §2.3 · p. 3, Figure 3 · p. 3

3. EXPERIMENTS AND RESULTS

Experiments on liver and spleen CT datasets compare UNet 3+ with UNet, UNet++, and other state-of-the-art methods. UNet 3+ achieves higher segmentation performance, while its hybrid loss and classification guidance improve boundary quality and suppress over-segmentation.

Datasets

  • The evaluation uses 131 LiTS 2017 liver CT volumes and 49 spleen CT volumes, with separate training and testing sets.The liver dataset has 103 training and 28 testing volumes; the spleen dataset has 40 training and 9 testing volumes.Source: §3.1 · p. 3

Comparison with UNet and UNet++

  • UNet 3+ without deep supervision improves average performance over UNet and UNet++ by 2.7 and 1.6 points across two backbones and two datasets.The comparison uses Vgg-16 and ResNet-101 backbones.Source: Figure 4 · p. 4
  • 0.4 point further improvement is obtained when UNet 3+ uses full-scale deep supervision for organs appearing at varying scales.This result is reported across the liver and spleen datasets.Source: Figure 4 · p. 4

Comparison with the State of the Art

  • The hybrid loss improves segmentation by jointly optimizing pixel-, patch-, and map-level objectives, with MS-SSIM emphasizing fuzzy boundaries.The reported design targets both large-scale and fine structures with clearer boundaries.Source: §3.3 · p. 4
  • UNet 3+ outperforms the second-best result on liver with Dice 0.9675 against 0.9341 and on spleen with Dice 0.9620 against 0.9324.The comparison is reported against five recent state-of-the-art methods.Source: §3.3 · p. 4, Table 2 · p. 4

4. CONCLUSIONS

The paper concludes that UNet 3+ combines full-scale feature use with deep supervision for accurate, efficient segmentation. Classification guidance and hybrid loss further support position-aware and boundary-aware outputs on liver and spleen datasets.

Conclusions

  • UNet 3+ uses full-scale feature maps and deep supervision to achieve accurate segmentation with fewer parameters.The architecture is designed to improve efficiency while retaining multi-scale information.Source: §4 · p. 4
  • Classification guidance and hybrid loss produce more accurate position-aware and boundary-aware segmentation maps.The conclusion attributes these additions to improved localization and boundary quality.Source: §4 · p. 4
  • Experiments on liver and spleen show that UNet 3+ surpasses previous state-of-the-art approaches and produces coherent organ boundaries.The reported outputs highlight organs and maintain coherent boundaries.Source: §4 · p. 4

Related papers

Loading 2004.08790v1…