PaperlayerView exact revision on arXiv

Source-linked AI summary

RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans

Qiangguo Jin, Zhaopeng Meng, Changming Sun, Leyi Wei, Ran Su

arXiv:1811.01328v1cs.CV

TL;DR

Segmenting heterogeneous, diffusive liver and tumors from CT is challenging, while 3D networks are computationally costly. RA-UNet combines 3D U-Net, attention modules, and residual learning for volumetric liver and tumor extraction, achieving strong liver and tumor segmentation results and competitive extension to brain tumors.

  • Problem

    Accurate automated liver and tumor segmentation from CT is needed for diagnosis, therapy planning, monitoring, and surgery, but heterogeneous, diffusive shapes make the task challenging.

  • Method

    RA-UNet is an end-to-end 3D hybrid architecture combining U-Net, stacked attention modules, and residual blocks to extract volumetric contextual features.

  • Results

    RA-UNet achieved Dice scores up to 0.961 on LiTS and 0.977 on 3DIRCADb for liver segmentation, with a mean Dice score of 0.830 for livers with tumors on 3DIRCADb.

  • Takeaways & Limitations

    The method achieved competitive liver-tumor challenge results and showed extension and generalization ability on brain tumor segmentation datasets.

Abstract

from arXiv · show

Automatic extraction of liver and tumor from CT volumes is a challenging task due to their heterogeneous and diffusive shapes. Recently, 2D and 3D deep convolutional neural networks have become popular in medical image segmentation tasks because of the utilization of large labeled datasets to learn hierarchical features. However, 3D networks have some drawbacks due to their high cost on computational resources. In this paper, we propose a 3D hybrid residual attention-aware segmentation method, named RA-UNet, to precisely extract the liver volume of interests (VOI) and segment tumors from the liver VOI. The proposed network has a basic architecture as a 3D U-Net which extracts contextual information combining low-level feature maps with high-level ones. Attention modules are stacked so that the attention-aware features change adaptively as the network goes "very deep" and this is made possible by residual learning. This is the first work that an attention residual mechanism is used to process medical volumetric images. We evaluated our framework on the public MICCAI 2017 Liver Tumor Segmentation dataset and the 3DIRCADb dataset. The results show that our architecture outperforms other state-of-the-art methods. We also extend our RA-UNet to brain tumor segmentation on the BraTS2018 and BraTS2017 datasets, and the results indicate that RA-UNet achieves good performance on a brain tumor segmentation task as well.

I. INTRODUCTION

Liver and tumor segmentation from CT is clinically important but difficult because of anatomical variability, low contrast, noise, and complex 3D structure. RA-UNet addresses computational and modeling challenges by combining U-Net, residual learning, and attention mechanisms.

  • Accurate liver and tumor extraction from CT supports diagnosis, treatment planning, and surgical intervention.
  • Manual segmentation is subjective, poorly reproducible, time-consuming, and dependent on highly skilled operators.
  • Automated segmentation remains difficult because of spatial and structural variability, low contrast, noise, partial-volume effects, complex 3D features, and nearby-organ similarity.
  • 3D fully convolutional networks face high computational cost and GPU-memory consumption, limiting their depth relative to 2D networks.
  • RA-UNet combines U-Net with residual attention-aware learning to extract 3D volumetric contextual features end-to-end.
  • Stacked residual blocks support deeper networks, while attention modules focus adaptively on relevant image regions and capture changing attention-aware features.

III. METHODOLOGY

The methodology uses a staged hybrid architecture that reduces computation before applying 3D models for precise liver and tumor extraction. RA-UNet combines coarse localization, volumetric liver segmentation, and tumor segmentation.

  • The architecture uses U-Net connections and residual attention mechanisms to process complicated volumes and extract multiscale contextual features.
  • RA-UNet-I uses a 2D residual attention-aware U-Net to mark a coarse liver boundary box and reduce overall computational time.
  • RA-UNet-II is trained on the localized region to obtain a precise liver volume of interest.
  • A second RA-UNet-II receives the liver volume of interest and extracts the tumor region.

B. Datasets and materials

The study evaluates RA-UNet on LiTS and 3DIRCADb CT datasets, using preprocessing to suppress irrelevant tissues and noise before segmentation. The pipeline includes coarse localization followed by 3D liver and tumor extraction.

  • LiTS contains 200 CT scans: 130 training scans and 70 test scans, all with 512×512 in-plane resolution but varying axial-slice counts.
  • 3DIRCADb provides 20 enhanced CT scans with manually segmented tumors and varying axial-slice counts at 512×512 in-plane resolution.
  • Preprocessing uses an initial segmentation to remove disturbing tissues and global HU windowing to retain a cleaner liver region.
  • The HU window is set from -100 to 200, after which zero-mean and min-max normalization are applied.
  • The pipeline first localizes the liver with 2D RA-UNet-I, then extracts liver VOIs and tumors with 3D RA-UNet-II.

D. RA-UNet architecture

RA-UNet combines a U-Net encoder–decoder with residual learning to build a deep architecture for volumetric liver and tumor segmentation. Residual blocks address gradient vanishing while preserving hierarchical contextual features through long-range connections.

  • D. RA-UNet architecture: The architecture combines attention mechanisms with residual learning so attention can focus on relevant object locations in volumetric segmentation.The paper presents this combination as the basis of the RA-UNet architecture.
  • D. RA-UNet architecture: RA-UNet uses a U-Net architecture with symmetric encoder and decoder paths to extract and reconstruct hierarchical contextual features.Long-range connections merge features of different complexity from corresponding encoder and decoder levels.
  • D. RA-UNet architecture: Residual blocks enable the network to become very deep by using identity mappings as skip connections to address gradient vanishing.The residual units propagate features from early to later convolutional layers.
  • D. RA-UNet architecture: Figure 3 compares raw and windowed CT scans alongside HU histograms for LiTS and 3DIRCADb.The figure organizes raw scans in the first row, windowed scans in the second, and HU histograms in the third.
  • D. RA-UNet architecture: Residual blocks contain batch normalization, ReLU activation, convolutional layers, and a convolutional identity mapping.The identity mapping is used to maintain accuracy as the network becomes deeper.

3) Attention residual mechanism:

The attention residual mechanism separates feature processing into a trunk branch and a soft mask branch. This design preserves original features while suppressing noise and emphasizing useful liver-tumor features.

  • 3) Attention residual mechanism:: The attention residual module uses a trunk branch for original features and a soft mask branch for constructing an identity mapping.This mechanism addresses the performance drop associated with naively stacking attention modules.
  • 3) Attention residual mechanism:: The soft mask S(x) takes values in [0,1], so values near zero make the attention output approximate the original feature maps.The soft mask selects identical features and suppresses noisy features from the trunk branch.
  • 3) Attention residual mechanism:: The soft mask branch uses an encoder–decoder structure to enhance useful features and reduce noise from the trunk branch.Its encoder includes max pooling and residual blocks with long-range connections to the decoder.
  • 3) Attention residual mechanism:: The attention residual mechanism preserves original feature information while directing attention toward liver-tumor features.The paper reports that using this mechanism improves RA-UNet performance significantly.

4) Loss function:

RA-UNet learns its weights by minimizing a Dice-coefficient-based loss that directly measures similarity between predicted segmentation and binary ground truth. The section also describes the coarse liver-localization stage and associated architectural components.

  • 4) Loss function:: RA-UNet minimizes a loss function based on the Dice coefficient to learn network weights.The loss is introduced for the segmentation task.
  • 4) Loss function:: The loss directly measures similarity between binary segmentation outputs and binary ground-truth voxel sets.N denotes the number of voxels, while s_i and g_i belong to the segmentation and ground-truth sets.
  • E. Liver localization using RA-UNet-I: RA-UNet-I first segments a coarse liver region to define the liver boundary box and reduce computation for subsequent processing.The coarse region removes redundant information and provides more effective input to RA-UNet-II.
  • E. Liver localization using RA-UNet-I: The RA-UNet-I architecture includes long-range connections, concatenation, convolution, up-sampling, residual blocks, and attention blocks.These operations are represented in the proposed architecture table.

F. Liver segmentation using RA-UNet-II

RA-UNet-II performs 3D liver and tumor segmentation within the localized liver volume. It combines volumetric patches, multiscale U-Net features, residual blocks, and overlap voting to reconstruct final regions.

  • F. Liver segmentation using RA-UNet-II: RA-UNet-II is a 3D model that uses volume information and spatial context to segment the liver within the boundary box.Its 3D U-Net architecture merges low- and high-resolution features.
  • F. Liver segmentation using RA-UNet-II: Residual blocks address gradient vanishing, while large 224×224×32 patches provide richer contextual information for more global segmentation.The architecture uses the residual blocks to support deeper networks without accuracy degradation.
  • F. Liver segmentation using RA-UNet-II: RA-UNet-II generates probability patches sequentially, then interpolates and stacks them to restore the original boundary-box size.This reconstruction follows patch-wise testing over the CT volume.
  • F. Liver segmentation using RA-UNet-II: Overlap voting combines sub-patch probabilities, after which 3D connected-component labeling selects the largest component as the final liver region.The network receives 224×224×32 patches and outputs probability volumes for those patches.
  • F. Liver segmentation using RA-UNet-II: Tumor extraction uses original tumor resolution and samples tumor and surrounding non-tumor patches to address class imbalance and learn tumor features.Interpolation and resizing are avoided because tumors are smaller than the liver.
  • F. Liver segmentation using RA-UNet-II: The final tumor segmentation is produced by voting over merged tumor patches and filtering out voxels outside the liver region.The tumor-testing routine otherwise follows the liver-segmentation procedure.

H. Evaluation metrics

The evaluation uses established segmentation metrics and a staged training pipeline for liver localization and segmentation. RA-UNet-I first constrains the liver region with a boundary box before later processing.

  • Evaluation metrics: The evaluation metrics include Dice score, Jaccard, VOE, RVD, ASSD, and MSD.Dice includes global and per-case variants.
  • Experimental setup: The study uses 5-fold cross-training on LiTS and evaluates generalization on 3DIRCADb using weights trained on LiTS.Liver and tumor models were trained for 50 epochs per fold.
  • Liver localization: RA-UNet-I uses downsampled 256×256 slices and liver-containing slices plus one-third of randomly selected slices without liver.The training data include 32,746 liver-containing slices, with 23,283 for training and 9,463 for validation.
  • Liver localization: The liver boundary box is computed from stacked slices using 3D connected-component labeling and expanded by 10 pixels in each spatial direction.This expansion is intended to ensure the entire liver region is included.

B. Liver segmentation using RA-UNet-II

RA-UNet-II segments the liver within a localized volume while balancing 3D contextual learning against hardware and memory constraints. The method achieves high reported liver Dice scores and supports subsequent tumor extraction.

  • Patch-based processing: RA-UNet-II interpolates the liver-box region to 224×224×M and randomly crops 224×224×32 volumetric patches.The design balances computational cost and efficiency while allowing deeper 3D processing.
  • Liver segmentation: RA-UNet-II learns 3D contextual information across adjacent slices and extracts the liver by selecting the largest connected component.The reported procedure follows 3D connected-component labeling.
  • Results: 0.961 and 0.977 Dice scores were reported on the LiTS test and 3DIRCADb datasets, respectively.The authors state that tumors can then be extracted from the segmented liver regions.
  • Patch-based processing: 128×128×32 patches achieved the best tumor-segmentation performance among the tested patch sizes.The compared sizes were 32×32×32, 64×64×32, and 128×128×32.
  • Qualitative results: The qualitative results show successful segmentation of large liver regions and identification of tiny, difficult-to-detect tumors.Low tumor contrast and very small tumor size still produce false positives and false negatives.

D. Comparison with other methods

RA-UNet reports strong liver and tumor segmentation results across LiTS and 3DIRCADb comparisons. The reported comparisons include higher tumor Dice than cited 2D methods and higher tumor-bearing-liver Dice than one 3DIRCADb baseline.

  • LiTS comparison: 0.961 Dice per case and 0.963 Dice global were reported for liver segmentation on LiTS, alongside Jaccard 0.926 and VOE 0.074.Additional reported metrics were RVD 0.002, ASSD 1.214, and MSD 26.948.
  • LiTS comparison: 0.595 Dice per case and 0.795 Dice global were reported for tumor segmentation on LiTS.The corresponding reported metrics were Jaccard 0.611, VOE 0.389, RVD -0.152, ASSD 1.289, and MSD 6.775.
  • LiTS comparison: RA-UNet outperformed Pandey et al. and Bellver et al. on tumor Dice per case, compared with their reported 0.587 and 0.59 values.The cited comparison methods were 2D segmentation methods, whereas RA-UNet was a full 3D technique.
  • 3DIRCADb comparison: 0.977 Dice per case was reported for liver segmentation on 3DIRCADb, with Jaccard 0.977 and VOE 0.045.The authors state that this performed better than the compared methods.
  • 3DIRCADb comparison: RA-UNet reached mean Dice 0.830 on livers with tumors versus 0.56 for Christ et al. on 3DIRCADb.The authors report this as a comparison for tumor segmentation on that dataset.

E. Extension to brain tumor segmentation

The study extends RA-UNet to multimodal brain-tumor MRI segmentation and tests transfer across BraTS datasets. The reported results indicate state-of-the-art performance and generalization to BraTS2017 without prior training there.

  • Dataset and modalities: BraTS2018 provides four 3D MRI modalities—T1, T1Gd, T2, and FLAIR—for 285 training patients and 66 validation patients.The training set includes 210 high-grade glioma and 75 low-grade glioma patients.
  • Generalization: RA-UNet was tested on BraTS2017 validation data using weights trained on BraTS2018.The authors use this setup to evaluate generalization across the two datasets.
  • Dataset and modalities: RA-UNet concatenates multimodal MRI data and normalizes them to [0,1] after sequentially testing single-modality and multimodality training.The authors report that multimodality input provided full tumor information.
  • Network adaptation: The brain-tumor version adds convolution filters to learn more tumor information and contains 12M parameters.Other hyperparameter settings remain the same as for liver-tumor segmentation.
  • Results: RA-UNet reached state-of-the-art performance on BraTS and outperformed some other leaderboard methods.The reported approach is a full 3D patch-based strategy with generalization to BraTS2017 without prior training.

V. CONCLUSION

RA-UNet is presented as an efficient hybrid architecture for automatic liver and tumor extraction from CT volumes. It combines attention, residual learning, and U-Net features, achieving competitive liver-tumor results and extending to brain tumor segmentation.

  • RA-UNet combines attention-aware features, stacked residual blocks, and U-Net multi-scale feature integration for 3D liver and tumor segmentation.Attention modules adapt features, residual blocks support deeper architectures, and U-Net integrates low- and high-level information.
  • The system uses three stages: 2D liver localization, precise liver segmentation, and 3D tumor-lesion segmentation.
  • RA-UNet is evaluated against methods from the LiTS challenge and the 3DIRCADb dataset.
  • The method achieves competitive results in liver tumor segmentation and demonstrates extension and generalization ability for brain tumor segmentation.Brain tumor experiments use the BraTS2018 and BraTS2017 datasets.
Loading 1811.01328v1…