PaperlayerView exact revision on arXiv

Source-linked AI summary

CE-Net: Context Encoder Network for 2D Medical Image Segmentation

Zaiwang Gu, Jun Cheng, Huazhu Fu, Kang Zhou, Huaying Hao, Yitian Zhao, Tianyang Zhang, Shenghua Gao, Jiang Liu

arXiv:1903.02740v1cs.CV

TL;DR

Medical image segmentation needs detailed spatial information, but U-Net downsampling can lose feature resolution. CE-Net combines a pretrained ResNet encoder with DAC and RMP context extraction and a feature decoder. Across five 2D medical segmentation tasks, the authors report improved performance over U-Net and other state-of-the-art methods.

  • Problem

    U-Net-style consecutive pooling and strided convolution reduce feature resolution, creating a challenge for segmentation tasks that require detailed spatial information.

  • Method

    CE-Net integrates a pretrained ResNet encoder, dense atrous convolution and residual multi-kernel pooling context blocks, and a feature decoder in an end-to-end segmentation framework.

  • Results

    CE-Net outperforms U-Net and other state-of-the-art methods across optic disc, retinal vessel, lung, cell contour, and retinal OCT layer segmentation tasks.

  • Takeaways & Limitations

    The proposed framework is presented as applicable to multiple 2D medical image segmentation tasks and adaptable to new applications through fine-tuning with new training data and manual ground truth.

Abstract

from arXiv · show

Medical image segmentation is an important step in medical image analysis. With the rapid development of convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, etc. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations lead to the loss of some spatial information. In this paper, we propose a context encoder network (referred to as CE-Net) to capture more high-level information and preserve spatial information for 2D medical image segmentation. CE-Net mainly contains three major components: a feature encoder module, a context extractor and a feature decoder module. We use pretrained ResNet block as the fixed feature extractor. The context extractor module is formed by a newly proposed dense atrous convolution (DAC) block and residual multi-kernel pooling (RMP) block. We applied the proposed CE-Net to different 2D medical image segmentation tasks. Comprehensive results show that the proposed method outperforms the original U-Net method and other state-of-the-art methods for optic disc segmentation, vessel detection, lung segmentation, cell contour segmentation and retinal optical coherence tomography layer segmentation.

I. INTRODUCTION

Medical image segmentation supports many clinical image-analysis tasks, but U-Net-style downsampling can discard spatial detail. CE-Net addresses this by combining pretrained encoding, multi-scale context extraction, and decoding, then evaluates the design across five segmentation tasks.

  • Traditional medical segmentation methods use hand-crafted features that may work for one image type but fail on another.
  • Patch-based deep learning methods introduce redundant sliding-window computation and cannot learn global features.
  • U-Net popularized end-to-end encoder-decoder segmentation, but consecutive pooling or strided convolutions reduce feature resolution and spatial detail.
  • CE-Net proposes DAC and RMP blocks within an encoder-decoder structure to capture high-level features and preserve spatial information.
  • The method is applied to optic disc, retinal vessel, lung, cell contour, and retinal OCT layer segmentation, where it outperforms U-Net and other state-of-the-art methods.

II. METHOD

CE-Net is organized as an encoder, context extractor, and decoder that together transform an input image into a segmentation output.

  • CE-Net consists of a feature encoder module, a context extractor module, and a feature decoder module.

A. Feature Encoder Module

The feature encoder replaces the original U-Net encoder with pretrained ResNet-34, while the context extractor uses DAC and RMP blocks to encode multi-scale semantic information.

  • A. Feature Encoder Module: The feature encoder replaces U-Net encoder blocks with a pretrained ResNet-34 while retaining feature-extracting blocks without average pooling or fully connected layers.
  • B. Context Extractor Module: DAC stacks atrous convolutions in four cascade branches with receptive fields of 3, 7, 9, and 19 to extract features at different scales.
  • B. Context Extractor Module: Atrous convolution changes the filter field of view through the atrous rate while preserving the input sampling structure for dense segmentation.
  • B. Context Extractor Module: RMP addresses medical-image object-size variation by combining multiple effective field-of-views through multi-kernel pooling.

C. Feature Decoder Module

The feature decoder restores semantic features and spatial detail, using skip connections and an efficient decoding block to produce the segmentation mask.

  • The decoder restores high-level features from the encoder and context extractor while using skip connections to recover details lost during pooling and strided convolution.

D. Loss Function

CE-Net replaces cross-entropy with Dice coefficient loss because medical-image targets can occupy small regions, and combines it with regularization in the final loss.

  • Loss design: Medical-image targets such as optic discs and retinal vessels often occupy small regions, making cross-entropy suboptimal for these tasks.The paper therefore uses Dice coefficient loss, which measures overlap against available ground truth.
  • Loss design: The final training objective combines Dice coefficient loss with regularization loss.The regularization term, also called weight decay, is used to avoid overfitting.
  • Evaluation scope: CE-Net is evaluated on five medical image segmentation tasks: optic disc, retinal vessel, lung, cell contour, and retinal OCT layer segmentation.This broad evaluation follows the proposed loss and network design.

III. EXPERIMENT

The experiments evaluate CE-Net across multiple medical image segmentation settings using augmentation, a pretrained ResNet-based implementation, and comparisons with established optic-disc methods. On optic-disc datasets, CE-Net generally outperforms the compared approaches, although it is slightly worse for two RIM-ONE-R1 annotations.

  • Experimental setup: Training augmentation includes horizontal, vertical, and diagonal flips, producing eight variants per image, plus scaling, HSV color jittering, and random shifting.Testing uses the same three flips and averages eight predictions; all baselines use this testing strategy.
  • Experimental setup: The implementation uses a pretrained ImageNet ResNet and trains with mini-batch SGD using batch size 8, momentum 0.9, and weight decay 0.0001.The reported setup is implemented in PyTorch.
  • Optic disc segmentation: CE-Net is evaluated on ORIGA, Messidor, and RIM-ONE-R1 for optic disc segmentation, with dataset-specific partitions and image formats.ORIGA and Messidor provide full images, whereas RIM-ONE-R1 provides cropped images.
  • Optic disc results: 0.058 overlapping error is achieved on ORIGA, a 15.9% relative reduction from 0.069 by Faster RCNN or DeepDisc.On Messidor, CE-Net achieves 0.051 versus 0.064 by DeepDisc, a 20.3% relative reduction.
  • Optic disc results: CE-Net generally outperforms the compared optic-disc methods, but performs slightly worse than DeepDisc for the Expert 2 and Expert 3 RIM-ONE-R1 annotations.The paper reports mean and standard deviation comparisons and qualitative examples showing more accurate segmentation results.

C. Retinal Vessel Detection

For retinal vessel detection, CE-Net is evaluated on the 40-image DRIVE dataset using sensitivity, accuracy, and AUC. It outperforms the compared methods on all three reported metrics and improves over its backbone.

  • Dataset and metrics: DRIVE contains 40 images, split into 20 training and 20 testing images, with the first of two expert annotations used as ground truth.Performance is evaluated using sensitivity and accuracy, consistent with prior vessel-detection studies.
  • Dataset and metrics: Retinal vessel detection is assessed with sensitivity, accuracy, and area under the receiver operating characteristic curve (AUC).The paper introduces AUC in addition to sensitivity and accuracy to measure segmentation performance.
  • Results: 0.8309 sensitivity, 0.9545 accuracy, and 0.9779 AUC are achieved by CE-Net, outperforming the other compared methods.The comparison includes state-of-the-art algorithms and classical deep-learning methods.
  • Results: 6.8% sensitivity improvement is obtained over the backbone, increasing from 0.7781 to 0.8309.Accuracy increases from 0.9477 to 0.9545, while AUC increases from 0.9705 to 0.9779.

D. Lung segmentation

CE-Net is evaluated for lung segmentation on 2D CT images from the LUNA dataset, using overlap, sensitivity, and accuracy metrics. It outperforms U-Net and the backbone, with improvements attributed to the DAC and RMP blocks.

  • Dataset and evaluation: The LUNA lung-segmentation dataset contains 534 512×512 2D CT samples, split into 80% training and the remainder testing, with cross-validation.Evaluation uses overlapping error, sensitivity, accuracy, and corresponding standard deviations.
  • Results: CE-Net achieves 0.038 overlapping error, 0.8309 Sensitivity score, and 0.9545 Accuracy score, outperforming U-Net.
  • Results: 13.6% lower overlapping error, from 0.044 to 0.038, is achieved by CE-Net compared with the backbone.Sensitivity increases from 0.967 to 0.980, while accuracy increases from 0.988 to 0.990.

E. Cell contour segmentation

CE-Net is tested on neuronal cell contour segmentation using the EM challenge dataset and is also applied to multi-class retinal OCT layer segmentation. It outperforms U-Net and the backbone in both reported applications.

  • Cell contour segmentation: The EM challenge cell-segmentation dataset contains 30 512×512 training images and 30 testing images, with testing ground truths unavailable.
  • Cell contour segmentation: CE-Net outperforms U-Net and the backbone for cell contour segmentation, although visual examples lack ground-truth masks.
  • Retinal OCT layer segmentation: CE-Net segments 11 retinal layers in the multi-class OCT task, using ten manually demarcated boundaries to divide each scan.
  • Retinal OCT layer segmentation: CE-Net achieves an overall mean absolute error of 1.68, a relative reduction of 31.4% from U-Net’s 2.45 and 22.9% from the backbone’s 2.18.It also performs better than the Topcon built-in and SRR methods in most scenarios.
  • Retinal OCT layer segmentation: Dice loss produces superior CE-Net performance compared with cross-entropy loss in the OCT layer experiments.
  • Qualitative comparison: Fig. 7 visually compares U-Net, the backbone, and CE-Net, with colored lines marking edges between retinal layers.

G. Ablation Study

Ablation studies evaluate pretrained ResNet blocks, the DAC block, the RMP module, and a similar-complexity network. The results support benefits from each proposed component and from the overall CE-Net design.

  • Pretrained ResNet: Pretrained ResNet blocks reduce OD overlapping error by 34.8%, from 0.115 to 0.075, and improve retinal vessel Acc and AUC from 0.939 and 0.960 to 0.943 and 0.971.
  • DAC block: The DAC block with atrous convolution reduces OD overlapping error by 16.4%, from 0.073 to 0.061, compared with regular convolution.It also improves vessel Acc and AUC.
  • DAC block: The DAC block reduces OD overlapping error by 10.3%, from 0.068 to 0.061, compared with a regular Inception-V2 block.
  • Interpretation: The DAC block extracts high-level semantic feature maps while retaining high resolution, which is useful for segmentation.
  • RMP module: The RMP module reduces OD overlapping error by 18.7%, from 0.075 to 0.061, and raises retinal vessel Acc and AUC from 0.943 and 0.971 to 0.952 and 0.974.The module encodes global information and changes feature-map combination.
  • Complexity-controlled comparison: Compared with a similar-complexity Inception-ResNet network, CE-Net reduces OD overlapping error from 0.065 to 0.058 and increases vessel Acc and AUC from 0.951 and 0.974 to 0.955 and 0.978.

IV. CONCLUSIONS

CE-Net is an end-to-end framework for 2D medical image segmentation that combines pretrained ResNet encoding with dense atrous convolution and residual multi-kernel pooling. It improves segmentation across several tasks, while validation remains limited to 2D images and extension to 3D data is future work.

  • CE-Net combines a pretrained ResNet feature encoder with dense atrous convolution and residual multi-kernel pooling in a modified U-Net structure.These components are designed to capture high-level features while preserving spatial information.
  • CE-Net improved segmentation across optic disc, retinal vessel, lung, cell contour, and retinal OCT layer tasks.The method can be adapted to new applications by fine-tuning with training data and manual ground truth.
  • Validation covers 2D medical images, while extension to 3D data remains possible future work.
Loading 1903.02740v1…