Source-linked AI summary
Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation
Michal Drozdzal, Gabriel Chartrand, Eugene Vorontsov, Lisa Di Jorio, An Tang, Adriana Romero, Yoshua Bengio, Chris Pal, Samuel Kadoury
TL;DR
Medical image segmentation performance depends on preprocessing, and FC-ResNets are particularly sensitive to data preparation. The paper uses a trainable low-capacity FCN to normalize raw images before iterative FC-ResNet refinement, achieving strong results across EM, CT liver-lesion, and MRI prostate tasks. The authors identify broader preprocessor and 3D-FCN exploration as a future direction.
Problem
Medical segmentation pipelines commonly require modality-specific preprocessing, while FC-ResNet performance depends strongly on proper data preparation.
Method
A low-capacity FCN learns to normalize raw medical images before a more than 100-layer FC-ResNet iteratively refines them into segmentation predictions.
Results
The pipeline reaches state-of-the-art performance on the EM benchmark, improves CT liver-lesion segmentation, and achieves competitive results on a 3D MRI prostate task.
Takeaways & Limitations
The approach provides a multimodality, end-to-end segmentation pipeline that removes hand-crafted preprocessing and operates on different imaging tasks.
Takeaways & Limitations
Future work should explore alternative preprocessing architectures, including expanding the approach to 3D FCN models.
Abstract
from arXiv · showhide
In this paper, we introduce a simple, yet powerful pipeline for medical image segmentation that combines Fully Convolutional Networks (FCNs) with Fully Convolutional Residual Networks (FC-ResNets). We propose and examine a design that takes particular advantage of recent advances in the understanding of both Convolutional Neural Networks as well as ResNets. Our approach focuses upon the importance of a trainable pre-processing when using FC-ResNets and we show that a low-capacity FCN model can serve as a pre-processor to normalize medical input data. In our image segmentation pipeline, we use FCNs to obtain normalized images, which are then iteratively refined by means of a FC-ResNet to generate a segmentation prediction. As in other fully convolutional approaches, our pipeline can be used off-the-shelf on different image modalities. We show that using this pipeline, we exhibit state-of-the-art performance on the challenging Electron Microscopy benchmark, when compared to other 2D methods. We improve segmentation results on CT images of liver lesions, when contrasting with standard FCN methods. Moreover, when applying our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we reach results that are competitive even when compared to 3D methods. The obtained results illustrate the strong potential and versatility of the pipeline by achieving highly accurate results on multi-modality images from different anatomical regions and organs.
I. INTRODUCTION
The paper proposes an end-to-end medical image segmentation pipeline that learns input normalization with an FCN before iterative refinement by an FC-ResNet. It reports strong results across EM, CT liver-lesion, and MRI prostate segmentation tasks.
- Motivation: Medical segmentation performance depends strongly on preprocessing and post-processing, which traditionally vary by imaging modality.Common preprocessing includes standardization, histogram equalization, clipping, and range normalization.
- Motivation: FC-ResNets add shortcut paths that improve segmentation accuracy and optimization, but their performance is highly dependent on proper data preparation.The paper frames FC-ResNets in terms of iterative estimation and notes their greater preprocessing sensitivity than FCNs.
- Proposed pipeline: The proposed pipeline uses an FCN to produce pre-normalized images that a FC-ResNet iteratively refines into a segmentation prediction.The design replaces hand-crafted preprocessing with a trainable component optimized through back-propagation.
- Results: The pipeline reaches state-of-the-art performance among 2D methods on the EM ISBI benchmark and outperforms standard FCN and FC-ResNet methods on CT liver lesions.These claims are reported as contributions for the EM and in-house CT datasets.
- Results: Applying the 2D pipeline off-the-shelf to a challenging 3D MRI prostate task produces results competitive with 3D methods.The authors separately report that the FCN-based preprocessor normalizes data to values adequate for FC-ResNet.
II. BACKGROUND
Medical segmentation systems commonly use fully convolutional architectures, modality-specific preprocessing, and output refinement. The paper situates its approach among FCNs, CNNs, and residual-network designs used across EM, MRI, and CT.
- Deep segmentation architectures: FCNs process whole images end-to-end and recover spatial information through expanding paths that merge features from contracting-path resolutions.Skip connections support full-resolution semantic segmentation outputs.
- Preprocessing and refinement: Medical imaging pipelines often preprocess inputs to address variability and post-process predictions with morphological operations, CRFs, or level sets.The specific choices differ across imaging modalities and organs.
- Applications: EM segmentation methods use patch-based CNNs, FCNs, UNets, and FC-ResNets, with intensity augmentation or preprocessing and several prediction-refinement techniques.Reported refinement methods include multicut, watershed, and median filtering.
- Applications: MRI and CT studies apply CNN, FCN8, and UNet variants to segment tumors, lesions, organs, and other pathological structures.The reviewed CT pipelines commonly use standardization, clipping, morphological operations, CRFs, or level sets.
III. METHOD
The method combines an FCN-based image preprocessor with a FC-ResNet so raw medical images can be transformed into inputs suitable for iterative segmentation refinement.
- Pipeline overview: The pipeline combines an FCN model with a FC-ResNet model, using the FCN to preprocess images into a format that FC-ResNet can iteratively refine.The section also introduces separate architecture descriptions and a loss function for training the combined system.
A. Fully Convolutional pre-processor
The pipeline uses a low-capacity FCN to process raw images before a fully convolutional ResNet iteratively refines them into segmentation predictions.
- FCN pre-processor: The FCN pre-processor accepts raw single-channel N × N images and outputs processed N × N × 1 feature maps without hand-crafted preprocessing.Its UNet-like architecture alternates convolutions and pooling in the contracting path with convolutions and repeat operations in the expanding path.
- Residual blocks: The residual formulation adds identity mappings to transformed features, enabling feature reuse and direct gradient flow to earlier layers.The transformation H consists of repeated batch normalization, ReLU, and convolution blocks.
- FC-ResNet: The FC-ResNet extends residual networks with an expanding path that recovers spatial information through skipped equal-resolution features.Its blocks support unchanged resolution, downsampling, or upsampling and include simple and bottleneck variants.
- Model scale: The complete model combines a 23-layer UNet-like FCN with a 140-layer FC-ResNet and contains 12.8 million trainable parameters.The FCN and FC-ResNet contribute 1.8 million and 11 million trainable parameters, respectively.
C. Dice Loss
The model is trained with batch-wise Dice loss, chosen because it matches a common segmentation metric and handles foreground–background imbalance without class-frequency balancing.
- Loss definition: The pipeline trains with Dice loss computed per batch from sigmoid outputs and corresponding binary ground-truth labels.The loss has a minimum value of −1.
- Motivation: Dice loss is preferred over binary crossentropy because the Dice coefficient is a common medical-image segmentation metric.Optimizing the evaluation metric directly motivates this choice.
- Motivation: Dice loss addresses high foreground–background class imbalance without requiring class-frequency balancing.
IV. EXPERIMENTS
The experiments evaluate the proposed pipeline across electron microscopy, CT liver-lesion, and MRI prostate segmentation settings, using validation-based early stopping.
- Experimental scope: The experiments test state-of-the-art performance on EM, comparisons against FCN and FC-ResNet baselines on CT liver lesions, and normalization effects on both datasets.
- Training protocol: Training uses early stopping at the highest validation Dice value with a patience of 50 epochs.The implementation uses Keras with the Theano backend.
A. Electron Microscopy dataset
On the EM benchmark, the pipeline is evaluated on separate labeled-training and unlabeled-test images using Vrand and Vinfo, and achieves the highest reported Vrand among published 2D methods.
- Dataset: The EM dataset contains 30 training images and a separate 30-image test set, each measuring 512 × 512 pixels.The images come from serial-section transmission electron microscopy of the Drosophila first instar larva ventral nerve cord.
- Evaluation: The official metrics are maximal foreground-restricted Rand and information-theoretic scores after thinning, with Vrand ranking leaderboard entries.
- Quantitative results: The pipeline outperforms fully convolutional alternatives on Vrand by 0.003 over FusionNet, the second-best fully convolutional approach.The comparison also indicates an advantage over standardization or rescaling before FC-ResNet.
- Quantitative results: The method reports the highest Vrand among published 2D methods, while IAL’s 2D entry reports 0.980 Vrand and 0.988 Vinfo.IAL’s leaderboard entry adds 3D context and improves those scores by 0.003 and 0.001, respectively.
- Qualitative results: Qualitative predictions from the proposed pipeline appear sharper and clearer than those from the compared FC-ResNet pipeline.The displayed predictions use white for cells, black for cell membranes, and gray for assignment uncertainty.
B. Liver lesion dataset
The liver lesion experiments evaluate the pipeline on manually annotated abdominal CT scans and compare it with standard FCN and FC-ResNet methods. The proposed approach achieves the strongest reported validation results and produces lesion predictions with fewer false positives and gaps.
- Dataset: 135 CT scans were manually annotated for colorectal metastases, with liver masks available for 58 scans.The dataset uses 77 training, 28 validation, and 30 test images; evaluation is restricted to liver regions.
- Results: The pipeline achieved the best validation loss of −0.795 and lesion validation Dice of 0.771.The second-best validation loss was obtained by UNet at −0.451.
- Comparison: The method was compared with FCN8, UNet, and FC-ResNet using the same training procedure and Dice loss.Results were reported for both validation and test sets.
- Qualitative analysis: Qualitative results show predictions with fewer false positives and no unsegmented holes or gaps within lesions.The comparison is made against the ground-truth annotation.
C. Prostate dataset
The pipeline was tested on heterogeneous prostate MRI data from the PROMISE12 challenge using a 2D training and inference setup. It achieved strong gland segmentation metrics, outperformed other 2D FCN methods, and remained competitive with 3D methods.
- Dataset: The prostate dataset contains 50 training T2-w MR images and 30 held-out test images acquired across hospitals, equipment, and protocols.The test ground truth was withheld by the challenge organizer for independent evaluation.
- Inference setup: The 2D framework used model averaging across 10 trained models and largest-component selection on each volume.A connected-component method was applied because the method remains based on 2D images.
- Results: 87.4 Dice, 2.17mm average boundary distance, and 12.37% volume difference were obtained for the entire gland.These are the reported results for the proposed method.
- Comparison: The pipeline outperformed other methods based on 2D FCNs and was competitive with methods based on 3D FCNs.The authors describe the results as ranking among the best automated approaches for prostate segmentation.
D. Data normalization
The FCN-based pre-processor changes the intensity distributions supplied to the FC-ResNet. Across prostate and liver data, intensities move from broad modality-specific ranges to narrower ranges before segmentation refinement.
- Distribution analysis: The analysis compares validation-set pixel-intensity distributions at the pipeline input and at the FC-ResNet input.Normal distributions fitted to the validation data are shown as dashed red lines.
- Prostate data: Prostate intensities shift from [0, 2000] at FCN input to [−5, 10] at FC-ResNet input.The plots show background and prostate-class distributions before and after pre-processing.
- Liver lesion data: Liver-lesion intensities shift from [0, 200] at FCN input to [−2, 3] at FC-ResNet input.The analysis plots liver and lesion distributions while ignoring regions outside the liver.
- Qualitative analysis: For liver images, the pre-processor transforms input images and changes intensities within the liver by removing void pixels.The qualitative visualization is provided in Figure 7.
V. DISCUSSION & CONCLUSION
The paper concludes that a low-capacity FCN followed by a deep FC-ResNet provides a versatile medical image segmentation pipeline. Across EM, CT liver-lesion, and MRI prostate tasks, it reports strong results while learning normalization end to end.
- Discussion & Conclusion: The pipeline combines a low-capacity FCN pre-processor with a very deep FC-ResNet of more than 100 layers.The FCN normalizes raw medical image data before the FC-ResNet refines the segmentation prediction.
- Discussion & Conclusion: FC-ResNets require data pre-processing because their identity paths forward input data while residual transformations are controlled by batch normalization.This is especially relevant for medical images with broader intensity ranges than standard RGB images.
- Future work: Future work includes testing alternative pre-processor architectures and expanding the framework to 3D FCN models.The paper presents these as potential directions rather than established results.