PaperlayerView exact revision on arXiv

Source-linked AI summary

A Bottom-up Approach for Pancreas Segmentation using Cascaded Superpixels and (Deep) Image Patch Labeling

Amal Farag, Le Lu, Holger R. Roth, Jiamin Liu, Evrim Turkbey, Ronald M. Summers

arXiv:1505.06236v2cs.CV

TL;DR

Pancreas segmentation remains difficult because the organ has high anatomical variability and previous approaches report limited accuracy. The paper proposes a bottom-up pipeline combining dense patch labeling, superpixels, and cascaded random forests. On 80 CT volumes, it achieved 70.7% Dice and 57.9% Jaccard, outperformed MALF, and was substantially faster.

  • Problem

    High variability in pancreas shape, size, location, and boundary contrast makes automated CT segmentation challenging.

  • Method

    A bottom-up pipeline uses boundary-preserving superpixels, dense random-forest and deep-CNN patch labeling, cascaded random forests, and connectivity post-processing.

  • Results

    70.7% Dice and 57.9% Jaccard were achieved on 80 manually segmented CT volumes under six-fold cross-validation, while deep labeling improved numerical stability.

  • Takeaways & Limitations

    The bottom-up method significantly outperformed MALF under the same six-fold cross-validation and required about 6–8 minutes rather than roughly 10 hours per case.

Abstract

from arXiv · show

Robust automated organ segmentation is a prerequisite for computer-aided diagnosis (CAD), quantitative imaging analysis and surgical assistance. For high-variability organs such as the pancreas, previous approaches report undesirably low accuracies. We present a bottom-up approach for pancreas segmentation in abdominal CT scans that is based on a hierarchy of information propagation by classifying image patches at different resolutions; and cascading superpixels. There are four stages: 1) decomposing CT slice images as a set of disjoint boundary-preserving superpixels; 2) computing pancreas class probability maps via dense patch labeling; 3) classifying superpixels by pooling both intensity and probability features to form empirical statistics in cascaded random forest frameworks; and 4) simple connectivity based post-processing. The dense image patch labeling are conducted by: efficient random forest classifier on image histogram, location and texture features; and more expensive (but with better specificity) deep convolutional neural network classification on larger image windows (with more spatial contexts). Evaluation of the approach is performed on a database of 80 manually segmented CT volumes in six-fold cross-validation (CV). Our achieved results are comparable, or better than the state-of-the-art methods (evaluated by "leave-one-patient-out"), with Dice 70.7% and Jaccard 57.9%. The computational efficiency has been drastically improved in the order of 6~8 minutes, comparing with others of ~10 hours per case. Finally, we implement a multi-atlas label fusion (MALF) approach for pancreas segmentation using the same datasets. Under six-fold CV, our bottom-up segmentation method significantly outperforms its MALF counterpart: (70.7 +/- 13.0%) versus (52.5 +/- 20.8%) in Dice. Deep CNN patch labeling confidences offer more numerical stability, reflected by smaller standard deviations.

I. INTRODUCTION

Pancreas segmentation is difficult because the organ varies substantially in shape, size, position, and boundary contrast. The paper proposes a bottom-up superpixel and patch-labeling approach, evaluated against existing methods and MALF under six-fold cross-validation.

  • Pancreas segmentation supports CAD, quantitative imaging analysis, surgical assistance, and pancreatic cancer detection.
  • Shape, size, location, and boundary-contrast variability make automated pancreas segmentation challenging and motivate approaches beyond conventional statistical shape models.
  • The proposed method propagates image and deep patch-labeling confidences through boundary-preserving superpixels and cascaded random forests.
  • The approach targets the computational burden of atlas-based methods, which can require more than 10 hours per testing case and scale poorly.
  • 70.7% Dice and 57.9% Jaccard were achieved on 80 manually segmented CT volumes under six-fold cross-validation.
  • 70.7 ± 13.0% versus 52.51 ± 20.84% Dice was obtained for the bottom-up method versus MALF on the same dataset and validation protocol.

III. METHODS

The method is organized as a modular pipeline whose extensions add sliding-window CNN patch labeling and framework variations to the core algorithm.

  • The core algorithm flow is presented first, covering the initial components of the overall framework.
  • Sliding-window CNN-based dense image patch labeling is described as a method extension.
  • Framework variations are included alongside the CNN patch-labeling extension.

A. Boundary-preserving over-segmentation

The method uses boundary-preserving superpixels as a middle-level representation for pancreas segmentation, balancing edge alignment, computational efficiency, and region size. SLIC is selected after comparing several superpixel algorithms and calibrating the superpixel scale.

  • Boundary-preserving representation: Superpixels decompose CT slices into irregular, perceptually meaningful regions that generally align with image edges.Unlike rectangular patches, superpixels preserve local boundaries while grouping pixels with similar color, texture, or intensity.
  • Algorithm selection: SLIC is fast and memory efficient, making it suitable for evaluating boundary-preserving regions in pancreas CT images.The study compares SLIC, efficient graph-based, entropy-rate, and watershed methods using superpixel quality criteria.
  • Algorithm selection: Watershed provides the least promising pancreas results because it does not combine boundary information with intensity information as graph-based methods do.The comparison evaluates boundary recall across distances from the manually annotated pancreas boundary.
  • Calibration: The number of superpixels per axial image is constrained to 100–200 to trade off superpixel dimensions and region size.This range is used to control the granularity of the 2D representation.
  • Calibration: Oracle superpixel labeling reaches 81.2% ± 3.3% mean Dice at overlap threshold τ = 0.50, establishing an upper bound for the labeling framework.The threshold and SLIC choice are calibrated on 20 scans, and the oracle requires ground-truth overlap information.

P RF

The RF patch-labeling stage combines texture, intensity, boundary-aware probability, and normalized spatial features to produce pancreas probability response maps. Dense RF classification provides informative specificity and recall in cross-validation.

  • Feature construction: The RF representation uses 46 patch-level image features describing pancreas appearance and its surrounding anatomy.The features include texture, intensity-distribution, boundary-aware probability, and normalized relative x- and y-position information.
  • Feature construction: Dense dSIFT descriptors are extracted on image grids with 3-pixel spacing to capture texture information.The descriptors are computed around patch centers and supplement other patch-level visual features.
  • Probability response maps: KDE intensity models convert positive and negative CT intensity distributions into pancreas probability values over H = [0 : 1 : 4095].A normalized likelihood ratio is precomputed as a lookup table for efficient response-map generation.
  • Probability response maps: The RF response maps use patch classification probabilities, with relative x- and y-position features improving spatial separation of positive and negative regions.The maps are generated with six-fold cross-validation and illustrate high pancreas probability in red.
  • Classification: Random forests with 50 trees and minimum leaf size 150 achieve good classification specificity and recall in cross-validation.The reported feature set includes dSIFT, pixel-level correlations, and other visual information from each image patch.

C. Patch-level Labeling via Deep Convolutional Neural Network: P CNN

The deep patch-labeling stage applies a CNN to larger 2.5D image windows, allowing richer spatial context than hand-crafted patch features. Its higher computational cost is managed by evaluating CNN patches only after initial superpixel pruning.

  • CNN architecture: The CNN uses five convolutional layers, max-pooling, fully connected layers, DropOut, and a two-way softmax for pancreas versus non-pancreas classification.The softmax outputs a probability for each image patch.
  • Patch labeling: CNN response maps are produced with sliding-window 2.5D patches combining axial, coronal, and sagittal planes.Training uses 64×64-pixel patches, which provide larger spatial context than the hand-crafted feature stage.
  • Computational strategy: Dense CNN patch labeling is more computationally expensive than RF labeling because it evaluates larger image windows.RF maps are computed exhaustively at 3-pixel intervals, whereas CNN evaluation is restricted to locations surviving the first cascade stage.
  • Computational strategy: The first cascade stage operates at high recall and low specificity to minimize false negatives before CNN labeling.This pruning strategy also reduces the class imbalance encountered when training the later CNN-dependent classifier.
  • Computational strategy: Superpixel cascading changes the non-pancreas-to-pancreas ratio from greater than 100 to approximately 5 before later CNN processing.The remaining candidates are enriched for difficult negatives and pancreas examples.

D. Superpixel-level Feature Extraction, Cascaded Classification and Pancreas Segmentation

The final segmentation framework pools CT intensities and patch-labeling probabilities within superpixels, then applies cascaded random forests to classify candidate regions. The cascade addresses severe class imbalance by pruning most negatives while retaining nearly all pancreas candidates.

  • Cascaded classification: Three superpixel-level random forests form two cascaded frameworks, F-1 and F-2, with superpixel labels derived from ground-truth overlap ratios.Positive superpixels have r ≥ 0.5, while negative superpixels have r ≤ 0.2.
  • Framework variants: The proposed CNN architecture outputs pancreas and non-pancreas patch probabilities through a final two-class softmax layer.These probabilities provide an additional channel for the superpixel-level framework.
  • Feature extraction: Superpixel classifiers model CT intensities and per-pixel pancreas probabilities as empirical distributions pooled within each superpixel.The pooled features are extracted from the original CT and RF or CNN probability channels.
  • Cascaded classification: The first cascade stage prunes 97% of negative superpixels while maintaining nearly 100% recall or sensitivity.The remaining hard negatives, together with all positives, train the second classifier.
  • Framework variants: F-1 uses CT and RF probability channels, whereas F-2 incorporates deep CNN probability maps after the shared initial pruning stage.The two frameworks share the first cascade layer and differ in the later information channels used for superpixel classification.

P RF S

The method combines superpixel-level cascaded classification with random-forest and CNN patch-labeling confidences, then produces a connected 3D pancreas mask. CNN probabilities improve the final classifier but are insufficient alone.

  • P_CNN consistently improves C2_SP results over P_RF, but P_CNN alone is not powerful enough for C3_SP.The reported findings support combining deep and conventional information rather than relying on CNN probabilities alone.
  • P CNN is integrated with intensity and percentile features from superpixels to train the final classifier C3_SP.The resulting feature set uses two information channels, I_CT and P_CNN.
  • F-1 uses P_RF, whereas F-2 uses both P_RF and P_CNN with little extra computational cost.P_RF is trained once using positive and negative image patches; the CNN uses hard-negative and positive patches.
  • The two-level cascaded random-forest hierarchy is used for F-1 and extended with CNN patch labeling in F-2.The hierarchy is reported as empirically sufficient, although deeper cascades are possible.
  • The binary 3D mask is formed by stacking slice-level superpixel labels and retaining the largest 3D connected component.This post-processing assumes overall pancreas connectivity in 3D.

A. Imaging Data

The evaluation uses 80 portal-venous contrast-enhanced abdominal CT volumes from 53 male and 27 female subjects, including healthy kidney-donor patients and radiologist-selected PACS cases.

  • 80 3D abdominal portal-venous contrast-enhanced CT scans are used for evaluation.The scans were acquired approximately 70 seconds after intravenous contrast injection.
  • The dataset includes 53 male and 27 female subjects.
  • 17 subjects are healthy patients from a kidney donor transplant list, while 63 are randomly selected from PACS by a radiologist.The donor-list patients had abdominal CT scans before nephrectomy.

B. Experiments

Experiments use six-fold cross-validation to assess segmentation accuracy, classifier behavior, training-set size, comparisons with prior methods, and a MALF baseline. The proposed framework reaches strong Dice performance while improving stability and runtime over MALF.

  • Classifier analysis: The F-2 classifier achieves an AUC of 0.931 after replacing P_RF with P_CNN, compared with 0.884 for C2_SP.The result demonstrates a performance benefit from CNN dense patch labeling over handcrafted image features.
  • Framework comparison: 70.7 ± 13.0% Dice is achieved by F-2, compared with 68.8 ± 25.6% for F-1 and 64.5 ± 12.3% for SP.These values are reported for the final pancreas segmentation accuracy.
  • Training-set size: Adding training patients yields steady improvements of approximately 4% in Dice and 5% in Jaccard from 40 to 60 and from 60 to 80 scans.The 80-patient results classify 15 cases as good, 49 as fair, and 16 as poor by Dice category.
  • Framework comparison: About 2% higher Dice is obtained with F-2, while Dice standard deviation drops from 25.6% to 13.0% versus F-1.The JI standard deviation also drops from 25.4% to 13.6%.
  • Comparison with prior methods: Under six-fold cross-validation, F-2 slightly but consistently outperforms prior methods, although direct numerical comparison is limited by different datasets.The paper explicitly contrasts its six-fold protocol with prior leave-one-patient-out evaluation.
  • Computational efficiency: The proposed method processes a patient scan in about 6–8 minutes, whereas other methods require 10 hours or more.The MALF experiments took approximately 33.5 days for six-fold validation, while the proposed approach processed 80 cases in approximately 9 hours.
  • MALF comparison: MALF obtains 52.51 ± 20.84% Dice, versus 70.7 ± 13.0% for the proposed method under the same six-fold cross-validation.MALF performance ranges from 0% to 80.56% Dice in the reported experiment.

V. CONCLUSION AND DISCUSSION

The fully automated bottom-up method propagates patch-level information through superpixels and cascaded random forests for pancreas segmentation. Its design also identifies weak boundary contrast and boundary refinement as important areas for further improvement.

  • Method: The pipeline decomposes CT slices into boundary-preserving superpixels, computes dense patch-level pancreas probabilities, pools intensity and probability features, and applies cascaded random forests with connectivity post-processing.Dense labeling can use either hand-crafted random-forest features or larger-context deep CNN windows.
  • Method: Superpixels are classified as pancreas or non-pancreas using pooled intensity values and supervised class probabilities such as P_RF and P_CNN.These probability maps function as supervised semantic class image embeddings for per-pixel responses.
  • Limitations and future work: Low image boundary contrast can impair superpixel generation, motivating supervised edge learning to strengthen semantic object boundaries.The proposed direction is intended to preserve weak boundaries and prevent segmentation leakage.
  • Limitations and future work: Voxel-level masks can be propagated from superpixel classifications and further improved with boundary-refinement methods such as narrow-band level sets or learned-intensity graph cuts.Further examination of spatial relationships among pancreatic and nearby major veins is also identified as future work.
Loading 1505.06236v2…