PaperlayerView exact revision on arXiv

Source-linked AI summary

Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection

Seyed Raein Hashemi, Seyed Sadegh Mohseni Salehi, Deniz Erdogmus, Sanjay P. Prabhu, Simon K. Warfield, Ali Gholipour

arXiv:1803.11078v4cs.CV

TL;DR

The paper targets the bias toward non-lesion predictions caused by severe class imbalance in medical-image segmentation. It combines asymmetric Tversky/Fβ losses with a patch-wise 3D FC-DenseNet and overlapping-patch fusion, achieving top results across MS lesion segmentation challenges.

  • Problem

    Severe lesion-to-non-lesion imbalance can bias segmentation toward high precision and low recall, while medical applications often place greater importance on false negatives.

  • Method

    The paper trains 3D fully convolutional networks with asymmetric Tversky-index Fβ losses, large overlapping patches, patch selection and augmentation, and B-spline weighted soft-voting fusion.

  • Results

    The method achieved top performance in MSSEG and ISBI; focal loss ranked first by ISBI overall score, while asymmetric loss achieved the best lesion true-positive rate and lowest surface distance.

  • Takeaways & Limitations

    Asymmetric similarity loss with large overlapping patches and prediction fusion is recommended for better precision–recall balancing in highly unbalanced medical-image segmentation.

Abstract

from arXiv · show

Fully convolutional deep neural networks have been asserted to be fast and precise frameworks with great potential in image segmentation. One of the major challenges in training such networks raises when data is unbalanced, which is common in many medical imaging applications such as lesion segmentation where lesion class voxels are often much lower in numbers than non-lesion voxels. A trained network with unbalanced data may make predictions with high precision and low recall, being severely biased towards the non-lesion class which is particularly undesired in most medical applications where FNs are more important than FPs. Various methods have been proposed to address this problem, more recently similarity loss functions and focal loss. In this work we trained fully convolutional deep neural networks using an asymmetric similarity loss function to mitigate the issue of data imbalance and achieve much better tradeoff between precision and recall. To this end, we developed a 3D FC-DenseNet with large overlapping image patches as input and an asymmetric similarity loss layer based on Tversky index (using Fbeta scores). We used large overlapping image patches as inputs for intrinsic and extrinsic data augmentation, a patch selection algorithm, and a patch prediction fusion strategy using B-spline weighted soft voting to account for the uncertainty of prediction in patch borders. We applied this method to MS lesion segmentation based on two different datasets of MSSEG and ISBI longitudinal MS lesion segmentation challenge, where we achieved top performance in both challenges. Our network trained with focal loss ranked first according to the ISBI challenge overall score and resulted in the lowest reported lesion false positive rate among all submitted methods. Our network trained with the asymmetric similarity loss led to the lowest surface distance and the best lesion true positive rate.

I. INTRODUCTION

The paper addresses highly unbalanced medical-image segmentation through asymmetric losses and patch-based FC-DenseNet designs, focusing on MS lesion segmentation as a benchmark. Its approach combines loss-function comparisons with overlapping patches and prediction fusion to improve precision–recall balance.

  • Motivation: MS lesion segmentation is a benchmark for semantic segmentation of highly unbalanced imaging data.
  • Contributions: Asymmetric similarity loss based on the Tversky index is compared with Dice, generalized Dice, and focal loss for unbalanced data.
  • Contributions: Large overlapping patches increase lesion-to-non-lesion sample ratios, provide intrinsic augmentation, and support efficient memory use for variable-size inputs.
  • Contributions: Patch borders have limited effective receptive fields, motivating prediction fusion to account for border uncertainty.
  • Contributions: The study evaluates 3D U-net and FC-DenseNet architectures, with the FC-DenseNet approach combining asymmetric losses and patch fusion.

III. MATERIALS AND METHODS

The methods use 3D fully convolutional U-net and DenseNet architectures, including a patch-wise FC-DenseNet designed for overlapping 3D patches. Dense connectivity, bottlenecks, transition layers, and patch prediction fusion support segmentation of large or arbitrary-size images.

  • 3D patch-wise FC-DenseNet: Each dense block uses bottleneck convolutions before 3 × 3 × 3 convolutions, while transition-down and transition-up blocks reduce or expand feature-map resolution.
  • Network architectures: The study trains 3D fully convolutional U-net and densely connected DenseNet architectures with asymmetric, focal, and generalized Dice losses.
  • 3D U-net: The 3D U-net uses contracting and expanding paths with skip connections to combine high-resolution local and low-resolution global features.
  • 3D patch-wise FC-DenseNet: The patch-wise FC-DenseNet uses overlapping patches, asymmetric loss, and patch prediction fusion, with dense blocks in contracting, middle, and expanding paths.
  • Patch processing: Overlapping patches, typically 64 × 64 × 64 or 128 × 128 × 128, are extracted to segment input images of any size.

B. ASYMMETRIC SIMILARITY LOSS FUNCTION

The paper proposes an asymmetric similarity loss based on Fβ scores to adjust false-positive and false-negative weighting for highly unbalanced lesion segmentation. The hyperparameter β controls the precision–recall trade-off, with larger values emphasizing recall.

  • The loss is computed from sigmoid voxel probabilities and binary ground-truth lesion labels.P denotes predicted labels or probabilities, while G denotes ground-truth labels.
  • The proposed asymmetric similarity loss uses Fβ scores to weight false positives and false negatives differently in highly unbalanced data.It is presented as an alternative to symmetric Dice-based similarity loss.
  • Fβ is a special case of the Tversky index and generalizes the Dice and Tanimoto coefficients.β = 1 yields F1, β = 2 yields F2, and β = 0 yields precision.
  • Larger β values place more emphasis on recall by increasing the penalty associated with false negatives.Adjusting β therefore controls the trade-off between precision and recall.

C. 3D PATCH PREDICTION FUSION

The method segments full-size images using augmented, overlapping 3D patches whose predictions are fused with spline-weighted soft voting. Overlap and border-aware weighting address reduced patch-border accuracy while trading off prediction time and memory.

  • Overlapping 3D patches of size 64 × 64 × 64 or 128 × 128 × 128 are augmented, processed, and fused to segment full-size images.Smaller patches use less memory.
  • Using 75% overlap takes roughly 45 minutes per example image, whereas 50% overlap keeps prediction time close to 5 minutes.The overlap amount is adjustable.
  • No-overlap tiling predicts each voxel once, whereas the fusion approach covers each voxel with 8 patches and reduces patch-border errors.Border voxels can be near the center of another overlapping patch.
  • 50% overlap produces 1,440 augmented 64 × 64 × 64 patch predictions and 32 prediction probabilities per voxel for the example image size.The four augmentations are the original image and three 180-degree plane rotations.
  • Second-order spline-weighted soft voting gives lower weights to border predictions and higher influence to predictions near patch centers.The weighting models Gaussian effective receptive fields while fusing overlapping and augmented predictions.

D. DATASETS

The experiments use the MSSEG and ISBI longitudinal MS lesion segmentation challenge datasets, with multimodal MRI inputs and differing subject, stack, and acquisition configurations.

  • The study evaluates networks on the MSSEG and ISBI longitudinal MS lesion segmentation challenge datasets.Both datasets provide challenge-based evaluation settings for MS lesion segmentation.
  • MSSEG uses five MRI channels from 15 subjects, while ISBI uses four channels from 5 subjects comprising 21 stacks.MSSEG includes Philips and Siemens scanners across three imaging domains; ISBI scans were acquired at 3.0 Tesla.

E. TRAINING

Training uses asymmetric-loss FCNs with five-fold cross-validation, patch selection requiring lesion voxels, and architecture-specific optimization procedures for 3D U-Net and patch-wise FC-DenseNet.

  • Both datasets use five-fold cross-validation, training on four-fifths and validating on the remaining fifth.Each MSSEG fold trains on 12 subjects and validates on 3; each ISBI fold uses 17 stacks for training and 4 for validation.
  • The 3D U-Net is trained end-to-end for 1,000 epochs with ADAM and a decaying learning rate.MSSEG training takes approximately 4 hours on an Nvidia Geforce GTX1080 GPU.
  • The patch-wise 3D FC-DenseNet is trained end-to-end with ADAM for 4,000 MSSEG epochs or 1,000 ISBI epochs.Its learning rate and step interval follow the schedule described for each dataset.
  • Training patches require at least 10 lesion voxels because similarity-loss networks cannot learn when the true-positive count is zero.An equal number of patches is selected from each image.

F. TESTING

The study used five-fold cross-validation because the datasets contained few subjects, retraining each network from scratch for every fold and thresholding voxel probabilities at 0.5.

  • F. TESTING: Five-fold cross-validation was used because the total number of subjects was very limited.Each MSSEG fold contained three subjects from each of three centers; each ISBI fold contained four stacks from one subject.
  • F. TESTING: Each test fold was evaluated with a network trained from scratch on the other four folds.Training used images from 12 MSSEG subjects or four ISBI subjects with four stacks each.
  • F. TESTING: Voxels with computed probabilities of 0.5 or more were classified as lesion, while lower probabilities were classified as non-lesion.

IV. EXPERIMENTS AND RESULTS

The experiments evaluated network architectures, loss-function asymmetry, and patch prediction fusion for lesion segmentation, comparing methods on two challenge datasets with reported literature and challenge results.

  • IV. EXPERIMENTS AND RESULTS: Experiments evaluated the relative effectiveness of different networks, loss-function asymmetry, and patch prediction fusion.
  • IV. EXPERIMENTS AND RESULTS: Methods were compared using results from two challenge datasets, the literature, and the challenges themselves.

A. EVALUATION METRICS

The evaluation used overlap, classification, lesion-wise, volume, and surface-distance metrics, with precision–recall analysis emphasized for skewed datasets. The ISBI overall score was treated cautiously because it may not represent the best criterion.

  • Overlap metrics: The Dice Similarity Coefficient measures twice the prediction–ground-truth intersection relative to their total voxel count.Prediction and ground truth are denoted by P and G.
  • Classification metrics: Sensitivity, specificity, and F2 score were reported, with F2 emphasizing recall more than precision compared with F1 or DSC.TP, FP, FN, and TN denote true positives, false positives, false negatives, and true negatives.
  • Imbalance-aware evaluation: Precision–Recall curves and area under the PR curve were preferred to ROC curves for evaluating highly skewed test datasets.
  • Challenge metrics: Jaccard index and Positive Predictive Value were included among the ISBI challenge metrics.PPV is defined as true positives divided by true and false positives.
  • Lesion-wise metrics: Lesion-wise true- and false-positive rates are more sensitive to accuracy in smaller lesions, which matter for early disease diagnosis.LTPR uses true positives and false negatives; LFPR uses false positives and true negatives, restricted to lesion voxels.
  • Geometric metrics: Volume Difference measures the absolute segmentation-volume difference relative to ground-truth volume, while Surface Difference averages bidirectional lesion distances.A Surface Difference of 0 corresponds to identical predicted and ground-truth lesions.
  • Challenge metrics: The ISBI overall score combines multiple metrics but does not necessarily represent the best criterion, so individual metrics were also examined.

B. RESULTS

Across MSSEG and ISBI evaluations, asymmetric loss and patch-based FC-DenseNet methods improved the sensitivity–specificity trade-off, with β = 1.5 generally performing best. Focal loss achieved the highest ISBI overall ranking, while asymmetric loss achieved the strongest recall and surface-distance outcomes.

  • MSSEG evaluation: β = 1.5 produced the best combined MSSEG performance, balancing higher sensitivity against a slight reduction in specificity.Higher β values increased sensitivity and reduced specificity; β = 3 reduced performance.
  • MSSEG evaluation: 70.5 vs. 71.6: F2 improved for the most complex FC-DenseNet with spline fusion when using asymmetric loss rather than Dice loss.The corresponding DSC difference was not significant, whereas F2 was more informative for highly imbalanced data.
  • Model and fusion comparisons: The 3D patch-wise FC-DenseNet outperformed 3D U-Net, and spline-weighted soft voting improved results over tiling with uniform fusion.The best configuration combined asymmetric loss at β = 1.5 with spline-weighted patch prediction fusion.
  • MSSEG evaluation: β = 1.5 gave the best precision–recall trade-off across the examined MSSEG approaches, outperforming Dice loss at β = 1.The advantage was smaller for large overlapping patches combined with patch selection and prediction fusion.
  • MSSEG evaluation: The asymmetric loss improvement was especially significant for subjects with very few lesion voxels, where Dice loss produced many false negatives.The same β = 1.5 setting was best for high-, medium-, and low-density example figures.
  • ISBI challenge evaluation: 1st, 6th, and 8th: focal-loss, GDL, and asymmetric-loss models ranked respectively on the ISBI challenge overall score as of September 2018.The overall score was not necessarily an optimal weighted average, so individual metrics distinguish the models’ behavior.
  • ISBI challenge evaluation: The focal-loss model achieved the best PPV and LFPR, whereas asymmetric loss achieved the best lesion TPR and lowest surface distance among the reported comparisons.The asymmetric-loss model also had competitive DSC and Jaccard indices and the best average lesion-volume estimation.

V. DISCUSSION

The discussion reports that asymmetric similarity loss improved test performance and precision–recall balance in highly imbalanced MS lesion segmentation, while different losses optimized different challenge metrics.

  • Loss-function effects: Asymmetric similarity loss improved almost all test metrics over Dice similarity loss, including DSC and F2 scores at β = 1.5.The loss deliberately weights recall more heavily than precision, while the observed improvements indicate better test-set generalization.
  • Evaluation metrics: F2 score is more informative than DSC for highly skewed lesion data because it balances precision and recall more appropriately.DSC corresponds to the F1 score and weights precision and recall equally.
  • Evaluation metrics: 0.6% separated the focal and Fβ methods’ average DSC scores for one subject, despite large differences in lesion-detection performance.The reported averages were 77.09 for focal loss and 76.45 for Fβ loss.
  • Challenge results: 69.9% and 65.74% were the average DSC scores achieved on the MSSEG 2016 and ISBI challenges, respectively, exceeding the highest-ranked techniques examined.The method also ranked above the top ten ISBI teams on four of nine metrics and second in DSC and Jaccard.
  • Challenge results: The asymmetric-loss model achieved the best ISBI lesion true-positive rate, whereas the focal-loss model achieved the lowest lesion false-positive rate.Both models maintained top performance according to the combination of challenge metrics.

VI. CONCLUSION

The conclusion presents a 3D patch-wise FC-DenseNet framework using asymmetric loss functions, patch selection and augmentation, and spline-weighted prediction fusion for imbalanced MS lesion segmentation. The approach achieved top performance across the MSSEG and ISBI challenges, with different losses favoring different metrics.

  • VI. CONCLUSION: The method adds asymmetric loss layers to 3D FC-DenseNet and U-net-based fully convolutional networks for highly unbalanced lesion segmentation.The compared losses include Dice, generalized Dice, focal, and asymmetric Fβ-based functions.
  • VI. CONCLUSION: Large overlapping patches support balanced sampling and intrinsic data augmentation, while patch prediction fusion handles variable-size 3D images and patch-border uncertainty.The fusion method uses spline-weighted soft voting.
  • VI. CONCLUSION: 69.9, 71.6, and 73.59 were the reported average DSC, F2, and APR scores for the MSSEG challenge.These metrics summarize the reported MSSEG performance of the proposed method.
  • VI. CONCLUSION: The authors recommend asymmetric similarity loss with large overlapping patches and patch prediction fusion for better precision–recall balancing in highly unbalanced medical image segmentation.The recommendation is based on results from two competitive MS lesion segmentation challenges.
Loading 1803.11078v4…