PaperlayerView exact revision on arXiv

Source-linked AI summary

A Mutual Bootstrapping Model for Automated Skin Lesion Segmentation and Classification

Yutong Xie, Jianpeng Zhang, Yong Xia, Chunhua Shen

arXiv:1903.03313v4cs.CV

TL;DR

The paper addresses the challenge of jointly improving automated skin lesion segmentation and classification, which are often handled separately despite their relationship. It proposes MB-DCNN, a mutual bootstrapping model linking coarse segmentation, mask-guided classification, and enhanced segmentation, and reports strong results on ISIC-2017 and PH2.

  • Problem

    Deep learning models for skin lesion segmentation and classification are usually designed for only one task, despite the tasks being related and potentially beneficial when performed jointly.

  • Method

    MB-DCNN mutually transfers information between coarse-SN, mask-CN, and enhanced-SN, while combining rank loss with Dice loss for segmentation.

  • Results

    The model achieves Jaccard indices of 80.4% and 89.4% and average AUCs of 93.8% and 97.7% on ISIC-2017 and PH2, respectively.

  • Takeaways & Limitations

    The results indicate that a unified model can simultaneously boost skin lesion segmentation and classification through mutual bootstrapping.

Abstract

from arXiv · show

Automated skin lesion segmentation and classification are two most essential and related tasks in the computer-aided diagnosis of skin cancer. Despite their prevalence, deep learning models are usually designed for only one task, ignoring the potential benefits in jointly performing both tasks. In this paper, we propose the mutual bootstrapping deep convolutional neural networks (MB-DCNN) model for simultaneous skin lesion segmentation and classification. This model consists of a coarse segmentation network (coarse-SN), a mask-guided classification network (mask-CN), and an enhanced segmentation network (enhanced-SN). On one hand, the coarse-SN generates coarse lesion masks that provide a prior bootstrapping for mask-CN to help it locate and classify skin lesions accurately. On the other hand, the lesion localization maps produced by mask-CN are then fed into enhanced-SN, aiming to transfer the localization information learned by mask-CN to enhanced-SN for accurate lesion segmentation. In this way, both segmentation and classification networks mutually transfer knowledge between each other and facilitate each other in a bootstrapping way. Meanwhile, we also design a novel rank loss and jointly use it with the Dice loss in segmentation networks to address the issues caused by class imbalance and hard-easy pixel imbalance. We evaluate the proposed MB-DCNN model on the ISIC-2017 and PH2 datasets, and achieve a Jaccard index of 80.4% and 89.4% in skin lesion segmentation and an average AUC of 93.8% and 97.7% in skin lesion classification, which are superior to the performance of representative state-of-the-art skin lesion segmentation and classification methods. Our results suggest that it is possible to boost the performance of skin lesion segmentation and classification simultaneously via training a unified model to perform both tasks in a mutual bootstrapping way.

I. INTRODUCTION

Skin lesion segmentation and classification are complementary but difficult CAD tasks, while most deep models address only one. MB-DCNN jointly trains both tasks through mutual bootstrapping and a hybrid segmentation loss.

  • Segmentation detects lesion locations and boundaries, whereas classification diagnoses lesion types such as melanoma, nevus, and seborrheic keratosis.
  • Low lesion–skin contrast, visual similarity between lesion types, and intra-type variation make both tasks challenging.
  • Most DCNN methods are designed for either segmentation or classification despite the tasks’ strong relationship and potential mutual benefits.
  • Classification localization maps can include partial-lesion and non-lesion regions, producing inaccurate proxy labels and unsatisfying segmentation.
  • MB-DCNN transfers coarse masks to classification and refined localization information back to segmentation, while combining Dice and rank losses.

II. RELATED WORK

Earlier skin-lesion systems relied on handcrafted features or separate DCNNs for segmentation and classification. MB-DCNN instead integrates the two tasks and transfers localization information without proxy pixel labels.

  • Traditional methods include clustering, thresholding, region growing, active contours, and classifiers trained on handcrafted features.
  • DCNNs provide unified learning-based feature extraction and classification, driving widespread use in medical image analysis.
  • Existing DCNN-based skin-lesion methods commonly perform either segmentation or classification, overlooking their intrinsic correlation.
  • Weakly supervised approaches transfer class-related localization cues, but sparse regions are insufficiently accurate for reliable segmentation training.
  • MB-DCNN uses coarse-SN, mask-CN, and enhanced-SN to transfer lesion masks and localization maps between classification and segmentation.

A. Coarse-SN

Coarse-SN first segments lesions roughly, then supplies lesion-location masks to mask-CN alongside images. This prior information reduces distracting background content and supports classification.

  • Coarse-SN is trained on pixel-level annotations to produce initial lesion masks and provide mask-CN with prior lesion locations.
  • Lesions occupy small image regions, while normal skin and artifacts such as hair, frames, blood vessels, and air bubbles can interfere with classification.
  • Concatenating each image with its coarse mask gives mask-CN region guidance while using only image-level class labels for classification training.
  • Mask-CN is built on pre-trained Xception, replacing its last pooling layer with two dilated convolutional layers using dilation rate 2.

C. Enhanced-SN

Enhanced-SN fuses encoder features with mask-CN localization maps before decoding segmentation results. Its segmentation networks use Dice loss with rank loss to emphasize difficult pixels.

  • Mask-CN produces class-specific localization maps through CAM by weighting and summing its final convolutional feature maps.
  • Enhanced-SN concatenates encoder feature maps with localization maps, then fuses them using a 1 × 1 convolution, batch normalization, and ReLU.
  • D. Hybrid loss for coarse- and enhanced-SN: The hybrid loss is L_hybrid = L_dice + λL_rank, with λ controlling the rank-loss contribution.
  • D. Hybrid loss for coarse- and enhanced-SN: Dice loss measures prediction–ground-truth agreement and handles class-imbalanced pixels, but alone struggles with hard-easy pixel imbalance.
  • D. Hybrid loss for coarse- and enhanced-SN: Online ranking selects the highest-error lesion and background pixels as hard pixels, adding constraints that focus learning on them.

IV. DATASETS

The study uses ISIC-2017, an annotated dataset supporting both segmentation and classification, an image-level-label ISIC additional dataset, and PH2.

  • ISIC-2017 dataset: ISIC-2017 contains 2,000 training, 150 validation, and 600 test dermoscopic images with lesion-boundary annotations and diagnostic labels.The diagnoses are melanoma, nevus, and seborrheic keratosis.
  • ISIC additional dataset: The ISIC additional dataset contributes 1,320 dermoscopic images with image-level labels: 466 melanoma, 32 seborrheic keratosis, and 822 nevus cases.
  • PH2 dataset: PH2 contains 200 dermoscopic images, including 160 nevus and 40 melanoma cases, acquired under the same imaging conditions.

V. EXPERIMENTS AND RESULTS

Experiments evaluate MB-DCNN using augmentation, segmentation and classification metrics, comparisons with recent segmentation methods, and qualitative mask examples.

  • Training phase: Training uses online augmentation including random crops, rotations, shearing, shifts, and zooming.The segmentation networks use pixel-level labels, while mask-CN uses image-level labels.
  • Evaluation metrics: Segmentation is evaluated with JA, DI, pixel-AC, pixel-SE, and pixel-SP, while classification uses AUC, AC, SE, and SP.ISIC-2017 ranking uses JA for segmentation and average AUC for classification.
  • Qualitative results: Figure 4 illustrates coarse-SN improving mask-CN localization maps and class-specific maps guiding enhanced-SN toward better segmentation.
  • Segmentation results: MB-DCNN achieves the best segmentation performance on both ISIC-2017 and PH2 in comparison with recent methods.
  • Segmentation results: 2.2% JA improvement over SLSDeep is achieved on ISIC-2017, while PH2 JA rises from 84.0% to 86.7% without fine-tuning and reaches 89.4% after fine-tuning.After fine-tuning on PH2, the model also achieves a DI of 94.2%.

D. Classification results

MB-DCNN classification results are compared with recent methods and challenge leaders on ISIC-2017 and with established systems on PH2.

  • Compared methods: The PH2 comparison includes CICS, MFLF, and CCS, while the ISIC-2017 comparison includes recent models and five top-ranking challenge solutions.
  • ISIC-2017: On ISIC-2017, MB-DCNN achieves the highest average AUC, exceeding ARL-CNN by 2.1% and surpassing the top-ranking challenge performance.
  • PH2: On PH2, direct application without fine-tuning raises AUC from 84.3% for CCS to 95.6%, while fine-tuning yields an AUC of 97.7%.The fine-tuned model also achieves AC 94.0%, SP 93.8%, and SE 95.0%.

E. Comparison to other multi-task methods

MB-DCNN transfers information between segmentation and classification rather than only sharing parameters, and its loss design addresses class and hard-easy pixel imbalance.

  • Multi-task comparison: Method I shares parameters between segmentation and classification, whereas Method II and MB-DCNN use different strategies to transfer information between tasks.
  • Rank loss: The rank-loss hyperparameters K and margin represent selected hard pixels and the constraint between hard background and foreground pixels.
  • Multi-task results: Table IV compares segmentation and classification performance for MB-DCNN and two multi-task methods on ISIC-2017 and PH2.ISIC-2017 classification metrics average melanoma and seborrheic keratosis performance.
  • Rank-loss settings: K=30 and margin=0.3 produce the highest JA on the ISIC-2017 validation set among the tested settings.
  • Hybrid-loss settings: λ controls the contribution of rank loss in the hybrid loss, with λ=0.05 producing the highest JA among tested values.
  • Loss comparison: The hybrid loss addresses both class imbalance and hard-easy pixel imbalance, unlike WCE and Dice loss, which mainly address class imbalance.

VI. DISCUSSIONS

Using coarse lesion masks improves the classification network’s lesion localization, making its CAMs more similar to segmentation ground truth.

  • Using coarse lesion masks improves the classification network’s lesion localization, producing CAMs more similar to segmentation ground truth.The comparison evaluates CAMs with and without coarse-SN masks against ground-truth lesion segmentation.

B. Using classification to boost segmentation

Classification-derived localization maps can improve segmentation, while segmentation masks improve classification and reduce reliance on densely annotated images. These benefits are strongest when training data are limited.

  • Using classification to boost segmentation: Transferring mask-CN localization maps to enhanced-SN improves segmentation when coarse-SN first bootstraps mask-CN.Without coarse-SN, mask-CN localization maps are less accurate and yield lower JA than the proposed model.
  • Using classification to boost segmentation: 3.8% improvement in JA over D-Net results from fusing transferred localization maps with encoder image features before decoding.The model also improves JA by 1.1% over MDC using the same segmentation network.
  • Using classification to boost segmentation: 2.9% to 1.1% is the performance gain over fully supervised segmentation as densely annotated training images increase from 100 to 2000.The proposed model uses image-level annotations as weak supervision for segmentation.
  • Using classification to boost segmentation: 0.2% lower JA is achieved by the proposed model with 1000 densely annotated images than by fully supervised training with 2000 samples.The result suggests weakly annotated images can replace almost half of densely annotated training images while maintaining segmentation performance.
  • Using classification to boost segmentation: Segmentation masks provide more accurate lesion regions of interest, improving classification performance compared with classification without coarse-SN.This advantage persists across different numbers of image-level annotated training images.
  • Using classification to boost segmentation: 7.5% to 2.1% is the classification gain from lesion masks as image-level training images increase from 500 to 3320.The smaller gain with more training data indicates that lesion masks play a greater role when data are limited.
  • Using classification to boost segmentation: Similar classification performance is obtained with masks and 2000 images as without masks and 3320 images.Lesion masks can largely compensate for reduced classification training data.

D. Robustness to ISIC archive dataset

The MB-DCNN model is evaluated on the larger ISIC archive dataset using separate segmentation and classification image sets, with comparisons against advanced task-specific models.

  • Robustness to ISIC archive dataset: 10,697 segmentation images and 21,153 classification images from the ISIC archive dataset are used for evaluation.Segmentation samples include expert lesion-boundary tracings, while the classification set contains nevus, melanoma, and seborrheic keratosis cases.
  • Robustness to ISIC archive dataset: MB-DCNN is compared with three advanced segmentation models and three advanced classification models using two-fold cross-validation.The competing models’ performance was obtained by running their source code.
  • Robustness to ISIC archive dataset: The ISIC archive experiments further justify the effectiveness and generality of MB-DCNN.The supplied result passage reports a classification range ending at 95.7%, but does not provide enough context to assign that value to a specific metric or model.

E. Complexity

MB-DCNN requires substantial offline training but performs segmentation and classification quickly during inference, supporting potential use in routine clinical workflows.

  • Complexity: 48 hours are required to train MB-DCNN, divided across coarse-SN, mask-CN, and enhanced-SN.Training takes 24 hours for coarse-SN, 12 hours for mask-CN, and 12 hours for enhanced-SN on four NVIDIA GTX 1080 Ti GPUs.
  • Complexity: Less than 1 second is required to segment and classify each lesion during online testing.The authors describe training as time-consuming but offline, while fast testing suggests potential routine clinical workflow use.
  • Complexity: MB-DCNN mutually bootstraps segmentation and classification and is reported to outperform representative state-of-the-art methods on ISIC 2017 and PH2.The model uses coarse-SN, mask-CN, and enhanced-SN together with a hybrid loss for the segmentation networks.
  • Complexity: Future work targets end-to-end iterative optimization and automated selection of the hard-pixel number K and weighting factor λ.The proposed mutual bootstrapping strategy is also intended for other medical image segmentation and classification tasks.
Loading 1903.03313v4…