Source-linked AI summary
Image Classification of Melanoma, Nevus and Seborrheic Keratosis by Deep Neural Network Ensemble
Kazuhisa Matsunaga, Akira Hamada, Akane Minagawa, Hiroshi Koga
TL;DR
The paper addresses lesion classification in Part 3 of the ISIC-ISBI 2017 challenge. It presents a classification system and machine-learning approach, reporting performance that significantly outperformed the previous state of the art despite no lesion segmentation or cropping.
Problem
The paper addresses the defined lesion-classification task in Part 3 of the ISIC-ISBI 2017 challenge.
Method
The authors present a proposed classification system and machine-learning approach, together with evaluation results.
Results
The proposed method significantly outperformed last year’s state of the art, with MM AUC improving from 0.917 to 0.924 on the validation set.
Takeaways & Limitations
The reported results indicate that the proposed method achieved strong challenge performance without using lesion segmentation or cropping.
Takeaways & Limitations
The method did not use lesion segmentation or cropping, and the authors expect reliable segmentation to improve results to a certain extent.
Abstract
from arXiv · showhide
This short paper reports the method and the evaluation results of Casio and Shinshu University joint team for the ISBI Challenge 2017 - Skin Lesion Analysis Towards Melanoma Detection - Part 3: Lesion Classification hosted by ISIC. Our online validation score was 0.958 with melanoma classifier AUC 0.924 and seborrheic keratosis classifier AUC 0.993.
1. Introduction
The challenge classified skin-lesion images into melanoma, nevus, and seborrheic keratosis using two binary classifiers, with participants ranked by mean AUC. The report presents the team's classification system and evaluation results.
- The task classified images into melanoma, nevus, and seborrheic keratosis classes.
- Two binary classifiers addressed melanoma versus the rest and seborrheic keratosis versus the rest.
- Participants were ranked by the mean of the two classifier area under the ROC curves.
- The task allowed external training data and age/sex information tagged with some provided samples.
- The report presents the proposed classification system, machine learning approach, and evaluation results.
2. Proposed Classification System
The proposed system normalizes lesion images, processes transformed inputs with CNN ensembles for two binary tasks, and optionally uses age/sex information for SK classification. It also integrates SK outputs into MM classification because the SK classifier was considered more reliable.
- Input images undergo luminance, color-balance, and color-constancy normalization before classification.
- Normalized images feed separate SK-versus-rest and MM-versus-rest base classifiers with identical composition.
- Geometric image transformations are processed in parallel by CNN ensembles that output prediction values in [0.0,1.0].
- The system uses 50-layer ResNet models implemented in Keras with modifications by the authors.
- Age/sex information may be thresholded before output for SK classification, while cross-validation found no significant increase for MM classification.
- The authors integrated SK-classifier information into MM classification because the SK classifier was more reliable despite training on the same sample set.
3. Machine Learning
The machine-learning pipeline augments provided data with external ISIC Archive samples, fine-tunes pretrained CNNs, and selects CNN combinations through cross-validation.
- The training data comprised 374 MM, 254 SK, and 1372 NCN provided samples.
- External training data added 409 MM, 66 SK, and 969 NCN samples from an ISIC Archive subset.
- CNNs were fine-tuned from an initial Keras model pretrained for generic object recognition.
- Different optimization methods were tested, and the best CNN combination was selected through cross-validation.
- The optimization methods were RMSProp and AdaGrad.
4. Results
The proposed method was evaluated on ISIC-ISBI Challenge 2017 validation data and Challenge 2016 test data. External data and complementary classifier use improved selected AUC results, while age/sex information helped SK classification but not MM classification.
- Age/sex information improved SK AUC from 0.957 to 0.960 in cross-validation but produced no improvement for MM.
- The complementary use of the SK classifier improved MM AUC from 0.917 to 0.924 on the validation set.
- Challenge 2016 Part 3 evaluated malignant-versus-benign binary classification using the 2016 training set with external samples.
- The reported evaluation included Challenge 2017 validation-set scores and Challenge 2016 test-set scores, with final results presented in Table 1.
5. Conclusions and Future Work
The proposed method significantly outperformed last year’s state-of-the-art despite using no lesion segmentation or cropping. The authors identify segmentation, age/sex handling, and multi-class integration as areas requiring further consideration.
- The proposed method significantly outperformed last year’s state-of-the-art.
- The system did not utilize lesion segmentation or cropping, although reliable segmentation could improve results to a certain extent.
- Age/sex information had a slight effect on seborrheic keratosis classification.
- Practical use may require more careful implementation than thresholding because seborrheic keratosis is rare at young ages but still possible.
- The multi-class integration may require greater mathematical sophistication and consideration of how to treat complex diseases.