PaperlayerView exact revision on arXiv

Source-linked AI summary

Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification

Peng Yao, Shuwei Shen, Mengjuan Xu, Peng Liu, Fan Zhang, Jinyu Xing, Pengfei Shao, Benjamin Kaffenberger, Ronald X. Xu

arXiv:2102.01284v2cs.CVcs.LG

TL;DR

Small, imbalanced skin-lesion datasets hinder DCNN-based diagnosis, especially where computational resources are limited. The paper combines moderate-complexity single models with regularization, Modified RandAugment, MWNL, and cumulative learning, achieving performance comparable or superior to ensembles across dermoscopic datasets at lower computational cost.

  • Problem

    Small, imbalanced public skin-lesion datasets hinder broad DCNN implementation for automated diagnosis.

  • Method

    The paper combines moderate-complexity DCNNs, DropOut and DropBlock, Modified RandAugment, Multi-Weighted New Loss, and cumulative learning.

  • Results

    The single-DCNN method achieved classification accuracy comparable or superior to multiple ensembling models across different dermoscopic datasets.

  • Takeaways & Limitations

    The method achieves high classification performance with low computational-resource and inference-time costs, supporting potential mobile screening applications.

Abstract

from arXiv · show

Deep convolutional neural network (DCNN) models have been widely explored for skin disease diagnosis and some of them have achieved the diagnostic outcomes comparable or even superior to those of dermatologists. However, broad implementation of DCNN in skin disease detection is hindered by small size and data imbalance of the publically accessible skin lesion datasets. This paper proposes a novel single-model based strategy for classification of skin lesions on small and imbalanced datasets. First, various DCNNs are trained on different small and imbalanced datasets to verify that the models with moderate complexity outperform the larger models. Second, regularization DropOut and DropBlock are added to reduce overfitting and a Modified RandAugment augmentation strategy is proposed to deal with the defects of sample underrepresentation in the small dataset. Finally, a novel Multi-Weighted New Loss (MWNL) function and an end-to-end cumulative learning strategy (CLS) are introduced to overcome the challenge of uneven sample size and classification difficulty and to reduce the impact of abnormal samples on training. By combining Modified RandAugment, MWNL and CLS, our single DCNN model method achieved the classification accuracy comparable or superior to those of multiple ensembling models on different dermoscopic image datasets. Our study shows that this method is able to achieve a high classification performance at a low cost of computational resources and inference time, potentially suitable to implement in mobile devices for automated screening of skin lesions and many other malignancies in low resource settings.

I. INTRODUCTION

Skin lesion classification is constrained by small, imbalanced datasets and difficult image characteristics, motivating a computationally efficient single-model strategy. The paper addresses model selection, overfitting, augmentation, class imbalance, and outlier effects through several coordinated methods.

  • Clinical motivation: Limited dermatologic workforces and pathology facilities restrict prompt skin-cancer detection in rural communities.Deep learning may automate screening despite shortages of dermatologists and laboratory facilities.
  • Data and diagnostic challenges: Public dermoscopic datasets are small, while lesions may have low contrast, fuzzy borders, and artifacts that complicate DCNN training.HAM contains 10,015 images, whereas BCN_20000 contains 19,424 images.
  • Research questions: Moderate-complexity DCNNs can outperform larger models on small dermoscopic datasets, where added capacity may cause overfitting.The authors experimentally compare structures and capacities to examine this pattern.
  • Research questions: The paper investigates regularization and augmentation to reduce overfitting caused by small datasets, artifacts, and problematic crops or labels.Random crops can create partial or empty lesions, producing very hard examples or outliers with large losses.
  • Research questions: Class imbalance involves both unequal sample counts and unequal classification difficulty across lesion categories.The paper considers sample-based and cost-sensitive strategies, including methods that emphasize difficult examples.
  • Contributions: The proposed Multi-weighted New Loss addresses class imbalance, emphasizes key classes such as melanoma, and limits very large losses from outliers.The contribution is presented as simultaneously handling data imbalance and outlier effects during training.
  • Contributions: An end-to-end cumulative learning strategy balances representation learning and classifier learning without additional computational cost.It first learns universal patterns for initialization before gradually focusing on imbalanced data.
  • Scope: The combined methods are intended to address insufficient sample size, class imbalance, and labeling noise across medical-image and computer-vision classification tasks.The authors state that few prior works handle these problems simultaneously.

II. RELATED WORK

Related work covers increasingly capable DCNN architectures and augmentation methods developed for image classification. The paper adopts this context while focusing on efficient augmentation for limited dermoscopic data.

  • DCNN models: ResNet, SENet, EfficientNet, and RegNet advanced DCNN feature extraction, classification performance, or computational efficiency.RegNet is reported to perform better and up to 5x faster than similarly sized EfficientNets on GPUs.
  • Framework: The proposed framework is organized as separate training and test pipelines.The flowchart presents the training pipeline in the upper part and the test pipeline in the lower part.
  • Data augmentation: Data augmentation expands training-data diversity and number to address limited training data.Traditional augmentation policies can be difficult to transfer across domains because they require domain-specific expertise.
  • Data augmentation: RandAugment reduces augmentation search complexity by applying N transformations of shared magnitude M.Its search space is reduced from 10^32 for AutoAugment and Fast AutoAugment to 10^2.

C. Class-Balanced Loss and Focal Loss

Class-Balanced Loss reweights classes using effective sample counts, while Focal Loss down-weights easy examples; CBfocal combines both mechanisms for imbalanced classification.

  • Class-Balanced Loss: Class-Balanced Loss uses the effective number of samples to define class-specific loss weights.The effective number is expressed as (1−β^n)/(1−β), where n is the class sample count.
  • Class-Balanced Loss: The class-balanced weight w_i varies with β and approaches 1/N_i as β approaches 1.N_i denotes the number of samples in class i.
  • Focal Loss: Focal Loss adds a modulating factor that increasingly down-weights easy examples as r increases.When r = 0, Focal Loss is equivalent to cross-entropy loss.
  • CBfocal: CBfocal combines class-balanced weighting with Focal Loss to address both sample-number imbalance and classification difficulty.Its weighting uses the ground-truth class sample count N_y.

III. METHODOLOGY

The methodology evaluates a single DCNN framework across four dermoscopic datasets with different sizes, class structures, and dataset compositions.

  • Methodology overview: The proposed framework covers datasets, evaluation metrics, DCNN models, augmentation, loss design, cumulative learning, and training and evaluation strategies.The methodology is organized around the classification framework illustrated in Fig. 2.
  • Datasets: ISIC 2018 uses HAM10000, containing 10,015 images across seven skin-lesion types.The test set contains 1,512 images without published labels and is evaluated through the ISIC website.
  • Datasets: ISIC 2017 contains 2,000 training, 150 validation, and 600 test images across MEL, SK, and NV.The class distributions are uneven, especially for NV relative to MEL and SK.
  • Datasets: ISIC 2019 combines HAM, MSK, and BCN_20000 into 25,331 images spanning eight lesion classes.Its 8,238 test labels are unpublished and require website-based evaluation.
  • Datasets: The 7-PT Dataset contains 1,011 cases across five categories, split into 413 training, 203 validation, and 395 test cases.Each case includes a dermoscopic image and associated metadata, with clinical images available for nearly all cases.

B. Evaluation Metrics

The study uses balanced accuracy as its main metric and supplements it with specificity, AUC, and melanoma sensitivity; augmentation transformations are documented separately.

  • Evaluation metrics: Balanced accuracy averages recall across classes, giving each class equal treatment.It is computed from true positives, false negatives, and the number of classes.
  • Evaluation metrics: The evaluation also reports averaged specificity and average ROC AUC for comparisons with state-of-the-art algorithms.These metrics complement balanced accuracy in the reported evaluations.
  • Evaluation metrics: Melanoma sensitivity is listed separately because MEL represents a life-threatening lesion category.This isolates performance on melanoma from the aggregate class metrics.
  • Augmentation operations: The transformation list includes geometric operations such as rotation, flipping, shearing, distortion, scaling, and Cutout.The listed operations are available in the augmentation search space.

C. DCNN Models

The study compares DCNN architectures and adds DropOut, DropBlock, and Modified RandAugment to improve performance and regularization on small dermoscopic datasets.

  • DCNN Models: Across four dermoscopic datasets, the study tests DCNNs from the VGG series through the RegNet series with dataset-specific output dimensions.RegNetY-3.2G achieves the best BACC on ISIC 2018, while other RegNetY variants lead on additional datasets.
  • Regularization: RegNetY-##-Drop inserts DropBlock after Stages 3 and 4 and DropOut before the final fully connected layer.DropOut is assigned to the fully connected layer, while DropBlock regularizes a convolutional layer.
  • Regularization: DropBlock layers share block-size parameter s, while all three regularization modules share drop parameter p.Parameter p controls how many features are dropped, and s controls the dropped block size.
  • Modified RandAugment: Modified RandAugment expands the transformation search space from 14 to 21 operations and adds regularization-oriented transformations.The added operations include Sample_pairing, Gauss_noise, and Cutout.
  • Modified RandAugment: Modified RandAugment introduces a shared execution probability P and applies transformations from both color and shape categories.Random cropping then produces a 224 x 224 training image.

E. Multi-weighted New Loss Method

The Multi-weighted New Loss method extends class-balanced weighting to address both class imbalance and unequal classification difficulty, while correcting excessive losses from outliers.

  • Multi-weighted New Loss: MWNL modifies class-balanced weighting with hyperparameter α, extending weighting strength beyond the conventional range when α > 1.The method combines this weighting with focal loss to form MWLfocal.
  • Multi-weighted New Loss: MWLfocal adds class-specific coefficient C_y to strengthen training for selected categories such as melanoma.For melanoma, C_MEL can exceed 1 while other class coefficients remain 1.
  • Outlier correction: The correction term in MWNL limits the influence of outliers whose predicted probability approaches zero and would otherwise produce unbounded loss.The threshold T controls the limiting operation, with T set to 0.1 in subsequent experiments.
  • Outlier correction: MWNL is introduced by modifying MWLfocal to reduce outlier interference during network optimization.The paper presents this correction as a response to the divergence of the focal-loss formulation for difficult outliers.

F. Cumulative Learning Strategy

The cumulative learning strategy gradually moves training from the original imbalanced distribution toward rebalancing, preserving early feature learning while reducing later class imbalance.

  • Cumulative Learning Strategy: CLS updates MWNL so class-weighting factors change progressively during training rather than switching abruptly.The strategy introduces epoch-dependent β, which ranges from 0 to α.
  • Cumulative Learning Strategy: CLS first trains on originally imbalanced data, then gradually transitions to rebalancing as training proceeds.The initial stage supports feature learning before rebalancing is emphasized.
  • Cumulative Learning Strategy: The rebalancing phase uses a decreasing learning rate so deep-feature weights undergo only minor changes during the later transition.The paper describes this as transferring the network toward rebalanced classification while retaining learned features.
  • Efficiency: The paper reports additional training time for the proposed strategy as almost half that of BBN.This comparison concerns training overhead rather than classification accuracy.

G. Training and Evaluation Strategies

The study fine-tunes pretrained DCNNs with image augmentation and multi-crop evaluation, then compares model capacity and regularization across datasets.

  • Training: Models are initialized with ImageNet-pretrained weights, fine-tuned on training sets, augmented, and randomly cropped to 224 x 224 inputs.The proposed data-enhancement strategy is applied before cropping.
  • Evaluation: Evaluation averages predictions from 16 evenly spaced crops of each test image.The authors report little improvement beyond 16 crops.
  • Model comparison: RegNetY-3.2G achieves the best ISIC 2018 BACC of 0.858, 0.005 above the best result from the compared alternatives.The comparison is based on the ISIC 2018 challenge test set.
  • Model comparison: The best network capacity varies with dataset size: RegNetY-8.0G for ISIC 2019, RegNetY-1.6G for ISIC 2017, and RegNetY-800M for 7-PT.The four dataset-specific best DCNNs are subsequently selected as baselines.
  • Regularization: Adding DropOut and DropBlock to RegNetY-3.2G yields an ISIC 2018 BACC of 0.860 at p=0.1 and s=5, 0.002 above the original model.Further increases in p reduce BACC below the original RegNetY-3.2G value.

B. Results on Modified RandAugment

Modified RandAugment improves skin-lesion classification by balancing color and shape transformations and selecting an appropriate transformation probability.

  • Execution probability: At N=2*^, BACC rises from 0.832 at P=0.1 to 0.860 at P=0.7.The results show that execution probability substantially affects performance.
  • Execution probability: The best execution probability is P=0.7 for N=1 or 2, but P=0.3 for N=4.The authors associate low probabilities with insufficient diversity and high probabilities with excessive changes to original characteristics.
  • Method comparison: Both RandAugment and Modified RandAugment outperform general augmentation across four test sets, with Modified RandAugment achieving higher BACC than RandAugment.General augmentation includes flipping, color changes, and random cropping.

C. Results on Multi-weighted New Loss

The MWNL experiments evaluate weighted-loss training across multiple dermoscopic datasets and class-sensitive operating points. Results indicate that MWLfocal can trade a small reduction in BACC for a substantial increase in melanoma sensitivity as CMEL rises.

  • The reported comparisons cover ISIC 2018, ISIC 2019, ISIC 2017, and 7-PT dermoscopic image classification datasets.
  • MWLfocal at CMEL=1.0 reaches a maximal BACC of 0.864, while melanoma detection sensitivity is 0.772.
  • At CMEL=2.0, BACC decreases to 0.848 while melanoma detection sensitivity increases to 0.871.
  • Table VIII evaluates MWNL with MWLfocal against standard training and state-of-the-art methods using RegNetY-##-Drop and Modified RandAugment.

D. Results on Our Cumulative Learning Strategy

The cumulative learning strategy is compared with BBN and DRW across four dermoscopic datasets and improves performance on the larger imbalanced datasets. Its effectiveness is limited on the smallest datasets, while the overall single-model approach achieves strong challenge results with substantially lower inference time than a large ensemble.

  • Results on Our Cumulative Learning Strategy: On ISIC 2018 and ISIC 2019, CLS outperforms other training strategies, and MWNL-CLS achieves the best performance.
  • Results on Our Cumulative Learning Strategy: On 7-PT and ISIC 2017, CLS performs poorly, with BACC dropping significantly on 7-PT, which has the fewest samples.
  • Results on Our Cumulative Learning Strategy: BBN is slightly better than DRW and CLS on 7-PT and ISIC 2017, but its training time is almost twice as long.
  • Comparison with Other Methods in the Challenge: MWNL-CLS ranks third overall and first among single-model methods without additional data on the ISIC 2018 and ISIC 2019 live leaderboards.
  • Comparison with Other Methods in the Challenge: MWNL surpasses all methods on the ISIC 2017 legacy leaderboard and outperforms published methods on 7-PT using only dermoscopic images.
  • Comparison with Other Methods in the Challenge: The proposed method requires 0.05 seconds per inference, compared with 13.9 seconds for an ensemble integrating 90 DCNNs.
  • Conclusion: The method combines modified DCNNs, DropOut, DropBlock, Modified RandAugment, MWNL, and end-to-end CLS in a single-model strategy.
Loading 2102.01284v2…