PaperlayerView exact revision on arXiv

Source-linked AI summary

Skin disease diagnosis with deep learning: a review

Hongfeng Li, Yini Pan, Jie Zhao, Li Zhang

arXiv:2011.05627v2eess.IVcs.CVcs.LG

TL;DR

Skin disease diagnosis is challenging because it requires varied visual evidence and can depend on subjective clinical judgment. This paper reviews deep learning methods, datasets, architectures, frameworks, metrics, and diagnostic applications, then examines unresolved challenges. Reported deep learning performance is similar to that of dermatologists in the reviewed literature, while the field remains constrained by limited labeled data, class imbalance, and incomplete clinical metadata.

  • Problem

    Skin disease diagnosis requires diverse visual evidence, and specialist judgments can vary with experience.

  • Method

    The paper systematically reviews deep learning concepts, data resources, architectures, frameworks, metrics, diagnostic applications, challenges, and future directions.

  • Results

    Deep learning methods achieve performance similar to dermatologists in the reviewed skin disease diagnosis literature.

  • Takeaways & Limitations

    The review provides a conceptual survey of deep learning for skin disease diagnosis and identifies directions for addressing persistent field challenges.

  • Takeaways & Limitations

    The field remains constrained by limited labeled data, imbalanced datasets, and omission of medical history and clinical metadata.

Abstract

from arXiv · show

Skin cancer is one of the most threatening diseases worldwide. However, diagnosing skin cancer correctly is challenging. Recently, deep learning algorithms have emerged to achieve excellent performance on various tasks. Particularly, they have been applied to the skin disease diagnosis tasks. In this paper, we present a review on deep learning methods and their applications in skin disease diagnosis. We first present a brief introduction to skin diseases and image acquisition methods in dermatology, and list several publicly available skin datasets for training and testing algorithms. Then, we introduce the conception of deep learning and review popular deep learning architectures. Thereafter, popular deep learning frameworks facilitating the implementation of deep learning algorithms and performance evaluation metrics are presented. As an important part of this article, we then review the literature involving deep learning methods for skin disease diagnosis from several aspects according to the specific tasks. Additionally, we discuss the challenges faced in the area and suggest possible future research directions. The major purpose of this article is to provide a conceptual and systematically review of the recent works on skin disease diagnosis with deep learning. Given the popularity of deep learning, there remains great challenges in the area, as well as opportunities that we can explore in the future.

1. Introduction

Skin disease diagnosis is difficult because it requires integrating diverse visual clues and can depend heavily on clinician experience. This review systematically surveys deep learning methods, datasets, architectures, evaluation, applications, challenges, and future directions in the field.

  • Skin cancer diagnosis requires integrating lesion morphology, body-site distribution, color, scaling, and lesion arrangement.
  • Handcrafted features are difficult to adapt across diverse skin diseases, motivating feature learning that automatically extracts effective representations.
  • The paper reviews recent deep learning research across skin diseases, public datasets, architectures, frameworks, metrics, and diagnosis tasks.
  • The review also identifies unresolved challenges and suggests future directions for skin disease diagnosis with deep learning.

2. Skin disease

The section introduces skin structure and major skin cancers, distinguishing common keratinocyte carcinomas from less common but more deadly melanoma.

  • Skin consists of the epidermis, dermis, and hypodermis, with functions including protection, sensation, and thermoregulation.
  • Melanocytes in the epidermis produce melanin, while unusual melanocyte growth causes melanoma.
  • The three major skin cancers are malignant melanoma, squamous cell carcinoma, and basal cell carcinoma.
  • Basal cell carcinoma and squamous cell carcinoma arise from keratinocytes and are more common, whereas melanoma is less common but more deadly.

3. Image acquisition and datasets

Dermatology uses multiple imaging modalities and publicly available datasets to support skin disease diagnosis research. These resources vary in imaging type, scale, resolution, and diagnostic labels.

  • 3.1. Image acquisition: Dermoscopy is a non-invasive magnifying technique that visualizes skin-surface colors and microstructures unavailable to the naked eye.
  • 3.1. Image acquisition: Other acquisition methods include confocal laser scanning microscopy, optical coherence tomography, ultrasound, MRI, spectral analysis, and ordinary camera or smartphone imaging.
  • 3.2. Datasets: High-quality, reliably labeled data are central to training deep neural networks, with clinical, dermoscopy, and pathological images serving as major modalities.
  • 3.2. Datasets: Public datasets include PH2, Dermnet/OLE, ISIC, HAM10000, IAD, and MED-NODE, covering different diseases, image sources, and annotation schemes.

4. Deep learning

Deep learning models learn representations from data through layered transformations, with CNNs especially suited to image-oriented problems. This section introduces neural-network fundamentals, training, and CNN components relevant to skin-disease diagnosis.

  • Foundations: Machine learning models learn data-driven rules from examples and are evaluated on separate validation and testing datasets to assess generalization.Supervised, semi-supervised, and unsupervised learning differ in how training data are used.
  • Foundations: Deep learning uses multiple layers to learn meaningful representations, reducing reliance on manually engineered features.Its growth was supported by improved GPU computation and accessible implementations.
  • Learning paradigms: Limited labeled healthcare data motivates semi-supervised and unsupervised learning approaches alongside conventional supervised training.Healthcare labels are expensive and difficult to obtain at scale.
  • Neural networks: Neural networks combine weighted inputs and biases with nonlinear activation functions, and multilayer transformations produce deep networks.The paper describes weights, biases, hidden layers, and activation functions as core components.
  • Training and prediction: Softmax maps final network activations to class probabilities, while stochastic gradient descent tunes parameters using mini-batches and task-specific losses.Binary and categorical cross-entropy are examples for two-class and multi-class classification.
  • Convolutional neural networks: CNNs preserve spatial relationships with sparse connections and extract image representations through convolution, activation, and pooling operations.Pooling supports translation invariance, while convolutional filters share weights across locations to reduce parameters.

5. Deep learning frameworks

Deep learning frameworks provide high-level building blocks for designing, training, and validating neural networks. The reviewed frameworks differ in interfaces, configurability, scalability, language support, and image-processing performance.

  • Framework role: Deep learning frameworks simplify implementation by providing building blocks for designing, training, and validating large-scale neural networks.They enable convenient implementation of complex models such as CNNs.
  • Major frameworks: TensorFlow supports Python, C++, and R and is described as convenient for creating and experimenting with deep learning architectures.The passage identifies TensorFlow as highly popular while noting that other frameworks are catching up.
  • Major frameworks: Keras offers a high-level Python interface for fast experimentation, trading configurability for ease of understanding and use.It can run on top of TensorFlow, CNTK, and Theano.
  • Major frameworks: PyTorch uses Python, supports GPU tensor computations, and employs dynamic computation graphs for building deep neural networks.It is presented as a major deep learning tool after TensorFlow.
  • Major frameworks: Caffe emphasizes image processing and offers advantages in processing and learning speed, but has weaker support for recurrent networks and language modeling.The passage also notes support for multiple interfaces.
  • Additional frameworks: Sonnet targets complex architectures through reusable Python objects connected to computational TensorFlow graphs, while MXNet emphasizes scalability and parallel computing.MXNet supports multiple languages and imperative or symbolic programming styles.
  • Additional frameworks: The review also notes other frameworks, including CNTK, Gluon, Swift, Chainer, DeepLearning4J, Theano, PaddlePaddle, and ONNX, but does not detail them because of space limitations.The frameworks are described as open source and built on CUDA and cuDNN.

6. Evaluation metrics

The review describes IoU as the primary segmentation metric and lists additional pixel-level indices, while classification uses image-level metrics including accuracy, sensitivity, specificity, AUC, and precision.

  • Segmentation metrics: Intersection-over-Union (IoU), or Jaccard Index, measures overlap between predicted and ground-truth segmented areas.IoU ranges from 0 to 1, with higher values indicating better algorithm performance.
  • Segmentation metrics: Segmentation evaluation additionally uses pixel-level indices derived from true-positive, true-negative, false-positive, and false-negative counts.Pixel values above 128 are treated as positive and values below 128 as negative.
  • Classification metrics: Classification commonly evaluates accuracy, sensitivity, and specificity at the whole-image level rather than the pixel level.Area under the receiver operating characteristic curve and precision are also commonly used.
  • Classification metrics: AUC measures how well a parameter distinguishes between two diverse groups by integrating the true-positive rate with respect to the false-positive rate.

7. Skin disease diagnosis with deep learning

This section reviews deep learning applications in skin disease diagnosis by first describing preprocessing and augmentation, then organizing prior work according to task type.

  • The review introduces common data preprocessing and augmentation methods before surveying deep learning applications in skin disease diagnosis.The literature review is organized according to the type of task.

7.1. Data preprocessing and augmentation

Preprocessing adapts heterogeneous skin-image data to network inputs while trying to limit distortion and information loss; augmentation addresses the scarcity of labeled medical images and overfitting.

  • Data preprocessing: Skin-image resolutions vary across datasets, whereas deep networks commonly require fixed square inputs such as 224 × 224 or 512 × 512.Images therefore need cropping or resizing before network input.
  • Data preprocessing: Direct resizing or cropping can distort objects or lose substantial information, so resizing along the shortest side while preserving aspect ratio is presented as feasible.Images are typically normalized by subtracting the mean and dividing by the standard deviation.
  • Data augmentation: Limited labeled medical data arise from disease rarity, patient privacy, expert-labeling requirements, and the high cost of acquiring medical data.Deep networks generally require large datasets to avoid overfitting and achieve strong performance.
  • Data augmentation: Augmentation methods include geometric and color transformations, kernel filters, image mixing, random erasing, feature-space augmentation, adversarial training, GANs, style transfer, and meta-learning.One example rotated 4,000 dermoscopy images by 0°, 90°, 180°, and 270° to reduce overfitting.

7.2. Applications of deep learning in skin disease diagnosis

The review covers deep learning for lesion segmentation and skin-disease classification, including encoder-decoder, U-net, residual, dense, adversarial, and end-to-end approaches. Reported applications span lesion-boundary extraction, multi-class segmentation, and classification systems, including a CNN that performed on par with dermatologists on clinical images.

  • Segmentation: Segmentation divides images into regions with similar pixel attributes, helping clinicians perceive lesion boundaries while remaining technically challenging.Reliable segmentation affects the success of subsequent image analysis.
  • Segmentation: Early segmentation approaches used fully convolutional encoder-decoder networks, including FCN and SegNet, for skin-lesion segmentation.FCN-based methods were also extended to multi-class segmentation of benign nevi, melanoma, and seborrhoeic keratoses.
  • Segmentation: U-net and its extensions were applied to dermoscopy-based melanoma and skin-lesion segmentation.The review also describes deep fully convolutional-deconvolutional networks and modified U-net architectures.
  • Segmentation: Residual and dense blocks were incorporated into segmentation architectures, while GAN-based networks were used for pigmented and skin-colored lesion segmentation.A mobile-device study using a GAN-based network reported segmentation accuracy of 91.4%.
  • Classification: Skin-disease classification can produce binary, ternary, or n ≥4-category outputs and is described as the final step in a typical CAD workflow.Early systems combined lesion detection and machine-learned features with a deep neural-network classifier.
  • Classification: End-to-end CNNs were developed for melanoma and other skin-disease classification using dermoscopy and non-dermoscopy images.One reported system achieved 93% testing accuracy, while another CNN performed on par with 145 dermatologists on clinical image classification.

8. Discussion

Deep learning has shown promising performance for skin disease diagnosis, but substantial challenges remain before clinical deployment. The review therefore focuses on fundamental barriers and possible solutions.

  • Deep learning methods for skin disease diagnosis have achieved promising performance, but extensive clinical application remains premature.
  • The discussion examines fundamental challenges and their root causes rather than isolated case-specific problems.
  • The review proposes directions for addressing these challenges and improving real-life clinical applicability.

8.1. Challenges

Clinical translation is constrained by limited and imbalanced data, heterogeneous image quality, insufficient demographic and disease diversity, missing clinical metadata, limited explainability, and unclear architecture selection.

  • 8.1.1. Limited labeled skin disease data: Limited labeled datasets make deep-network training vulnerable to overfitting and create demand for larger datasets or methods requiring fewer labels.The largest cited public dataset contains more than 20,000 skin images, while expert labeling remains costly and time-consuming.
  • 8.1.2. Imbalanced skin disease datasets: Imbalanced datasets dominated by benign lesions can bias diagnostic results, even when weighted loss functions penalize false negatives.Obtaining balanced data may be as difficult as creating a large-scale dataset because some positive conditions are infrequent.
  • 8.1.3. Noisy data obtained from heterogeneous sources: Models trained on high-quality DSLR images may lose performance on noisy smartphone images captured under different lighting, distances, or equipment.
  • 8.1.4. Lack of diversity among cases in existing skin disease datasets: Datasets underrepresent dark-skinned individuals and rare diseases, increasing the risk of misdiagnosis for these conditions.The review identifies broader dataset diversity as important for effective diagnosis systems.
  • 8.1.5. Missing of medical history and clinical meta-data of patients: Most systems use images alone despite clinicians also relying on medical history, risk factors, and clinical metadata that can improve diagnostic performance.
  • 8.1.6. Explainability of deep learning methods: Deep neural networks remain difficult to interpret, creating demands for explainability in clinical diagnosis and leaving architecture selection insufficiently established.

8.2. What can we do next?

The review recommends expanding research beyond limited disease categories and image-only inputs while addressing the shortage of labeled data. These directions aim to better reflect clinical diagnosis and support future system development.

  • 8.2. What can we do next?: The field is expected to produce many further studies, but unresolved challenges must be addressed to obtain satisfactory diagnostic performance.
  • 8.2. What can we do next?: Limited labeled data can be addressed through clinician annotation or automated and semi-automated labeling tools.Manual labeling is described as expensive and time-consuming.
  • 8.2. What can we do next?: Future datasets and models should cover broader differential diagnoses instead of only predefined high-frequency skin disease categories.
  • 8. What can we do next?: Clinical systems should incorporate lesion history, individual risk profiles, and global skin assessment alongside dermoscopy or histopathological images.

8.2.4. Fuse handcrafted features with deep networks extracted features

Future work can combine handcrafted clinical or geometric cues with learned deep features and use data-generation and transfer strategies to address limited labeled data. The review presents fusion as a way to extend current deep-learning systems.

  • 8.2.4. Fuse handcrafted features with deep networks extracted features: Handcrafted features such as lesion elevation, evolution, geometry, and color-space information can complement deep features in skin disease diagnosis.
  • 8.2.4. Fuse handcrafted features with deep networks extracted features: Integrating handcrafted-feature extraction with deep-network learning is identified as a promising research direction.
  • 8.2.4. Fuse handcrafted features with deep networks extracted features: GANs can generate realistic synthetic images as additional labeled data when large-scale labeled datasets are unavailable.The review states that models commonly perform better with synthetic additions than with limited original data alone.
  • 8.2.4. Fuse handcrafted features with deep networks extracted features: Transfer learning and domain adaptation have been used to address shortages of large-scale labeled data and improve skin-diagnosis models.

8.2.7. Develop semi-supervised deep learning methods for skin disease diagnosis

The review identifies semi-supervised learning, reinforcement learning, and explainability as directions for addressing limited labels, sequential decisions, and clinical trust in skin disease diagnosis.

  • Semi-supervised learning can leverage abundant unlabeled skin data to alleviate the expense and time required for expert annotation.Large labeled datasets are required for deep learning, whereas unlabeled skin data are easier or cheaper to obtain.
  • Reinforcement learning is relevant to skin disease diagnosis because many medical decision problems are sequential.
  • Explainable deep learning should provide clinically reasonable reasons for predictions rather than only confidence scores.The review points to explanations based on ABCDE criteria or the 7-point skin lesion criteria.

9. Summary

The review systematically surveys recent deep learning applications in skin disease diagnosis, covering technical foundations, datasets, evaluation, applications, challenges, and future directions. It concludes that many systems show comparable or superior performance on experimental datasets, while reliable clinical deployment remains an important concern.

  • The review covers skin disease fundamentals, image acquisition, public datasets, deep learning architectures, frameworks, evaluation metrics, applications, challenges, and future directions.
  • Compared with related reviews, the article provides a systematic survey focused on recent deep learning applications in skin disease diagnosis.It also aims to clarify essential concepts and challenges for future researchers.
  • Numerous deep learning systems achieve comparable or superior diagnosis performance on experimental skin disease datasets, while automated diagnosis still demands reliability for expert and non-expert clinicians.
Loading 2011.05627v2…