Source-linked AI summary
The impact of patient clinical information on automated skin cancer detection
Andre G. C. Pacheco, Renato A. Krohling
TL;DR
Skin-cancer detection systems commonly rely on images while omitting patient clinical information. This paper introduces a smartphone-based clinical-image dataset and a straightforward deep-learning aggregation approach, then compares image-only and combined models. Combining both information sources produced an approximately 7% general improvement in balanced accuracy across models, with statistically significant differences reported between conditions.
Problem
Most skin-cancer detection systems rely on images and do not incorporate patient clinical information used by dermatologists.
Method
The study builds a smartphone-acquired clinical-image dataset with patient information and aggregates both sources using several CNN models.
Results
Combining clinical information with images yielded a general improvement of around 7% in balanced accuracy across models, with significant differences between conditions.
Takeaways & Limitations
Patient clinical information is an important information source that may help skin-cancer detection systems and overcome limited data availability.
Takeaways & Limitations
The approach treats clinical features as a support source, keeps their amount unreduced, and selects cf = 0.8 for the next experiment.
Abstract
from arXiv · showhide
Skin cancer is one of the most common types of cancer around the world. For this reason, over the past years, different approaches have been proposed to assist detect it. Nonetheless, most of them are based only on dermoscopy images and do not take into account the patient clinical information. In this work, first, we present a new dataset that contains clinical images, acquired from smartphones, and patient clinical information of the skin lesions. Next, we introduce a straightforward approach to combine the clinical data and the images using different well-known deep learning models. These models are applied to the presented dataset using only the images and combining them with the patient clinical information. We present a comprehensive study to show the impact of the clinical data on the final predictions. The results obtained by combining both sets of information show a general improvement of around 7% in the balanced accuracy for all models. In addition, the statistical test indicates significant differences between the models with and without considering both data. The improvement achieved shows the potential of using patient clinical information in skin cancer detection and indicates that this piece of information is important to leverage skin cancer detection systems.
1 Introduction
Skin cancer detection remains challenging, especially where dermatologists and dermatoscopes are scarce, while existing systems often rely mainly on images and omit patient clinical information. This work introduces a dataset and aggregation approach to study how clinical information affects deep-learning predictions.
- Skin cancer is highly prevalent, accounting for an estimated one in three diagnosed cancers worldwide.
- Dermatologists combine lesion images with clinical information, but differentiating skin lesions from cancer remains challenging.
- Dermatoscopy can improve diagnostic accuracy, yet its benefit depends on examiner experience and access is limited in many Brazilian countryside cities.
- Most existing skin-cancer CAD systems use images without incorporating patient clinical information, despite its importance for diagnosis.
- The study contributes a smartphone-collected dataset and a deep-learning approach that aggregates clinical images with patient information for comparative evaluation.
2 Material and methods
The study constructs a smartphone-based PAD dataset pairing clinical lesion images with patient information, explores feature patterns, and proposes CNN-based aggregation of both sources for skin-lesion diagnosis.
- Dataset: The PAD dataset contains smartphone-acquired clinical images, diagnoses, and eight patient clinical features for each lesion sample.The data were collected through a smartphone application used by PAD doctors and medical students.
- Dataset: The study uses six lesion categories formed by clustering Bowen’s disease with SCC and Lentigo Maligna with MEL, comprising three cancers and three skin diseases.The categories are BCC, SCC, ACK, SEK, MEL, and NEV.
- Dataset: The dataset is presented as a public resource containing smartphone images together with corresponding clinical information.The dataset is available upon request.
- Clinical features analysis: Clinical-feature exploration examines associations involving pain, bleeding, itching, age, lesion pigmentation, and anatomical region.The analysis identifies feature differences across lesion types, including age distributions and preferred body regions.
- Convolutional Neural Networks: The evaluated CNN architectures include GoogleNet, ResNet50/101, VGGNet, and MobileNet.These networks provide the image-processing backbones for investigating the impact of clinical information.
- Proposed aggregation: The proposed aggregation extracts CNN image features, reduces them with a neural network, concatenates them with clinical data, and sends the combined representation to a classifier.A combination factor controls the amount of image features included while retaining the clinical features.
3 Experiments and results
The experiments evaluate data preparation, feature-combination settings, and CNN performance with images alone versus images combined with patient clinical features. Clinical features generally improve performance, although SCC and BCC remain difficult to distinguish.
- Experimental setup: The experiments use smartphone clinical images, patient features, transfer learning, augmentation, weighted loss, and 5-fold cross-validation across six evaluation metrics.Clinical features are one-hot encoded into 28 values, and the experiments compare image-only and combined-information scenarios.
- Experimental setup: Scenario 1 uses only clinical images, whereas Scenario 2 combines clinical images with patient clinical features.The image feature extractor is shared across scenarios, while Scenario 2 adds a reducer block before classification.
- Sensitivity analysis: For ResNet-50, cf = 0.7 and cf = 0.8 perform best, with no statistical difference between them; cf = 0.5 and cf = 0.9 are slightly worse.The Friedman test indicated significant differences requiring pairwise Wilcoxon comparisons.
- Impact of clinical features: Around 7%: average balanced accuracy improves from Scenario 1 to Scenario 2, with improvement across all metrics and statistically different models.The authors conclude that including clinical features is the best option for this experiment.
- Impact of clinical features: ResNet-50 is around 4% above the other models in Scenario 2 for balanced accuracy, while the models otherwise perform nearly similarly.The detailed ResNet-50 analysis shows improvement for all labels but frequent confusion between SCC and BCC.
- Impact of clinical features: Clinical features improve probability distributions for ACK, MEL, NEV, and SEK, but distributions for SCC and BCC remain almost unchanged.The results therefore indicate that clinical information is not equally effective for every lesion type.
4 Conclusion
The paper presents a smartphone-based clinical-image dataset with patient information and combines both sources in CNN models for skin cancer detection. Clinical features substantially improve the investigated models overall, but do not improve SCC/BCC classification.
- Contributions: The study introduces a dataset of smartphone-acquired clinical images and patient clinical information for skin cancer detection.It also evaluates a straightforward CNN-based approach for combining image and clinical features.
- Findings: Combining clinical features with images substantially improves performance across all investigated CNN models.The authors state that patient clinical information may help address the limited amount of training data.
- Findings: Clinical features do not improve SCC/BCC classification because those lesions have quite similar features.The conclusion identifies this lesion pair as a specific boundary of the observed benefit.