A Comparative Evaluation of Pre-trained Convolutional Neural Networks for Melanoma Detection

Clinical motivation

• More than 50% growth in melanoma cases is projected between 2020 and 2040, with over 500,000 new cases and approximately 100,000 annual deaths expected worldwide.Effective public-health strategies, including early detection, could prevent a substantial proportion of these cases.Source: §1 · p. 2
• Approximately 99% of patients diagnosed with early-stage melanoma survive at least five years, whereas naked-eye diagnosis achieves only around 60% accuracy.Dermoscopy improves visualization and diagnostic accuracy, but its effectiveness depends strongly on examiner expertise.Source: §1 · p. 2

Study rationale

• The study addresses inconsistent prior evaluations by comparing pre-trained CNN architectures across imaging modalities under identical training and hyperparameter configurations.This controlled design isolates architectural effects and examines how modality influences performance and generalization.Source: §1 · p. 3

Study scope

• Five pre-trained CNN architectures are evaluated on three datasets: two dermatoscopic and one histopathological, enabling comparisons across datasets and modalities.The analysis investigates model consistency, robustness, generalization, and statistically significant performance differences.Source: §1 · p. 4

Statistical evaluation

• Friedman and Nemenyi hypothesis tests are applied to determine whether performance differences among architectures are statistically significant.The study systematically compares CNNs used as feature extractors on dermoscopic and histopathological data.Source: §1 · p. 4

Review scope and selection

• 27 of 278 retrieved articles met the selection protocol and inclusion criteria for detailed analysis, providing a comparative baseline for the current research.The review searched IEEE Xplore and Expert Systems with Applications using predefined CNN, medical-image, deep-learning, and skin-lesion terms, then screened studies by healthcare relevance, English language, CNN use, access conditions, and topic.Source: §2 · pp. 5–6

Recurring methodological practices

• HAM10000 was the most frequently used dataset, followed by ISIC editions, while private datasets limited reproducibility and direct comparison; hold-out splitting was predominant.Common splits included 80% training and 20% testing or 80% training, 10% validation, and 10% testing.Source: §2 · pp. 8–9

Effective strategies

• Transfer learning, attention and feature fusion, and augmentation—including GAN-based generation—were recurring strategies associated with strong performance and improved representation or robustness.The review reports accuracy above 98% on HAM10000 for cited transfer-learning studies and identifies ensemble methods as outperforming individual architectures in imbalanced classification scenarios.Source: §2 · p. 16

Best practices

• Ensemble strategies and interpretability tools such as Grad-CAM and LIME emerged as best practices for improving performance assessment and supporting clinical trust.The reviewed examples include SynthraXCoreNet and Sonune et al.’s soft-voting ensemble, alongside qualitative or quantitative explainability components.Source: §2 · p. 16

Limitations

• Inconsistent metrics and data splits, imbalanced public datasets, and limited external or prospective validation hindered cross-study comparison and restricted conclusions about real-world clinical applicability.The review specifically links imbalanced datasets to weak minority-class recall and notes recurring absence of external validation in studies including Saeed et al. (2025) and Adeel Ajmal Khan et al. (2026).Source: §2 · p. 16

Classification methods

• Prior systems report 99.14% and 98.62% accuracy on balanced binary melanoma-versus-benign datasets using an 80/20 hold-out split, while Bayesian hyperparameter search adds non-negligible training overhead.Sethanan et al. report over 99.4% accuracy on aggregated data using an 80/20 split and 95.078% through a chatbot, with a computationally expensive six-CNN ensemble.Source: §2 · pp. 17, 21

Segmentation methods

• Segmentation studies improve lesion boundaries and contextual modeling with attention, cascaded context, hybrid CNN-ViT, and transformer architectures, but often trade accuracy for parameter count, training cost, or deployment practicality.Examples include AS-Net’s 88.07% Dice on ISIC2017, EM-Net’s 94.03% Dice on PH2, and AMST-Net’s 88.53% Dice on ISIC2018, alongside reported computational or deployment constraints.Source: §2 · pp. 17, 18, 22

Efficiency and feature engineering

• Lightweight and handcrafted-feature approaches reduce computational demands or improve benchmark scores, but may impose accuracy ceilings, depend on manual processing, or lack validation beyond small challenge datasets.DSEUNet uses 23.306K parameters and 9.497M FLOPs but trails TransUnet, while contourlet-based segmentation gains 12% Jaccard yet remains dependent on handcrafted features and manual postprocessing.Source: §2 · pp. 19–21

Limitations of prior work

• Across the literature, reported gains are qualified by dataset imbalance, limited cross-dataset testing, restricted skin-tone representation, resolution constraints, and weak comparability with challenge or clinical benchmarks.Examples include PH2 results based on only eight melanoma test samples, validation confined to related ISIC datasets, and augmentation experiments limited to 256×256 images.Source: §2 · pp. 18–20

Cross-domain methods

• Methods developed for other cancers and imaging modalities demonstrate transferable ideas such as domain knowledge and multimodal fusion, but their direct relevance to dermoscopic skin-lesion classification is limited.DKNet targets nasopharyngeal-carcinoma MRI/CT segmentation, whereas M2CR targets primary liver-cancer CECT diagnosis rather than dermoscopic analysis.Source: §2 · pp. 22–23

Study contribution

• This study establishes a unified and reproducible framework for controlled skin-cancer classification comparisons using standardized preprocessing, dataset splits, class balancing, and multiple performance metrics.The protocol evaluates accuracy, precision, recall, F1-score, and AUC across multiple CNN architectures under identical conditions.Source: §2 · p. 23

3.1. Datasets

• Three public datasets combined dermoscopic and histopathological images in a standardized binary task distinguishing melanoma from benign lesions.HAM10000 and ISIC 2018 are dermoscopic; CR-AI4SkIN is histopathological, enabling robustness evaluation across imaging modalities.Source: §3.1 · pp. 23–24

3.1.1. HAM10000

• HAM10000 contains 10,015 dermoscopic images across seven diagnostic classes, with melanocytic nevi dominant and substantial class imbalance.The study grouped mel as melanoma and akiec, bcc, bkl, df, nv, and vasc as non-melanoma.Source: §3.1.1 · p. 24
• HAM10000’s varied lighting, devices, color, contrast, and resolution support evaluating model robustness and generalization in dermoscopic classification.Source: §3.1.1 · p. 24

3.1.2. ISIC 2018

• ISIC 2018’s Training Ground Truth subset contains 10,015 dermoscopic images across seven diagnostic categories and is imbalanced toward melanocytic nevi.Its original categories were reorganized so mel represented melanoma and nv, bcc, akiec, bkl, df, and vasc represented non-melanoma.Source: §3.1.2 · pp. 24–25
• ISIC 2018 includes acquisition variation in clinical settings, illumination, color, resolution, and lesion appearance, supporting robustness and generalization assessment.It is also widely used as an international benchmark for automated skin-lesion classification.Source: §3.1.2 · pp. 24–25

3.1.3. CR-AI4SkIN

• CR-AI4SkIN comprises histopathological patches from 215 patients, with multiple tissue regions and morphological structures per whole-slide case.This patch-based organization increases within-case variability and reflects realistic histopathological analysis conditions.Source: §3.1.3 · p. 25
• CR-AI4SkIN used dermatopathologist expert ground-truth annotations for training and evaluation, excluding majority-vote and individual non-expert labels.Its cellular structures, staining patterns, and complex tissue morphology provide a more detailed modality for assessing cross-modality generalization.Source: §3.1.3 · p. 25

Input standardization

• 224×224 RGB images were normalized for most CNNs, while InceptionV3 used its specified 299×299 input resolution.This standardization supported consistent comparison across datasets acquired under different imaging conditions while preserving lesion detail.Source: §3.2 · p. 26

Data augmentation

• Augmentation generated synthetic samples for underrepresented classes until target class sizes were reached, using flips, intensity changes, rotations, affine transformations, blur, and resized crops.The Albumentations pipeline was designed to increase variability, reduce overfitting, and strengthen generalization.Source: §3.2 · p. 26

Class balancing and labeling

• 3000 samples per class were used for HAM10000 and 1800 per class for ISIC 2018, with melanoma labeled malignant and all other lesions benign.The resulting balanced datasets contained 6000 HAM10000 images and 3600 ISIC 2018 images.Source: §3.2 · p. 26

Dataset splitting

• HAM10000 and ISIC 2018 used 70%/15%/15% training, validation, and testing splits, whereas CR-AI4SkIN followed its authors’ patient allocation of 150/23/43.Patient-level separation in CR-AI4SkIN prevented data leakage and supported consistent evaluation.Source: §3.2 · p. 26

Dataset characteristics

• CR-AI4SkIN contained approximately 60 to over 1,000 image crops per patient, reflecting substantial variation in data volume across analyzed patches.The number of images per patient was not standardized because it depended on the specific area analyzed.Source: §3.2 · p. 27

3.3. Pre-trained Models

• Five ImageNet-pretrained CNNs generate feature embeddings for supervised metric learning: ResNet50, MobileNet, VGG16, VGG19, and InceptionV3.Embeddings are classified with k-NN, separating feature extraction from classification and assessing representation separability without additional trainable parameters.Source: §3.3 · pp. 27–28
• A standardized protocol fixes hyperparameters, projection-head structure, and training strategy so performance differences primarily reflect backbone architecture.The models use fixed learning rate, batch size, optimizer, loss function, and embedding dimensionality.Source: §3.3 · p. 28

3.3.1. ResNet50 Architecture

• ResNet50 uses skip connections across 50 convolutional layers to improve gradient propagation, convergence stability, and hierarchical feature representation (Py-Torch Foundation, 2025).Residual learning supports deeper-network optimization and makes ResNet50 a baseline for comparative image-classification evaluation.Source: §3.3.1 · pp. 28–29, Figure 1 · p. 28

3.3.2. MobileNet Architecture

• MobileNet uses depthwise separable convolutions plus width and resolution multipliers to reduce computation and parameters while balancing accuracy and efficiency.This design supports deployment on resource-constrained devices while preserving discriminative capacity.Source: §3.3.2 · pp. 29–30, Figure 2 · p. 29

3.3.3. VGG16 and VGG19 Architectures

• VGG16 and VGG19 stack small 3 × 3 convolutions with ReLU and pooling, while VGG19 adds depth for more detailed hierarchical feature extraction.VGG16 has 16 weight layers, whereas VGG19 extends the same design to 19 layers for fine-grained visual discrimination.Source: §3.3.3 · p. 30

3.3.4. InceptionV3 Architecture

• InceptionV3 combines different kernel sizes in parallel branches and factorizes convolutions to capture multi-scale features efficiently.Dimensionality reduction preserves representational capacity, supporting robust feature extraction for automated diagnostic systems (Admass et al., 2024).Source: §3.3.4 · p. 30

Backbone configuration

• ImageNet-pre-trained ResNet50, MobileNet, VGG16, VGG19, and InceptionV3 backbones were frozen, with include_top=False and Global Average Pooling.Only the embedding head was optimized during training.Source: §3.4 · p. 31

Embedding head

• The embedding head used dropout rates of 0.5 and 0.3, fully connected layers with 256 and 128 units, ReLU activation, batch normalization, and L2-normalized embeddings.The L2 normalization constrained the feature space.Source: §3.4 · p. 31

Metric learning

• Triplet Loss used a 0.5 margin and Adam optimization with a 1 × 10−4 learning rate to separate anchor-positive and anchor-negative pairs.The loss used squared Euclidean distances and enforced a minimum margin between pair types.Source: §3.4 · p. 31

Triplet construction

• Triplets were generated dynamically by sampling same-class anchors and positives and different-class negatives, with optional hard-negative mining based on embedding distance.Hard-negative mining selected the closest negative among a candidate subset when enabled.Source: §3.4 · p. 31

Training procedure

• Training used rotations up to 30 degrees, shifts, shear, zoom, horizontal flips, reflective padding, and early stopping for up to 30 epochs with patience 15.The best weights were restored according to validation loss.Source: §3.4 · p. 31

Experimental protocol

• 15 models were evaluated: five backbone architectures trained independently on HAM10000, ISIC 2018, and CR-AI4SkIN under a consistent protocol.Each dataset used five distinct models, enabling comparable experiments across all data sources.Source: §3.4 · p. 31

• Evaluation combined classification metrics with embedding interpretation, training stability, and statistical model comparisons.Together, these analyses assessed both discriminative capacity and result consistency.Source: §3.5 · pp. 31–32
• Metrics were derived from confusion-matrix entries comprising true positives, true negatives, false positives, and false negatives.The matrix compares predicted and true labels, providing the basis for subsequent measures.Source: §3.5 · p. 32
• Accuracy measured overall correct-prediction proportion, but was insufficient alone when medical datasets were class-imbalanced.Accuracy was computed from TP, TN, FP, and FN.Source: §3.5 · p. 32
• Class-wise precision, recall, and F1-score characterized malignant and non-malignant classification, emphasizing malignant-lesion sensitivity and balanced performance under class imbalance.Precision measured true positives among predicted class members, recall measured correctly identified actual samples, and F1-score was their harmonic mean.Source: §3.5 · p. 33
• ROC curves and AUC evaluated discrimination across decision thresholds, with values closer to 1 indicating better separation of malignant from non-malignant lesions (Li, 2024).TPR denotes recall, FPR denotes false-positive rate, and t denotes the decision threshold.Source: §3.5 · p. 33

Preprocessing and balancing

• HAM10000 and ISIC 2018 images were resized per architecture, balanced through augmentation, and processed by ImageNet-pretrained CNNs with frozen backbones.HAM10000 formed 3000 benign and 3000 malignant samples, while ISIC 2018 formed 1800 benign and 1800 malignant samples.Source: §3.6 · pp. 34–35

CR-AI4SkIN workflow

• CR-AI4SkIN used precomputed PKL embeddings directly, without image preprocessing, new feature extraction, augmentation, or metric-learning optimization.Its patches were mapped structurally by clinical case, and classification used expert-defined ground-truth labels rather than crowdsourced annotations.Source: §3.6 · pp. 34–36

Preprocessing and balancing

• Class imbalance was addressed with Albumentations sampling and augmentation, using flips, intensity changes, rotations, affine transformations, blur, and random resized cropping.The transformations expanded minority classes to predefined targets and increased intra- and inter-class variability.Source: §3.6 · p. 34

Metric learning

• Triplet Loss refined HAM10000 and ISIC 2018 embeddings by pulling same-class samples together and separating different-class samples by a margin.Only projection layers were trained, organizing the embedding space for class similarity while maximizing inter-class separation and reducing intra-class variability.Source: §3.6 · pp. 35–36

Classification and evaluation

• KNN classified the architecture embeddings with k=5 and Euclidean distance, while accuracy and macro F1-score evaluated binary benign-versus-malignant predictions.KNN was selected to assess embedding quality through feature similarity without additional parameter learning.Source: §3.6 · p. 36

Model performance

• ResNet50 led all datasets, achieving 83.8% accuracy/84.0% macro F1-score on HAM10000, 76.8%/77.0% on ISIC 2018, and 83.0%/84.0% on CR-AI4SkIN.Table 3 reports accuracy and average F1-score across the three datasets.Source: Table 3 · p. 37, §4 · p. 37
• All models performed approximately 7%–10% worse on ISIC 2018, while MobileNet ranked second in most experiments except where VGG16 and VGG19 achieved higher accuracy.The ISIC 2018 decline was attributed to greater lesion variability, acquisition differences, or class-distribution differences; MobileNet remained notable for lightweight computational efficiency.Source: §4 · p. 38

Metric behavior

• Accuracy and macro F1-score remained similar across models, while recall generally detected many malignant lesions but decreased for VGG16 and VGG19 in borderline cases.These errors reflect difficulty distinguishing benign and malignant lesions with similar visual characteristics.Source: §4 · pp. 38–39

Clinical and cross-domain robustness

• ResNet50 showed greater stability across datasets, whereas VGG architectures and MobileNet exhibited lower reliability under domain shifts; ResNet50 and VGG16 also showed better cross-modality stability than VGG19 and MobileNet.Modality shifts affected architectures differently across dermatoscopic and histopathological scenarios.Source: §4 · p. 39

Statistical analysis

• Friedman tests found significant accuracy differences for every dataset: p<0.0001 for ISIC 2018 and CR-AI4SkIN, and p=0.0077 for HAM10000.The test evaluated multiple architectures under the same experimental conditions.Source: §4 · p. 39, Table 5 · p. 40
• At 95% confidence, Nemenyi comparisons found no significant HAM10000 pair (lowest p=0.487), while ISIC 2018 had one significant pair: ResNet50 × InceptionV3 (p=0.017).Source: §4 · p. 40

Conclusion

• ResNet50 consistently achieved the best performance across HAM10000, ISIC 2018, and CR-AI4SkIN, demonstrating greater stability across dermoscopic and histopathological modalities.On HAM10000, architectures performed statistically similarly; on ISIC 2018, only ResNet50 and InceptionV3 differed significantly (p=0.017), whereas all pairwise comparisons were significant on CR-AI4SkIN.Source: §5 · p. 42
• Performance differences were strongly modulated by dataset characteristics, with histopathological complexity producing more distinct architecture responses than dermoscopic datasets.The transition from statistically homogeneous behavior on HAM10000 to differentiated performance groups on CR-AI4SkIN shows that modality shifts affect architectures differently.Source: §5 · p. 42
• The study’s contributions include comparing architectures across modalities, examining modality-shift effects on CNN generalization, and clarifying implications for computer-aided melanoma detection.Architectures that behave similarly in one visual domain may not preserve that behavior in another, so conclusions from a single dataset may not represent diverse scenarios.Source: §5 · pp. 42–43
• Model selection should consider stability, consistency, and adaptability across datasets rather than isolated accuracy, requiring comprehensive multimodal validation for safer and more reliable melanoma diagnosis.ResNet50’s superior performance and greater statistical separation under domain shift reinforce the need for validation across distinct clinical contexts.Source: §5 · p. 43
• Future work should investigate attention, transformer, augmentation, domain adaptation, ensembles, and broader datasets to improve generalization across imaging modalities and clinical variability.These approaches may mitigate modality-shift performance drops, particularly for VGG19 and MobileNet, while additional class diversity and acquisition variability could clarify real-world generalization boundaries.Source: §5 · p. 43

• Wagner M. Schmitz led conceptualization, formal analysis, methodology, software, investigation, and original-draft writing.Source: CRediT authorship contribution statement · p. 43
• Marco A. C. Barbosa, Thiago M. Amaral, and D. Casanova contributed to review and editing and methodology, while D. Casanova also supervised; J. T. Oliva additionally investigated, validated, and supervised.Source: CRediT authorship contribution statement · p. 44

Declaration of generative AI use

• Generative AI assisted with scientific-language adaptation, translation and synthesis of author-drafted text, and related-literature searches, but did not independently generate content or draw conclusions.The authors retained their own critical reading and interpretation of the reviewed works.Source: Declaration of generative AI use · p. 44