Source-linked AI summary
Monkeypox virus detection using pre-trained deep learning-based approaches
Chiranjibi Sitaula, Tej Bahadur Shahi
TL;DR
Monkeypox detection research has been limited, including reliance on a single pre-trained model and insufficient interpretability. This paper compares and fine-tunes 13 pre-trained deep-learning models, ensembles the best performers using probabilistic majority voting, and reports the highest performance with the ensemble. The dataset is comparatively small, limiting the scope of the evaluation.
Problem
Monkeypox detection research is limited, with prior work relying on VGG-16, limited binary-classification performance, and insufficient interpretability.
Method
The paper fine-tunes 13 pre-trained deep-learning models using a common approach, evaluates them over five folds, and ensembles the best-performing models through probabilistic majority voting.
Results
The ensemble achieves Precision of 85.44%, Recall of 85.47%, F1-score of 85.40%, and Accuracy of 87.13%, the highest reported performance.
Takeaways & Limitations
Comparing and ensembling pre-trained models provides the paper’s reported overall performance improvement for Monkeypox virus detection.
Takeaways & Limitations
The dataset is comparatively small, and adding more data could improve performance.
Abstract
from arXiv · showhide
Monkeypox virus is emerging slowly with the decline of COVID-19 virus infections around the world. People are afraid of it, thinking that it would appear as a pandemic like COVID-19. As such, it is crucial to detect them earlier before widespread community transmission. AI-based detection could help identify them at the early stage. In this paper, we aim to compare 13 different pre-trained deep learning (DL) models for the Monkeypox virus detection. For this, we initially fine-tune them with the addition of universal custom layers for all of them and analyse the results using four well-established measures: Precision, Recall, F1-score, and Accuracy. After the identification of the best-performing DL models, we ensemble them to improve the overall performance using a majority voting over the probabilistic outputs obtained from them. We perform our experiments on a publicly available dataset, which results in average Precision, Recall, F1-score, and Accuracy of 85.44\%, 85.47\%, 85.40\%, and 87.13\%, respectively with the help of our proposed ensemble approach. These encouraging results, which outperform the state-of-the-art methods, suggest that the proposed approach is applicable to health practitioners for mass screening.
1 Introduction
Monkeypox detection research has used deep learning, but prior work was limited in model coverage, classification performance, and interpretability. This paper compares 13 pre-trained models, evaluates them consistently, ensembles the best performers, and examines explainability.
- Motivation: Monkeypox has spread beyond its historically predominant African region, while diagnosis commonly relies on PCR or skin-lesion testing.The passage reports more than 50 countries and 3,413 confirmed cases with one death.
- Related work: Prior virus- and skin-disease studies applied transfer learning and deep learning models to medical image classification.Examples include COVID-19 detection with ResNet-50 and skin-disease classification using CNNs and VGG-16.
- Research gap: Existing Monkeypox work was limited to binary classification, VGG-16, and insufficient interpretability, leaving model selection and combinations underexplored.The authors identify these as three main limitations of the prior approach.
- Approach: The paper fine-tunes 13 pre-trained models using a common architecture and compares them using averaged Precision, Recall, F1-score, and Accuracy over five folds.This design supports consistent evaluation across candidate models.
- Contributions: The best-performing models are ensembled to improve overall performance, with comparisons against state-of-the-art methods and explainability using Grad-CAM and LIME.The stated contributions include ensemble learning, comparison with existing methods, and visual explanations.
2.1 Dataset
The study uses a publicly available Monkeypox image dataset and selects images from its augmented-data folder for model training and evaluation.
- Dataset: The experiment uses a publicly available Monkeypox image dataset containing folders with and without image augmentations.The dataset is organized into different sub-folders by augmentation status.
- Dataset: The study uses images from the augmented folder because deep learning models are described as benefiting from augmented images when learning meaningful information.Table 1 reports the number of augmented images in each category.
2.2 Evaluation metrics
The paper evaluates detection models with Precision, Recall, F1-score, and Accuracy, using standard confusion-matrix quantities and metric definitions.
- Metrics: The evaluation uses four widely used metrics: Precision, Recall, F1-score, and Accuracy.The paper references separate equations for each metric.
- Metrics: Accuracy is computed as (TP + TN) divided by (TP + TN + FP + FN).TP and TN denote true positives and true negatives, while FP and FN denote false positives and false negatives.
- Metrics: F1-score is computed from Precision and Recall as 2 × P × R divided by P + R.P, R, and F denote Precision, Recall, and F1-score, respectively.
2.3 Pre-trained DL models
The study uses transfer learning with 13 pre-trained DL models, applying the same customization across models. The architectures span heavyweight and lightweight CNN designs with different convolution, pooling, residual, and scaling strategies.
- Transfer learning is used because pre-trained ImageNet models can transfer learned knowledge to small domain-specific datasets with limited expert-labelled data.
- The study compares 13 pre-trained DL models, ranging from heavyweight VGG-16, InceptionV3, and Xception to lightweight MobileNet and EfficientNet models.
- VGG: VGG-16 contains 13 convolution, 5 max-pooling, and 3 dense layers, while VGG-19 extends this design with 16 convolution layers.
- ResNet: ResNet-50 and ResNet-101 use residual blocks and skip connections to address vanishing gradients in very deep convolutional networks.
- Inception and InceptionResNet: Inception-v3 extracts features at different scales using four parallel convolutions with different kernel sizes, while InceptionResNetV2 combines Inception architecture with residual connections.
- Xception: Xception improves convolution efficiency with modified depth-wise separable convolution, using point-wise convolution followed by channelwise spatial convolution.
- MobileNet: MobileNetV2 extends MobileNet with expand, residual, and projection layers alongside depth-wise convolution layers in bottleneck residual blocks.
- DenseNet: DenseNet connects each layer to all forward layers through dense blocks, and the study uses DenseNet-121 with 120 convolution and 4 average-pooling layers.
2.4 Implementation
The implementation resizes images, applies online augmentation, and trains models with Adam under a five-fold cross-validation design. Each fold uses a 70/30 train/test split, and reported results are averaged.
- Images are resized to 150*150 before training, following the cited prior recommendation.
- Online augmentation applies rescaling, rotations, shifts, shearing, zooming, and channel shifting to the images.
- The models use the Adam optimizer with batch size 16 and an initially specified learning rate.
- Five random folds use a 70/30 train/test ratio, with average performance reported across folds.
2.5 Ensemble approach
The ensemble combines probabilistic outputs from selected fine-tuned models using majority voting. The described example combines Xception and DenseNet-169, with the voting block performing max-voting.
- The ensemble extracts probabilistic output values from each fine-tuned pre-trained model and applies majority voting.
- The ensemble models are fine-tuned to learn optimal features during training and testing, as illustrated by the accompanying plots.
- Xception and DenseNet-169 are selected as the two best-performing fine-tuned models based on empirical evaluation.
- The voting block in the ensemble diagram represents max-voting between the Xception and DenseNet-169 models.
- For an input image I, Xception produces vector X and DenseNet-169 produces vector D, each sized by the number of classes.
3 Results and discussion
The study compares pre-trained deep learning models using five-fold averaged metrics, then evaluates model combinations and explainability. The ensemble achieves the strongest reported performance, while visual analyses examine discriminative regions and class-wise errors.
- Comparative performance: 87.13% Accuracy is achieved by the proposed ensemble, alongside 85.44% Precision, 85.47% Recall, and 85.40% F1-score.These results are averaged over five folds, and the ensemble exceeds Xception by 0.43, 0.33, and 0.38 percentage points for Precision, Recall, and F1-score.
- Comparative performance: 85.01% Precision, 85.14% Recall, 85.02% F1-score, and 86.51% Accuracy make Xception the best among the 13 pre-trained models.Xception is described as second-best among all contenders but best among the pre-trained models.
- Model tuning: Common custom layers improve overall performance across the evaluated models, with Accuracy increasing from 82.22% to 86.06%.Reported increases also reach 80.18% to 84.07% for Precision, 79.17% to 83.74% for Recall, and 70.01% to 83.83% for F1-score.
- Model combination: The top five models are combined for decision fusion, with Xception and DenseNet-169 providing the best-performing two-model combination.The probabilistic outputs of the selected models are fused through majority voting.
- Explainability: Grad-CAM and LIME visualisations identify discriminative regions for Xception-based classification, including potentially virus-infected areas.Grad-CAM highlights regions, while LIME encircles potentially infected superpixels.
- Class-wise study: Across five confusion matrices, chickenpox and normal images are highly discriminated, with all 66 chickenpox and 111 normal test images recognised correctly.Measles and monkeypox remain less perfectly separated, including errors in fold 1.
4 Conclusion and future works
The paper compares 13 pre-trained deep learning models through transfer learning, selects strong models for ensembling, and reports the ensemble's highest detection performance. It also identifies dataset size and memory constraints as important future-work boundaries.
- Conclusion: The proposed ensemble achieves 85.44% Precision, 85.47% Recall, 85.40% F1-score, and 87.13% Accuracy for Monkeypox detection.Xception is the second-best overall method with 85.01% Precision, 85.14% Recall, 85.02% F1-score, and 86.51% Accuracy.
- Future works: The comparatively smaller dataset may limit performance, so adding more data could improve results further.This is identified as the first major limitation.
- Future works: Reliance on pre-trained deep learning models may hinder deployment in memory-constrained settings, motivating lightweight model design.This is identified as the second major limitation and future-work direction.
5 Declarations
The declarations state that the study uses publicly available data, requires no ethical approval, reports no relevant competing interests, and received no funding.
- Declarations: No ethical approval is required because the data used in the work are publicly available.
- Declarations: The authors declare no competing interests relevant to the article's content.
- Declarations: No funding was received for conducting the study.
5.8 Authors’ information
The authors are affiliated with Monash University, Central Queensland University, and Tribhuvan University in Australia and Nepal.
- Authors’ information: C. Sitaula is affiliated with the Department of Electrical and Computer Systems Engineering at Monash University.
- Authors’ information: TB. Shahi is affiliated with Central Queensland University and Tribhuvan University's Central Department of Computer Science and IT.