Source-linked AI summary
Soft-Attention Improves Skin Cancer Classification Performance
Soumyya Kanti Datta, Seyed Mohammad Abuzar Hashemi, Sargur N. Srihari, Mingchen Gao
TL;DR
Skin-lesion classification is difficult because malignant and benign diseases look similar, while large dermoscopy images contain artifacts and limited relevant features. The paper evaluates Soft Attention across deep architectures and reports that attention-based models outperform state-of-the-art approaches on HAM10000 and ISIC-2017, including 93.7% precision on HAM10000.
Problem
Skin-lesion classification is difficult because similar-looking diseases, limited annotated data, low inter-class variation, image-property differences, and artifacts complicate feature extraction and diagnosis.
Method
The paper applies Soft Attention during image encoding and compares VGG, ResNet, InceptionResNetv2, and DenseNet models with and without attention for skin-cancer classification.
Results
Soft Attention-based models outperform the reported state-of-the-art approaches on HAM10000 and ISIC-2017; on HAM10000, IRv2+SA achieves 93.7% precision and exceeds the baseline by 4.7%.
Takeaways & Limitations
Soft Attention can internally identify where the model focuses, handle image noise, and boost the performance of the underlying network without external mechanisms such as GradCAM.
Abstract
from arXiv · showhide
In clinical applications, neural networks must focus on and highlight the most important parts of an input image. Soft-Attention mechanism enables a neural network toachieve this goal. This paper investigates the effectiveness of Soft-Attention in deep neural architectures. The central aim of Soft-Attention is to boost the value of important features and suppress the noise-inducing features. We compare the performance of VGG, ResNet, InceptionResNetv2 and DenseNet architectures with and without the Soft-Attention mechanism, while classifying skin lesions. The original network when coupled with Soft-Attention outperforms the baseline[16] by 4.7% while achieving a precision of 93.7% on HAM10000 dataset [25]. Additionally, Soft-Attention coupling improves the sensitivity score by 3.8% compared to baseline[31] and achieves 91.6% on ISIC-2017 dataset [2]. The code is publicly available at github.
1. Introduction
Skin cancer classification matters because the disease is common and early diagnosis affects survival, yet malignant and benign lesions can look similar. Dermoscopy provides high-resolution images, but large images and visual artifacts make relevant-feature extraction difficult.
- Skin cancer is common worldwide, and early diagnosis significantly affects patient survival rates.
- Malignant and benign skin diseases are difficult to distinguish because their appearances are highly similar.
- Large dermoscopy images make it difficult for feature extractors to isolate relevant classification features.
- Skin lesion classification is complicated by limited annotated data, low inter-class variation, visual changes, and artifacts such as hair and veins.
2. Related Work
Prior skin-cancer classification work uses computer-vision pipelines and deep learning, but low inter-class variation and class imbalance remain limiting challenges. This paper proposes soft attention to identify fine-grained lesion-feature variability.
- Soft attention is suggested as a way to identify fine-grained variability in skin-lesion visual features.
- Existing approaches use preprocessing, segmentation, class balancing, cross-validation, augmentation, and varied deep-learning methodologies for skin-lesion classification.
- Low inter-class variance and class imbalance remain limitations for deep-learning models on skin-lesion datasets.
3. Experiment Settings And Method
The proposed method applies soft attention to multiple ImageNet-trained deep neural networks for skin-cancer image classification. The evaluated architectures are ResNet34, ResNet50, Inception ResNet v2, DenseNet201, and VGG16.
- Five ImageNet-trained feature extractors are implemented with soft attention for skin-cancer image classification.The architectures are ResNet34, ResNet50, Inception ResNet v2, DenseNet201, and VGG16.
3.1. Dataset
Experiments use HAM10000 and ISIC 2017 separately, with different dataset sizes, diagnostic categories, and image-processing settings. The data are balanced through over- and under-sampling before pixel normalization.
- Experiments are performed separately on the HAM10000 and ISIC 2017 datasets.
- HAM10000 contains 10,015 dermatoscopic images across seven diagnostic categories.Images are resized to 299 x 299 for Inception ResNet v2 and 224 x 224 for other architectures.
- ISIC 2017 contains 2,600 images across three categories, but this experiment trains classification only for benign nevi and seborrheic keratosis.The test dataset contains 600 images, and images are resized to 224 x 224.
- Both datasets are balanced using over-sampling and under-sampling, then normalized by dividing each pixel by 255.
3.2. Soft Attention
Soft Attention focuses skin-lesion classification on relevant image regions by amplifying salient features and suppressing artifacts. It generates attention maps from intermediate feature tensors, scales those features, and retains the original representation through a residual branch.
- Soft Attention suppresses irrelevant artifacts such as veins and hair because only a small fraction of lesion-image pixels are relevant.The mechanism is intended to concentrate the model on diagnostically useful image regions.
- Higher-attention regions are shown in red, while low-attention regions receive weights closer to 0.The resulting focused representation is described as improving model performance.
- Attention maps are normalized and aggregated into a unified weighting function that scales salient feature values.The module produces 16 attention maps before forming the unified map α.
- The scaled features are multiplied by a learnable scalar and concatenated with the original feature tensor through a residual branch.The scalar γ is initialized at 0.01 so the network gradually regulates the amount of attention during training.
3.3. Model Setup
The study integrates Soft Attention into five ImageNet-trained architectures using architecture-specific insertion points and shared training components. The resulting designs are illustrated for the different network families and attention-map outputs.
- Training uses Adam with a 0.01 learning rate and 0.1 epsilon, batch normalization after each layer, seven output units with softmax, and Keras.The HAM10000 setup uses seven skin-cancer classes.
- The model schematics present the Soft Attention Block and end-to-end architectures for Inception ResNet v2, ResNet34, DenseNet201, and VGG16.The VGG16 schematic identifies convolution blocks containing two or three convolution layers followed by max pooling.
- The attention layer is placed after architecture-specific feature blocks: Inception ResNet C, DenseNet's fourth dense block, ResNet convolution blocks, and VGG16's fourth convolution layer.The corresponding feature-map sizes are 8 × 8, 7 × 7, 28 × 28 or 7 × 7, and 28 × 28, respectively.
- Soft Attention maps for Inception ResNet v2 on HAM10000 are shown as standalone maps and as maps superimposed on input lesions.The figure’s bottom row contains inputs, the middle row overlays attention on inputs, and the top row shows the attention maps.
- ResNet34, ResNet50, and Inception ResNet v2 concatenate attention-derived features with standard features, followed by ReLU activation and 0.5 dropout.This integration is explicitly described for the ResNet variants and Inception ResNet v2.
3.4. Loss Function
Because the experiment classifies seven skin-cancer categories, it uses categorical cross-entropy to optimize the neural network. The loss uses ground-truth labels and softmax-derived class scores.
- Categorical cross-entropy loss is used to optimize the network across seven skin-cancer classes.The class index ranges from 0 to 6.
- The loss compares each class's ground-truth value with the CNN score after softmax activation.The formulation defines t_i as ground truth, s_i as the CNN score, and f(s)_i as the softmax output.
3.5. Evaluation Metrics
The paper evaluates its models with classification metrics covering precision, accuracy, sensitivity, specificity, and AUC. An ablation table reports model-selection results on HAM10000, but the supplied passage does not provide its cell values.
- The evaluation uses precision, accuracy, sensitivity, specificity, and AUC scores.The reported formulas use true positives, true negatives, false positives, false negatives, and total samples.
- The HAM10000 ablation table is used to choose the best model among the evaluated architectures.The supplied table caption identifies the comparison scope but does not state individual results.
4. Discussion
Soft Attention consistently improves or matches the underlying networks across HAM10000 and ISIC-2017 evaluations, with IRv2+SA achieving the strongest reported results. The mechanism also produces more lesion-focused maps than Grad-CAM and improves clinically prioritized sensitivity.
- 4.1. Ablation Analysis: 93.7% precision and 98.4% AUC make IRv2+SA the highest-scoring model on HAM10000, while Soft Attention raises IRv2 precision by 3.2%.Soft Attention also improves precision for VGG16, ResNet34, ResNet50, and DenseNet201.
- 4.1. Ablation Analysis: 17%, 3%, 33%, and 4% precision gains are reported for AKIEC, BCC, DF, and NV, respectively, with IRv2+SA on HAM10000.For AUC, IRv2+SA improves BKL and MEL by 1.2% and 0.9%, while matching the original model for BCC, NV, and VASC.
- 4.1. Ablation Analysis: The 85/15% train-test split outperforms the 80/20% and 70/30% splits by 2.2% and 2.6%, respectively, and is selected for subsequent experiments.The split comparison is reported in Table 2.
- 4.2. Quantitative Analysis: 4.7% higher precision and 0.5%–4.3% higher AUC than state-of-the-art baselines are reported for the Soft Attention approach on HAM10000.The comparison is presented in Table 3.
- 4.2. Quantitative Analysis: 3.8% higher sensitivity than ARL-CNN50 is achieved on ISIC-2017, although the baseline retains a 3.4% specificity advantage.IRv25x5+SA performs on par in AUC and improves accuracy by 3.6%; IRv212x12+SA instead has 2.9% higher sensitivity than IRv25x5+SA.
- 4.3. Qualitative Analysis: Soft Attention maps focus more on relevant lesion locations than Grad-CAM heatmaps, which are described as shifted toward uninfected skin in the compared examples.The qualitative comparison is shown in Figure 9.
5. Conclusion
The paper applies Soft Attention during image encoding for high-resolution skin cancer classification and reports state-of-the-art performance on HAM10000 and ISIC-2017. It argues that the mechanism both improves classification and internally identifies relevant image locations while handling noise.
- 5. Conclusion: Soft Attention is implemented during image encoding to address high-resolution skin cancer image classification.
- 5. Conclusion: The Soft Attention architecture outperforms current state-of-the-art approaches on both HAM10000 and ISIC-2017 datasets.
- 5. Conclusion: Soft Attention internally provides the model’s focus location, boosts the main network, and naturally handles image noise without external mechanisms such as Grad-CAM.The authors identify dermoscopy assistance and application to other medical databases as future uses.