Source-linked AI summary
Data Augmentation for Skin Lesion using Self-Attention based Progressive Generative Adversarial Network
Ibrahim Saad Ali, Mamdouh Farouk Mohamed, Yousef Bassyouni Mahdy
TL;DR
Imbalanced class frequencies and limited manually designed augmentations hinder robust, well-generalizing skin cancer classifiers. The paper enhances PGAN with self-attention for long-range feature dependencies and applies TTUR to stabilize high-resolution synthetic skin-lesion generation. APGAN+TTUR improved classification accuracy by 2.8% on average after generated samples augmented the ISIC2018 training data.
Problem
Imbalanced class frequencies and limited manually designed augmentations hinder robust, well-generalizing skin cancer classifiers.
Method
The paper enhances PGAN with self-attention for long-range feature dependencies and applies TTUR to stabilize high-resolution synthetic skin-lesion generation.
Results
APGAN+TTUR improved classification accuracy by 2.8% on average after generated samples augmented the ISIC2018 training data.
Takeaways & Limitations
APGAN+TTUR augmentation is reported to provide higher information gain and improve classification accuracy over standard augmentation methods.
Takeaways & Limitations
Generated samples can contain blur, high-frequency artifacts, mode collapse, and melanoma bright spots associated with similar real training images.
Abstract
from arXiv · showhide
Deep Neural Networks (DNNs) show a significant impact on medical imaging. One significant problem with adopting DNNs for skin cancer classification is that the class frequencies in the existing datasets are imbalanced. This problem hinders the training of robust and well-generalizing models. Data Augmentation addresses this by using existing data more effectively. However, standard data augmentation implementations are manually designed and produce only limited reasonably alternative data. Instead, Generative Adversarial Networks (GANs) is utilized to generate a much broader set of augmentations. This paper proposes a novel enhancement for the progressive generative adversarial networks (PGAN) using self-attention mechanism. Self-attention mechanism is used to directly model the long-range dependencies in the feature maps. Accordingly, self-attention complements PGAN to generate fine-grained samples that comprise clinically-meaningful information. Moreover, the stabilization technique was applied to the enhanced generative model. To train the generative models, ISIC 2018 skin lesion challenge dataset was used to synthesize highly realistic skin lesion samples for boosting further the classification result. We achieve an accuracy of 70.1% which is 2.8% better than the non-augmented one of 67.3%.
1. Introduction
Skin lesion classification faces severely imbalanced labeled data, while manually designed augmentation offers limited, dataset-specific alternatives. The paper proposes APGAN enhanced with self-attention and stabilized with TTUR to generate clinically meaningful synthetic images for augmentation.
- Motivation: ISIC2018 contains 10015 images across seven lesion types with heavily skewed class frequencies, motivating augmentation for classification.The imbalance can bias DNNs toward dominant classes and hinder generalization.
- Motivation: Standard augmentation uses transformations such as flips, rotations, translations, and Gaussian noise, but requires dataset-specific policy design and may transfer poorly across datasets.The paper illustrates this limitation by contrasting rotation effectiveness on CIFAR-10 and MNIST.
- Method: GANs are proposed to learn class-preserving transformations and model the real image distribution, enabling broader synthetic augmentation than manually designed transformations.The paper positions GAN-based augmentation as a way to generate samples intended to remain indistinguishable from real images.
- Method: The method combines progressive GAN training, self-attention, and TTUR to generate high-definition synthetic skin lesion images.Self-attention models attention-driven long-range dependencies, while TTUR stabilizes training at 256 × 256 resolution.
- Results: 2.8% average classification-accuracy improvement is reported after augmenting ISIC2018 with the best-performing APGAN variant.The models are trained on ISIC2018, compared using GAN-train and GAN-test, and the selected generator augments the classification training data.
2. Proposed Approach
APGAN+TTUR extends progressive GANs with self-attention and stabilized training to synthesize high-resolution, clinically meaningful skin-lesion images. The approach uses progressive growth, long-range feature dependencies, and TTUR to address high-resolution generation challenges.
- 2. Proposed Approach: APGAN+TTUR combines progressive growing, self-attention, and TTUR to generate 256 × 256 skin-lesion images with class-preserving fine detail.The framework targets high-resolution lesion structure relevant to distinguishing benign and malignant cases.
- 2. Proposed Approach: The overall pipeline trains APGAN+TTUR on ISIC 2018 data, then combines real and generated samples to train a CNN classifier.Figure 3 presents the synthesis-and-classification workflow.
- 2.2. Self-Attention Progressive GAN: Self-attention models long-range dependencies beyond convolutional local neighborhoods, helping the generator focus on salient structures across varying shapes and sizes.The discriminator suppresses irrelevant regions and highlights task-relevant features through the inserted attention block.
- 2.2. Self-Attention Progressive GAN: The simplified non-local block reduces attention computation by sharing a global query-independent attention map and moving Wv outside the attention pooling.The paper identifies Equation 3 as the simplified block used to introduce self-attention into PGAN.
- 2.3. Two Time Update Rule: TTUR addresses high-resolution training instability by setting the discriminator learning rate to 5x the generator rate while keeping update steps at a 1:1 ratio.The paper presents TTUR as a stabilization technique intended to improve quantitative and qualitative GAN results.
3. Experiments and Results
Experiments evaluate APGAN variants, stabilization with TTUR, generated-image artifacts, and classification augmentation on ISIC2018. APGAN+TTUR achieves the strongest reported augmentation result while reducing instability-related artifacts.
- 3.1. Dataset: The study used 10,015 ISIC2018 images from seven skin-disease classes, split into 9,514 training and 501 validation images.Images were center-cropped to 450 × 450 pixels and downsampled to 256 × 256 pixels.
- 3.2. Evaluation Metrics: GAN-train measures generated-image diversity and realism through real-test classification, whereas GAN-test measures realism through classification on generated images.The evaluation uses classifiers trained on GAN images for GAN-train and real images for GAN-test.
- 3.4. Self-Attention mechanism: APGAN models placing self-attention at stage 64 or 128 outperformed placements at lower-level feature maps in GAN-train and GAN-test.For APGAN feat 64, GAN-train rose from 67.7 to 70.1 and GAN-test from 60.8 to 62.2 relative to PGAN.
- 3.5. High Resolution Skin Lesions: APGAN+TTUR produced the best quantitative and qualitative results among the evaluated models after TTUR stabilized APGAN training.Each model generated 1,000 synthetic images per class for GAN-train and GAN-test evaluation.
- 3.6. Artifacts: Unstable training produced blur, high-frequency artifacts, and mode collapse; TTUR mitigated instability, although bright spots remained in some melanoma samples.The bright spots were attributed to corresponding patterns in real melanoma training images.
- 3.7. GAN data augmentation: 70.1% validation accuracy with APGAN+TTUR augmentation exceeded real-only training by 2.8% and standard augmentation by 1.4%.The classifier used 100 real images per class and 1k added images per class, with ResNet-18 evaluated on 501 validation images.
4. Conclusion.
The paper proposes a self-attention-enhanced progressive GAN with TTUR to generate clinically meaningful, high-definition skin-lesion images. APGAN+TTUR augmentation improves classification accuracy over real-only and standard augmentation approaches.
- 4. Conclusion.: The proposed framework combines progressive GAN growth, self-attention, and TTUR to generate high-definition, visually appealing, clinically meaningful skin-lesion images.Self-attention models long-range dependencies and guides attention toward malignancy-related fine-grained details.
- 4. Conclusion.: APGAN+TTUR augmentation improves classification accuracy by adding synthetic samples with reported information gain over standard augmentation.The conclusion also identifies future evaluation on multiple datasets as planned work.
- 4. Conclusion.: The generated samples still contain some artifacts caused by unstable training behavior, which TTUR is used to address.The paper reports that some residual artifacts remain despite applying self-attention and TTUR.