Source-linked AI summary
Skin Lesion Synthesis with Generative Adversarial Networks
Alceu Bissoto, Fábio Perez, Eduardo Valle, Sandra Avila
TL;DR
Skin-lesion classification faces costly annotation needs and requires image detail to show malignancy markers. This paper uses GANs to synthesize lesions from semantic and instance maps, reporting visually appealing, clinically meaningful images and a 1.3-percentage-point average classification improvement.
Problem
Skin-lesion image classification is challenging, annotated data is costly to obtain, and high image resolution is needed to show malignancy markers.
Source: §1 · p. 1Method
The paper uses GANs to generate synthetic lesions from semantic and instance maps, with feature matching loss to stabilize training.
Source: §2.1 · p. 3Results
The synthetic images improve the classification network by an average of 1.3 percentage points and keep it more stable.
Source: §3.4 · p. 6Takeaways & Limitations
Synthetic images contain features that characterize lesions as malignant or benign and are appropriate for classification purposes.
Source: §4 · p. 7The pix2pixHD-based solution requires annotated data to generate images.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Skin cancer is by far the most common type of cancer. Early detection is the key to increase the chances for successful treatment significantly. Currently, Deep Neural Networks are the state-of-the-art results on automated skin cancer classification. To push the results further, we need to address the lack of annotated data, which is expensive and require much effort from specialists. To bypass this problem, we propose using Generative Adversarial Networks for generating realistic synthetic skin lesion images. To the best of our knowledge, our results are the first to show visually-appealing synthetic images that comprise clinically-meaningful information.
1 Introduction
Skin-lesion classification is challenging and depends on detailed images, while annotated data is costly to obtain. The paper proposes GAN-generated synthetic lesions and reports a one-percentage-point classification improvement using synthetic and real images.
1 Introduction
- Annotated skin-lesion data is costly and labor-intensive to obtain, motivating synthetic-image generation with GANs.Image-based classification is challenging because lesions vary in fine-grained appearance; DNN classification had improved rapidly [7] [19].Source: §1 · p. 1
- High-resolution images are needed to display malignancy markers that distinguish benign from malignant lesions.Most GAN work was evaluated on low-resolution datasets, while few works had shown promising high-resolution generation results.Source: §1 · pp. 1–2
- A classification network trained with synthetic and real images improves by 1 percentage point.The method generates high-definition synthetic skin lesions described as visually appealing and clinically meaningful.Source: §1 · p. 2
2 Proposed Approach
The approach translates semantic and instance maps into detailed synthetic lesion images, using pix2pixHD and lesion-specific annotations. Its architecture combines multi-scale discrimination and feature matching with maps encoding malignancy markers and lesion structure.
2 Proposed Approach
- The method synthesizes images from semantic and instance maps rather than noise, making image synthesis an image-to-image translation task.These inputs explicitly provide malignancy markers and lesion-border information.Source: §2 · p. 2
2.1 GAN Architecture: The pix2pixHD Baseline
- Three discriminators process the same inputs at different resolutions to handle global and finer image details.The second and third discriminators receive images downsampled by factors of 2 and 4.Source: §2.1 · p. 3
- Feature matching loss compares real and synthetic image features across discriminator layers to stabilize training and is combined with conditional GAN loss.Matching features encourages the generator to reproduce real-image statistics at multiple scales.Source: §2.1 · p. 3
2.2 Modeling Skin Lesion Knowledge
- Semantic maps label skin, lesions, and five malignancy markers, while a separate border label preserves lesion aspect ratio at fixed input size.The labels encode lesion content and boundary information for image-to-image translation.Source: §2.2 · pp. 3–4
- Instance maps use superpixels as lesion instances; specialists annotate each superpixel for the presence or absence of five malignancy markers.SLIC creates the superpixels, which serve as annotation units and distinguish instances within each class.Source: §2.2 · p. 4
3 Experiments
The experiments compare four GAN approaches for generating skin lesions, then test their visual quality and usefulness for classification. Synthetic augmentation improved classification, with images generated from both semantic and instance maps performing best among the synthetic sets.
3 Experiments
- The evaluation compared DCGAN, conditional PGAN, and pix2pixHD conditioned on semantic maps alone or on both semantic and instance maps.DCGAN represented low-resolution GANs; the authors report that other low-resolution GANs offered little improvement.Source: §3 · p. 4
3.1 Datasets
- pix2pixHD used 2,594 pattern-mask images and lesion masks split into 2,346 training/248 test; DCGAN/PGAN used four datasets (2,000, 13,000, 1,300, 200 images).The classifier used 2,346 training images and was tested on 900 Atlas images, including 270 melanomas.Source: §3.1 · pp. 4–5
3.2 Experimental Setup
- PGAN received benign/melanoma labels at all but its final layer and reached 256 × 256 after 330 epochs; classification used augmented Inception-v4.The implementations otherwise kept their default parameters; classification also used 50-replica test augmentation.Source: §3.2 · p. 5
3.3 Qualitative Evaluation
- DCGAN lacks fine detail, while noise-generated PGAN's malignancy markers lack cohesion; semantic maps markedly improve their arrangement.The authors report that PGAN looked better than other work they knew, but its marker placement lacked proper criteria.Source: §3.3 · p. 5, Figure 4 · p. 6
- Combining semantic and instance maps produced the overall best visual result, with the network learning patterns of both skin and lesion.Figure 4 compares the two-map approach with semantic-only, PGAN, DCGAN, and a real image.Source: §3.3 · p. 6, Figure 4 · p. 6
3.4 Quantitative Evaluation
- 1.3 percentage points average improvement and greater stability followed synthetic augmentation; Instance samples led synthetics, with all compared differences significant (p-value < 0.05).The comparison used ten training runs for each set.Source: §3.4 · p. 6, Table 1 · p. 7
4 Conclusion
The paper presents GAN-generated skin lesions with high-resolution fine detail and clinically meaningful features. It reports that pix2pixHD generation requires annotated data, while approaches to generate diversified images without additional data remain in development.
4 Conclusion
- Synthetic images reached resolutions up to 1024 × 512 and showed fine-grained details, with malignancy markers placed coherently and rendered sharply.The authors describe the images as visually appealing.Source: §4 · p. 7
- The authors report that synthetic images carry features characterizing lesions as malignant or benign and are appropriate for classification.A classification network was used to evaluate lesion-specific information.Source: §4 · p. 7
- The pix2pixHD-based solution requires annotated data; generating diversified images without additional data is still under investigation.Proposed directions include combining or distorting existing masks and generating masks with GANs.Source: §4 · p. 7