Source-linked AI summary
A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context
Veronica Rotemberg, Nicholas Kurtansky, Brigid Betz-Stablein, Liam Caffery, Emmanouil Chousakos, Noel Codella, Marc Combalia, Stephen Dusza, Pascale Guitera, David Gutman, Allan Halpern, Harald Kittler, Kivanc Kose, Steve Langer, Konstantinos Lioprys, Josep Malvehy, Shenara Musthaq, Jabpani Nanda, Ofer Reiter, George Shih, Alexander Stratigos, Philipp Tschandl, Jochen Weber, H. Peter Soyer
TL;DR
Existing melanoma AI studies largely evaluate single lesions, whereas clinicians interpret lesions in the context of a patient’s other lesions. This paper presents a multicenter dataset linking multiple dermoscopic images per patient, designed to support context-aware melanoma recognition and improve translational potential.
Problem
Prior AI evaluations and datasets largely omit the multiple-lesion patient context clinicians use when assessing melanoma, including the ugly duckling sign.
Method
The authors constructed a multicenter dermoscopic dataset by linking multiple patient lesions and labeling diagnoses using reviewed histopathology or longitudinal monitoring.
Results
The resulting dataset labels a mean of 16 lesions per patient, enabling algorithms to assess multiple images from the same patient for malignancy.
Takeaways & Limitations
The dataset supports studying whether patient- and lesion-related clinical context improves recognition performance and helps algorithms evaluate lesions in clinical context.
Takeaways & Limitations
The dataset under-represents darker skin types, so algorithms trained on it may not generalize reliably across skin tones without prospective study.
Abstract
from arXiv · showhide
Prior skin image datasets have not addressed patient-level information obtained from multiple skin lesions from the same patient. Though artificial intelligence classification algorithms have achieved expert-level performance in controlled studies examining single images, in practice dermatologists base their judgment holistically from multiple lesions on the same patient. The 2020 SIIM-ISIC Melanoma Classification challenge dataset described herein was constructed to address this discrepancy between prior challenges and clinical practice, providing for each image in the dataset an identifier allowing lesions from the same patient to be mapped to one another. This patient-level contextual information is frequently used by clinicians to diagnose melanoma and is especially useful in ruling out false positives in patients with many atypical nevi. The dataset represents 2,056 patients from three continents with an average of 16 lesions per patient, consisting of 33,126 dermoscopic images and 584 histopathologically confirmed melanomas compared with benign melanoma mimickers.
Background & Summary
AI may improve melanoma care by expanding access to expertise, diagnostic accuracy, and screening efficiency. This work presents a multicenter dermatology image dataset incorporating patient- and lesion-related clinical context to test whether such information improves recognition performance.
- AI in medical imaging could reduce melanoma-associated mortality, morbidity, and healthcare costs by improving expertise access, diagnostic accuracy, and screening efficiency.
- Controlled studies show AI can match or outperform clinicians on individual skin-lesion images, including algorithms that outperformed over 500 clinical readers and experts.These reader studies did not reflect scenarios in which clinicians examine all lesions on a patient.
- Clinicians assess biopsy candidates in context with a patient’s other lesions and their overall biologic skin ecosystem.An unusual lesion among otherwise benign-looking lesions constitutes the dermatologic “ugly duckling sign.”
- The study presents methods for creating a multicenter dermatology image dataset with clinical contextual information.
Methods
The dataset was assembled from clinical imaging databases across six centers, linking multiple lesions per patient and assigning diagnoses from histopathology or longitudinal follow-up. Curators applied image-selection and expert quality-assurance procedures, while accounting for timepoint variability, contextual-lesion imbalance, and duplicate ingestion.
- Dataset assembly: Images from six centers were compiled by identifying patients with multiple lesions and reviewing histopathology reports for internally biopsied lesions.Non-biopsied lesions monitored for at least six months were considered benign.
- Dataset assembly: A training subset from six sites across five geographic locations was allocated for the 2020 ISIC Grand Challenge to test algorithm generalizability.
- Lesion representation: Each lesion was represented by one image, with selected timepoints minimizing imaging-date variability between benign and melanoma patient classes.This was intended to reduce potential bias from image lighting, camera type, or other factors.
- Lesion context: Retrospective acquisition produced unequal lesion counts between patients with and without melanoma images, potentially reflecting selection bias in imaged lesions.The dataset does not represent all lesions existing on these patients.
- Data integrity: A clerical ingestion error introduced 425 pixelwise-identical duplicate images into the dataset.The redundant cases and deidentified patient labels were listed on the 2020 SIIM-ISIC Melanoma Classification competition page and were available from the authors.
Data Records
The dataset is publicly distributed under defined access, licensing, and update policies, with image metadata supplied in linked records. It is available in both DICOM and JPEG/TIF-plus-CSV formats.
- Access and licensing: The dataset was available through Kaggle until August 20, 2020, and is permanently accessible via the ISIC Archive at DOI 10.34970/2020-ds01.It is licensed CC-BY-NC under the ISIC Terms of Use; modifications are recorded at the DOI landing page.
- Metadata: Each image’s metadata includes approximate age, biological sex, lesion site, anonymized patient ID, benign/malignant category, and available specific diagnosis.Dataset characteristics are summarized at both patient and lesion levels in Table 1.
- Dataset format: The dataset is available in two formats: DICOM, or JPEG/TIF images with metadata in a linked CSV file.DICOM follows Part 10 of the international DICOM standard.
Technical Validation
Technical validation established lesion labels through histopathology review, expert dermoscopic confirmation, and defined follow-up or consensus criteria for benign lesions. Melanoma in situ and invasive melanoma were coded as melanoma, while other qualifying lesions were coded as benign.
- Ground-truth labeling: Histopathology reports confirmed malignant-lesion labels, with suspicious reports double checked and dermoscopic experts visually verifying diagnosis plausibility.Melanoma in situ and invasive melanoma were both coded as melanoma.
- Ground-truth labeling: All other qualifying images were coded as benign, including lesions diagnosed as severely dysplastic nevi.This coding followed the stated melanoma definition, which included both melanoma in situ and invasive melanoma.
- Benign-label validation: Non-biopsied lesions were labeled benign when expert consensus supported benignity or follow-up of at least six months showed no malignant changes.Certain lesion types were not monitored because doing so would not reflect clinical practice, but experts visually verified their labels.
Usage Notes
The dataset links multiple dermoscopic images from each patient to better mimic clinical assessment, especially for patients with multiple atypical nevi. Its varied imaging modalities and under-representation of darker skin types affect interpretation and generalization to clinical use.
- Clinical context: Patient-level labels link multiple images from the same patient, supporting malignancy assessment in cases with multiple atypical nevi.The dataset reports a mean of 16, median of 12, and standard deviation of 16 images per patient.
- Imaging modalities: The dataset includes contact nonpolarized, contact polarized, and non-contact polarized dermoscopic imaging modalities.Polarized light more often reveals deeper skin structures, while colors, structures, and patterns vary in accessibility across modalities.
- Generalization: Under-representation of darker skin types may cause melanoma overdiagnosis or underdiagnosis and limit generalization to broad clinical use.The passage attributes this risk to low population prevalence and unequal access to care across populations.
Competing Interests
The authors disclose consultancy, advisory, reporting, and shareholding relationships with several dermatology and medical-technology companies.
- HPS is a shareholder of MoleMap NZ Limited and e-derm consult GmbH and regularly provides teledermatological reporting for both.
- HPS consults for Canfield Scientific Inc. and Revenio Research Oy and advises First Derm.
- AH consults for Canfield Scientific Inc. and serves on the SciBase advisory panel.