PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Dataset Diagnostic Utility of Clinical Visual Concepts in AI Systems for Dermatology

Linda Wermelinger, Simone Lionetti, Fabian Gröger, Nipun Ranasekara, Philippe Gottfrois, Ludovic Amruthalingam, Labelling Consortium, Marc Pouly, Alexander A. Navarini

arXiv:2610.08086v1cs.CV

TL;DR

Scattered, heterogeneous clinical-concept annotations limit research on their diagnostic utility in dermatology AI. The paper harmonizes four datasets into SkinLex and evaluates diagnosis from visual concepts; varied concept categories support classification, while the 48-concept set shows some redundancy for the examined task.

  • Problem

    Clinical concept annotations are scattered and heterogeneous across datasets, limiting large-scale study of their utility, complementarity, and redundancy.

    Source: §1 · p. 2

  • Method

    SkinLex harmonizes four datasets into a shared set of 48 clinical visual concepts and nine diagnosis categories, using text extraction when structured annotations are unavailable.

    Source: §1 · p. 3, §2.1 · p. 4

  • Results

    Classification requires varied concept categories, and the 48 concepts show some redundancy for the examined nine-category diagnosis task.

    Source: §1 · p. 3, §4.2 · p. 9

  • Takeaways & Limitations

    The findings suggest that coarse diagnosis on this dataset uses a relatively small but varied concept set, while removing redundant concepts may reduce annotation effort and improve efficiency and interpretability.

    Source: §4.2 · p. 9

    The flat 48-concept set and nine-diagnosis partition omit disease hierarchies and context-dependent vocabularies and may not reflect consensus across medical institutions.

    Source: §4.1 · p. 8

Abstract

from arXiv · show

The clinical integration of AI systems in digital dermatology relies heavily on human trust. Clinically interpretable visual concepts can act as intermediate representations enhancing trust and reliability. However, research in this domain is currently limited by scattered, heterogeneous dataset annotations. In this work, we introduce SkinLex, a harmonized dataset of 48 clinical morphological attributes across four public datasets (SkinCon, DermaCon-IN, MM-Skin, and PASSION) for a total of 20,411 records. Supervised nine-partition classification of skin conditions shows that limiting features to specific visual groups, like shapes or colors alone, reduces diagnostic accuracy. Bootstrapped backward elimination reveals that the set of 48 visual concepts has some degree of redundancy for algorithmic nine-partition diagnosis on the examined dataset. This demonstrates that coarse diagnosis on the selected dataset requires a relatively small but varied combination of clinical concepts, and motivates further research to improve concept taxonomy. Results can be translated into clinical benefits by reducing inputs for concept-based models, improving efficiency for annotation and modeling, and further enhancing interpretability. Code and prompt templates are available at https://github.com/Digital-Dermatology/SkinLex.

1 Introduction

Clinical visual concepts can provide an interpretable layer for dermatology AI, but scattered and heterogeneous annotations limit cross-dataset study of their utility. This paper introduces SkinLex and evaluates how harmonized concepts support diagnosis and where the concept set may be redundant.

1 Introduction

  • Scattered, heterogeneous, and sparse concept annotations hinder large-scale analysis, motivating cross-dataset harmonization before studying concept utility, complementarity, and redundancy.Annotations reflect diverse populations, clinical settings, local label sets, and practices.Source: §1 · pp. 2–3
  • The study tests clinical concepts as primary diagnostic inputs across disease categories, rather than only as explanatory support, while informing concept-bottleneck approaches.The authors frame their empirical study as an initial step, with further evidence needed to guide concept selection and annotation.Source: §1 · p. 3
  • SkinLex harmonizes four datasets into a unified 48-concept vocabulary and maps diagnoses to nine Fitzpatrick17k categories for a dataset five times larger than SkinCon.Sources include clinician-annotated datasets and datasets with expert-authored text.Source: §1 · p. 3
  • Classification requires varied concept categories, and the 48-concept set has some redundancy for the examined nine-category task.The paper presents the work as separating concept utility from image-based extraction and as a path toward refining concept sets.Source: §1 · p. 3

2 Methodology

The methodology harmonizes four dataset sources into SkinLex by mapping concepts and diagnoses to shared vocabularies, including concepts extracted from text when structured labels are unavailable. It evaluates concept utility and redundancy using diagnostic metrics and bootstrapped feature elimination.

2.1 Dataset Harmonization

  • Samples are mapped to the 48-concept SkinCon vocabulary, while Qwen extracts concepts from captions when structured labels are unavailable and maps granular diagnoses to nine Fitzpatrick17k categories.The mapping addresses more than 6,000 separate condition identifiers; Qwen uses a Fitzpatrick17k mapping dictionary as an in-context reference.Source: §2.1 · p. 4
  • For MM-Skin, concepts and diagnoses are extracted from captions and VQA dialogues; after discarding extraneous files and cases with multiple or no extracted diagnoses, 7,500 instances remain.Source: §2.1 · p. 5
  • Derm1M is excluded because 378,678 of its 423,016 captions are repeated at least twice, creating repeated concept-diagnosis entries that would reweight data without variance.The most frequent caption appears in 2,945 copies.Source: §2.1 · p. 5
  • Aggregation fills localized concept gaps: MM-Skin supplies 87.1% of Sclerosis, 96.7% of Blue, and 79.9% of Warty/Papillomatous instances absent from DermaCon-IN.PASSION supplies 92.4% of all Excoriation instances.Source: §2.1 · p. 5

2.2 Experimental Setup

  • Backward elimination uses 100 bootstrapped training and validation sets, removing at each step the feature whose exclusion yields the best median score until no features remain.A one-sided paired Wilcoxon signed-rank test with a 5% threshold determines the minimal set retaining full performance; no multiple-testing correction is applied.Source: §2.2 · p. 6

3 Experiments and Results

Across the examined nine-partition diagnosis task, restricted concept groups weaken performance, while backward elimination indicates some redundancy in the full set. The results favor varied combinations of concepts and show metric-dependent limits to how many can be removed without lower performance.

3.1 Reference models

  • LightGBM was selected as the primary model, achieving 0.8555 ROC-AUC and 0.3236 AP on validation; logistic regression reached 0.8384 ROC-AUC and 0.2947 AP.LightGBM captured 90.9% of the ROC-AUC and 58.1% of the AP ceiling.Source: §3.1 · p. 6

3.2 Concept Groups and Sensitivity Analysis

  • Restricting inputs to individual concept groups degraded performance; Shapes and Colors retained the strongest standalone signals, while Others approached random guessing.The table reports median performance and 95% confidence intervals from 100 validation bootstraps.Source: §3.2 · p. 7, Table 3 · p. 7
  • Leave-one-out sensitivity showed small performance shifts, with Brown (Hyperpigmentation) causing the largest degradation, followed by Plaque, Papule, Nodule, and Scale.The figure plots mean absolute decreases in macro AP and ROC-AUC when each concept is omitted; error bars show standard deviations across 100 validation bootstraps.Source: §3.2 · p. 7, Figure 2 · p. 7

3.3 Backward Elimination

  • Backward-elimination scores grew sub-linearly as the number of concepts used for classification increased.Figure 3 reports LightGBM macro AP and ROC-AUC across concept counts using 100 training and validation bootstraps.Source: §3.3 · p. 7, Figure 3 · p. 8
  • Removing 8 concepts did not significantly lower AP, while up to 18 could be omitted without decreasing ROC-AUC, indicating metric-dependent redundancy.These thresholds were measured against the full concept set on this dataset.Source: §3.3 · p. 8, Figure 3 · p. 8

4 Discussion

The discussion qualifies the findings by noting that the flat concept set and nine-diagnosis partition simplify clinical variation, and that harmonization and generalization have constraints. It concludes that SkinLex supports analysis of concept utility for a specific task, with potential clinical relevance from removing redundant concepts.

4.1 Limitations

  • The flat 48-concept set and nine-diagnosis partition omit disease hierarchies and context-dependent vocabularies, and may not reflect consensus across institutions.More granular concepts or disease targets could capture greater diagnostic nuance and increase task complexity; some condition-specific annotation bias may remain.Source: §4.1 · p. 8
  • Caption-based extraction can conflate unmentioned features with negative findings, and Qwen-based extraction and harmonization need cross-model and human verification.The source datasets emphasize concept presence over absence.Source: §4.1 · p. 8
  • Generalization is constrained by possible hidden subgroups and demographic constraints in the combined sources, and humans or other models may rank concepts differently.A concept’s lack of utility for coarse algorithmic classification does not establish low general relevance, because redundancy is defined within a concept set.Source: §4.1 · pp. 8–9

4.2 Conclusions and Outlook

  • SkinLex combines four datasets into 20,411 samples, and the analysis suggests that SkinCon’s 48 concepts have some redundancy for coarse algorithmic classification.The authors state that removing redundant concepts can reduce annotation effort and improve computational efficiency, interpretability, and clinical trust.Source: §4.2 · p. 9

Related papers

Loading 2610.08086v1…