PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief

Robert Engel

arXiv:2609.03095v1eess.IVcs.CVcs.HCcs.LG

TL;DR

Teledermatology needs a reliable way to screen the diagnostic suitability of variable smartphone photo sets. The paper introduces an interpretable tri-view pipeline using micro-relief segmentation and case-level classification, achieving higher discrimination on unanimous-consensus than majority-consensus cases while supporting submission-time filtering.

  • Problem

    Smartphone variability makes assessing the diagnostic suitability, or gradability, of teledermatology case submissions a critical bottleneck.

  • Method

    The Semantic Tri-view Pipeline segments micro-relief fidelity with DeepLabV3+ and aggregates interpretable spatial signals across up to three views using logistic regression.

  • Results

    AUC improves from 0.81 on majority-consensus cases to 0.96 on unanimous-consensus cases, with 97.7% PPV for passed cases at the prespecified >90% sensitivity operating point.

  • Takeaways & Limitations

    The system supports interpretable, privacy-by-design, edge-ready screening that can provide immediate feedback and filter ungradable photo sets before physician review.

  • Takeaways & Limitations

    Because SCIN was not designed as a case-level gradability benchmark, the results provide feasibility and clinical-relevance evidence rather than definitive strict-benchmark performance.

Abstract

from arXiv · show

Smartphone skin photographs are indispensable to teledermatology, yet assessing the diagnostic suitability of submitted cases (gradability) remains a critical bottleneck in mobile care workflows. Dermatologists routinely review multiple photographic views (regional, angled, and close-up) to identify consistent textural detail rather than relying on a single image. We present the Semantic Tri-view Pipeline, an interpretable architecture for automated teledermatology gradability screening that formalizes epidermal micro-relief as a computable biomarker of image quality. Using an expert-annotated subset of the public SCIN dataset, we train a lightweight DeepLabV3+ model to segment micro-relief fidelity. These spatial masks are then aggregated across up to three case views with a logistic regression classifier, leveraging viewpoint redundancy to support robustness under uncontrolled smartphone acquisition. This approach learns context-aware, clinically intelligible heuristics, such as penalizing high-fidelity texture in regional distance views. Evaluated at a predefined 90% sensitivity operating point, the system's apparent errors largely reflect subjective clinical variance on borderline cases where clinicians rely on non-visual metadata. On SCIN, performance improves from an AUC of 0.81 (80.6% PPV) on variance-heavy majority-consensus cases to 0.96 (97.7% PPV) on optically unambiguous unanimous cases. Overall, this work delivers an interpretable, privacy-by-design, edge-ready system that can provide real-time feedback during case submission to filter ungradable photo sets before review.

I. INTRODUCTION

Smartphone variability makes teledermatology gradability difficult, motivating an interpretable, multi-view screening pipeline based on micro-relief fidelity. The system combines semantic visual cues across views and shows stronger discrimination on unanimous cases than on majority-consensus cases.

  • Smartphone submissions vary in resolution, lighting, focal distance, angle, and background clutter, complicating assessment of diagnostic suitability.
  • The pipeline formalizes epidermal micro-relief fidelity as a computational biomarker of photographic quality using lightweight DeepLabV3+ segmentation.
  • Semantic signals are aggregated across up to three photographic views to connect pixel-level quality with case-level diagnostic utility.
  • Logistic regression over spatial feature vectors produces interpretable, context-aware decision boundaries, including penalizing high-fidelity texture in regional distance views.
  • AUC rises from 0.81 on majority-consensus cases to 0.96 on unanimous-consensus cases, with thresholded performance evaluated at 90% sensitivity.
  • The segmentation model runs in 19.99 ms per image on an iPhone 12 mini, while downstream case-level scoring uses compact spatial summaries.

II. BACKGROUND AND RELATED WORK

Micro-relief fidelity offers a clinically meaningful quality signal, but existing approaches are poorly aligned with uncontrolled smartphone submissions and multi-view clinical review. The proposed framework addresses these gaps with semantic, case-level aggregation across complementary views.

  • Skin micro-relief reflects optical focus and illumination, making its visibility a biomarker of photographic adequacy.
  • Controlled-hardware micro-relief studies do not trivially generalize to smartphone photos with variable resolution, focal distance, angle, and background clutter.
  • Prior DIQA approaches commonly use neural classifiers to identify broad quality problems such as bad light, blur, low resolution, or excessive distance.
  • Dermatology evaluates holistic cases through regional, close-up, and angled views, while prior SCIN analysis linked diagnostic confidence more strongly to contextual information than intrinsic sharpness.
  • Semantic segmentation of micro-relief fidelity is proposed to replace opaque scoring with localized, context-aware evidence about skin-texture distribution.
  • The tri-view design independently processes clinical views, converts masks into spatial features, and aggregates them for patient-level gradability classification.

III. PROPOSED SEMANTIC TRI-VIEW PIPELINE

The Semantic Tri-view Pipeline maps multi-view smartphone images to case-level gradability through three interpretable stages: semantic segmentation, spatial feature extraction, and logistic-regression classification.

  • Semantic Micro-relief Segmentation: DeepLabV3+ independently segments regions with varying micro-relief fidelity in each clinical view.
  • Spatial Feature Extraction: Spatial masks are compressed into interpretable 1D metrics, avoiding opaque latent-space fusion.
  • Multi-view Case Classification: Logistic regression maps the multi-view semantic signals to a final case-level gradability outcome.

A. Clinical Taxonomy & Expert-Annotated Dataset

The method uses a five-class ordinal taxonomy to represent progressive micro-relief degradation, with background added as a sixth semantic class. Ground truth comes from dense expert annotations of a sampled SCIN subset.

  • Clinical Taxonomy: The taxonomy orders micro-relief fidelity from Excellent (1) through Good, Fair, and Poor to None (5).
  • Clinical Taxonomy: Excellent requires crisp resolution of deep primary and shallow secondary micro-relief lines, whereas intermediate classes represent progressive optical degradation.
  • Clinical Taxonomy: Representative 240 × 240 px SCIN crops illustrate the five fidelity classes from crisp micro-relief lines to complete loss of topological geometry.
  • Clinical Taxonomy: None designates regions where defocus or artifacts completely obscure topological geometry, and non-skin background forms class 0.
  • Expert-Annotated Dataset: The annotated ground truth contains 132 randomly sampled SCIN images, with five images annotated twice for 137 total masks by a board-certified dermatologist.

B. Stage 1: Semantic Micro-relief Segmentation

Stage 1 uses a DeepLabV3+ segmentation network to estimate skin micro-relief fidelity while preserving edge-ready inference and handling ambiguous intermediate texture classes.

  • Network architecture: DeepLabV3+ with an EfficientNet-B0 backbone segments subtle micro-relief details at a fixed 640 × 640 input resolution.Noisy Student initialization is used to improve robustness to noise and ambiguous intensity transitions.
  • Training design: Photometric augmentation is omitted because brightness, contrast, and hue changes could distort the high-frequency details underlying fidelity labels.Training instead uses crops, flips, and orthogonal rotations to reduce overfitting on the small annotated subset.
  • Inference and optimization: Inference uses sliding-window prediction with 640×640 tiles and 50% overlap, averaging logits before argmax to reduce boundary artifacts.The optimization setup uses AdamW, ReduceLROnPlateau, early stopping, and best-epoch weights per fold.
  • Segmentation evaluation: 5-fold cross-validation on 137 annotated images achieves a mean IoU of 0.259, with performance decreasing monotonically from Excellent (0.297) to None (0.093).Non-skin background achieves the highest IoU at 0.722; intermediate fidelity boundaries are continuous spectral transitions rather than hard anatomical edges.

C. Stages 2 & 3: Spatial Feature Extraction and Multi-view Case Classification

Stages 2 and 3 convert segmentation masks from up to three photographic views into interpretable case-level features and classify gradability with regularized logistic regression.

  • Spatial feature extraction: Stage 2 compresses high-dimensional segmentation masks into compact features that preserve clinically meaningful spatial structure.This design avoids the opacity of latent-space fusion typical of deep learning ensembles.
  • Multi-view representation: Each case uses nine engineered features for CLOSE_UP, AT_AN_ANGLE, and AT_DISTANCE views, producing a fixed x ∈ R27 vector with zero-padding for absent views.Explicit case-level metadata such as image count is excluded so gradability information remains encoded in per-view mask statistics.
  • Multi-view representation: The features quantify micro-relief composition, spatial concentration near the image center, class-1 centrality, and entropy over class 1–3 proportions.These statistics summarize both fidelity distribution and its spatial organization within each view.
  • Case classification: Stage 3 applies ℓ2-regularized logistic regression to map the interpretable feature space to case-level gradability decisions.A StandardScaler pipeline and randomized hyperparameter search optimize ROC-AUC under 5-fold stratified cross-validation.

IV. EXPERIMENTAL SETUP

The experiment separates training, threshold calibration, and evaluation by vote agreement, contrasting a variance-heavy Majority cohort with a held-out unanimous Gold cohort.

  • Dataset partitioning: After removing 132 Stage 1 cases from SCIN, 4,901 unseen cases remain for Stage 3 training and evaluation.Ground truth derives from either one or three independent dermatologist gradability votes.
  • Dataset partitioning: The Noisy pool contains 4,213 one-vote cases and is used exclusively for model training and hyperparameter search.This pool represents lower-confidence labels relative to the multi-vote evaluation cohorts.
  • Dataset partitioning: The Majority pool contains 331 non-unanimous three-vote cases and supplies the operating threshold through 5-fold cross-validated Platt calibration.It is evaluated as a pooled cohort to quantify baseline clinical performance.
  • Dataset partitioning: The Gold pool contains 357 unanimous three-vote cases and is held out for a single-pass evaluation using a global Platt scaler fit on the Majority pool.This high-consensus subset approximates performance under higher clinician agreement.
  • Evaluation protocol: A single global calibrator is applied to the unseen Gold pool, with strict partition boundaries preventing leakage between training and evaluation.The setup evaluates models trained on lower-confidence labels against higher-agreement ground truth.

B. The Clinical Operating Point

The Clinical Operating Point prioritizes sensitivity to avoid rejecting viable images, then compares thresholded performance and error patterns across clinically variable and unanimous cohorts.

  • Operating-point rationale: A sensitivity target of at least 90% defines the highest allowable threshold, reflecting asymmetric costs between false negatives and false positives.The design treats rejected viable images as a potential access-to-care concern, while passed poor images primarily create physician inefficiency.
  • Thresholded performance: 50.0% specificity and 80.6% PPV are achieved on the Majority cohort at the fixed Clinical Operating Point, passing 258 of 331 cases.The threshold is derived on the Majority pool and then applied across both evaluation sets.
  • Cohort comparison: AUC improves from 0.81 on the Majority pool to 0.96 on the Gold cohort, where cases are optically unambiguous and clinician agreement is unanimous.The same-threshold Gold evaluation also shows marked gains in thresholded metrics.
  • Error analysis: False positives fall from 50/100 (50%) in the Majority pool to 7/45 (15.6%) in the unanimous Gold standard.The paper attributes many apparent Majority-set errors to annotator disagreement on borderline cases rather than algorithmic failures.

B. Case-Level Model Interpretability and Semantic Context

The case-level classifier produces interpretable, shot-specific heuristics by combining micro-relief segmentation features across views. Its coefficients indicate that close-up detail is most influential, while distance-view texture can be penalized when it conflicts with regional context.

  • Case-Level Decision Boundary: Logistic regression coefficients make the case-level decision boundary interpretable as clinically meaningful photographic heuristics rather than spurious correlations.The classifier evaluates compact spatial summaries derived from segmentation masks.
  • Shot-Specific Semantics: CLOSE_UP coefficients favor Excellent micro-relief (+0.07) and penalize Fair/Poor/None texture (all ≈−0.05).The pattern matches the expectation that macro images must resolve high-fidelity micro-relief for diagnostic use.
  • Shot-Specific Semantics: AT_DISTANCE coefficients reverse this pattern: Excellent texture is penalized (- 0.03), while Poor texture becomes mildly positive (+0.01).Distance shots are intended to preserve regional anatomical context rather than resolve fine micro-relief.
  • Feature Influence: The CLOSE_UP slot carries the largest-magnitude coefficients overall, making macro-detail quality the strongest case-level determinant.Class 1–3 entropy (+0.08) is also among the most influential spatial features.
  • Spatial Features: Central high-quality texture promotes CLOSE_UP gradability, whereas concentrated poor-quality texture is penalized; these effects weaken or reverse at distance.Class 1–2 center concentration is +0.08 in CLOSE_UP views but -0.02 in AT_DISTANCE views.
  • Multi-View Context: View count alone contributes little: 57.7%, 57.8%, and 54.2% of 1-, 2-, and 3-view cases were gradable, while viewpresence-only baselines remained near chance.The baseline AUCs were 0.525 on Gold and 0.516 on Majority.

C. Algorithmic Fairness

Performance remains strong across the reported Fitzpatrick skin-type groups on the Gold holdout set, while clinical subjectivity and dataset scope constrain interpretation. The system also offers vision-only, privacy-oriented deployment advantages, but SCIN was not designed as a case-level gradability benchmark.

  • Fitzpatrick Skin-Type Evaluation: Gold-set discrimination remained strong across Fitzpatrick groups I–II, III–IV, and V–VI, with AUCs of 0.906, 0.979, and 0.974, respectively.Sensitivity at τCOP was 93.5%, 95.0%, and 94.7%, while specificity ranged from 76.9% to 90.0%.
  • Fitzpatrick Skin-Type Evaluation: On the noisier Majority set, AUCs were lower but similar across the reported skin-type groups.The supplied passage reports Majority AUCs of 0.796, 0.811, and 0.753 for I–II, III–IV, and V–VI.
  • Clinical Variance: The 97.7% PPV Gold cohort outperformed the 80.6% PPV Majority cohort, consistent with greater clinical variance in borderline cases.The discussion attributes borderline disagreement to clinicians using non-visual metadata unavailable to the model.
  • Deployment Scope: The vision-only design supports privacy by avoiding PHI and enables edge inference, with segmentation taking 19.99 ms per image on an iPhone 12 mini.Downstream logistic-regression scoring adds negligible cost.
  • Limitations and Future Validation: Stage 1 segmentation relies on a highly curated expert dataset of N = 137 masks, motivating broader multi-clinician validation.The paper identifies inter-rater agreement and more balanced Fitzpatrick representation as future priorities.
  • Limitations and Future Validation: SCIN was not originally designed as a case-level gradability benchmark, so the results are evidence of feasibility and clinical relevance rather than definitive benchmark performance.The Tri-view formulation repurposes real-world teledermatology data to approximate submission quality.

VII. CONCLUSION

The Semantic Tri-view Pipeline links micro-relief segmentation to case-level teledermatology gradability assessment in an interpretable architecture. At a prespecified >90% sensitivity, it rejects most ungradable cases while preserving high PPV among cases passed to physicians.

  • The interpretable architecture links micro-relief segmentation to case-level teledermatology gradability assessment.
  • 84.4% of ungradable cases were rejected at a prespecified >90% sensitivity on SCIN’s held-out unanimous-consensus cohort.The operating point was designed to preserve patient access to care.
  • 97.7% PPV was achieved for cases passed to physicians at the same operating point.
  • The results support accurate, interpretable, privacy-by-design gradability screening in real-world mobile teledermatology workflows.
  • Research code, the feature extraction pipeline, and evaluation artifacts are available for non-commercial use.
Loading 2609.03095v1…