PaperlayerView exact revision on arXiv

Source-linked AI summary

CG-HAF: An Interpretable Global-Local Lesion-Burden Fusion Framework for Ordinal Acne Severity Grading in Agentic Skincare Support

Muhammad Muhtasim Shahriar, Md. Naimur Asif Borno, Saad Aloteibi, Mohammad Ali Moni

arXiv:2609.31326v1cs.CVcs.AI

TL;DR

Acne severity grading must combine holistic appearance and lesion burden, but existing approaches do not establish the value of keeping both observable in the final decision. CG-HAF fuses global severity probabilities with explicit detector-derived burden descriptors. On ACNE04, it improves over global-probability-only stacking, including in Severe-class recall.

  • Problem

    Existing approaches do not establish the incremental value of jointly observable global severity probabilities and detector-derived lesion-burden measurements in a compact, interpretable final representation.

    Source: §1 · p. 2

  • Method

    CG-HAF combines probabilities from three independently trained global branches with detector-derived lesion counts, confidence statistics, and area evidence in a 12-dimensional representation for a balanced logistic-regression meta-classifier.

    Source: §1 · p. 2

  • Results

    On ACNE04, the retained F4 predictor achieved 87.33% accuracy and improved over global-probability-only stacking by 6.16 accuracy points and 0.1944 Severe-class recall.

    Source: §8 · p. 19

  • Takeaways & Limitations

    Structured detector-derived lesion evidence adds decision-relevant information beyond global severity probabilities in the evaluated ACNE04 setting.

    Source: §8 · p. 19

    No subgroup analysis was available to attribute individual errors to skin tone, imaging conditions, or demographic factors.

    Source: §7.4 · p. 18

Abstract

from arXiv · show

Ordinal acne severity grading requires distinguishing visually similar neighboring grades while jointly weighing holistic facial appearance and localized lesion burden - evidence that most existing approaches collapse into a single opaque representation. We introduce CG-HAF, a global-local fusion framework that instead keeps this evidence explicit: averaged holistic severity probabilities from independently trained classifiers are combined with structured lesion-burden descriptors from an object detector (lesion count, detection confidence, lesion area) into a compact representation, from which a lightweight, interpretable classifier produces the final grade. On a widely used benchmark, this fusion yields a clear, statistically supported improvement over global-evidence-only baselines, with the largest gains on the most severe cases. Testing on an independent dataset with a different grading standard shows that strong within-dataset performance does not transfer automatically, and a follow-up diagnostic attributes much of this gap to mismatched grading criteria rather than detection failure alone. These findings support interpretable global-local fusion as an effective strategy for ordinal acne grading while highlighting criterion alignment as key to cross-dataset portability, with a further illustration of how the resulting severity signal can support transparent, non-diagnostic decision-making in skincare applications.

1. Introduction

Acne grading must weigh holistic facial appearance and localized lesion burden despite visually similar neighboring grades and limitations in each evidence source alone. CG-HAF makes both sources explicit in a compact representation, and the study evaluates fusion and cross-criterion transfer separately.

1. Introduction

  • Holistic appearance and lesion burden are complementary evidence sources, while lesion localization is difficult and higher grades are often underrepresented.The introduction notes that small, dense lesions are sensitive to illumination and scale, and class imbalance can obscure Severe and Very Severe cases.Source: §1 · p. 2
  • Existing approaches do not establish the incremental value of jointly observable global probabilities and detector-derived burden in a compact final representation.The paper identifies limited controlled feature ablation and evaluation beyond a single endpoint score as additional gaps.Source: §1 · p. 2
  • CG-HAF fuses holistic severity probabilities with detector-derived lesion evidence in an interpretable 12-dimensional representation for final grading.Three independently trained global branches supply averaged probabilities; a validation-fitted balanced logistic-regression meta-classifier produces the final grade.Source: §1 · p. 2
  • The study separately tests frozen cross-dataset transfer to PLSBRACNE01 and criterion adaptation using expert lesion evidence.ACNE04 uses a Hayashi-aligned, count-oriented formulation, whereas PLSBRACNE01 uses a Pillsbury-based criterion combining lesion type and burden.Source: §1 · p. 2
  • The evaluation reports ordinal and class-specific metrics and rejects candidates that reduce Severe or Very Severe sensitivity despite aggregate gains.The contributions also include a non-diagnostic skincare orchestration example using structured CG-HAF outputs.Source: §1 · p. 2

2. Related Work

Prior acne methods address ordinal grading, local lesions, or global-local fusion in different ways, but local evidence can be difficult to interpret and evaluate across settings. CG-HAF positions its contribution as a compact, observable final-stage fusion whose feature contributions can be tested progressively.

2.1. Ordinal Grading and Local Lesion Evidence

  • Acne studies use ordinal-aware grading, lesion detection and counting, or lesion-derived scores, but detection performance alone does not establish reliable ordinal grading.Local detection also faces small, dense lesions, illumination and scale sensitivity, and errors for low-contrast or underrepresented lesions.Source: §2.1 · pp. 2–3

2.2. Global–Local and Evidence-Fusion Approaches

  • CG-HAF combines global severity probabilities with explicit detector-derived burden cues in a compact final-stage representation, unlike hidden-feature fusion or detector-only grading.Progressive feature addition tests whether count-derived, continuous-burden, confidence, and area evidence adds information beyond global probabilities.Source: §2.2 · p. 3

2.3. Reliability, Evaluation, and Research Gap

  • CG-HAF targets the gap of isolating observable lesion-burden contributions while protecting high-severity sensitivity; differing protocols preclude direct performance rankings across studies.Reliable evaluation also considers class-specific behavior, ordinal error, calibration, and evidence boundaries alongside aggregate performance.Source: §2.3 · pp. 3–4, Table 1 · p. 4

3. Materials

The study uses fixed ACNE04 partitions for internal evaluation and PLSBRACNE01 as an independent cohort with a different grading criterion. The datasets provide distinct settings for internal confirmation and external transport assessment.

3.1. ACNE04 Dataset, Annotations, and Fixed Partition

  • ACNE04 contains 1,457 images and 18,983 lesion boxes across four ordered grades, divided into fixed train, validation, and test partitions.The image is the analysis unit; patient identifiers were unavailable, so subject-level independence could not be established.Source: §3.1 · p. 4, Table 2 · p. 4
  • The ACNE04 test split provides internal confirmation but is not an untouched benchmark because it was inspected during prior development.Marked imbalance at higher grades motivated class-balanced sampling and evaluation.Source: §3.1 · p. 4

3.2. PLSBRACNE01 External Evaluation Cohort

  • PLSBRACNE01 provides an independent cohort of 200 subjects and 600 images under a different, four-grade consensus labeling standard.A SHA-256 audit found no image overlap with ACNE04; expert lesion annotations were reserved for a separate annotation-assisted diagnostic.Source: §3.2 · p. 5

4. Methods

The methods evaluate a fixed-partition, four-class acne-grading pipeline that combines global image evidence with localized lesion-burden evidence. They also specify controlled ablations, a safety-constrained application illustration, and separate protocols for internal and cross-criterion evaluation.

4.1. Study Design and Evidence Scope

  • The retrospective study uses a fixed ACNE04 partition, three global models, a local detector, a 12-dimensional representation, and a validation-fitted balanced logistic-regression meta-classifier.ACNE04 analyses are treated as internal confirmation because the test partition had historically been inspected; the study makes no prospective or clinical-validation claim.Source: §4.1 · p. 5

4.2. Global ConvNeXtV2-Large Severity-Count Branch

  • ConvNeXtV2-Large predicts four severity classes and an auxiliary 65-bin lesion-count distribution, with count guidance kept out of the final fusion stack.Training also uses deterministic skin-region masking, Gaussian label-distribution learning, count/agreement losses, and Teacher-EMA consistency; the count head is training-only.Source: Figure 1 · p. 5, §4.2 · p. 6, Figure 2 · p. 6

4.3. Local YOLOv12s Lesion-Evidence Branch

  • YOLOv12s detects lesions and exports predicted count, confidence statistics, and summed lesion area; its count maps to Hayashi-aligned severity intervals, but the detector does not decide the final grade.The final export uses confidence threshold 0.10, IoU 0.50, and up to 300 detections per image.Source: §4.3 · p. 7, Figure 3 · p. 7

4.4. Three-Seed Aggregation and 12-Dimensional CG-HAF Fusion

  • Three independently trained global models’ aligned test probabilities are averaged, then combined with detector-derived category, normalized count, confidence, and area features in a 12-feature representation.A balanced multinomial logistic-regression classifier is fitted on validation data and applied unchanged to test features.Source: Figure 1 · p. 5, §4.4 · pp. 7–8

4.5. Controlled Ablations and Safety-Constrained Candidate Evaluation

  • The secondary count-conditioned candidate was not adopted because selection required Severe and Very Severe recall not to degrade, even when aggregate or calibration metrics improved.Training-component ablations remove one component at a time; fusion ablations progressively add detector features, refitting balanced logistic regression at each step.Source: §4.5 · p. 8, Table 3 · p. 8

4.6. Integration within Agentic Skincare Support

  • The SkinAgent prototype passes CG-HAF severity evidence to deterministic-guard, LLM-orchestrated routing without changing CG-HAF weights; this integration is outside the grading experiments.The example is not quantitative or clinical validation of the complete agentic system.Source: §4.6 · p. 9, Figure 4 · p. 9

4.7. Evaluation, External Transport, and Criterion-Adaptation Diagnostic

  • Evaluation spans class balance, ordinal error, calibration, and class-specific recall, with paired significance tests and bootstrap confidence intervals.Interpretability uses detection boxes, named fusion features, and inspectable logistic-regression coefficients as evidence, not causal explanations.Source: §4.7 · p. 9
  • External transport is tested by applying the frozen ACNE04 system label-blind to PLSBRACNE01, while a separate target-label diagnostic compares expert-annotation features with the locked count-based rule.The annotation-assisted diagnostic uses privileged target information and is not automatic external-performance evaluation.Source: §4.7 · pp. 9–10

5. Experimental Results

The retained F4 model delivered strong internal ACNE04 performance, with most errors confined to adjacent grades. Qualitative examples show F4 often matched ground truth where comparison systems missed by an adjacent grade, but not in every case.

5.1. Evidence Integrity and Historical Reference

  • The historical single-run model achieved 84.93% accuracy (QWK 0.9042); the higher current score was not significantly different (McNemar p=0.210), and the test split was development-inspected.These comparisons are internal confirmation, not an untouched-benchmark evaluation.Source: §5.1 · p. 10

5.2. Final Retained CG-HAF Performance

  • 87.33% accuracy (255/292 correct), balanced accuracy 0.8699, Macro-F1 0.8576, QWK 0.9173, and grade MAE 0.1301 define retained F4; 36/37 errors were between adjacent grades.The confusion matrix shows integer counts on the fixed ACNE04 test split.Source: Figure 5 · p. 10, §5.2 · p. 10

5.3. Qualitative Comparison with Principal Baselines

  • F4 often matched ground truth where comparison systems missed by an adjacent grade, but a challenging case remained; these examples are qualitative, not independent evidence of statistical superiority.Figure 6 aligns predictions from the YOLOv12s count-only reference, historical CG-HAF, and retained F4.Source: §5.3 · p. 11, Figure 6 · p. 11

5.4. Multi-Seed Stability and Probability Aggregation

The probability-aggregated F4 predictor outperformed the seed-level mean and each individual seed on reported aggregate metrics. Seed-level variation and class-specific recall show why the aggregate predictor is a distinct result rather than a mean of run scores.

5.4. Multi-Seed Stability and Probability Aggregation

  • Across seeds, F4 accuracy ranged 84.9–86.3%, Macro-F1 0.822–0.845, and QWK 0.902–0.913, documenting run-level variation.The 85.50% ± 0.71 pp accuracy mean is a descriptive summary across seeds, not the final ensemble output.Source: §5.4 · p. 11
  • Aggregated Severe recall was 0.8333 and Very Severe recall 0.8846, not simply the best seed for each class; no formal variance-reduction analysis was conducted.The four class probabilities were averaged image-wise before refitting the F4 meta-classifier.Source: §5.4 · p. 12
  • 87.33% accuracy, 0.8576 Macro-F1, and 0.9173 QWK for probability-aggregated F4 exceeded both the 85.50% ± 0.71 pp seed mean and every individual seed.The figure and table distinguish seed-level performance from the distinct probability-aggregated predictor.Source: §5.4 · pp. 11–12, Figure 7 · p. 12, Table 4 · p. 12

5.5. Safety-Constrained Candidate Evaluation

The frozen dual-stack candidate improved some metrics but failed the high-severity recall gate, so the original three-seed F4 remained the retained predictor. Its calibration and interpretability evidence is descriptive and includes explicit limits on attribution.

5.5. Safety-Constrained Candidate Evaluation

  • The frozen dual-stack matched retained F4 accuracy at 87.33% and improved QWK, grade MAE, and calibration, but Severe recall fell from 0.8333 to 0.7500, so it failed the selection gate.It produced no net gain in correct predictions (McNemar p=1.000); the original-branch three-seed F4 remained the final predictor.Source: Table 5 · p. 12, §5.5 · p. 12

5.6. Calibration, Error Patterns, and Interpretability

  • NLL 0.4750, Brier 0.2528, and ECE 0.1453 show residual calibration error; calibration gains did not justify selecting a candidate with lower Severe recall.The metrics are descriptive confidence-quality measures, not evidence of full calibration.Source: §5.6 · p. 12
  • Errors were overwhelmingly between adjacent grades, with one wider Mild-to-Severe error; no subgroup analysis attributed errors to morphology, skin tone, or other factors.The confusion matrix and its accompanying interpretation describe the error pattern, not its subgroup causes.Source: §5.6 · p. 13
  • Global-probability features carried the largest, consistently positive weights for their corresponding classes, while detector features had class-dependent positive and negative weights.The fitted coefficients describe the decision surface; differently scaled feature blocks prevent interpreting them as a causal importance ranking.Source: §5.6 · p. 13
  • Historical-branch Grad-CAM and YOLO examples concentrate on acne-bearing regions; they are not ensemble attributions, and scale-dependent coefficients are descriptive, not causal.Figure 8 presents the historical visualizations alongside coefficients from the retained validation-fitted F4 stack.Source: Figure 8 · p. 13, §5.6 · p. 13

5.7. Frozen Cross-Criterion Performance on PLSBRACNE01

The frozen ACNE04-trained system showed limited performance on PLSBRACNE01 under dataset and grading-criterion shift. An annotation-assisted diagnostic found higher accuracy with criterion-adapted burden and lesion-type information, separately from CG-HAF’s automatic external evaluation.

5.7. Frozen Cross-Criterion Performance on PLSBRACNE01

  • Under frozen cross-criterion testing, all configurations reached only 35–40% subject-level accuracy; F4 led in balanced accuracy, Macro-F1, QWK, and grade MAE, with a 32.5–45.5% accuracy CI.This was a deliberately challenging setting with both image-distribution and grading-criterion differences, not a like-for-like replication.Source: Table 6 · p. 13, §5.7 · p. 13

5.8. Annotation-Assisted Criterion-Adaptation Analysis

  • The diagnostic indicates criterion-specific mapping and lesion-type composition captured target-severity information that the fixed ACNE04 count-to-grade rule did not.The finding does not attribute the external transport gap solely to automatic detector error.Source: §5.8 · pp. 13–14
  • With expert-derived lesion evidence, accuracy rose from 43.5% for the locked Hayashi count rule to 58.5% for criterion-adapted total burden and 72–74% for lesion-type composition.The primary TYPE10 ordinal model achieved 74.0% accuracy, Macro-F1 0.7364, and QWK 0.8726; this annotation-assisted diagnostic is separate from automatic CG-HAF performance.Source: §5.8 · pp. 13–14, Table 7 · p. 14

6. Ablation Studies

The ablations test the contribution of global and local evidence, progressively added lesion-burden features, and selected global-branch training components. The complete structured fusion outperforms global-probability-only stacking, while label distribution learning is the training component with independently supported benefit.

6. Ablation Studies

  • The ablations separately examine evidence-stream contributions, progressive detector-feature additions, and selected global-branch training components rather than treating them as one hierarchy.The progressive feature analysis held averaged global probabilities and detector export fixed; the training-component analysis used the probability-mean endpoint.Source: §6 · p. 14, §6.1 · p. 14, §6.3 · p. 16

6.1. Global–Local Branch Ablation

  • Full CG-HAF improved accuracy by 6.16 percentage points over Global-only and 6.85 points over Local-only, with gains in Macro-F1, QWK, and Severe-class recall.Neither single stream dominated the other across metrics, and Holm-adjusted McNemar tests found significant correctness changes when either stream was removed.Source: Table 8 · p. 14, §6.1 · p. 14

6.2. Qualitative Global–Local Model Comparison

  • The qualitative comparison shows complementary single-stream successes and a case where fusion corrected both streams’ incorrect Very Severe prediction, but fusion did not eliminate image-level errors.All three configurations agreed and were correct on 201 of 292 test images; Full-only-correct occurred once.Source: Figure 9 · p. 15, §6.2 · pp. 15–16

6.3. Progressive Fusion-Feature Ablation

  • The complete F4 representation improved accuracy by 6.16 points, Macro-F1 by 0.0918, QWK by 0.0467, and Severe recall by 0.1944 over F0 (Holm-adjusted p=0.001109).Intermediate F1–F3 increments were not individually validated as significant after multiplicity correction.Source: Table 9 · p. 16, §6.3 · p. 16, Figure 10 · p. 16

6.4. Training-Component Ablation

  • Removing label distribution learning produced the clearest deterioration, with accuracy falling from 81.16% to 78.42% and Severe recall from 0.6111 to 0.5000.A paired bootstrap supported A2 across balanced accuracy, Macro-F1, QWK, and Severe recall; no Full-versus-ablation Holm-adjusted McNemar comparison was significant.Source: §6.4 · p. 17, Table 10 · p. 17

7. Discussion

The discussion identifies structured lesion-burden fusion as the principal contribution, while noting constraints on internal comparisons, calibration, interpretability, and external transport. It also positions prior-study results as context rather than a controlled ranking.

7.1. Principal Findings

  • F4 increased accuracy by 6.16 percentage points, Macro-F1 by 0.0918, QWK by 0.0467, and Severe-class recall by 0.1944 over F0 (Holm-adjusted p=0.001109).The final predictor’s higher accuracy than the historical single-run result was not statistically established (McNemar p=0.210).Source: §7.1 · pp. 17–18

7.2. Interpreting Global–Local Lesion-Burden Fusion

  • The fusion gains are attributed to complementarity: detector-derived count, confidence, and area variables expose lesion burden that holistic global probabilities do not.The largest progressive gain came from adding the count-derived grade; the low-capacity logistic regression makes the result evidence about the structured representation, not added model capacity.Source: §7.2 · p. 18

7.3. Model Selection Under Stochasticity

  • Three-seed probability averaging reduces dependence on initialization but is not a formal robustness guarantee; only label distribution learning had independently supported benefit among the examined training components.The rejected count-conditioned candidate illustrates selection under a high-severity non-degradation rule, not clinical validation.Source: §7.3 · p. 18

7.4. Calibration, Error Behavior, and Interpretability

  • Residual calibration error means strong classification and ordinal agreement do not establish reliable confidence estimates, and the study lacked subgroup analysis for attributing individual errors.Attention visualizations, detector boxes, and fusion coefficients improve auditability but are not causal or clinically validating evidence.Source: §7.4 · p. 18

7.5. Positioning Relative to Prior Acne-Grading Research

  • Reported performance from representative studies is contextual rather than a controlled ranking because datasets, grading criteria, class definitions, and evaluation protocols differ.The cited comparison also includes results that are not directly comparable to four-class whole-image severity accuracy.Source: §7.5 · pp. 18–19, Table 11 · p. 19

7.6. External Transport and Applications

  • Frozen evaluation showed limited zero-shot transport under simultaneous dataset and grading-criterion shift; the diagnostic used privileged annotations and cannot isolate the effects of domain and criterion shift.The SkinAgent integration demonstrates implementation-level use as a non-diagnostic evidence component, not validated recommendation quality, agent performance, or clinical safety.Source: §7.6 · p. 19

8. Conclusion

The conclusion presents CG-HAF as an interpretable global–local framework and reports improved ACNE04 results over global-probability-only stacking. It limits the claim to a non-diagnostic research framework given restricted external and clinical validation.

8. Conclusion

  • On ACNE04, the retained three-seed F4 predictor achieved 87.33% accuracy and improved over global-probability-only stacking by 6.16 accuracy points and 0.1944 Severe-class recall (Holm-adjusted p=0.001109).The predictor fuses ConvNeXtV2-Large severity probabilities with YOLOv12s lesion-burden evidence in a semantically defined 12-dimensional representation.Source: §8 · p. 19
  • Limited zero-shot transfer, one inspected benchmark, unverified subject independence, and absent demographic and multicenter validation leave CG-HAF a non-diagnostic research framework rather than a clinically validated tool.The conclusion calls for criterion-aware lesion-type evidence, external calibration, uncertainty-aware deferral, and prospective subject-identified multicenter evaluation before broader claims.Source: §8 · p. 19

CRediT authorship contribution statement

The contribution statement assigns broad methodological, analytical, and writing responsibilities to Shahriar, with supervision and specialized project contributions from Borno, Aloteibi, and Moni.

CRediT authorship contribution statement

  • Muhammad Muhtasim Shahriar is credited with conceptualization, methodology, software, validation, analysis, investigation, data curation, visualization, project administration, and drafting and revising the manuscript.His listed roles span the full research and writing workflow.Source: CRediT authorship contribution statement · p. 20
  • Md. Naimur Asif Borno is credited with supervision, validation, visualization, and writing review and editing.Source: CRediT authorship contribution statement · p. 20
  • Saad Aloteibi is credited with supervision, project administration, and funding acquisition, while Mohammad Ali Moni is credited with supervision, project administration, methodology, and resources.Source: CRediT authorship contribution statement · p. 20
Loading 2609.31326v1…