PaperlayerView exact revision on arXiv

Source-linked AI summary

Sparse concept attribution for histomorphological hypothesis generation from whole-slide classifiers

Tristan Lazard, Kenza Bouzid, Julius Hense, Shruthi Bannur, Daniel Coelho de Castro, Daniel Shao, Rajesh Jena, Drew Williamson, Stephanie Hyland

arXiv:2609.02985v1q-bio.QMeess.IV

TL;DR

Morphology-to-clinical hypothesis generation is limited by the manual interpretation of whole-slide images and classifier decisions. SCOPE combines pathology vision–language models with sparse attribution over a generalist concept bank, evaluated by MorphoRecoveryBench. Sparse attribution recovers known morphology while pooled-embedding decomposition preserves correctness at lower computational cost, supporting scalable hypothesis generation for expert validation.

  • Problem

    Existing whole-slide classifiers can predict biological or clinical attributes, but the morphological patterns supporting those predictions still require interpretable, verifiable explanations.

  • Method

    SCOPE interprets whole-slide classifiers by combining pathology-specific vision–language models, sparse concept attribution, a generalist histomorphological concept bank, and MorphoRecoveryBench.

  • Results

    Across seven tasks and six organs, sparse attribution recovered substantial known morphology, while dense attribution was indistinguishable from random selection; the best method reached 0.50 average Judge-F1.

  • Takeaways & Limitations

    Post-hoc interpretation can generate candidate morphological hypotheses at scale without task-specific concept curation, shifting expert effort toward selecting and validating them.

  • Takeaways & Limitations

    The benchmark is a proxy for novel discovery, and its reference granularity and support can limit measured explanation correctness.

Abstract

from arXiv · show

Histology images contain rich morphological information and can provide insights into pathological processes. However, deriving hypotheses relating morphological phenotypes to clinical attributes is bottlenecked by a manual image interpretation step. Here, we demonstrate that this process can be automated through interpretable deep learning. We present SCOPE, a method to interpret slide-level classifiers by combining pathology-specific vision--language models with sparse concept attribution onto a generalist histomorphological concept bank. To measure whether such explanations recover known morphology, we introduce MorphoRecoveryBench, a benchmark of seven tasks with pathologist-curated reference descriptions. On this benchmark, dense concept attribution is indistinguishable from a random baseline, whereas sparse attribution recovers substantial known morphology; decomposing the pooled slide embedding reaches similar explanation correctness at a fraction of the computational cost. Post-hoc interpretation of whole-slide classifiers can thus generate morphological hypotheses at scale, for expert validation.

1 Introduction

Whole-slide classifiers can predict clinical and biological attributes, but interpreting the specific morphology behind their decisions remains difficult. SCOPE addresses this gap with sparse language-based concept attribution and a benchmark for measuring recovery of known morphology.

  • 1 Introduction: Heatmaps identify regions important to a model but do not specify the histological features responsible, leaving interpretation laborious and subjective.The approach instead targets named, human-interpretable morphological hypotheses that experts can verify.
  • 1 Introduction: SCOPE combines pathology vision–language models, sparse concept attribution, and a general histomorphological concept bank to express whole-slide classifier decisions in natural language.Sparse attribution represents image embeddings as a sparse, non-negative combination of concepts rather than dense similarity scores.
  • 1 Introduction: A broad fixed concept bank supports discovery beyond task-specific concepts, while sparse attribution addresses spurious activations that grow with vocabulary size.The bank composes standard pathological terms into hypotheses spanning a broad space of histological patterns.
  • 1 Introduction: The method supports slide-level interpretation by combining concept attribution with multiple-instance learning and aggregating explanations into dataset-level hypotheses and subgroup analyses.SCOPE can bypass tile-level decomposition by operating directly on pooled slide embeddings and can be adjusted post hoc without retraining the classifier.
  • 1 Introduction: MorphoRecoveryBench quantifies whether generated explanations recover known morphological differences across whole-slide classification tasks.The benchmark uses binary tasks with pathologist-curated ground-truth explanations and associated metrics.
  • 1 Introduction: Across seven tasks and six organs, sparse attribution recovered substantial known morphology, whereas dense attribution was indistinguishable from random concept selection.The study also reports that pooled slide-embedding decomposition achieved similar correctness at a fraction of the computational cost.

2 Results

Across MorphoRecoveryBench, sparse concept attribution recovers known morphology substantially better than dense attribution, while slide-level SCOPE preserves explanation quality at far lower cost. Interpretable concept-based models also retain near-black-box classification performance, but explanation quality depends on backbone, sparsity, and concept-bank coverage.

  • Sparse concept attribution recovers known class morphology: Sparse attribution substantially outperforms dense attribution, which is indistinguishable from random concept selection on explanation quality.Dense tile attribution achieves jF1 0.11 versus a random floor of 0.09, while both sparse methods significantly improve over random.
  • Sparse concept attribution recovers known class morphology: SCOPE matches tile-sparse explanation quality while decomposing the pooled slide embedding instead of individual tiles.SCOPE reaches jF1 0.50 versus 0.47 and nDCG@10 0.62 versus 0.53, with nonsignificant differences for both metrics.
  • Sparse concept attribution recovers known class morphology: 5.5 milliseconds versus 66.5 seconds: slide-level SCOPE reduces decomposition runtime for an average 12,040-tile slide.The comparison uses a single-thread CPU benchmark and contrasts pooled-slide decomposition with tile-level decomposition.
  • Interpretable models retain predictive performance and transfer to an external cohort: Interpretable pipelines retain strong predictive performance, with tile-sparse macro AUROC 0.90 versus 0.96 for the black-box upper bound and CBM-SCOPE only 2 points lower on average.CBM-SCOPE remains 3 points below the black-box upper bound while classifying from concept reconstruction alone.
  • What drives explanation quality: Explanation quality varies across classes and backbones: clear-cell renal carcinoma reaches jF1 0.69, while cholangiocarcinoma reaches 0.38, and PathGen-CLIP-L reaches macro jF1 0.50 versus 0.27 for MUSK and 0.22 for CONCH.The backbone comparison shows that classification AUROC exceeds 0.86 for all three models despite marked differences in explanation correctness.
  • What drives explanation quality: Explanation correctness peaks at intermediate sparsity and saturates near a 989-concept bank, while optimal sparsity varies from 2 to 100 concepts per class.The shared default of approximately 25 concepts per slide is a compromise because precision rises with sparsity but recall has an interior maximum.

3 Discussion

SCOPE converts trained whole-slide classifiers into cohort-level histopathology explanations, recovering known morphology while exposing important representation, benchmark, and architectural limits. Its scalable hypotheses are intended for expert selection and validation within broader discovery workflows.

  • Results: Judge-F1 reaches 0.50 on average while the underlying classifiers achieve AUROC 0.93 across tasks, showing that explanation remains challenging despite strong discrimination.The benchmark re-discovers known cancer subtypes and morphologically distinct molecular groups.
  • Scale and use: SCOPE generates 34 candidate morphologies across 8 organs from 26 tasks without task-specific input, shifting experts toward selecting and validating hypotheses.The proposed concepts remain hypotheses until validated outside the model that produced them.
  • Method: Sparse decomposition is crucial when the discovery-oriented concept bank is overcomplete, unlike methods based on limited task-specific vocabularies.The method uses a generalist concept bank rather than requiring the relevant concepts to be specified in advance.
  • Representation limits: Explanation quality depends on the vision-language backbone and concept coverage: PathGen-CLIP-L nearly doubles explanation correctness versus two CLIP competitors despite similar classification accuracy.The concept bank derived from a large pathology-text corpus does not cover the full range of task-relevant morphology.
  • Evaluation limits: MorphoRecoveryBench is bounded by granularity and cohort support: automated explanations may be more specific than references, while absent cohort morphology cannot be surfaced.Separate precision and recall expose these distinct ceilings rather than merging them into one score.
  • Architectural limit: Because MIL explanations cannot represent slide-wide spatial organization, SCOPE cannot address tissue architecture, co-localization, or spatial organization beyond the tile scale.Such hypotheses require spatially aware classifiers and explainability methods.
  • Implications: SCOPE provides a complementary discovery layer by converting trained classifiers into cohort-level histopathology hypotheses within broader workflows combining explicit measurements and tool orchestration.The authors position it as one tool rather than a complete discovery system.

4 Methods

The methods define shared concept-based slide representations, compare dense and sparse attribution at tile or pooled-slide level, and evaluate generated explanations against expert-grounded relevance metrics.

  • 4.1 Describing tiles and slides with concepts: The framework represents each tile or slide with a fixed, shared bank of K morphological concepts, enabling task-comparable concept profiles.The default medium bank contains 989 concepts embedded with PathGen-CLIP-L.
  • 4.1 Describing tiles and slides with concepts: The generalist bank is mined from Quilt-1M captions, filtered for visually identifiable morphology, deduplicated, and frequency-trimmed to avoid task-label leakage.The construction is deliberately overcomplete so relevant concepts can be selected rather than prespecified.
  • 4.4 From profiles to class explanations: Class profiles are converted into natural-language explanations and evaluated using judge-based precision, recall, F1, and direct profile-ranking metrics such as nDCG@k.The statistical protocol generally reports mean and standard deviation across five random seeds.
  • 4.2 Classifier models and interpretable features: Tile-level pipelines use SI-MIL attention to identify relevant regions, then compare dense cosine attribution with sparse SpLiCE reconstruction of tile embeddings.The interpretable branch aggregates per-concept contributions across tiles into a slide profile.
  • 4.2 Classifier models and interpretable features: SCOPE instead decomposes an attention-pooled slide embedding, producing concept contributions without per-tile decomposition while preserving exact additive attribution through a residual term.CBM-SCOPE drops the residual, retaining concept-only predictions and requiring calibration for thresholded classification.

Extended Data

The extended data documents classification performance, explanation-evaluation reliability, concept-bank and sparsity sweeps, and transfer settings for the proposed models.

  • Classification performance: Extended Data Table 1 reports held-out AUROC per task across five seeds, distinguishing inherently interpretable models from black-box AbMIL using Virchow2.The table’s comparison concerns predictive performance across the evaluated tasks and model settings.
  • Evaluation reliability: Extended Data Table 2 compares human checklist metrics with LLM-judge metrics over 47 graded explanations from tile-level pipelines and SCOPE.This assesses whether automated judging tracks expert evaluation.
  • Concept and sparsity sweeps: Extended Data Tables 4 and 5 sweep concept-bank size and SpLiCE active-concept budget, measuring explanation quality across the eleven clinical classes.The sparsity figure shows class-specific optima spanning 2–100 concepts per slide, alongside the shared default L0 = 25.
  • Ground-truth augmentation: Extended Data Table 6 evaluates per-task augmentation of the default 989-concept bank with canonical concepts derived from each task’s ground-truth description.The comparison is paired per metric and class, with five-seed means and macro averages across classes.
  • External transfer: Extended Data Table 7 reports internal and external-cohort AUROCs together with explanation metrics for full SCOPE, CBM-SCOPE, and tile-sparse settings.It contrasts TCGA-trained transfer to CPTAC with models trained and tested within CPTAC.

S1 CBM-SCOPE margin calibration

CBM-SCOPE preserves much of the original classifier’s ranking behavior, but removing the residual changes margin scale and can substantially affect uncalibrated threshold decisions.

  • Margin calibration: Across tasks, the affine fit between full and concept-only margins has mean R2 values ranging from 0.72 to 0.97 across seeds.High- and low-R2 examples illustrate tightly and weakly affine relationships.
  • Margin calibration: AUROC changes comparatively little after dropping the residual, whereas balanced accuracy can shift substantially under the uncalibrated default decision rule.CBM-SCOPE therefore requires calibration before thresholded classification.

S2 Benchmark creation details

MorphoRecoveryBench uses pathologist-curated literature descriptions to evaluate explanations across seven morphology-grounded classification tasks spanning multiple organs and cancer subtypes.

  • Ground-truth construction: Each ground truth combines literature-derived candidate features with board-certified pathologist filtering and condensation into a one- to three-sentence reference description.These references support both generated-text judging and expert checklist annotation.
  • Task coverage: The benchmark covers breast, lung, kidney, liver, brain, PAM50 breast, and colorectal microsatellite-instability classification tasks.The task list spans subtype comparisons and a colorectal molecular-status comparison.
  • Ground-truth scope: The benchmark includes bilateral tasks with morphology described for both class directions and unilateral tasks where only the informative positive class is evaluated.The table explicitly records whether each task is bilateral or unilateral.
  • Ground-truth scope: Reference descriptions specify discriminative morphology for evaluated positive directions, while uninformative negative classes in unilateral tasks have no ground-truth explanation.This applies to the BRCA non-basal and CRC MSS directions shown in the benchmark table.

S3 Human evaluation details

The expert checklist evaluates 47 explanations with pathologist annotations, while Table S2 reports method-level means across the three tile-level pipelines and SCOPE.

  • 47 explanations were graded by the same pathologist, including 36 tile-level outputs across eight datasets and 11 SCOPE outputs from seven TCGA tasks.The SCOPE explanations used a single seed and matching-seed judge scores.
  • Table S2 summarizes human precision, recall, and F1 across the three tile-level pipelines and SCOPE, alongside corresponding automated judge scores.

S4 Judge and templating stochasticity

The reported across-seed variability includes stochasticity from both templating and judging, with the combined LLM noise comparable to total observed seed variation.

  • 0.09 Judge-F1 standard deviation from templating and judging is comparable to the 0.08 across-seed standard deviation, indicating substantial LLM sampling noise.Each per-seed Table 2 value combines one stochastic templating call and one stochastic judge call.
  • 0.05 judge standard deviation is isolated by freezing the templater and re-judging each class 10 times.
  • The templater variance is estimated by quadrature difference after separately measuring combined and judge variance.
  • 61% of the combined templating-and-judge variance is attributed to the templater, compared with 39% for the judge.The templater therefore contributes the larger share, although both stages are described as modest and comparable.

S5 Qualitative explanation panels

The qualitative panels make automated grading inspectable by linking model concepts to ground-truth phrases and marking hallucinated or missed morphology. Across examples, explanations capture substantial disease-specific architecture and cytology with variable judged F1.

  • The panels use shared tints for matched concepts, muted red for hallucinated claims, and grey for missed ground-truth concepts, with each header reporting precision, recall, and F1.These annotations are parsed deterministically from stored judge rationales.
  • 0.63 F1 is reported for BRCA lobular, with an explanation emphasizing lobular architecture, dense fibrosis, and infiltrative plasmacytoid or signet-ring-like cells.The ground truth centers on non-cohesive cells dispersed through fibrous tissue or arranged in single-file cords.
  • 0.71 F1 is reported for LGG oligodendroglioma, whose explanation captures uniform cells, perivascular pseudorosettes, clear-cell change, and salt-and-pepper chromatin.
  • 0.60 F1 is reported for liver HCC, whose explanation describes trabecular or solid growth, polygonal cells, sinusoidal vasculature, and bile production.The ground truth emphasizes malignant hepatocytes, loss of normal hepatic architecture, arterialization, and several growth patterns.
  • 0.53 F1 is reported for NSCLC adenocarcinoma, with glandular, acinar, alveolar, papillary, and micropapillary features identified in the explanation.The reference description specifically lists glandular, lepidic, acinar, papillary, and complex gland patterns.

S6 Qualitative TCGA-to-CPTAC explanation transfer

The panels compare explanations from the same TCGA-trained models on internal TCGA and external CPTAC cohorts, showing cohort-dependent changes in judged precision, recall, and F1.

  • The panels define cohort comparisons through shared tints for common ground-truth concepts, red for unsupported content, and grey for omitted ground-truth content.
  • 0.33 CPTAC F1 for BRCA basal-like falls from 0.58 on TCGA, with precision decreasing from 0.56 to 0.36 and recall from 0.60 to 0.30.
  • 0.23 CPTAC F1 for CRC MSI-high falls from 0.40 on TCGA, driven by recall decreasing from 0.38 to 0.14 despite precision increasing from 0.43 to 0.60.
  • NSCLC adenocarcinoma changes modestly from 0.56 TCGA F1 to 0.51 CPTAC F1, with recall decreasing from 0.80 to 0.60.
  • The TCGA and CPTAC explanations for NSCLC adenocarcinoma retain shared glandular, alveolar, papillary or micropapillary, and cytologic features.
  • 0.57 CPTAC F1 for NSCLC squamous exceeds its 0.47 TCGA F1, while recall rises from 0.67 to 1.00.

S7 Evaluation of concept scoring

Concept scoring is evaluated for accuracy and co-prediction noise, showing that dense similarity scores produce many spurious activations that obscure the ground-truth concept in explanations.

  • The evaluation uses representative positive and negative patches, scores all bank concepts with PathGen-CLIP-L, and compares scores using AUROC.The patch assessment sampled 200 positive and 200 control images for each of 200 kidney-relevant concepts, with positive labels visually checked using GPT-5.2.
  • Target Concept AUROC measures the best AUROC among semantically equivalent concepts, whereas co-predicted concepts count concepts reaching AUROC ≥ 0.65.
  • Target concepts for “clear cytoplasm” exceeded 0.88 AUROC, while unrelated concepts were also highly predictive.The result indicates accurate target detection alongside systematic spurious concept activations.
  • 0.731 mean AUROC still co-occurred with 83.0 co-predicted concepts on average across 200 evaluated concepts.In 63 cases, more than 100 concepts were predictive for the evaluated concept.
  • In simulation, increasing spurious concepts rapidly reduced the ground-truth concept’s explanation rank, especially when its AUROC was imperfect.
  • With ground-truth AUROC 0.75 and at least 15 spurious co-predictions, the ground-truth concept usually failed to rank first, with MRR ≤0.3.Such target AUROCs frequently coincided with more than 50 co-predicted concepts for PathGen-CLIP-L cosine scores.

S8 Supplementary atlas of generated morphological hypotheses

The supplementary atlas contains generated morphological hypotheses for 26 tasks across eight organs, after excluding classifiers with test AUROC ≤0.6. These outputs cover diverse class-associated patterns but remain unvalidated hypotheses.

  • Tasks with held-out classifier test AUROC ≤0.6 were excluded before hypothesis generation.SCOPE was initially applied to 35 candidate TCGA molecular classification tasks.
  • 34 class-level morphological hypotheses span 26 tasks and eight organs, including positive- and negative-class hypotheses.
  • The atlas outputs are unvalidated hypotheses, and test AUROC measures classifier discrimination at the task level.
  • Table S3 reports task-level test AUROC and repeats it when both class hypotheses are available.
  • The atlas links molecular or clinical classes to varied morphology, including clear-cell, papillary, fibrotic, inflammatory, mucinous, and keratinizing patterns.Examples include BAP1-mutant clear-cell features, PIK3CA-associated sclerotic fibrosis, MSI-high mucinous morphology, and HPV-positive basaloid lesions.
Loading 2609.02985v1…