Source-linked AI summary
Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images
Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri, Faisal Mahmood
TL;DR
Weakly supervised computational pathology faces annotation and data bottlenecks. CLAM uses attention and instance-level clustering to analyze whole slides from slide-level labels, with reported data efficiency, adaptability, and interpretability across applications.
Problem
Weakly supervised multiple-instance learning faces a major bottleneck in computational pathology.
Method
CLAM uses attention-based learning to identify diagnostically valuable sub-regions and instance-level clustering to refine whole-slide classification.
Results
CLAM models were adaptable to independent cohorts, biopsy slides, and cellphone microscopy images, while remaining interpretable for generating morphological features.
Takeaways & Limitations
CLAM supports interpretable and adaptable whole-slide analysis across clinical and research settings.
Takeaways & Limitations
Cellphone microscopy images may be affected by imperfect capture conditions, including poor focus.
Abstract
from arXiv · showhide
The rapidly emerging field of computational pathology has the potential to enable objective diagnosis, therapeutic response prediction and identification of new morphological features of clinical relevance. However, deep learning-based computational pathology approaches either require manual annotation of gigapixel whole slide images (WSIs) in fully-supervised settings or thousands of WSIs with slide-level labels in a weakly-supervised setting. Moreover, whole slide level computational pathology methods also suffer from domain adaptation and interpretability issues. These challenges have prevented the broad adaptation of computational pathology for clinical and research purposes. Here we present CLAM - Clustering-constrained attention multiple instance learning, an easy-to-use, high-throughput, and interpretable WSI-level processing and learning method that only requires slide-level labels while being data efficient, adaptable and capable of handling multi-class subtyping problems. CLAM is a deep-learning-based weakly-supervised method that uses attention-based learning to automatically identify sub-regions of high diagnostic value in order to accurately classify the whole slide, while also utilizing instance-level clustering over the representative regions identified to constrain and refine the feature space. In three separate analyses, we demonstrate the data efficiency and adaptability of CLAM and its superior performance over standard weakly-supervised classification. We demonstrate that CLAM models are interpretable and can be used to identify well-known and new morphological features. We further show that models trained using CLAM are adaptable to independent test cohorts, cell phone microscopy images, and biopsies. CLAM is a general-purpose and adaptable method that can be used for a variety of different computational pathology tasks in both clinical and research settings.
CLAM: High-throughput, interpretable, weakly-supervised and data-efficient whole slide analysis
CLAM is an interpretable, weakly supervised WSI method that combines attention-based pooling with instance-level clustering to identify diagnostically valuable regions and classify whole slides. Its feature-extraction pipeline reduces computation substantially, enabling high-throughput training and adaptation across pathology image types.
- CLAM uses attention-based learning to identify diagnostically valuable sub-regions and instance-level clustering to constrain and refine the feature space.Attention-based pooling aggregates patch-level features into slide-level representations for classification.
- CLAM supports generic multi-class classification through parallel attention branches that produce class-specific slide representations and final whole-slide probability predictions.Each branch emphasizes regions viewed as strong positive evidence for one diagnostic class.
- CLAM improves weakly supervised learning efficiency by generating pseudo-labels for highly and weakly attended patches, enriching supervision for a separable patch-level feature space.The method adds a supervised clustering task for the most and least attended patches of each class.
- The WSI pipeline decreases the data-space volume nearly 200-fold by extracting low-dimensional feature embeddings before training and inference.This shifts subsequent computation from high-dimensional pixels to low-dimensional feature space.
- Thousands of gigapixel-sized resection slides can be used to train models within hours on modern consumer-grade workstations.The reduced feature space substantially lowers the computation required for supervised deep-learning models.
- CLAM models trained on WSIs are adaptable to cell phone microscopy images and biopsy slides.The paper evaluates data efficiency, adaptability, and interpretability across renal cell carcinoma subtyping, non-small cell lung cancer subtyping, and breast cancer lymph node metastasis detection.
Results
CLAM achieved high slide-level classification performance across cancer detection and subtyping tasks despite moderate training-set sizes. It also retained useful performance and interpretable attention patterns across independent cohorts, cellphone images, and biopsy slides.
- Data efficiency: 25% of available training cases, averaging around 170 slides per fold, achieved an average test AUC of > 0.94 for RCC subtyping.NSCLC subtyping and lymph-node metastasis detection required 50% of training data, corresponding to ∼853 and ∼289 slides, respectively.
- Adaptation to cellphone microscopy: On cellphone microscopy images, CLAM achieved an average test AUC of 0.873 for lung and an average one-vs-rest macro-averaged AUC of 0.921 for kidney.The best models scored AUCs of 0.899 and 0.958, respectively, despite performance drops attributed to imperfect cellphone-image capture conditions.
Discussion
CLAM addresses key computational pathology challenges through weak supervision, data-efficient learning, multi-class subtyping, adaptability across specimen and imaging domains, and interpretable whole-slide analysis. Its models identify diagnostically relevant morphology without pixel-level annotation and support research and clinical applications.
- a) Weak supervision: CLAM trains models using only slide-level labels, while identifying relevant morphological features without pixel-, patch-, ROI-level, or subjective annotation.The models identify well-known features and may reveal new features with diagnostic, prognostic, or therapeutic relevance.
- b) Data efficiency: Quantitative experiments show CLAM is more data efficient than established weakly supervised whole-slide classification approaches, reducing the trade-off between supervision and labeled-slide requirements.Data-efficient training enables classification of rare conditions and patient stratification from small cohorts.
- c) Applicability to multi-class subtyping problems: CLAM applies to multi-class subtyping and can stratify patients between predominant and relatively rare classes, including Clear Cell versus Chromophobe renal cell carcinoma.This extends beyond weakly supervised binary positive/negative diagnosis settings.
- d) Adaptability: CLAM models adapt to independent cohorts, biopsy slides, and cellphone microscopy images, including adaptation from large-tissue-content resections to limited-field-of-view cellphone images.Training on resections and adapting to biopsies enables limited slides while maximizing available training tissue.
- e) Interpretability and explainability as a clinical and research tool: CLAM generates fine-grained heatmaps that isolate tumor and other classification-relevant regions without pixel-level annotation, supporting morphological discovery and secondary pathology review.The paper presents whole-slide heatmaps at high resolution and fine scale as a proof of concept for weakly supervised computer-aided diagnosis.
Online Methods
CLAM is a weakly supervised WSI method that treats each slide as a labeled bag of unlabeled patches and uses trainable class-specific attention pooling for general multi-class classification. It further refines class-specific representations through attention-derived instance clustering, with supervision adapted to mutual-exclusivity assumptions.
- CLAM framework: CLAM treats each WSI as a slide-level labeled bag containing up to hundreds of thousands of unlabeled patches, without pixel- or region-level annotations.This formulation targets weakly supervised computational pathology classification.
- Class-specific attention pooling: Unlike max-pooling MIL, CLAM uses trainable attention-based pooling to aggregate patch representations separately for each class, enabling general multi-class classification.The attention network produces n class-specific attention-score sets and corresponding slide-level representations.
- Instance-level clustering: CLAM adds an instance-level binary clustering objective that uses attention outputs to generate pseudo labels during each training iteration.Clustering is optimized on selected patches rather than all patches, focusing on strongly attended or nearly ignored regions.
- Instance-level clustering: For the ground-truth class, the B lowest-attention patches receive negative-cluster labels and the B highest-attention patches receive positive-cluster labels.This constrains class-characterizing evidence to separate from negative evidence in the patch-level feature space.
- Instance-level clustering: Under mutual exclusivity, CLAM labels top-attended patches from out-of-class branches as negative evidence; without exclusivity, it does not supervise those patches.The clustering task uses the smooth top1 SVM loss for instance-level classification.