Source-linked AI summary
Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images
Noriaki Hashimoto, Daisuke Fukushima, Ryoichi Koga, Yusuke Takagi, Kaho Ko, Kei Kohno, Masato Nakaguro, Shigeo Nakamura, Hidekata Hontani, Ichiro Takeuchi
TL;DR
Cancer subtype classification from whole slide images is difficult because tumor regions are mixed with non-tumor tissue, patch annotations are costly, staining varies, and diagnostically relevant features occur at multiple scales. The paper combines multiple-instance, domain-adversarial, and multi-scale learning in a CNN-based method. On 196 malignant lymphoma cases, it outperformed standard CNN and conventional methods and compared favorably with standard pathologists.
Problem
Unannotated whole slide images make cancer subtype classification difficult because tumor regions are mixed with non-tumor tissue, staining varies, and relevant features occur across magnifications.
Method
The method combines multiple-instance learning, domain-adversarial training, and simultaneous multi-scale learning in a CNN-based classifier.
Results
The method performed significantly better than standard CNN and conventional methods on malignant lymphoma subtype classification and compared favorably with standard pathologists.
Takeaways & Limitations
The combined approach can classify malignant lymphoma subtypes from unannotated histopathological images while attending to tumor regions across multiple magnifications.
Takeaways & Limitations
The formulation assumes each positive bag contains at least a few positive class-specific patches and each negative bag contains none.
Abstract
from arXiv · showhide
We propose a new method for cancer subtype classification from histopathological images, which can automatically detect tumor-specific features in a given whole slide image (WSI). The cancer subtype should be classified by referring to a WSI, i.e., a large-sized image (typically 40,000x40,000 pixels) of an entire pathological tissue slide, which consists of cancer and non-cancer portions. One difficulty arises from the high cost associated with annotating tumor regions in WSIs. Furthermore, both global and local image features must be extracted from the WSI by changing the magnifications of the image. In addition, the image features should be stably detected against the differences of staining conditions among the hospitals/specimens. In this paper, we develop a new CNN-based cancer subtype classification method by effectively combining multiple-instance, domain adversarial, and multi-scale learning frameworks in order to overcome these practical difficulties. When the proposed method was applied to malignant lymphoma subtype classifications of 196 cases collected from multiple hospitals, the classification performance was significantly better than the standard CNN or other conventional methods, and the accuracy compared favorably with that of standard pathologists.
1. Introduction
The paper addresses unannotated histopathological image classification by combining learning mechanisms for tumor localization, staining variation, and multi-scale analysis. Applied to 196 lymphoma cases from 80 hospitals, the method outperformed conventional approaches and compared favorably with pathologists.
- Motivation: Patch-level annotation is prohibitively costly, making subtype classification challenging when tumor labels are unavailable.Whole slide images contain many patches, but annotating each patch is too expensive.
- Challenges: Tumor and non-tumor regions are mixed within whole slide images, so subtype classification must identify tumor-containing regions.The paper frames tumor-region identification as a practical requirement for diagnosis.
- Approach: The proposed CNN combines multiple-instance learning, domain-adversarial normalization, and multi-scale learning to address the three practical difficulties simultaneously.The design is intended to mimic pathologists’ diagnostic practices.
- Evaluation: 196 lymphoma cases collected from 80 hospitals were used to evaluate the proposed method.The cases were H&E-stained histological tissue slides.
- Results: The method performed significantly better than standard CNN and conventional methods, with accuracy comparable to standard pathologists.It also attended to true tumor regions across images at various magnifications, confirmed by immunostaining.
2. Preliminaries
The preliminaries define the weakly supervised, patch-based problem and motivate combining patient-domain normalization with simultaneous multi-scale analysis. MIL uses bag labels without instance labels, while multi-scale patches capture complementary tissue features.
- Problem setup: Whole slide images are too large for direct CNN input, so the method represents them using 224 × 224-pixel patches grouped into bags.Each bag contains instances sampled from one or more image scales.
- Problem setup: The problem assumes positive bags contain at least a few tumor-specific patches, whereas negative bags contain none.This weak supervision avoids requiring labels for individual patches.
- Multiple-instance learning: Multiple-instance learning observes bag labels while leaving instance labels unobserved.A bag is positive when it contains at least one positive instance and negative when all instances are negative.
- Domain variation: Staining differences across specimens and hospitals can substantially degrade classification accuracy, motivating domain-adversarial training and other color-handling methods.Domain-adversarial training treats each patient as an individual domain to reduce reliance on non-task-relevant differences.
- Multi-scale analysis: Pathologists use changing magnifications because low and high scales reveal global tissue structure and detailed nuclear shapes, respectively.The proposed approach uses multiple scales simultaneously within MIL rather than hierarchically or selectively.
3. Proposed method
The proposed method combines single-scale domain-adversarial MIL with a second-stage multi-scale network to classify WSI subtypes from aggregated patch evidence. Attention weights identify informative instances, while domain-adversarial training reduces sensitivity to staining differences.
- Patient-level prediction: The method predicts a patient's subtype by summarizing class-label probabilities from bags of patches sampled from the H&E-stained WSI.For test WSI Xn, the patient-level probability is computed from the bag probabilities p1 and p0.
- Bag prediction: An attention-based bag predictor aggregates feature vectors extracted by a CNN from 224×224-pixel patches into bag class probabilities.The feature extractor maps each patch to a Q-dimensional vector, and the predictor uses attention-weighted features.
- Stage 2: multi-scale learning: In stage 2, the trained feature extractors for all scales are plugged into a multi-scale DA-MIL network whose bags contain patches across scales.Feature vectors from all scales are aggregated to calculate attention weights for multi-scale bag prediction.
- Stage 1: single-scale learning: In stage 1, a separate single-scale DA-MIL network is trained for each image scale using bags containing patches from that scale.The resulting feature extractors are later reused by the multi-scale network.
- Stage 1: single-scale learning: The stage-1 objective combines bag-label prediction loss with attention-weighted domain-adversarial regularization so feature extractors can ignore staining-condition differences.Bag labels use highly attended instances, while the domain regularization penalizes domain-prediction capability.
- Training procedure: Each parameter update uses the patches in each bag as a mini-batch.
4. Experiments
The experiments evaluate malignant lymphoma subtype classification using multi-scale images and compare conventional, MIL, domain-adversarial, and proposed methods. MS-DA-MIL achieved the highest accuracy and its attention maps aligned with tumor-associated CD20-positive regions across magnifications.
- Dataset: The database contained 196 difficult malignant lymphoma cases from 80 institutions, obtained through expert-pathologist consultation.
- Task: The task first discriminated DLBCL from AITL, HLMC, and HLNS using H&E-stained tissue images.
- Experimental setup: The experimental setup used 10x and 20x images, patientwise dataset separation, and 100 randomly extracted 224×224-pixel patches per scale for each bag.
- Results: MS-DA-MIL outperformed DA-MIL, supporting the use of multi-scale input for pathology image classification.
- Attention visualization: DA-MIL attention weights were higher in CD20-positive regions and lower in CD20-negative regions, while MS-DA-MIL cases showed differing contributions from 10x and 20x scales.
- Results: MS-DA-MIL showed the highest classification accuracy among patch-based, attention-based MIL, DA-MIL, and MS-DA-MIL methods.
5. Conclusion
The study combines multiple-instance, domain-adversarial, and multi-scale learning in a CNN for cancer subtype classification from unannotated histopathological images. Applied to 196 malignant lymphoma cases, the method outperformed standard CNN and conventional methods and compared favorably with standard pathologists.
- The proposed CNN combines multiple-instance, domain-adversarial, and multi-scale learning for cancer subtype classification from unannotated histopathological images.
- Applied to 196 malignant lymphoma cases, the method performed significantly better than standard CNN and other conventional methods.
- The proposed method’s accuracy compared favorably with that of standard pathologists.