Source-linked AI summary
RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification
Shujun Wang, Yaxi Zhu, Lequan Yu, Hao Chen, Huangjing Lin, Xiangbo Wan, Xinjuan Fan, Pheng-Ann Hen
TL;DR
Gastric WSI classification must identify informative abnormal regions despite very large slides and lesions of varying size. The paper proposes a two-stage discriminative-patch selection and RMDL pipeline, achieving 86.5% test accuracy and a 4.0% margin over a state-of-the-art multi-instance learning method. The study also introduces a pixel-level annotated gastric WSI dataset, while noting that the two stages were trained separately because of technical constraints.
Problem
Large WSIs and small abnormal regions make it difficult to select informative regions and aggregate patch information for automated gastric classification.
Method
The framework selects discriminative patches with a localization network and uses RMDL to recalibrate instance features according to learned importance coefficients for image-level prediction.
Results
86.5% classification accuracy was achieved on testing images, with RMDL outperforming a state-of-the-art multi-instance learning method by 4.0%.
Takeaways & Limitations
The method achieved the best performance on the constructed WSGI dataset and provides an option for considering differing patch contributions in gastric WSI classification.
Takeaways & Limitations
The two-stage framework was trained separately because of technical issues such as limited GPU memory, so localization-derived instance features may not be optimal for RMDL.
Abstract
from arXiv · showhide
The whole slide histopathology images (WSIs) play a critical role in gastric cancer diagnosis. However, due to the large scale of WSIs and various sizes of the abnormal area, how to select informative regions and analyze them are quite challenging during the automatic diagnosis process. The multi-instance learning based on the most discriminative instances can be of great benefit for whole slide gastric image diagnosis. In this paper, we design a recalibrated multi-instance deep learning method (RMDL) to address this challenging problem. We first select the discriminative instances, and then utilize these instances to diagnose diseases based on the proposed RMDL approach. The designed RMDL network is capable of capturing instance-wise dependencies and recalibrating instance features according to the importance coefficient learned from the fused features. Furthermore, we build a large whole-slide gastric histopathology image dataset with detailed pixel-level annotations. Experimental results on the constructed gastric dataset demonstrate the significant improvement on the accuracy of our proposed framework compared with other state-of-the-art multi-instance learning methods. Moreover, our method is general and can be extended to other diagnosis tasks of different cancer types based on WSIs.
1. Introduction
Automated gastric WSI classification must identify informative abnormal regions despite enormous slide size, subtle patch-level variation, and abnormal areas occupying only a small fraction of each slide. The paper addresses these challenges with a two-stage framework that selects discriminative patches and recalibrates their contributions for image-level prediction.
- Gastric WSI analysis matters clinically, but manual examination is subjective and time-consuming because slides can reach 100,000 × 100,000 pixels.
- Classification is difficult because histopathology patches have large intra-class and small inter-class texture and morphology variation.
- Abnormal regions may occupy a small portion of a WSI, allowing normal tissue to suppress discriminative information during classification.
- The proposed framework first selects discriminative patches with a localization network, then performs image-level prediction using recalibrated multi-instance learning.
- RMDL captures interrelationships among patches and weights their differing contributions through instance-feature recalibration before image-level classification.
- The study constructs a 608-image gastric WSI dataset and reports that RMDL outperforms other multi-instance learning methods.
2. Dataset
The WSGI dataset is a newly constructed gastric whole-slide image resource with pixel-level annotations and a defined training/testing split. Its accompanying workflow illustrates localization followed by RMDL-based image-level prediction.
- The WSGI dataset contains 608 whole-slide images collected from different patients and stained with Hematoxylin and Eosin.
- WSGI slides were scanned at 40X magnification with 0.2517 µm/pixel resolution and stored in a multi-resolution pyramid.
- A pathologist with seven years of experience manually created polygon annotations around dysplasia and cancer regions.
- The dataset was divided into 408 training images and 200 testing images.
- The method flowchart shows localization-based discriminative-instance selection followed by RMDL modules for image-level label prediction.
3. Methodology
The methodology uses a two-stage pipeline that selects discriminative patches before applying RMDL to aggregate image-level evidence. RMDL fuses local and global information, recalibrates instance contributions, and pools the resulting features for classification.
- Framework: The two-stage framework first selects discriminative patches and then predicts the whole-slide image label with RMDL.Patch selection uses a localization network, while RMDL performs image-level classification.
- Discriminative Instance Selection: A fully convolutional localization network detects abnormal regions efficiently across large whole-slide images.It adapts Inception ResNet v2 by replacing the original classification components with pooling and convolutional layers.
- Discriminative Instance Selection: Training patch selection uses annotated abnormal regions and randomly sampled normal regions to supervise localization.Abnormal patches are extracted with an overlapped sliding window, while normal patches come from remaining or normal-slide regions.
- Discriminative Instance Selection: Non-maximum suppression selects the top discriminative patches from the localization probability map while limiting excessive overlap.The selected instances form the input to the RMDL image-level prediction stage.
- RMDL Network: RMDL combines local instance features with global slide features to capture dependencies among selected instances.Global features are formed by pooling intermediate representations and are tiled with instance features before recalibration.
- RMDL Network: Instance recalibration learns importance coefficients from fused features and applies them element-wise to the original instance features.The mechanism is applied at the instance level rather than the channel level and uses a simple fully connected recalibration layer with softmax normalization.
- RMDL Network: Average pooling aggregates recalibrated instance features into a global image-level feature, followed by a fully connected layer and softmax prediction.This pooling operation produces the final whole-slide classification output.
4. Experiments and Results
Experiments evaluate RMDL through qualitative localization, comparisons with multi-instance learning methods, ablations, instance-count sensitivity, and testing-time computation. Results show effective discriminative-region selection, strong classification performance, robustness to selected-instance counts, and an average total testing cost of 93.79 seconds.
- Qualitative Evaluation of Discriminative Instance Selection: The localization network accurately identifies dispersed and tiny cancer and dysplasia regions, while selection removes low-probability false-positive outliers.These visualizations support extracting discriminative regions for image-level prediction.
- Quantitative Comparison with Other Methods: RMDL achieves the best performance among compared multi-instance deep learning methods, while MAXMIN-Layer also produces competitive results.The comparison includes handcrafted-feature and CNN-based methods, including Attention-MIP, MISVM, and MIMLNN.
- Quantitative Comparison with Other Methods: RMDL achieves the highest accuracy for each image type, and its dysplasia and cancer AUC values exceed those of Attention-MIP and MAXMIN-Layer.The ROC curves use a one-vs.-rest strategy, and the confusion matrices compare the three methods across image types.
- Analysis of Our Method: The ablation study evaluates multi-instance pooling, instance recalibration, and local-global feature fusion as components of the RMDL network.The baseline contains multi-instance pooling and a classification layer; the supplied results state that combining both modules boosts image-level accuracy and average score.
- Analysis of Our Method: Classification accuracy fluctuates only slightly across different numbers of selected discriminative instances, indicating robustness when the most important instances are selected.The experiment analyzes the influence of the selected-instance count m′.
- Computation Cost Analysis: 93.79 seconds is the average total testing cost for slides averaging 61858 × 43834 pixel2, split between instance selection and image-level prediction.Table 5 reports average, smallest-slide, and largest-slide costs for the two testing stages.
5. Discussion
The paper addresses automated gastric WSI diagnosis by combining discriminative region selection with recalibrated multi-instance learning, while demonstrating attention-based instance weighting and identifying training and dataset limitations.
- Challenges: Gastric WSI diagnosis is difficult because large images, small lesion proportions, and weakly supervised label assumptions hinder automated classification.The paper highlights large image scale, disproportionate lesion areas, and limited generalization of methods assuming most patches share the WSI label.
- Evaluation: The framework uses confusion matrices and one-vs.-rest ROC analysis to evaluate classification results across Normal, Dysplasia, and Cancer categories.The figures define the class abbreviations and explain that true- and false-positive rates are calculated one-vs.-rest.
- Method: RMDL recalibrates instance features using learned importance coefficients, automatically emphasizing crucial instances for image-level prediction.The recalibration process is described as selective attention to different aspects of the image.
- Dataset: The authors built the WSGI dataset with pixel-level annotations because no public benchmark dataset existed for gastric histopathology image classification.The dataset was created in response to the time and labor required to annotate whole-slide images.
- Limitation: The two-stage framework was trained separately because of technical constraints, so localization-derived instance features may not be optimal for RMDL.The authors identify end-to-end parallel training as future work.
6. Conclusions
The study proposes a two-stage gastric WSI classification pipeline that selects discriminative patches and aggregates them with RMDL. On testing images, it achieved 86.5% accuracy and exceeded the state-of-the-art multi-instance learning method by 4.0%.
- Method: The proposed pipeline selects discriminative patches before predicting image-level labels with recalibrated multi-instance learning.RMDL adaptively aggregates patch information by recalibrating instance features according to learned importance coefficients.
- Results: 86.5% classification accuracy was achieved on the testing images.The result is reported for the proposed method during evaluation.
- Results: 4.0% was the reported margin by which RMDL outperformed the state-of-the-art multi-instance learning method.The comparison is stated as a large margin in the evaluation.
- Future work: Further work is needed on semi-supervised learning to reduce annotation effort and model compression to speed detection.These directions are identified as necessary future investigations.