Source-linked AI summary
DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification
Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah E. Coupland, Yalin Zheng
TL;DR
WSI MIL must learn from few slide labels despite enormous slides containing many patches and sparse positive regions. The paper introduces pseudo-bags, double-tier feature distillation, and AB-MIL instance-probability derivation to address this setting. DTFD-MIL reports superior performance on CAMELYON-16 and TCGA lung cancer datasets, while pseudo-bag labeling introduces controllable noise.
Problem
WSI classification has few labeled slides but many patches, sparse positive regions, and overfitting risks that complicate positive-instance recognition.
Method
DTFD-MIL randomly partitions slide patches into pseudo-bags, derives AB-MIL instance probabilities, and uses double-tier feature distillation across pseudo-bags.
Results
DTFD-MIL provides superior performance to other latest methods on the CAMELYON-16 and TCGA lung cancer datasets, while derived instance probabilities more reliably localize positive activations than attention scores.
Takeaways & Limitations
The framework offers a new perspective for WSI MIL beyond mutual-instance relations, and its instance-probability derivation may support related MIL extensions.
Takeaways & Limitations
Random pseudo-bag allocation introduces a proportion of noisy pseudo-bag labels, with noise roughly controlled by the number of pseudo-bags M.
Abstract
from arXiv · showhide
Multiple instance learning (MIL) has been increasingly used in the classification of histopathology whole slide images (WSIs). However, MIL approaches for this specific classification problem still face unique challenges, particularly those related to small sample cohorts. In these, there are limited number of WSI slides (bags), while the resolution of a single WSI is huge, which leads to a large number of patches (instances) cropped from this slide. To address this issue, we propose to virtually enlarge the number of bags by introducing the concept of pseudo-bags, on which a double-tier MIL framework is built to effectively use the intrinsic features. Besides, we also contribute to deriving the instance probability under the framework of attention-based MIL, and utilize the derivation to help construct and analyze the proposed framework. The proposed method outperforms other latest methods on the CAMELYON-16 by substantially large margins, and is also better in performance on the TCGA lung cancer dataset. The proposed framework is ready to be extended for wider MIL applications. The code is available at: https://github.com/hrzhang1123/DTFD-MIL
1. Introduction
WSI classification is difficult because slides are enormous, weakly labeled, and contain many patches but relatively few bags, worsening overfitting and positive-instance identification. DTFD-MIL addresses these constraints with pseudo-bags, instance-probability derivation, and a double-tier feature-distillation framework.
- WSIs range from 100M to 10G pixels, making direct transfer of existing machine-learning techniques infeasible.
- MIL treats each WSI as a bag of hundreds or thousands of patches, using slide-level labels to determine the bag class.
- Only a few hundred slide labels guide training, creating overfitting and inferior generalization when learned features are less relevant to disease.
- The framework derives instance probabilities under AB-MIL and uses them for feature distillation in Tier-1 and Tier-2 MIL models.
- Positive tissue often occupies a small portion of diseased slides, making positive-instance recognition difficult when slides contain many patches but few labels.
- Pseudo-bags randomly partition each slide’s patches into smaller labeled bags, virtually increasing bag count while enabling double-tier MIL.
2. Related Works
Related work covers weakly supervised MIL for WSI analysis, distinguishing instance-prediction aggregation from feature-based bag embeddings. The paper connects AB-MIL to image-classification architectures and uses Grad-CAM to derive instance probabilities.
- WSI MIL methods use only slide labels and aggregate either instance predictions or instance features to produce bag predictions.
- AB-MIL obtains a bag embedding by weighting individual instance features, with methods differing in how attention scores are generated.
- Grad-CAM extends class activation maps to complex multilayer architectures and supports localization using image-level labels.
- The paper shows AB-MIL is a special case of deep image classification, enabling Grad-CAM-based derivation of positive instance probabilities.
3. Method
DTFD-MIL randomly divides slide patches into pseudo-bags, applies a Tier-1 attention-based MIL model, distills features, and classifies the parent slide with Tier-2 MIL. It derives instance probabilities for AB-MIL using Grad-CAM and evaluates four feature-distillation strategies.
- MIL formulation: MIL models represent a bag from instance predictions or aggregated instance features, with attention-based MIL learning instance weights for bag representation.The extracted instance feature h_k is aggregated into a bag representation F before classification.
- Instance probability derivation: AB-MIL is formulated as a special case of a deep-learning image-classification network, enabling Grad-CAM to derive each instance’s class signal and probability.The derivation uses the attention-weighted instance feature and applies soft-max to obtain the corresponding probability.
- Double-tier framework: Slide instances are randomly split into M approximately even pseudo-bags, each inheriting its parent slide label before separate Tier-1 AB-MIL processing.The pseudo-bag strategy virtually increases the number of bags while retaining slide-level supervision.
- Double-tier framework: Tier-1 outputs produce one distilled feature per pseudo-bag, which are forwarded to Tier-2 AB-MIL for parent-bag inference and jointly supervised with Tier-1.The framework defines separate Tier-1 and Tier-2 losses and an overall optimization process.
- Training consideration: Pseudo-bag labels introduce noise when random allocation omits positive instances, while the noise level is roughly controlled by the number of pseudo-bags M.The effect of M is examined in an ablation study.
- Feature distillation strategies: Four distillation strategies are evaluated: maximum probability, maximum-and-minimum probability, maximum attention score, and aggregated features.MaxS selects one instance, MaxMinS concatenates two instance features, MAS uses maximum attention, and AFS aggregates all instances.
4. Experiments
Experiments evaluate DTFD-MIL on CAMELYON-16 and TCGA lung cancer using AUC as the primary metric, alongside accuracy and F1. Results show strong performance, improved localization from derived instance probabilities, and dataset-dependent effects of pseudo-bag and distillation choices.
- Evaluation setup: The study evaluates DTFD-MIL on CAMELYON-16 and TCGA lung cancer, with AUC as the primary metric plus slide-level accuracy and F1.CAMELYON-16 uses repeated experiments with 95% confidence intervals; TCGA uses patient-level splitting and four-fold cross-validation with standard variances.
- Performance comparison: Instance-level Mean Pooling and Max Pooling generally underperform bag embedding-based methods on both datasets.The comparison includes conventional MIL, RNN-MIL, AB-MIL, and other recent methods with broadly similar complexity except Trans-MIL.
- Performance comparison: At least 4% higher AUC is reported for DTFD-MIL(AFS) than other existing methods on CAMELYON-16.The other three DTFD-MIL variants show similar strong performance, while MaxS is weaker but generally remains competitive.
- Performance comparison: 96.1% AUC is achieved by DTFD-MIL(MaxMinS) on TCGA lung cancer.The proposed methods generally achieve leading performance, but their advantage is less pronounced because positive slides contain larger tumor regions.
- Visualization of detection results: Derived instance probabilities produce more consistent, accurate, and higher-contrast positive-region localization than normalized attention scores.They also reduce strong false-positive regions observed in attention-score heatmaps for ground-truth negative slides.
- Ablation study: Tier-2 MIL generally outperforms Tier-1 MIL, especially on CAMELYON-16, where small tumor regions can leave pseudo-bags without positive instances.Pseudo-bags benefit both tiers, but Tier-1 performance is more sensitive to their number on CAMELYON-16 than on TCGA lung cancer.
5. Conclusion
The paper derives instance probabilities for AB-MIL and uses them with pseudo-bags in a double-tier DTFD-MIL framework. Experiments support the framework’s superior performance and the derived probability’s usefulness for positive-region detection and future MIL development.
- Conclusion: The paper derives instance probabilities under AB-MIL and qualitatively finds them more reliable than attention scores for positive-region detection.The derivation is motivated by viewing AB-MIL as a special case of a deep learning architecture compatible with Grad-CAM.
- Conclusion: DTFD-MIL combines pseudo-bags, double-tier MIL, and the derived instance probability for feature distillation.Pseudo-bags virtually increase the number of bags by splitting slide patches into smaller labeled groups.
- Conclusion: Experiments report superior performance to other latest MIL methods on two public histopathology WSI datasets.The conclusion presents DTFD-MIL as a different perspective from approaches based on mutual-instance relations.
- Conclusion: The instance probability derivation is presented as a potentially useful tool for developing and analyzing related MIL models.This anticipated use extends beyond its role in constructing and analyzing DTFD-MIL.