Source-linked AI summary
Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images
Tiancheng Lin, Zhimiao Yu, Hongyu Hu, Yi Xu, Chang Wen Chen
TL;DR
Existing WSI MIL methods can learn spurious bag-label correlations from contextual priors rather than label-relevant content. IBMIL uses structural causal modeling and backdoor adjustment for interventional bag prediction, and experiments report consistent improvements across existing MIL methods, while its confounder prior remains an assumption.
Problem
Bag contextual priors can confound WSI bag-label prediction, causing models to rely on irrelevant correlations instead of key instances.
Method
IBMIL adds a third interventional-training stage that approximates contextual confounders with clustered bag features and applies backdoor adjustment.
Results
IBMIL consistently improves compared MIL methods across feature extractors and aggregators on Camelyon16 and TCGA-NSCLC.
Takeaways & Limitations
IBMIL is presented as a general deconfounded bag-level prediction framework that is orthogonal to existing bag MIL methods.
Takeaways & Limitations
IBMIL assumes a uniform prior over confounders, while more reasonable expert-informed priors are left for future work.
Abstract
from arXiv · showhide
Multi-instance learning (MIL) is an effective paradigm for whole-slide pathological images (WSIs) classification to handle the gigapixel resolution and slide-level label. Prevailing MIL methods primarily focus on improving the feature extractor and aggregator. However, one deficiency of these methods is that the bag contextual prior may trick the model into capturing spurious correlations between bags and labels. This deficiency is a confounder that limits the performance of existing MIL methods. In this paper, we propose a novel scheme, Interventional Bag Multi-Instance Learning (IBMIL), to achieve deconfounded bag-level prediction. Unlike traditional likelihood-based strategies, the proposed scheme is based on the backdoor adjustment to achieve the interventional training, thus is capable of suppressing the bias caused by the bag contextual prior. Note that the principle of IBMIL is orthogonal to existing bag MIL methods. Therefore, IBMIL is able to bring consistent performance boosting to existing schemes, achieving new state-of-the-art performance. Code is available at https://github.com/HHHedo/IBMIL.
1. Introduction
WSI classification uses MIL to handle gigapixel slides and unlabeled patches, but bag contextual priors can create spurious bag-label correlations. IBMIL addresses this confounding through causal intervention and improves existing MIL methods across WSI datasets.
- Motivation: WSIs are modeled as labeled bags of unlabeled image patches because their gigapixel resolution and expensive annotation complicate direct analysis.The framework applies to WSI classification and other downstream tasks.
- Problem: Conventional MIL improves feature extractors and aggregators but leaves bag contextual priors that can associate irrelevant shared information with labels.For example, staining colors may correlate with positive or negative bags.
- Problem: Contextual bias can produce correct predictions with unreasonable attention, such as emphasizing disease-irrelevant instances outside relevant tissue regions.The causal view treats this context as a confounder opening a backdoor path between bags and labels.
- Contribution: IBMIL adds interventional training using a structural causal model and backdoor adjustment instead of directly inferring with P(Y |X).The intervention targets P(Y |do(X)) to suppress contextual confounding.
- Results: Experiments on Camelyon16 and TCGA-NSCLC show consistent performance boosting across existing feature extractors and aggregation networks.Ablation studies and analyses further support the effectiveness of interventional training.
2. Related Work
Related MIL work addresses noisy instance supervision, bag-level modeling, and causal inference, while IBMIL differs by applying backdoor adjustment as a general bag-level WSI classification framework.
- Instance-level MIL: Instance-level MIL assigns scores to instances and aggregates them, but unbalanced WSIs can make propagated bag labels noisy at the instance level.Camelyon16 may contain only a small portion of disease-positive tissue in a positive slide.
- Instance-level MIL: Methods improve instance selection or annotation quality through cleaner labels, thresholding, or top-k selection, each modifying the instance-level baseline.These approaches seek to preserve informative instances while reducing noisy supervision.
- Two-stage MIL: WSI MIL commonly separates feature-extractor and aggregation training because storing all patch intermediates for backpropagation is impractical.Recent frameworks therefore use a two-stage modeling procedure.
- Causal MIL: Existing causal MIL methods address different treatments or instance-level tasks, whereas IBMIL uses backdoor adjustment to empower bag-level WSI classification.IBMIL is presented as a general framework compatible with existing bag-level MIL methods.
3. Method
The method represents WSIs as bags, extracts and aggregates instance features, then adds causal intervention to remove contextual confounding. IBMIL approximates confounders with clustered bag features and performs backdoor adjustment efficiently.
- MIL formulation: MIL treats each WSI as a labeled bag of unlabeled instances, with the bag label determined by the presence of positive instances.This formulation supports downstream classification despite unavailable instance-level labels.
- MIL formulation: A standard MIL pipeline transforms instances with f, combines features using permutation-invariant pooling σ, and predicts with classifier g.The resulting representation is b_i = f(x_i), B = σ(b_1,...,b_n), and Ŷ = g(B).
- MIL formulation: WSI MIL uses learnable aggregators and staged training because bags can contain thousands of instances and may represent multiple tissue concepts.The average bag length can be about 8,000 instances.
- Causal formulation: The causal graph distinguishes content-driven prediction X →Y from scanner, staining, and preparation effects that can bias WSI appearance.Contextual priors can make models predict from color rather than label-relevant content.
- Causal formulation: IBMIL replaces conventional P(Y |X) with P(Y |do(X)), conceptually cutting the backdoor path through contextual confounders.Randomized enumeration of all contexts is impractical, motivating an approximate intervention.
- Interventional training: Backdoor adjustment averages predictions over confounder strata using their prior probabilities rather than letting confounders depend on X.The formulation sums P(Y | X,h(X,c_i))P(c_i) over the confounder set.
- Interventional training: IBMIL retains feature-extractor and aggregator training, then adds a third causal-intervention stage based on practical backdoor adjustment.The method is not restricted to particular extractor architectures, aggregators, or training paradigms.
- Confounder approximation: The confounder dictionary is built by K-means clustering of training-set bag features and representing each cluster with its average feature.The resulting d × K dictionary approximates global visual-bias strata.
4. Experiments
Experiments on Camelyon16 and TCGA-NSCLC evaluate IBMIL across datasets, feature extractors, aggregators, and design variants. Results show consistent gains, robustness to several choices, and improvements attributable to interventional training rather than simply longer training or post-processing.
- Experimental Results: IBMIL improves all 12 feature-extractor and aggregator combinations across Camelyon16 and TCGA-NSCLC.The evaluation covers binary-class MIL with unbalanced bags and multiclass MIL with balanced bags.
- Experimental Results: 5.4% AUC gain on Camelyon16 and 1.5% on TCGA-NSCLC are reported for ImageNet-pre-trained ResNet.The larger ResNet gains are attributed to greater susceptibility to context patterns than the self-supervised alternatives.
- Ablation on Model Design Variants: IBMIL performance is relatively robust to the confounder-dictionary size across ResNet-18, CTransPath, and ViT.The authors report that a wide range of dictionary sizes can improve performance without elaborate tuning.
- Ablation on Model Design Variants: Freezing confounders exceeds learnable confounders by 1.43% accuracy, with both outperforming the 81.43% baseline.The default configuration therefore keeps confounders unlearnable during interventional training.
- Ablation on Model Design Variants: All tested backdoor-adjustment implementations improve performance, supporting the stability and effectiveness of the intervention.The variants combine the bag feature with the weighted confounder representation in different ways.
- Ablation on Model Design Variants: Training baselines for more epochs does not improve performance and often degrades it, unlike IBMIL’s additional interventional stage.The degradation is associated with overfitting in MIL, while IBMIL’s gains are attributed to backdoor adjustment rather than extra epochs.
- Analysis and Discussion: More unlabeled bags improve AUC for both max- and mean-pooling confounder construction, indicating better confounder approximation.The additional bags combine TCGA and Camelyon16 data.
- Analysis and Discussion: KNN-based evaluation yields limited gains or performance degradation, indicating that IBMIL’s improvements come from interventional training rather than post-processing.The comparison uses a confounder dictionary obtained with class-specific K-means.
5. Conclusions
IBMIL addresses MIL’s bag-context confounding through causal intervention and backdoor adjustment, providing deconfounded bag-level prediction. Experiments show it can significantly boost existing MIL methods, while future work targets more efficient confounder-set approximation.
- 5. Conclusions: IBMIL analyzes confounders between bags and labels to suppress bias from the bag contextual prior.The method uses a structural causal model to reveal causalities and eliminate their effect through backdoor adjustment.
- 5. Conclusions: IBMIL introduces causal intervention as a general approach for bag-level prediction in MIL.The paper presents this intervention as a fresh perspective on the MIL problem.
- 5. Conclusions: IBMIL can significantly boost existing methods across various MIL benchmarks.The conclusion reports comprehensive experiments supporting performance improvements over existing methods.
- 5. Conclusions: Future work will approximate the confounder set more efficiently and elegantly.