Source-linked AI summary
AutoMIL: Automated Multiple Instance Learning for Whole-Slide Image Classification
Jonas Waibel, Frank Kramer, Dominik Müller
TL;DR
Whole-slide image classification with MIL involves many design decisions and requires pathology and machine-learning expertise. AutoMIL automates pipeline steps and, with default settings, achieved results comparable to published methods across three datasets, with ensembling most beneficial on imbalanced data.
Problem
MIL pipelines for whole-slide image classification involve many interdependent design choices and require expertise in pathology and machine learning.
Source: §1 · p. 1Method
AutoMIL automates the workflow from raw slides and annotations through preprocessing, training, and evaluation, using defaults that adapt to data and hardware.
Source: §1 · p. 2, §2.1 · p. 2Results
Takeaways & Limitations
Ensembling performed best or equal-best on minority classes in imbalanced UBC-OCEAN but gave no gain on balanced TCGA-NSCLC.
Source: §4 · p. 4PCam patches were split randomly because patient information was unavailable, so patches from the same lymph-node section may occur in both training and test data.
Source: §4 · p. 4
Abstract
from arXiv · showhide
Developing multiple instance learning pipelines for the classification of whole-slide images involves many interdependent design decisions and expertise in both pathology and machine learning. We present AutoMIL, an open-source framework that applies principles from the field of automated machine learning to tackle this task by automating integral pipeline steps, such as preprocessing, training and evaluation. In a benchmark evaluation with default settings, AutoMIL reached ROC-AUC scores of 0.94 on UBC-OCEAN, 0.99 on PCam and 0.93-0.96 on a TCGA-NSCLC subset, comparable to specialised pipelines in related work. Ensembling mainly improved performance on imbalanced data, showing that an automated pipeline must adapt to dataset characteristics. AutoMIL thus provides an accessible baseline for whole-slide image classification without manual pipeline configuration.
1. Introduction
Whole-slide images are too large and weakly annotated for conventional CNNs, while MIL pipelines require many choices and domain expertise. AutoMIL automates this workflow and evaluates its performance across three public datasets.
1. Introduction
- Gigapixel whole-slide images with slide-level annotations are unsuitable for conventional CNN models, while MIL treats each slide as a bag of tiles.Building an MIL pipeline also involves choices about tissue detection, tile resolution, feature extraction, architecture, and training hyperparameters.Source: §1 · p. 1
- General-purpose AutoML does not handle gigapixel images or bag construction, and WSI toolkits leave users to select and configure building blocks.Developing a pipeline for a new dataset therefore takes time and domain knowledge.Source: §1 · p. 1
- AutoMIL automates the workflow with defaults that adapt to data and hardware, enabling training and evaluation from raw slides and annotations with one command.The authors evaluate it on three public datasets with different class balance, slide heterogeneity, and bag structure, comparing it with published methods.Source: §1 · p. 2
2. Methods
AutoMIL automates WSI processing, model training, and evaluation, with resource-aware training and automatic ensembling. The experiments use three datasets and default settings with specified splits and model configurations.
2.1. AutoMIL
- AutoMIL runs preprocessing, training, and evaluation from one command, selects a batch size to fit GPU memory, and automatically averages class probabilities across folds or resolutions.It is built on Slideflow for WSI processing and FastAI for training; defaults include class-weighted cross-entropy and AdamW.Source: §2.1 · p. 2
- Table 1 maps AutoMIL's automation scope onto the four AutoML stages described by He et al..The table is identified as an overview of automation scope per stage.Source: §2.1 · p. 2, Table 1 · p. 2
2.2. Datasets
- UBC-OCEAN has 538 ovarian cancer slides and tissue microarrays across five imbalanced subtypes; PCam has 220,025 labelled patches, and TCGA-NSCLC uses 100 sampled slides balanced across two cancer types.PCam patches are treated as single-instance bags because labels are per patch.Source: §2.2 · p. 2
2.3. Experimental Setup
- Experiments use default settings and 80/20 splits, with patient-level splits for UBC-OCEAN and TCGA-NSCLC; each configuration is trained once.UBC-OCEAN uses an AttMIL ensemble across 5×, 10×, and 20×; PCam compares three architectures, while TCGA-NSCLC uses TransMIL with three-fold cross-validation and 20 held-out slides.Source: §2.3 · p. 3
3. Results
Results varied by dataset: AttMIL achieved a high ROC-AUC on PCam, while the UBC-OCEAN ensemble improved overall accuracy and F1-score and performed well on rare classes. On TCGA-NSCLC, ensembling did not exceed the best fold.
3. Results
- 0.81 balanced accuracy was reached by the UBC-OCEAN ensemble, whose per-class accuracy ranged from 0.92 for LGSC to 0.67 for endometrioid carcinoma.The ensemble improved accuracy and F1-score; endometrioid carcinoma was most often confused with high-grade serous carcinoma.Source: §3 · p. 3, Figure 1 · p. 3
- 0.99 ROC-AUC on full PCam was achieved by AttMIL; on the smaller subset, TransMIL reached 0.92 accuracy, while the TCGA-NSCLC ensemble matched the best fold at 0.90 accuracy.Both transformer models stopped after 10 epochs on full PCam when validation loss failed to improve.Source: §3 · p. 3, Table 2 · p. 4
- On full PCam, TransMIL assigned every patch to the negative class, yielding an F1-score of 0 despite a ROC-AUC of 0.94; the threshold maximizing correct classifications was about 0.22.On the two rarest UBC-OCEAN classes, the ensemble achieved the highest or equal-highest accuracy, mainly by improving LGSC, while the 20× model performed best on EC.Source: §3 · p. 3
4. Discussion
With shared defaults and no task-specific tuning, AutoMIL produced models within the range of published results across three datasets. Ensembling’s benefit varied with dataset characteristics, while the authors identify data-split and evaluation constraints that limit comparisons.
4. Discussion
- With shared defaults and no task-specific tuning, AutoMIL produced models within the range of published results on all three datasets.On UBC-OCEAN, its balanced accuracy lay within the winning challenge solution’s cross-validation range, while that solution reached only 0.66 on the hidden test set [14].Source: §4 · p. 4
- Ensembling performed best or equal-best on minority classes in imbalanced UBC-OCEAN but gave no gain on balanced TCGA, with gains more likely from probability-averaging variance reduction than multiscale information.Source: §4 · p. 4
- On PCam’s single-instance bags, transformers seemed unable to use inter-instance relations and overfit quickly, pointing to architecture selection by bag structure and validation-threshold calibration.TransMIL’s failure despite high ROC-AUC suggests a thresholding or calibration issue rather than poor class separation.Source: §4 · p. 4
- Table 3 indicates a range, not a ranking, because published results used different protocols.The UBC-OCEAN challenge test set is not public, the TCGA test set has 20 slides, PCam’s random patch split may place patches from one section in both sets, and each configuration was trained once.Source: §4 · p. 4
5. Conclusions
AutoMIL’s default settings reached performance comparable to published methods on three heterogeneous datasets, prioritizing accessibility and standardization over task-specific optimization. The authors identify additional optimization and configurability as possible future work.
5. Conclusions
- With default settings, AutoMIL reached performance comparable to published methods on three heterogeneous datasets.Source: §5 · p. 5
- The pipeline prioritizes accessibility and standardization over task-specific performance optimization.Source: §5 · p. 5
- Future work may add hyperparameter optimization, architecture selection, threshold calibration, and configurability.Source: §5 · p. 5