Source-linked AI summary
Deep Learning for Whole Slide Image Analysis: An Overview
Neofytos Dimitriou, Ognjen Arandjelović, Peter D Caie
TL;DR
Whole slide images create a difficult deep-learning setting because they are massive, heterogeneous, artefact-prone, and often weakly annotated. The paper reviews strategies for selecting and learning from informative patches under slide- or patient-level supervision, reporting promising performance and broader task opportunities while identifying important scope boundaries.
Problem
WSI analysis is challenged by gigapixel scale, morphological heterogeneity, artefacts, limited labelled data, scanner variation, and the computational burden of direct deep-learning application.
Method
The paper reviews patch-based, weakly supervised, unsupervised, reinforcement, transfer, attention, and aggregation methods for training deep networks on WSI data.
Results
Patch-level supervision with hard-negative mining can in many cases match or surpass pathologist accuracy, while reviewed methods also support learning from slide- or patient-level labels.
Takeaways & Limitations
Moving beyond strong supervision opens opportunities for WSI tasks whose ground truth is known only at higher levels, including survivability and recurrence prediction.
Takeaways & Limitations
Patch-based analysis cannot capture information distributed over scales larger than the patch, and some proposed remedies require patch-level annotations.
Abstract
from arXiv · showhide
The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artefacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies.
1 Introduction
Digital pathology is expanding through whole slide imaging, creating opportunities for automated analysis and deep-learning-based diagnostic support. However, WSI scale, morphological variation, artefacts, and limited data challenge direct application of conventional deep learning.
- Digital pathology enables tissue images to be distributed globally for diagnostic, teaching, and research purposes.
- Automated tissue analysis could quantify biomarkers, recognize morphological patterns, reduce reporting workload, shorten turnaround times, and standardize clinical practice.
- WSIs typically reach 100,000 × 100,000 pixels and contain substantial morphological variance and artefacts.
- These properties create simultaneous challenges in visual understanding and hardware capacity for processing WSI data.
2 Whole slide images
Whole slide images provide complementary brightfield and fluorescence information but remain difficult to analyze because of data scarcity, scanner variability, artefacts, and staining variation. Preprocessing and augmentation are used to improve robustness.
- 2.1 Tissue visualization: H&E staining is widely accessible and supports morphological characterization, whereas multiplexed IF adds in situ molecular information.
- 2.1 Tissue visualization: IF WSI datasets are limited because reagents and fluorescence scanners are expensive, while images can exceed 10 gigabytes.
- 2.2 Data availability: Labelled data are harder to obtain in digital pathology, challenging the practicability of supervised approaches, and many studies still use proprietary datasets.
- 2.3 Image format: Different scanners produce incompatible image formats, compression, illumination, objectives, and resolutions, delaying large-dataset curation.
- 2.4 Artefacts and colour variability: Clinical translation requires generalization across patient populations, artefacts, and staining variability introduced during preparation and imaging.
- 2.4 Artefacts and colour variability: Artefact handling uses either explicit filtering and colour normalization or data augmentation with synthetic variability.
3 Deep Learning
WSI deep-learning pipelines reduce gigapixel images to informative patches, then aggregate patch evidence or learn patch selection from slide-level labels. Methods address annotation scarcity through weak, unsupervised, reinforcement, transfer, and attention-based learning, but patch analysis loses larger-scale structure.
- 3.1 Patch extraction: Most successful WSI methods extract a small number of patches rather than processing the entire image.Common patch sizes range from 32 × 32 to 10,000 × 10,000 pixels, with many methods using approximately 256 × 256 pixels.
- 3.1.1 Patch level annotation: Patch-level annotations provide strong supervision but require experts to localize and label relevant tissue at fine spatial resolution.
- 3.1.1 Patch level annotation: Using all tiled patches creates excessive computational and memory overhead, while random sampling can worsen class imbalance.
- 3.1.2 Slide level annotation: Slide-level labels are handled with multiple-instance, unsupervised, reinforcement, transfer, or combined learning approaches that identify informative patches.
- 3.1.2 Slide level annotation: Patch predictions are aggregated through operations such as maximum or average pooling to infer slide-level outcomes.
- 3.2 Beyond patch extraction: Independent patch analysis cannot capture information distributed across scales larger than the patch, including higher-level tumour structure.Proposed remedies include spatial correlations, multiple magnifications, global context, and larger patches, with some requiring patch-level annotations.
4 Discussion
The review situates WSI analysis within deep learning’s progress in automated feature extraction and synthesizes approaches for learning from slide- or patient-level labels. It concludes that moving beyond strong supervision broadens the problems WSI models can address.
- Deep learning advanced computer vision partly by transforming feature extraction from a human-designed stage into a learned process.
- Patch-level labels with hard-negative mining can produce models that in many cases match or surpass pathologist accuracy.
- For slide- or patient-level labels, reviewed methods primarily address where to find signal, either separately from or jointly with visual understanding.
- Weakly supervised, unsupervised, reinforcement, and transfer learning reduce reliance on detailed annotations for massive, heterogeneous images and small datasets.
- This shift opens WSI analysis to tasks whose ground truth is available only above the patch level, such as survivability and recurrence prediction.