Source-linked AI summary
Machine learning methods for histopathological image analysis
Daisuke Komura, Shumpei Ishikawa
TL;DR
The accumulation of digital histopathological images creates demand for machine-learning analysis, especially computer-aided diagnosis, while pathology-specific image and task characteristics require special processing. This mini-review presents applications, learning methods, and domain-specific problems with possible solutions. It concludes that these developments support more sophisticated digital histopathological image analysis, although important limitations remain.
Problem
The growing volume of digital pathological images requires analysis, but their unique characteristics and tasks often demand processing beyond general image-recognition methods.
Method
The review surveys machine-learning applications, learning paradigms, pathology-specific problems, and proposed computational solutions for digital pathological image analysis.
Results
The review identifies machine learning as supporting increasingly sophisticated digital histopathological image analysis, including applications involving multiple magnifications.
Takeaways & Limitations
Digital pathology analysis should account for pathology-specific image characteristics and can use multiple levels of magnification depending on disease type.
Takeaways & Limitations
The review notes that many identified problems remain open and that the optimal magnification level is still controversial.
Abstract
from arXiv · showhide
Abundant accumulation of digital histopathological images has led to the increased demand for their analysis, such as computer-aided diagnosis using machine learning techniques. However, digital pathological images and related tasks have some issues to be considered. In this mini-review, we introduce the application of digital pathological image analysis using machine learning algorithms, address some problems specific to such analysis, and propose possible solutions.
1. Introduction
Digital histopathological images are accumulating rapidly, creating demand for machine-learning analysis to assist diagnosis. This review takes a pathology-oriented view of applications, image-specific problems, and possible solutions.
- Whole slide scanners convert entire glass slides into digital images, enabling machine-learning analysis of accumulating WSIs for diagnostic assistance.
- Digital pathological images and tasks have unique characteristics, so general image-recognition methods often require special processing techniques.
- The review describes machine-learning applications in digital pathological image analysis and problems specific to this domain.
- Rather than reviewing general machine-learning algorithms broadly, the article provides a more pathology-oriented perspective than recent related reviews.
- Deep learning has replaced many image-recognition techniques since its strong ImageNet 2012 performance, and this shift also applies to pathological image analysis.
2. Machine learning methods
Histopathological image analysis uses supervised, unsupervised, and derivative learning paradigms, with preprocessing, feature extraction, and classification adapted to digital pathology tasks. Deep learning jointly optimizes features and classifiers, and learned features often outperform traditional ones.
- A typical workflow samples local mini patches from large WSIs, extracts features, and classifies each patch as cancer or non-cancer.
- Traditional local features include GLCM and LBP, while deep learning begins with feature extraction and simultaneously optimizes features and classifiers.
- Learned deep-learning features often outperform other traditional features in histopathological image analysis.
- Supervised learning maps input images to labels, whereas unsupervised learning describes hidden structures in unlabeled images.
- Supervised algorithms include support vector machines, random forests, and convolutional neural networks; unsupervised methods include k-means, autoencoders, and principal component analysis.
- Derivative paradigms include semi-supervised learning and multiple instance learning.
3. Machine learning application in digital pathology
Digital pathology applications span diagnosis, image retrieval, and discovery of clinicopathological relationships. Machine learning can support interpretation and large-scale analysis, but labeled data remain difficult to obtain.
- Computer-assisted diagnosis: Computer-assisted diagnosis maps one or more WSIs to disease categories and is therefore essentially a supervised learning task.
- Computer-assisted diagnosis: Machine-learning errors reportedly differ from human pathologist errors, while CAD may improve accuracy, reduce interpretation variability, and investigate all WSI pixels.
- Other pathology tasks: Other diagnosis-related applications include ROI detection or segmentation, immunostaining scoring, cancer staging, mitosis detection, gland segmentation, and vascular-invasion quantification.
- Content Based Image Retrieval: CBIR retrieves similar images for diagnosis, education, and research, including relevant cases for learners and rare cases for professional pathologists.
- Content Based Image Retrieval: CBIR can use unsupervised learning without labels, while supervised approaches may learn better similarity measures when labels are available.
- Discovering new clinicopathological relationships: Machine learning can analyze relationships between pathological morphology and genomic or clinical information, including stromal texture correlations with breast-cancer prognosis.
4. Problems specific to histopathological image analysis
Histopathological image analysis faces domain-specific challenges involving scale, representation, color, artifacts, and annotation. The review surveys computational solutions including transfer learning, multiscale methods, normalization, artifact detection, and reduced-label learning.
- The review organizes pathology-specific problems and corresponding solutions in an overview of prior studies.
- Learning and interaction: The reviewed learning strategies also include active learning, multiple instance learning, semi-supervised learning, GUI tools, and tracking pathologists’ behavior.
- Computational solutions: Transfer learning uses feature extraction and fine-tuning to adapt existing models for digital pathology analysis.
- Scale and representation: Different magnifications provide different information, motivating multiscale analysis with CNNs, dictionary learning, and texture features.
- Scale and representation: WSIs can be treated as orderless texture-like images, motivating traditional texture features and CNN-based textures.
- Image quality: Color variation and artifacts such as blur and tissue folds are addressed with color normalization, color augmentation, and artifact detection.
4.1. Very large image size
Whole-slide images are too large for direct analysis: resizing loses cellular information, while retaining resolution creates major computational and integration challenges. Patch-based analysis is therefore common, but combining patch decisions remains difficult.
- WSIs can contain tens of billions of pixels, making direct analysis computationally and memory intensive.Resizing an entire WSI to 256 × 256 can markedly reduce identification accuracy by losing cellular-level information.
- Dividing WSIs into approximately 256 × 256 patches preserves local detail while enabling independent regional analysis.
- Hundreds of thousands of patches per WSI make false positives likely even when individual patches are accurately classified.
- Regional averaging can suppress isolated patch errors, but may miss small ROIs such as isolated tumor cells.
- IHC scoring and cancer staging require more sophisticated integration of patch-level or object-level decisions across multiple regions.
4.2. Insufficient labeled images
Histopathological machine learning is constrained by scarce, costly regional labels: most WSIs have only case-level labels, while accurate annotation requires pathologists and substantial labor. The review describes data-efficient labeling, weak or unlabeled supervision, and reuse of models from other tasks as responses.
- Only small amounts of patch- or pixel-level annotation are available, while regional labeling requires pathologists and substantial labor.
- Public datasets provide useful annotations when disease, stain, magnification, and resolution match the intended analysis, but their disease- and cell-specific coverage leaves many tasks unsupported.
- Case-level labels in large WSI repositories require preprocessing or specialized algorithms before they can be used for training.
- Efficient labeling: Active learning selects valuable unlabeled samples for pathologist annotation and may shorten labeling time by achieving comparable discrimination performance with fewer labeled images.
- Weak or unlabeled information: Weakly supervised and multiple instance learning use WSI-level bag labels, where a positive bag contains at least one positive patch.
- Reusing parameters from another task: Transfer learning initializes models with parameters learned on another task, and ImageNet-pretrained features have achieved state-of-the-art performance in pathological image classification and segmentation.
4.3. Different levels of magnification result in different levels of information
Different magnifications capture complementary histopathological information: high magnification emphasizes cellular shape, whereas lower magnification captures larger tissue structures. Combining magnifications can improve accuracy in some disease and tissue settings.
- High-power images capture cell shape, while lower-power images better capture structures such as glands composed of many cells.
- Cancerous tissues contain both cellular and structural atypia, making information from multiple magnifications potentially important for diagnosis.
- Images are often resized to represent different magnifications because original-resolution images are difficult to process directly.
- Using high- and low-magnification images simultaneously sometimes improves accuracy, depending on disease, tissue, and machine learning algorithm.
4.4. WSI as orderless texture-like image
Histopathological images have repetitive, texture-like structure rather than a straightforward object-centered organization. CNNs and deep texture representations exploit this structure, although the best representation depends on the analyzed object.
- Pathological images contain repetitive patterns and are described as closer to texture than to conventional objects.
- CNN pooling and shifted data augmentation provide a degree of shift invariance and can learn texture-like tissue structure.
- Deep texture representations use feature-map correlations from a CNN layer to encode tissue texture.
- Converting CNN features into texture representations can provide cell-position invariance while retaining CNN-learned representations.
- The appropriate representation depends on whether the target is a single cell, a larger structure, or another analyzed object.
4.5. Color variation and artifacts
Digital pathological images acquire artifacts and color variation during slide preparation, staining, and scanning, requiring task-appropriate preprocessing and artifact handling.
- Artifacts: Slide preparation and scanning can introduce folds, dust, blur, tissue-thickness effects, and marker artifacts that adversely affect interpretation.Dedicated algorithms have been proposed to detect blur and tissue folds.
- Color variation: Color variation arises from staining reagents, tissue-section thickness, staining conditions, and scanner models, even when images depict the same tissue type.The review illustrates this issue using lymphocyte images.
- Color variation: Learning without accounting for color variation can worsen machine-learning performance, while fully covering every stain and scanner is currently unlikely.The influence of color variation may become negligible only with sufficient data spanning every stained tissue and scanner.
- Color preprocessing: Color preprocessing options include grayscale conversion, color normalization, and color augmentation, each with distinct trade-offs.Grayscale conversion is easiest but discards color information routinely used by pathologists.
- Color preprocessing: Color normalization matches source-image pixel distributions to a reference but may require component adjustment when tissue compositions differ.Nuclear detection may be needed before adjustment, and normalization is most suitable when target WSIs share similar tissue compositions.
- Color preprocessing: Color augmentation randomly changes hue, saturation, brightness, and contrast and is easy to implement, but excessive changes can erase diagnostic color information.It seems most suitable for WSIs with smaller color variation; normalization and augmentation may be complementary.
5. Summary and Outlook
The review identifies unresolved challenges in digital histopathological image analysis, including outliers, interpretability, speed, and limited data, and outlines emerging applications and future directions.
- Summary and Outlook: The review concludes that many problems are becoming tractable but still require improvement, and it suggests future topics that remain insufficiently studied.The paper explicitly identifies discovery of novel objects as a future research topic.
- Summary and Outlook: More sophisticated analysis could be accelerated by gathering WSIs from multiple institutes, annotating them collaboratively with consistent criteria, and making the data public.The review presents this as a future need once large numbers of well-annotated WSIs become available.
- Summary and Outlook: Unexpected objects and rare tumors can be forcibly assigned to predefined categories by discriminative models, motivating outlier detection.Only a few studies have addressed this problem in histopathological image analysis, and reconstruction-error methods from other domains have not yet been applied here.
- Summary and Outlook: Deep learning remains difficult to interpret because its decision-making process is not understandable to humans, although visualization, influence functions, and joint image-report learning offer partial approaches.The review states that this problem has not been completely solved.
- Summary and Outlook: Intraoperative diagnosis requires rapid classification while maintaining accuracy because diagnostic time is very limited.Frozen sections are used instead of longer FFPE processing, but few frozen-section WSIs are available and the task is more challenging.
- Summary and Outlook: Quantitative tumor-infiltrating immune-cell analysis is an emerging application, spanning cell detection, immune-cell typing, and spatial-relationship analysis.These analyses may examine relationships with prognosis or response to immunotherapy.