Source-linked AI summary
Methods for Segmentation and Classification of Digital Microscopy Tissue Images
Quoc Dang Vu, Simon Graham, Minh Nguyen Nhat To, Muhammad Shaban, Talha Qaiser, Navid Alemi Koohbanani, Syed Ali Khurram, Tahsin Kurc, Keyvan Farahani, Tianhao Zhao, Rajarsi Gupta, Jin Tae Kwak, Nasir Rajpoot, Joel Saltz
TL;DR
Accurate and efficient analysis of tissue morphology is challenging because tissue structure is complex and heterogeneous, while manual examination is subjective and difficult to scale. The paper presents algorithms for nuclear segmentation and whole-slide tissue classification, achieving scores of 0.78 and 0.81, respectively, in the MICCAI 2017 Digital Pathology challenge.
Problem
Accurate, scalable computational methods are needed for nuclear segmentation and tissue-image classification because tissue morphology is complex, heterogeneous, and manual analysis is subjective and not scalable.
Method
The paper combines a multiscale deep residual aggregation network for nuclear segmentation with patch-level deep learning and random forest regression for whole-slide classification.
Results
The segmentation algorithm achieved a combined DICE_1 and DICE_2 score of 0.78, while the classification algorithm achieved an accuracy score of 0.81 in the MICCAI 2017 challenge.
Takeaways & Limitations
The evaluated algorithms provide automated approaches for quantifying nuclear morphology and classifying whole-slide NSCLC tissue images.
Takeaways & Limitations
Segmentation performance is constrained by staining variation, instability, and densely overlapping nuclei that produce fragmented or imperfect boundaries.
Abstract
from arXiv · showhide
High-resolution microscopy images of tissue specimens provide detailed information about the morphology of normal and diseased tissue. Image analysis of tissue morphology can help cancer researchers develop a better understanding of cancer biology. Segmentation of nuclei and classification of tissue images are two common tasks in tissue image analysis. Development of accurate and efficient algorithms for these tasks is a challenging problem because of the complexity of tissue morphology and tumor heterogeneity. In this paper we present two computer algorithms; one designed for segmentation of nuclei and the other for classification of whole slide tissue images. The segmentation algorithm implements a multiscale deep residual aggregation network to accurately segment nuclear material and then separate clumped nuclei into individual nuclei. The classification algorithm initially carries out patch-level classification via a deep learning method, then patch-level statistical and morphological features are used as input to a random forest regression model for whole slide image classification. The segmentation and classification algorithms were evaluated in the MICCAI 2017 Digital Pathology challenge. The segmentation algorithm achieved an accuracy score of 0.78. The classification algorithm achieved an accuracy score of 0.81.
1. INTRODUCTION
Whole-slide tissue analysis addresses the need for scalable, reliable quantitative methods alongside subjective manual examination. This paper presents separate algorithms for nuclei segmentation and whole-slide NSCLC classification.
- Motivation: 20,000x20,000 to over 100,000x100,000 pixels are captured rapidly in whole-slide tissue images, increasing demand for efficient automated analysis.Quantitative analysis can characterize tumor structure and heterogeneity across large numbers of samples.
- Analysis tasks: Nuclei segmentation and image classification are the two most common whole-slide tissue image analysis tasks.Segmentation supports extraction of sub-cellular morphology, while classification assigns labels to regions or whole images.
- Challenges: Tumor and normal tissue morphology varies across and within cancer types, making digital slide information difficult to extract, mine, and interpret.Tissue images also contain dense, multiscale information and existing methods may rely on parameter-sensitive heuristics.
- Contributions: The paper presents one algorithm for nuclei segmentation and another for whole-slide tissue image classification.The segmentation method targets accurate nuclei segmentation and clump separation, while the classification method targets NSCLC histology images.
- Evaluation: The two algorithms achieved the highest scores in the MICCAI 2017 Computational Precision Medicine digital pathology challenge.The challenge covered NSCLC, HNSCC, GBM, and LGG tissue images.
2.1 Segmentation of nuclei by a deep learning method
The nuclei segmentation pipeline detects blobs and boundaries, separates clumped nuclei, and refines individual-nucleus masks. Its DRAN-based architecture combines contracting and expanding paths with multiscale inputs.
- Segmentation procedure: The segmentation workflow detects nuclear blobs and boundaries with CNNs, separates clumped nuclei using watershed, and produces final individual-nucleus segmentation.Boundary removal and watershed-based core identification support assignment of boundary pixels to individual nuclei.
- Network architecture: DRAN uses contracting and expanding paths for feature extraction, up-sampling, and production of finer segmentation maps.The contracting path uses a modified pre-activated ResNet50, while decoder layers combine inputs from preceding and contracting-path layers.
- Expanding path: The expanding path uses addition rather than concatenation to reduce memory usage without substantially losing learning capability.Nearest-neighbor resizing doubles input size and is described as computationally inexpensive.
- Expanding path: The decoder architecture divides input and output channels into disjoint convolutional paths, with Table 1 detailing three decoders.The table caption defines C as the number of paths or groups and gives kernel-size and channel notation.
- Multiscale aggregation: MDRAN composes three DRANs at x0.5, x1.0, and x2.0 scales and aggregates them into an x1.0 segmentation map.The multiscale design addresses variation in nuclei sizes across and within tissue specimens.
2.2 Classification of whole slide images by a two-part automated method
The classification framework combines patch-level deep learning with whole-slide statistical and morphological analysis to classify NSCLC images while focusing on diagnostic regions. A residual CNN produces class probability maps, whose features are then classified by a random forest model.
- Patch-based NSCLC methods classify image regions because whole-slide images are computationally difficult to process directly.
- The proposed framework focuses primarily on diagnostic areas to determine whether a whole slide is lung adenocarcinoma or lung squamous cell carcinoma.
- Test-slide patches are processed into three probability maps, one for each class.
- 2.2.2 Extraction of statistical and morphological features: Features from LUAD and LUSC probability maps are classified with either max voting or a random forest regression model.The random forest uses statistical and morphological map features, including probability summaries, class ratios, and connected-component sizes.
- 2.2.3 Random forest regression model: The random forest model combines decision trees fitted on data subsamples and converts its output into a binary LUAD-versus-LUSC classification.The optimized model uses 10 bagged trees, one third of variables per split, and minimum leaf size 5.
3. RESULTS
The challenge evaluated nuclei segmentation and whole-slide image classification using curated tissue-image datasets. The reported results show improved nuclei segmentation from multiscale aggregation and improved whole-slide classification from statistical and morphological features.
- Evaluation: The segmentation evaluation averaged DICE_1 and Ensemble Dice DICE_2 scores for each tile, then averaged tile scores across the test dataset.DICE_2 captures mismatches caused by splits and merges that ordinary DICE does not distinguish.
- Nuclei segmentation: 0.783 average score was achieved by the multiscale MDRANBL+DRANBD nuclei segmentation method, comprising DICE_1 of 0.862 and DICE_2 of 0.703.The method outperformed the single-scale DRANBL+DRANBD approach, which achieved an average score of 0.777.
- Nuclei segmentation: 0.777 average score was achieved by the single-scale DRANBL+DRANBD nuclei segmentation method, compared with 0.783 for multiscale aggregation.The corresponding single-scale results were DICE_1 of 0.853 and DICE_2 of 0.701.
- Whole-slide classification: 0.776 average LUAD and LUSC patch-level accuracy was obtained with ResNet32, versus 0.678 with InceptionV3, motivating use of ResNet32 for test images.The classification pipeline assigned patches to LUAD, LUSC, or non-diagnostic classes before whole-slide modeling.
- Whole-slide classification: Random forest regression using statistical and morphological features from labeled WSI probability maps increased whole-slide classification accuracy over max voting when no class was dominant.Max voting was sufficient when LUAD or LUSC dominated the labeled WSI.
4. DISCUSSION
The paper addresses laborious, non-scalable tissue examination by presenting automated nucleus-segmentation and whole-slide classification methods evaluated in the MICCAI 2017 challenge. The segmentation method achieved a combined DICE_1 and DICE_2 score of 0.78, while the classification method achieved 0.81; future work targets contextual information and larger datasets.
- Discussion: The paper presents two automated methods contributed by the two top-scoring participants in the MICCAI 2017 Digital Pathology Challenge.The methods address nucleus segmentation and whole-slide tissue classification.
- Discussion: The segmentation network detects nuclear blobs and boundaries, separates clumped nuclei with watershed processing, and refines individual-nucleus segmentation.Its multiscale design addresses variation in nucleus size across and within tissue specimens.
- Discussion: 0.78 combined DICE_1 and DICE_2 score was achieved by the multiscale deep residual aggregation network.The evaluation suggests multiscale aggregation improves segmentation of relatively smaller nuclei.
- Discussion: The classification method labels patches as LUAD, LUSC, or non-diagnostic, then uses statistical and morphological probability-map features for whole-slide classification.A random forest regression model performs the final whole-slide classification.
- Discussion: 0.81 accuracy was achieved by the automated NSCLC classification method in the MICCAI 2017 digital pathology challenge.The training set lacked normal cases, making non-diagnostic-region handling important for this implementation.
- Discussion: Future work proposes adding contextual information and using a larger dataset to improve patch-level and overall classification accuracy.The larger dataset is intended to provide more representative features for subsequent NSCLC classification.