PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks

Xiaomin Zhou, Chen Li, Md Mamunur Rahaman, Yudong Yao, Shiliang Ai, Changhao Sun, Xiaoyan Li, Qian Wang, Tao Jiang

arXiv:2003.12255v2eess.IVcs.CV

TL;DR

Breast histopathological image analysis uses artificial neural networks, but the field lacked a dedicated review of these approaches. This paper synthesizes more than 150 studies across classical and deep networks, finding that newer network frameworks tend to perform better across datasets.

  • Problem

    Before this review, no dedicated synthesis focused on artificial-neural-network approaches for breast histopathological image analysis.

  • Method

    The review categorizes more than 150 studies into classical and deep neural networks and analyzes them across breast histopathology datasets.

  • Results

    Novel network frameworks tend to perform better across different datasets, while MLP and PNN are the most applied classical classifiers.

  • Takeaways & Limitations

    The review provides a structured account of ANN-based breast histopathology methods and identifies publicly available datasets for future research.

  • Takeaways & Limitations

    Large-scale, comprehensive, fully labeled whole-slide-image datasets remain lacking, and molecular-subtype, treatment-response, and prognosis analysis remains challenging.

Abstract

from arXiv · show

Breast cancer is one of the most common and deadliest cancers among women. Since histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of breast cancers. To improve the accuracy and objectivity of Breast Histopathological Image Analysis (BHIA), Artificial Neural Network (ANN) approaches are widely used in the segmentation and classification tasks of breast histopathological images. In this review, we present a comprehensive overview of the BHIA techniques based on ANNs. First of all, we categorize the BHIA systems into classical and deep neural networks for in-depth investigation. Then, the relevant studies based on BHIA systems are presented. After that, we analyze the existing models to discover the most suitable algorithms. Finally, publicly accessible datasets, along with their download links, are provided for the convenience of future researchers.

I. INTRODUCTION · A. GENERAL DEVELOPMENT OF EXISTING AI ANALYSIS HISTOPATHOLOGY · B. MOTIVATION OF OUR REVIEW PAPER

Breast histopathological analysis is clinically important but manual interpretation is time-consuming and subjective. This review surveys classical and deep neural-network approaches for BHIA, motivated by the need to clarify developments, technologies, trends, and future potential.

  • I. INTRODUCTION: Breast cancer is the most commonly diagnosed cancer and leading cause of cancer deaths among women; 2.1 million women develop it worldwide annually.In 2018, an estimated 627,000 women died, representing about 15% of all cancer deaths among women.
  • I. INTRODUCTION: Pathological diagnosis is often regarded as the “golden standard” among breast-cancer detection techniques.Other techniques include X-ray mammography, 3-D ultrasound, CT, PET, MRI, and breast temperature measurement.
  • I. INTRODUCTION: Traditional histopathological assessment relies on histopathologists’ manual visual analysis, which is time-consuming and has unstable objectivity because images are complex and diverse.Sections are examined under a microscope using prior medical knowledge.
  • I. INTRODUCTION: AI-assisted histopathological diagnosis can perform tedious focus screening, rapidly extract diagnostically valuable information from massive data, and reduce subjective differences through objective analysis.AI has shown potential advantages in histopathological analysis, particularly in computer vision, image processing, and analysis.
  • A. GENERAL DEVELOPMENT OF EXISTING AI ANALYSIS HISTOPATHOLOGY: Machine learning supports pathological diagnosis across cervical, gastric, colon, lung, and breast cancers, including benign–malignant diagnosis, disease grading, staining analysis, and early tumor screening.Artificial neural networks are a machine-learning branch that imitates biological neural-network structure and function.
  • A. GENERAL DEVELOPMENT OF EXISTING AI ANALYSIS HISTOPATHOLOGY: ANNs are widely used in breast histopathological image analysis for image segmentation, feature extraction, and classification through classical and deep neural networks.The paper focuses on ANN methods for breast histopathology image analysis and examines their development history, popular technologies, application trends, and future potential.
  • B. MOTIVATION OF OUR REVIEW PAPER: Existing surveys address histopathological or breast-image analysis but provide limited coverage of BHIA with ANNs or omit results for individual methods.One survey summarized more than 130 histopathological image-analysis papers but only five BHIA studies with ANNs; another included 106 related works but only 20 BHIA works with ANNs.
  • B. MOTIVATION OF OUR REVIEW PAPER: As of early 2020, this review summarizes more than 150 related works to provide a comprehensive account of recent BHIA using ANN approaches.It extends the authors’ previous brief review of more than 60 works covering classical ANNs, deep ANNs, and methodology analysis.

II. BHIA USING CLASSICAL ANNS

This section overviews breast histopathological image analysis using classical artificial neural network methods and analyzes and summarizes the related work.

  • II. BHIA USING CLASSICAL ANNS: The section compiles an overview of BHIA studies using classical ANN methods before analyzing and summarizing the chapter.

A. RELATED WORKS · B. SUMMARY

The review organizes prior BHIA research into classification and segmentation tasks, summarizing each study’s contributions, methods, and results. It concludes that classical neural networks dominated BHIA around 2012, while limited data and computational speed constrained feature extraction and encouraged classifier-focused use.

  • A. RELATED WORKS: Related work is divided into classification and segmentation tasks, with each paper summarized by its contributions, methods, and results.
  • A. RELATED WORKS: Other classification experiments report 90% accuracy, around 95% overall accuracy, around 87% overall accuracy, and 80% accuracy with 82.9% sensitivity and 89.2% AUC.These studies use neural-network classifiers for texture features, H&E histopathology images, low-magnification malignancy grades, and morphological cell features.
  • A. RELATED WORKS: Classification studies combine handcrafted texture, morphological, edge, intensity, and selected features with neural-network, SVM, KNN, or PNN classifiers.Reported approaches include feature ranking, sequential forward selection, principal component analysis, random-subspace ensembles, and multilayer perceptrons.
  • A. RELATED WORKS: 98.80% accuracy is achieved on dataset 1 and 96.33% on dataset 2 using the combined classifier and feature-processing scheme in.Dataset 1 contains 692 fine-needle aspirates of breast lumps, while dataset 2 contains 295 microarrays.
  • A. RELATED WORKS: Segmentation research applies competitive neural networks to cluster breast-cancer regions using shape, texture, and topological features, achieving around 98.7% accuracy on over 500 images.
  • B. SUMMARY: Around 2012, BHIA primarily used classical neural networks, which performed well but were easier to overfit, trained slowly, and required experience-based parameter settings.
  • B. SUMMARY: Limited computational speed and insufficient data made effective feature extraction from raw data impossible, so most classical neural networks served as classifiers using texture and morphological features.Table 1 summarizes classical-neural-network applications for breast histopathological image analysis.

III. BHIA USING DEEP NEURAL NETWORKS · A. RELATED WORKS

The review organizes deep-neural-network breast histopathology studies by dataset and summarizes their motivations, methods, and results chronologically. It covers publicly available datasets and reports classification and detection performance across BreaKHis, Camelyon, BACH, and related challenges.

  • III. BHIA USING DEEP NEURAL NETWORKS: Publicly available datasets are frequently used for deep-neural-network breast histopathology analysis, with detailed dataset information and download links provided.The review presents these datasets in Table 2.
  • A. RELATED WORKS: Related studies are grouped by applied dataset and summarized chronologically according to motivation, contribution, methods, and results.
  • A. RELATED WORKS: 83.25% average recognition was achieved by the single-task CNN for benign/malignant classification, while the multi-task CNN achieved 82.13% and 80.10% for classification and magnification estimation.The single-task CNN predicted malignancy; the multi-task CNN predicted malignancy and image magnification simultaneously.
  • A. RELATED WORKS: 94.67% average accuracy, 96% sensitivity, 92% specificity, and 96.2% F-score were obtained by a DCNN whole-slide classifier using patient-level 5-fold cross-validation.The method combined posterior estimates across random multi-views and multiple magnifications using voting.
  • A. RELATED WORKS: 25 of 32 Camelyon 2016 submissions used deep learning, and all 19 top-performing algorithms were DCNN approaches.Camelyon evaluates automated metastatic breast-cancer detection in sentinel lymph-node whole-slide images.
  • A. RELATED WORKS: 98.75% AUC and 0.8533 FROC were obtained by ScanNet for metastatic breast-cancer detection in whole-slide images.ScanNet modified VGG-16 by replacing its final three fully connected layers with fully convolutional layers.
  • A. RELATED WORKS: 99.84% accuracy was achieved by an SVM classifier in a two-stage BACH classification approach, while another transfer-learning method reached 97.08% average test accuracy.The BACH task classified four histology classes; the transfer-learning comparison used Inception-V3 and ResNet-50.

1) Classification

The reviewed classification studies apply classical and deep neural networks to distinguish breast cancer types, diagnostic styles, tumor characteristics, and cell categories. Reported accuracies range from 55% for five carcinoma categories to around 92% for four breast cancer types, while CNN-based methods dominate HER2 scoring submissions.

  • Classification: PCANet classified DCIS and UDH images with around 79% accuracy using 20 DCIS and 31 UDH images.The training set supplied 10,000 randomly sampled patches for model learning.
  • Classification: A hierarchical-loss, global-pooling deep learning method using VGG-16 and VGG-19 achieved around 92% average accuracy across four breast cancer types.The dataset contained 400 images: 280 for training, 60 for validation, and 60 for testing.
  • Classification: A saliency detector and CNN classified five carcinoma types with 55% accuracy after fusing saliency and classification maps.The method used multi-scale localization of diagnostically relevant regions and was evaluated on 240 images, with 45 pathologists participating in final test evaluation.
  • Classification: Deep learning classified tumor grade, ER status, PAM50 subtype, histological subtype, and recurrence risk using 571 training and 288 testing tumor examples.Reported accuracies included 82% for tumor grade, 84% for ER status, 77% for Base-like versus non-Base-like, and 94% for ductal versus lobular classification.
  • Classification: CNN-based learning methods accounted for 8 of the top 10 teams in the HER2 Scoring Contest, which received 18 submissions from 14 teams.The review identifies CNN-based learning as important for automatic HER2 scoring, a clinically significant prognostic task.

2) Segmentation

Breast histopathology segmentation studies use CNN, DCNN, and full-CNN methods for nuclei, regions, and whole-slide images. Reported results include high segmentation accuracy, while some approaches emphasize fast processing, morphological refinement, or publicly available datasets.

  • Classical and deep neural network methods: A three-hidden-layer CNN achieved around 95% overall accuracy for breast cancer cell-nucleus segmentation in 58 H&E-stained images using both RGB and Lab color spaces.The reported accuracy was similar across the two color spaces.
  • Deep neural network methods: A fast scanning deep convolutional neural network segmented 1000 × 1000-pixel images in 2.3 seconds, with mean precision of 91%, mean recall of 89%, and mean F1-score of 85%.The study used 92 images from 20 patients, with 75 for training and 17 for testing.
  • Deep neural network methods: A DCN cascade combined with sparse reconstruction, gradient descent, and morphological operations achieved 92.45% pixel segmentation accuracy and an 83.93% F1-measure.Sparse reconstruction highlights nuclei, while morphological operations and prior knowledge reduce segmentation errors.
  • Deep neural network methods: A deep-learning and mathematical-morphology method used 33 images with 2754 annotated cells and processed posterior probability maps for joint nuclear segmentation.The reported results include accuracy of 95.4% and recall of 77.3%; the precision value is truncated in the supplied passage.
  • Whole-slide image segmentation: A DCNN-based whole-slide-image segmentation method trained and tested on 12 H&E-stained breast cancer WSIs achieved an estimated segmentation accuracy of 95.62%.The framework is described as effective regardless of malignant-tumor texture features and available online as a computer-assisted diagnosis tool.

3) Detection

The reviewed detection methods use classical-deep hybrid and deep neural architectures for nuclei, metastasis, invasive cancer, and mitosis detection in breast histopathology images. Reported results include sensitivity up to 99.9%, average DC of 76%, and F-scores reaching 90%.

  • Nuclei detection: SSAE combined with a softmax classifier detects nuclei in high-resolution breast cancer images using sliding-window image patches.Detected nuclei are marked with green center dots.
  • Metastasis detection: 99.9% sensitivity is achieved by a DCNN model detecting breast cancer metastasis in sentinel lymph nodes.The experiment uses 100 training examples, 50 validation examples, and 75 testing examples.
  • Invasive cancer detection: 76% average DC is obtained by HASHI for automatic invasive breast cancer detection on an independent test dataset.The study compares intensive sampling of 6 million samples in 24 hours with 2000 samples in 1 minute across 500 cases.
  • Mitotic detection: A hybrid of handcrafted and deep-learning features achieves 92% precision, 88% recall, and 90% F-score for mitotic detection.Experiments use the MITOS-ATYPIA-14, ICPR-2012, and AMIDA-13 datasets.
  • Mitotic detection: 84.49% accuracy, 80.55% TPR, 11.66% FNR, and 15.50% loss are achieved on average with a 17-layer CNN for mitosis detection.The study reports that deeper CNN layers perform better for breast cancer image detection.
  • Mitotic detection: Deep transfer learning with a pre-trained VGGNet and random forest classifier produces average F-scores of 88.6% and 89.66% for mitosis detection.Results are reported on the MITOS-ATYPIA-14 and RCC datasets.

B. SUMMARY

Deep artificial neural networks have been increasingly used for breast histopathological image analysis since 2012, with convolutional neural network methods predominating. This trend is associated with high-performance GPU computing and the growing release of breast histopathology image datasets.

  • Deep ANN methods have been increasingly used in BHIA since 2012.
  • CNN-based methods are dominant among deep ANN approaches for BHIA.
  • High-performance GPU computing enables the training of networks with more layers.
  • Growing releases of breast histopathological image datasets partly alleviate the shortage of labeled public datasets.

IV. METHODOLOGY ANALYSIS

This section analyzes classical and deep artificial neural networks for breast histopathological image analysis, including notable methods across different tasks.

  • IV. METHODOLOGY ANALYSIS: The methodology analysis reviews classical and deep ANNs and examines outstanding methods for different tasks.

A. ANALYSIS OF CLASSICAL ANN METHODS

Classical ANN-based breast histopathological image analysis most frequently uses Multi-Layer Perceptron (MLP) and Probabilistic Neural Network (PNN) methods. Because studies use different datasets, methods cannot be evaluated longitudinally and are instead analyzed from the neural-network perspective.

  • A. ANALYSIS OF CLASSICAL ANN METHODS: MLP and PNN are the most frequently used classical ANN methods for breast histopathological image analysis.Their popularity is shown in FIGURE 18.
  • A. ANALYSIS OF CLASSICAL ANN METHODS: Different datasets across studies prevent longitudinal evaluation of classical ANN methods.The survey therefore analyzes methods from the perspective of the neural network itself.
  • A. ANALYSIS OF CLASSICAL ANN METHODS: MLP is a feed-forward neural network.The passage introduces MLP as a classical ANN method and begins describing its problem-solving capability.

B. ANALYSIS OF DEEP ANN METHODS

Deep ANN methods for breast histopathological image analysis increasingly use transfer learning, particularly for classification. VGG16, VGG19, and ResNet50 are prominent pretrained CNNs because breast tissue images are complex and diverse, while expert labeling is costly and large labeled datasets are scarce.

  • Transfer Learning: Transfer learning has been increasingly applied to deep ANN classification of breast histopathological images during the recent four years.It transfers knowledge acquired from one task to resolve another.
  • Transfer Learning: VGG16, VGG19, and ResNet50 are popular pretrained CNN models used in transfer learning for BHIA.Their popularity is attributed to their more in-depth architectures.
  • Transfer Learning: Complex and diverse breast tissue pathological images are difficult to label, while expert annotation is expensive and large-scale labeled image datasets are scarce.These constraints motivate the use of pretrained models and transfer learning.

C. ANALYSIS OF THE OUTSTANDING METHODS IN EACH REVIEWED TASK

The review identifies outstanding methods for specific breast histopathology tasks, including parameter-efficient SE-ResNet and IRRCNN-based approaches. CasCNN is also noted for reducing detection time while maintaining satisfactory accuracy.

  • BreakHis dataset: For the BreakHis dataset, achieves the best results with a small SE-ResNet combining residual modules and Squeeze-and-Excitation blocks.The design can effectively reduce model training parameters.
  • BreakHis dataset: The Gaussian error scheduler delivers excellent performance without complicated fine-tuning.
  • Detection methods: CasCNN significantly reduces detection time while achieving satisfactory accuracy.
  • Bioimaging 2015 Breast Histology Classification Challenge dataset: For the Bioimaging 2015 Breast Histology Classification Challenge dataset, [129] achieves the best results using an Inception Recurrent Residual Convolutional Neural Network.IRRCNN is an improved hybrid DCNN architecture based on inception, residual networks, and RCNN.

D. THE POTENTIAL OF THE METHODS MENTIONED IN THIS REVIEW IN OTHER FIELDS · V. CONCLUSION AND FUTURE WORK

The review highlights that deep ANN methods extend beyond breast histopathological image analysis to several related microscopic-image domains. It concludes that newer network frameworks often perform better, while future progress requires tailored models and larger, comprehensive, fully labeled WSI datasets.

  • D. THE POTENTIAL OF THE METHODS MENTIONED IN THIS REVIEW IN OTHER FIELDS: Deep ANN methods can also support cervical histopathological and cytopathological analysis, stem-cell analysis, microbiological image analysis, and sperm-quality analysis.The review also mentions a web-based platform for computer-assisted diagnosis, but the supplied passage is truncated.
  • V. CONCLUSION AND FUTURE WORK: The review comprehensively summarizes breast cancer histopathological image-analysis methods based on artificial neural networks.Methods are grouped into classical artificial neural networks and deep neural networks.
  • V. CONCLUSION AND FUTURE WORK: Deep neural-network studies are grouped according to the applied datasets and arranged in ascending chronological order within each dataset.This organization is presented as part of the review’s summary structure.
  • V. CONCLUSION AND FUTURE WORK: Novel network frameworks tend to perform better in different datasets.The passage states this as a general conclusion across datasets without reporting numerical metrics.
  • V. CONCLUSION AND FUTURE WORK: Future research can combine pathological-image characteristics to develop new network models for breast cancer histopathological-image analysis.The review identifies this as the first area for improvement.
  • V. CONCLUSION AND FUTURE WORK: Large-scale, comprehensive, and fully labeled WSI datasets remain lacking, making large public datasets valuable for future research.The review identifies dataset establishment as a second priority for improvement.
  • V. CONCLUSION AND FUTURE WORK: The breast-cancer classification system is complex and contains many subtypes.The supplied passage ends after stating that many subtypes exist, so further implications are not available here.
Loading 2003.12255v2…