Source-linked AI summary
Deep-Learning for Classification of Colorectal Polyps on Whole-Slide Images
Bruno Korbar, Andrea M. Olofson, Allen P. Miraflor, Katherine M. Nicka, Matthew A. Suriawinata, Lorenzo Torresani, Arief A. Suriawinata, Saeed Hassanpour
TL;DR
Pathologists face challenging and variable colorectal-polyp classification, including difficulty distinguishing sessile serrated from innocuous hyperplastic polyps. The paper presents a deep-learning image-understanding system for colorectal-polyp classification and reports 93.0% overall accuracy, with 89.7% precision, 88.3% recall, and 88.8% F1 score.
Problem
Colorectal-polyp classification can be challenging, with substantial variability among pathologists and frequent difficulty distinguishing sessile serrated from innocuous hyperplastic polyps.
Method
The paper presents an image-understanding system based on a deep-learning approach and state-of-the-art deep-neural-network architectures, covering five common colorectal-polyp classes.
Results
93.0% overall accuracy, 89.7% overall precision, 88.3% overall recall, and 88.8% overall F1 score were reported for colorectal-polyp classification.
Takeaways & Limitations
The system was presented to assist pathologists with colorectal-polyp image understanding and classification.
Takeaways & Limitations
The authors identify the black-box nature of deep-learning outcomes as a general limitation and note mistakes between hyperplastic polyps and sessile serrated polyps.
Abstract
from arXiv · showhide
Histopathological characterization of colorectal polyps is an important principle for determining the risk of colorectal cancer and future rates of surveillance for patients. This characterization is time-intensive, requires years of specialized training, and suffers from significant inter-observer and intra-observer variability. In this work, we built an automatic image-understanding method that can accurately classify different types of colorectal polyps in whole-slide histology images to help pathologists with histopathological characterization and diagnosis of colorectal polyps. The proposed image-understanding method is based on deep-learning techniques, which rely on numerous levels of abstraction for data representation and have shown state-of-the-art results for various image analysis tasks. Our image-understanding method covers all five polyp types (hyperplastic polyp, sessile serrated polyp, traditional serrated adenoma, tubular adenoma, and tubulovillous/villous adenoma) that are included in the US multi-society task force guidelines for colorectal cancer risk assessment and surveillance, and encompasses the most common occurrences of colorectal polyps. Our evaluation on 239 independent test samples shows our proposed method can identify the types of colorectal polyps in whole-slide images with a high efficacy (accuracy: 93.0%, precision: 89.7%, recall: 88.3%, F1 score: 88.8%). The presented method in this paper can reduce the cognitive burden on pathologists and improve their accuracy and efficiency in histopathological characterization of colorectal polyps, and in subsequent risk assessment and follow-up recommendations.
1. Introduction
Colorectal polyp characterization is clinically important but difficult and variable, especially for distinguishing sessile serrated from hyperplastic polyps. The paper proposes deep learning to assist whole-slide histopathological analysis and diagnosis.
- Histopathological characterization distinguishes high-risk from low-risk polyps and informs colorectal cancer risk and follow-up colonoscopy timing.
- Accurate characterization is challenging because pathologists show substantial variability in how they characterize and diagnose colorectal polyps.
- Existing screening methods such as fecal blood, fecal DNA, and virtual colonoscopy are not well suited to differentiating sessile serrated polyps from other polyps.
- Sessile serrated polyps are especially difficult to distinguish from innocuous hyperplastic polyps, despite the importance of appropriate surveillance.
- Prior computational pathology methods commonly used structural segmentation, hand-constructed features, and standard machine-learning classifiers.
- The paper proposes a deep-learning approach for colorectal polyp characterization and diagnosis to support more efficient and accurate colorectal cancer screening.
2. Materials and Methods
The study constructs a whole-slide colorectal polyp dataset and trains a modified ResNet-based classifier using pathologist annotations and cropped H&E-stained image regions. The workflow evaluates architecture choices, training behavior, and final whole-slide classification.
- Dataset: The dataset covers five common colorectal polyp types included in US multi-society task force guidelines, plus normal samples.
- Dataset: The study collected 1,723 whole-slide images, used 85% for training and 15% for validation, and evaluated an additional 239 images after training.
- Annotation and preprocessing: Pathologists independently labeled polyp types and outlined polyp regions to generate smaller crops used as reference standards and training inputs.
- Annotation and preprocessing: Disagreements in classification or cropping were resolved through pathologist discussion and senior gastrointestinal pathology consultation; unresolved cases were discarded and replaced.
- Architecture: The modified ResNet uses shortcut mappings that bypass convolutional layers, helping address vanishing gradients and support greater network depth.
- Training and inference: A 152-layer ResNet model was evaluated with a hold-out validation set, followed by retraining on the augmented training set and testing on the independent test set.
3. Results
The experiments compare candidate deep-learning architectures and report results for the selected model on colorectal polyp classification. The supplied results passages identify the ablation comparison but do not provide its numerical table values.
- Table 1 reports an ablation test on raw image crops over 50 epochs for selecting the best architecture.
- Table 2 presents results for the best model, identified as ResNet-D, for colorectal polyp classification.
- Table 3 presents final-model results for classification of colorectal polyps in 239 whole-slide test images.
- Table 4 presents the confusion matrix for the final model on the 239-image test set.
4. Discussion
The study presents an automated deep-learning system for colorectal polyp characterization on H&E-stained whole-slide images, achieving strong performance across polyp classes. The discussion highlights practical benefits, architectural findings, error patterns, and limitations including limited interpretability and evaluation scope.
- Clinical utility: The automated system is intended to assist histopathological characterization on H&E-stained whole-slide images and reduce clinicians’ manual and cognitive burden.The proposed system is discussed as a tool to support pathologists rather than replace their interpretations.
- Model architecture: Data augmentation positively affected classification accuracy, supporting its use in the whole-slide inference pipeline.The discussion attributes the small evaluation-time difference between models to ResNet’s relatively simple computational layers.
- Error patterns and limitations: Low-confidence examples tended to be classified as normal, possibly because the training set contained diverse normal whole-slide images.Hyperplastic and sessile serrated polyps were another major source of classification mistakes, consistent with pathologists’ experience.
- Error patterns and limitations: The evaluation used a test set collected at the authors’ organization, motivating planned external validation on an additional New Hampshire Colonoscopy Registry dataset.The authors also identify black-box outputs as a limitation and plan visualization methods to expose influential image regions and features.
5. Conclusion
The system assists pathologists by classifying five colorectal polyp types in H&E-stained whole-slide images using a deep-neural-network architecture. Evaluation on 239 images produced strong reported performance and supports potential use as a low-burden diagnostic and assessment tool.
- The system assists pathologists in characterizing colorectal polyps on H&E-stained whole-slide images.
- A deep-neural-network architecture identifies colorectal polyp types in whole-slide, H&E-stained images.
- The evaluation used 239 H&E-stained whole-slide images covering five colorectal polyp classes specified in US task-force guidelines.
- Accuracy: 93.0%, precision: 89.7%, recall: 88.3%, and F1 score: 88.8% were reported for the classification task.
- The authors describe the system as potentially low-burden, efficient, and accurate for colorectal-polyp diagnosis and assessment.
- The reported outcomes could potentially increase colorectal-cancer screening coverage and accuracy and reduce colorectal-cancer mortality.