Source-linked AI summary
Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks
Jason W. Wei, Laura J. Tafe, Yevgeniy A. Linnik, Louis J. Vaickus, Naofumi Tomita, Saeed Hassanpour
TL;DR
Lung adenocarcinoma pattern classification is clinically important but difficult because tumors commonly contain mixed patterns and evaluations vary among pathologists. The paper develops a residual convolutional neural-network patch classifier with whole-slide aggregation for predominant and minor patterns, achieving pathologist-level agreement on 143 independent slides. The authors describe potential use for pre-screening and highlighting regions, while noting single-center and relatively small data limitations.
Problem
Mixed histologic patterns and subjective criteria make lung adenocarcinoma subtype classification difficult despite its importance for prognosis, grading, and treatment.
Method
A residual convolutional neural network classifies patches, and a whole-slide inference mechanism aggregates predictions to determine predominant and minor subtypes.
Results
0.525 average kappa and 66.6% predominant agreement were achieved on 143 independent whole-slide images, with performance on par with pathologists.
Takeaways & Limitations
The model could aid pathologists by pre-screening slides and highlighting cancerous or elusive high-grade pattern regions for review.
Takeaways & Limitations
The study used data from a single medical center, and its dataset was relatively small, with papillary and micropapillary classes extremely rare.
Abstract
from arXiv · showhide
Classification of histologic patterns in lung adenocarcinoma is critical for determining tumor grade and treatment for patients. However, this task is often challenging due to the heterogeneous nature of lung adenocarcinoma and the subjective criteria for evaluation. In this study, we propose a deep learning model that automatically classifies the histologic patterns of lung adenocarcinoma on surgical resection slides. Our model uses a convolutional neural network to identify regions of neoplastic cells, then aggregates those classifications to infer predominant and minor histologic patterns for any given whole-slide image. We evaluated our model on an independent set of 143 whole-slide images. It achieved a kappa score of 0.525 and an agreement of 66.6% with three pathologists for classifying the predominant patterns, slightly higher than the inter-pathologist kappa score of 0.485 and agreement of 62.7% on this test set. All evaluation metrics for our model and the three pathologists were within 95% confidence intervals of agreement. If confirmed in clinical practice, our model can assist pathologists in improving classification of lung adenocarcinoma patterns by automatically pre-screening and highlighting cancerous regions prior to review. Our approach can be generalized to any whole-slide image classification task, and code is made publicly available at https://github.com/BMIRDS/deepslide.
1. Introduction
Lung adenocarcinoma patterns guide prognosis, grading, and treatment, but heterogeneous tumors and subjective criteria make classification difficult and variable among pathologists. Deep learning offers a data-driven route to automated subtype classification on resection slides.
- Adenocarcinoma is the most common lung carcinoma histologic type, accounting for about half of cases.
- The WHO identifies five non-mucinous patterns: lepidic, acinar, papillary, micropapillary, and solid.Guidelines recommend documenting minor components alongside the predominant subtype because tumors often mix multiple patterns.
- Pattern classification informs prognosis and treatment: lepidic is typically low grade, acinar and papillary intermediate grade, and micropapillary and solid high grade.Micropapillary or solid predominant tumors have lower survival rates and are more likely to undergo and benefit from adjuvant chemotherapy.
- About 80% of adenocarcinomas contain multiple histologic patterns, complicating accurate subtype assignment.Small amounts of high-grade patterns can be overlooked, while mixed acinar-lepidic and papillary-micropapillary patterns can be contentious.
- Reported agreement varies among evaluators, with expert-pathologist kappa scores of 0.24-0.52 for difficult cases.This contrasts with kappa scores of 0.70-0.84 for classical images in the same survey.
- The study presents deep learning as an automated approach for classifying histologic subtypes on lung adenocarcinoma surgical resection slides.Convolutional neural networks automatically extract image features through a data-driven process.
2. Results
The study combines patch-level residual neural-network classification with whole-slide aggregation to identify predominant and minor lung adenocarcinoma patterns. On independent whole-slide evaluation, the model achieved patch-level accuracy and agreement measures comparable to pathologists.
- The model combines a patch classifier with sliding-window whole-slide inference to identify predominant and minor patterns.It applies a heuristic to aggregate patch predictions into whole-slide subtype assignments.
- 422 whole-slide images were split into training, development, and independent test sets, with 143 slides used for final comparison against three pathologists.
- 90.4% was the best model’s F1 score on 1,068 classic development patches.The patch classifier achieved an AUC greater than or equal to 0.97 for every histologic class.
- 0.525 was the model’s average kappa score for predominant classification on the independent test set, with 66.6% average predominant agreement.These measures slightly exceeded inter-pathologist agreement measures reported for the same evaluation.
- All model and pathologist evaluation metrics fell within 95% confidence intervals of agreement for every pairwise comparison.The study also calculated robust agreement, defined as agreement with at least two of the three other annotators.
3. Discussion
The model performed on par with pathologists on predominant and minor histologic subtype classification, while offering a potential aid for reviewing lung adenocarcinoma slides. Its broader clinical utility remains constrained by single-center, relatively small data and requires further validation and development.
- Performance: Moderate agreement was observed for predominant subtype classification, with the model slightly outperforming pathologists on kappa and agreement metrics.Across annotator/model pairs, Kpredom was 0.41–0.60 and predominant agreement was approximately 60–65%.
- Performance: 39.5% of predominant subtype disagreements occurred between acinar and lepidic patterns, which often coexist and have difficult boundaries.The authors relate this disagreement to the challenge of defining an exact border between the two patterns.
- Performance: Minor-pattern detection was more challenging for both the model and pathologists, likely because small components can be interpreted differently or overlooked.This difficulty is clinically relevant because even small amounts of micropapillary pattern have been associated with poor prognosis.
- Clinical implications: The model could support clinical workflows by pre-populating histologic pattern diagnoses, requesting genetic testing, and assisting quality assurance.The proposed applications depend on successful clinical implementation and could facilitate faster diagnosis and treatment.
- Limitations: The study’s data came from a single medical center and were relatively small, with papillary and micropapillary patterns rare in the training set.Papillary and micropapillary patterns represented 4% and 17% of training whole-slide images, respectively; collaboration across centers is proposed.
- Future directions: Future extensions include more localized cancer-region visualization and models predicting genetic mutations or survival outcomes.The authors propose R-CNN or Mask R-CNN approaches and additional outcome-linked datasets for these directions.
4. Materials and Methods
The study develops and evaluates a ResNet-based pipeline that classifies lung adenocarcinoma histologic patterns from whole-slide images. It trains on pathologist-annotated crops, aggregates patch predictions for slide-level classification, and compares the model with pathologists using agreement metrics.
- Data Collection: 422 whole-slide images were collected, with 279 used for training and 143 reserved for testing.The training images were split into 245 training and 34 development images.
- Slide Annotation: Three pathologists annotated 4,161 crops from 245 training images, while all three independently labeled predominant and minor patterns on the 143-image test set.The test set contained slides with one or more of five histological patterns.
- Model Training: A ResNet convolutional neural network was trained on fixed-size patches generated from variable-sized annotated crops.Sliding windows, class-balancing overlap, and color normalization were used before training.
- Whole-slide Inference: Whole-slide inference used overlapping fixed-size patches, discarded low-confidence predictions, and aggregated classifications to detect predominant and minor patterns.The model was designed to classify patches rather than entire slides, with inference taking around thirty seconds per slide.
- Statistical Analysis: The model was evaluated against three pathologists on predominant and minor pattern classifications using kappa, predominant agreement, per-class kappa, and robust agreement.Predominant agreement measures the percentage of slides with matching predominant patterns, while robust agreement measures agreement with at least two of the other three annotators.
- Visualization: Color-coded overlays of predicted patches visualized the detected histologic patterns and provided pathologists insight into the model’s classification method.The visualization was used to confirm the model’s generated decisions.
6. Author Contributions
The listed authors contributed across data collection, model implementation, study design, annotation, literature review, and evaluation of model predictions.
- Author Contributions: JW facilitated data collection and implemented the deep learning model, while LT and YL contributed to data collection, annotation, and evaluation.LT also advised study design and conducted the literature search.