Source-linked AI summary
Artificial Intelligence Assistance Significantly Improves Gleason Grading of Prostate Biopsies by Pathologists
Wouter Bulten, Maschenka Balkenhol, Jean-Joël Awoumou Belinga, Américo Brilhante, Aslı Çakır, Xavier Farré, Katerina Geronatsiou, Vincent Molinié, Guilherme Pereira, Paromita Roy, Günter Saile, Paulo Salles, Ewout Schaafsma, Joëlle Tschui, Anne-Marie Vos, Hester van Boven, Robert Vink, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, Geert Litjens
TL;DR
Gleason grading is important but variable, and evidence for integrating AI into pathologists’ workflow is limited. The study compares assisted and unassisted grading of prostate biopsies and reports improved performance with AI assistance, while noting that grading time was not directly measured.
Problem
Gleason grading has significant observer variability, while evidence for the merit of embedding deep-learning systems in pathologists’ workflow is limited.
Method
The study evaluates AI-assisted Gleason grading by integrating a deep-learning system into pathologists’ diagnostic workflow.
Results
AI assistance improved pathologists’ performance at histological tumor grading and produced higher median performance than the standalone AI.
Takeaways & Limitations
The results support an added benefit of using AI as a supportive tool for pathologists in Gleason grading.
Takeaways & Limitations
The study did not directly measure time taken per case, so the reported faster grading remains a direction for future research.
Abstract
from arXiv · showhide
While the Gleason score is the most important prognostic marker for prostate cancer patients, it suffers from significant observer variability. Artificial Intelligence (AI) systems, based on deep learning, have proven to achieve pathologist-level performance at Gleason grading. However, the performance of such systems can degrade in the presence of artifacts, foreign tissue, or other anomalies. Pathologists integrating their expertise with feedback from an AI system could result in a synergy that outperforms both the individual pathologist and the system. Despite the hype around AI assistance, existing literature on this topic within the pathology domain is limited. We investigated the value of AI assistance for grading prostate biopsies. A panel of fourteen observers graded 160 biopsies with and without AI assistance. Using AI, the agreement of the panel with an expert reference standard significantly increased (quadratically weighted Cohen's kappa, 0.799 vs 0.872; p=0.018). Our results show the added value of AI systems for Gleason grading, but more importantly, show the benefits of pathologist-AI synergy.
3. Salomão Zoppi Diagnostics/DASA, São Paulo, Brazil
The passage identifies Salomão Zoppi Diagnostics/DASA in São Paulo, Brazil, as an institutional affiliation.
- Salomão Zoppi Diagnostics/DASA is listed as an affiliated institution.The affiliation is located in São Paulo, Brazil.
- The institution is based in São Paulo.
- The listed country is Brazil.
9. Tata Medical Center, Department of Pathology, Kolkata, India
The passage identifies Labor Team WAG as an institution in Goldach, Switzerland.
- Labor Team WAG is listed as an affiliated institution.
- The institution is associated with histopathology and cytology.
- The institution is located in Goldach SG, Switzerland.
14. Laboratory of Pathology East Netherlands, Hengelo, The Netherlands
The passages list affiliations with Linköping University in Sweden and Radboud University Medical Center.
- Linköping University’s Center for Medical Image Science and Visualization is listed in Linköping, Sweden.
- The affiliations span medical image science and medical-center research institutions.
- Radboud University Medical Center is listed as an affiliated institution.
Introduction
Gleason grading is clinically important but variable, while AI systems show pathologist-level performance yet remain vulnerable to tissue and imaging anomalies. This study addresses limited evidence on AI-assisted Gleason grading by examining whether integrating AI feedback with pathologists improves grading performance.
- Clinical context: The biopsy Gleason score is the most important prognostic marker for prostate cancer patients.
- Clinical context: Gleason grading suffers from significant inter- and intraobserver variability, despite specialized uropathologists showing higher concordance rates.
- Existing AI evidence: Deep learning systems have achieved pathologist-level performance in prostate grading within study-specific limits.
- Research gap: Evidence for embedding AI systems in pathologists’ workflows is limited, and no prior study had evaluated AI-assisted Gleason grading of prostate biopsies to the authors’ knowledge.
- Existing AI evidence: AI systems can be affected by non-prostate tissue, atypical patterns, ink, fixation, scanning or cutting artifacts, and rare cancer subtypes.
- Study objective: The study compares pathologists’ diagnostic performance with and without deep-learning assistance to investigate pathologist-AI synergy.
Results
Fourteen panel members graded 160 prostate-biopsy cases with and without AI assistance using an online viewer and an expert reference standard. AI assistance increased agreement with the reference standard, reduced variability, and produced higher group performance than the standalone system.
- Study design: 14 panel members from 12 independent laboratories and eight countries graded 160 cases, including 100 reused cases and 60 unseen controls.The panel comprised 11 certified pathologists and three residents with varying Gleason-grading experience.
- Study design: The viewer presented the original biopsy, a color overlay highlighting AI-predicted growth patterns, and an automatically predicted biopsy-level grade group.Panel members also completed questionnaires about the grading process and AI feedback.
- AI feedback: 11 of 14 (79%) panel members used AI feedback during grading; the growth pattern overlay was most useful, while the final grade group was least helpful.Most members reported that AI assistance did not distract them and instead made grading faster.
- Performance: In the unassisted read, the AI system exceeded 10 of 14 (71%) panel members; in the assisted read, only five (36%) remained below it.Nine of the 10 panel members initially below the AI system scored higher with assistance, while none of those initially above it improved.
- Performance: The largest improvement occurred among panel members with less than 15 years of experience, and 64% of members scored higher in the assisted read.The control cases had higher panel and system kappa values than the test cases, with panel values of 0.910 and 0.872, respectively.
Discussion
The study found that AI assistance improved pathologists’ Gleason grading performance and supported higher, more consistent panel-level accuracy than unassisted reading. Benefits were greatest among lower-performing pathologists, while the study’s single-center, single-biopsy design and unmeasured grading time constrain interpretation.
- The study reports that AI assistance improves pathologists’ performance in histological tumor grading.
- 0.799 vs 0.872 quadratically weighted Cohen’s kappa: agreement with the expert reference standard increased significantly with AI assistance.
- AI-assisted reads achieved higher median performance than both unassisted reads and the standalone AI system, indicating pathologist-AI synergy.
- The largest performance increase occurred among panel members who initially scored lower than the AI system, whereas highly experienced pathologists showed no diagnostic-accuracy gain.
- Almost all pathologists found the AI overlay useful and used it most, while the biopsy-level grade group was rated least useful because it duplicated the Gleason score.
- The observer study used 100 test and 60 control cases from one center, and grading time was reported as faster but not directly measured.
Methods
The study evaluated pathologists’ prostate-biopsy grading with and without deep-learning assistance using a panel, an expert consensus reference standard, and paired statistical comparisons.
- Reference standard: Three expert uropathologists graded test cases using ISUP 2014 guidelines to establish a consensus reference standard.Disagreements were resolved through majority voting, additional review, or consensus discussion depending on the case.
- Analysis: Agreement with the consensus reference standard was assessed using quadratically weighted Cohen’s kappa, with median kappa used for group-level comparisons.Paired kappa differences for the 100 cases used in both reads were compared with a Wilcoxon signed-rank test.
Competing interests
The authors reported grants, personal fees, and other financial relationships involving cancer organizations and medical-technology companies.
- Disclosures: Several authors reported grants or personal fees from the Dutch Cancer Society, Philips, ContextVision, AbbVie, Sectra, and Novartis.Some relationships occurred during the study, while others were reported outside the submitted work.
- Disclosures: Reported relationships included funding from Philips Digital Pathology Solutions and personal fees from medical-technology and pharmaceutical companies.The disclosure statement distinguishes relationships during the study from those outside the submitted work.
- Disclosures: The disclosure statement lists financial relationships for named authors and indicates no additional details for the remaining listed authors in the supplied passage.The passage ends before the complete author disclosure list is shown.