PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial Intelligence Assistance Significantly Improves Gleason Grading of Prostate Biopsies by Pathologists

Wouter Bulten, Maschenka Balkenhol, Jean-Joël Awoumou Belinga, Américo Brilhante, Aslı Çakır, Xavier Farré, Katerina Geronatsiou, Vincent Molinié, Guilherme Pereira, Paromita Roy, Günter Saile, Paulo Salles, Ewout Schaafsma, Joëlle Tschui, Anne-Marie Vos, Hester van Boven, Robert Vink, Jeroen van der Laak, Christina Hulsbergen-van de Kaa, Geert Litjens

arXiv:2002.04500v1eess.IVcs.CVq-bio.QM

TL;DR

Gleason grading varies between observers, while evidence on AI assistance in pathology is limited. This study tested AI-assisted prostate-biopsy grading and found significantly higher agreement with an expert reference standard when pathologists used AI.

  • Problem

    Gleason grading has significant inter- and intraobserver variability, making evaluation of AI assistance for prostate-biopsy grading relevant.

    Source: Introduction · p. 4

  • Method

    A panel of 14 observers graded prostate biopsies with and without AI assistance.

    Source: The observer panel · p. 7

  • Results

    Agreement with an expert reference standard increased significantly with AI assistance.

    Source: Discussion · p. 13

  • Takeaways & Limitations

    The findings support AI assistance as a tool that can add benefit to pathologists’ Gleason grading.

    Source: Discussion · p. 13

    The study did not directly measure grading time per case, leaving whether AI makes grading faster for future research.

    Source: Discussion · p. 13

Abstract

from arXiv · show

While the Gleason score is the most important prognostic marker for prostate cancer patients, it suffers from significant observer variability. Artificial Intelligence (AI) systems, based on deep learning, have proven to achieve pathologist-level performance at Gleason grading. However, the performance of such systems can degrade in the presence of artifacts, foreign tissue, or other anomalies. Pathologists integrating their expertise with feedback from an AI system could result in a synergy that outperforms both the individual pathologist and the system. Despite the hype around AI assistance, existing literature on this topic within the pathology domain is limited. We investigated the value of AI assistance for grading prostate biopsies. A panel of fourteen observers graded 160 biopsies with and without AI assistance. Using AI, the agreement of the panel with an expert reference standard significantly increased (quadratically weighted Cohen's kappa, 0.799 vs 0.872; p=0.018). Our results show the added value of AI systems for Gleason grading, but more importantly, show the benefits of pathologist-AI synergy.

5. Department of Health. Public Health Agency of Catalonia. Lleida, Catalonia, Spain … 9. Tata Medical Center, Department of Pathology, Kolkata, India

The supplied excerpt lists a Swiss histopathology and cytology laboratory affiliation under the Tata Medical Center subsection.

9. Tata Medical Center, Department of Pathology, Kolkata, India

  • A labor team is affiliated with the Department of Histopathology and Cytology in Goldach, Switzerland.Source: §9 · p. 1

11. Instituto Mário Penna, Belo Horizonte, Brazil … Introduction

Gleason grading is clinically important but variable, and deep-learning systems can achieve pathologist-level performance within study limits. This study investigates whether AI feedback improves pathologists’ grading of prostate biopsies.

Introduction

  • The biopsy Gleason score is the most important prognostic marker for prostate cancer, yet grading shows inter- and intraobserver variability that grade groups have not resolved.Specialized uropathologists show higher concordance, but their expertise is not always available.Source: Introduction · p. 4
  • Deep-learning systems have shown pathologist-level performance in prostate cancer grading tasks, but only within the limits of study setups [5, 7, 8].The cited applications include grading prostatectomies, tissue microarrays, and biopsies.Source: Introduction · pp. 4–5
  • Evidence for deep-learning systems embedded in pathologists’ workflows remains limited; pathology studies combining expert opinions with automated-system feedback include lymph-node metastasis detection, where micrometastasis sensitivity increased and overall case-reading time decreased.Source: Introduction · p. 5
  • To the authors’ knowledge, no prior study had evaluated AI assistance for Gleason grading of prostate biopsies, a prognostic measure rather than computer-aided detection or diagnosis.The authors frame AI systems as tools for pathologists that should be evaluated in that role.Source: Introduction · p. 5
  • AI performance can be affected by non-prostate or atypical tissue, ink, fixation, scanning and cutting artifacts, and rare cancer subtypes.Many errors, especially those caused by artifacts, are easily spotted by a human observer.Source: Introduction · p. 5
  • A previously developed system achieved pathologist-level performance in grade-group assessment and patient risk stratification, outperforming 10 of 15 observers on grade-group determination [7].This observer comparison was part of the system’s initial validation.Source: Introduction · pp. 5–6
  • The study found merit in integrating AI feedback into pathologists’ diagnostic process, with pathologist-AI synergy achieving the best grading performance.The study compares pathologists’ diagnostic performance with and without assistance from a deep-learning system.Source: Introduction · p. 6

Results

AI assistance increased the panel’s median agreement with the expert reference standard, while most panelists reported using AI feedback. The Results also describe the panel, reviewed cases, and readers’ assessments of the assistance.

The observer panel

  • The 14-member panel comprised 11 certified pathologists and three residents from 12 independent labs and eight countries.Source: The observer panel · p. 7

Dataset under review and reference standard

AI-assisted Gleason grading

  • Eleven of 14 panelists used AI feedback; they rated the growth-pattern overlay most useful and easy to interpret, the grade-group prediction least helpful, and generally found grading faster without distraction.Source: Figure 2 · p. 9, AI-assisted Gleason grading · p. 10

Performance of the panel with and without AI feedback

Discussion

The discussion concludes that AI assistance improves Gleason grading agreement and consistency, with the largest gains among lower-performing readers, while noting workflow findings, study limitations, and future research needs.

Discussion

  • Quadratically weighted Cohen’s kappa increased from 0.799 to 0.872 with AI assistance, while panel-member performance variance decreased, indicating more consistent Gleason grading.The authors describe this as a significant increase in agreement with the expert reference standard.Source: Discussion · p. 13
  • The AI system outperformed 10 of 14 pathologists in unassisted reads but only 5 of 14 in AI-assisted reads, and assisted readers’ median performance exceeded the standalone AI.The discussion presents these findings as evidence of an added benefit from using AI as a supportive diagnostic tool.Source: Discussion · p. 13
  • Performance gains were greatest among initially lower-scoring readers; most pathologists with more than 15 years’ experience scored comparably in both reads and reported faster grading, which was not timed.The study did not directly measure time per case.Source: Discussion · p. 13
  • Almost all pathologists found the system’s gland-level overlays useful and used them most, whereas the biopsy-level grade group was rated least useful because it duplicated information from the Gleason score.The overlays supplied more detailed feedback alongside the system’s overall assessment.Source: Discussion · p. 14
  • Possible second-read effects, including greater familiarity with digital viewing or Gleason grading, could have influenced performance, although the increase remained when more-experienced readers were excluded and unseen control cases also showed high performance.Most pathologists predicted that they scored the same between reads.Source: Discussion · p. 14
  • The study had no AI training phase and used 100 test and 60 control cases from one center; median kappa differed by set, supporting the need for larger multicenter evaluations.The reported median kappa was 0.872 for the test set and 0.910 for the control set.Source: Discussion · p. 15
  • Future research should measure whether AI assistance saves time and examine patient-level grading, since this study assessed single biopsies rather than integrating multiple biopsies from one patient.Patient-level assessment could enable approaches such as automatically prioritizing slides.Source: Discussion · p. 15

Methods

The methods established a consensus reference, selected biopsies for observer grading, supplied AI predictions during a second read, and compared paired agreement with the reference standard.

Collection of the dataset and setting the reference standard

Observer panel and case selection

  • The 160-case set combined 100 biopsies already graded by the pathologists with 60 new cases unseen by panel members as controls; ten benign cases were manually selected, and 50 sampled uniformly across five grade groups.Source: Observer panel and case selection · p. 17

Feedback of the AI system

  • The AI supplied biopsy-level volume, Gleason-score, and grade-group predictions alongside automatically processed overlays labeling malignant Gleason 3, 4, and 5 glands by color.Source: Feedback of the AI system · pp. 17–18

Second-read with AI assistance

  • After at least three months, pathologists reread cases alongside AI overlays and numerical predictions, without access to their prior grades or reference-standard feedback.Source: Second-read with AI assistance · p. 18

Statistical analysis

  • Performance used quadratic-weighted Cohen’s kappa, summarized by median across observers; paired reads were compared with a Wilcoxon signed-rank test on the 100 shared cases.Source: Statistical analysis · p. 19

Competing interests

Several authors reported financial relationships during or outside the study; the remaining listed authors disclosed nothing to report.

Competing interests

  • W.B., J.v.d.L. and G.L. reported financial relationships; 14 other listed authors reported nothing to disclose.W.B. reported Dutch Cancer Society grants during the study. J.v.d.L. reported fees or grants from Philips, ContextVision, AbbVie and Sectra outside the submitted work; G.L. reported Dutch Cancer Society grants during the study and Philips Digital Pathology Solutions grants and Novartis fees outside the submitted work.Source: Competing interests · p. 20
Loading 2002.04500v1…