PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning-based algorithm for assessment of knee osteoarthritis severity in radiographs matches performance of radiologists

Albert Swiecicki, Nianyi Li, Jonathan O'Donnell, Nicholas Said, Jichen Yang, Richard C. Mather, William A. Jiranek, Maciej A. Mazurowski

arXiv:2207.12521v1eess.IVcs.CV

TL;DR

Radiographic knee osteoarthritis severity grading is subject to inter-reader variability, while prior approaches did not include lateral views or compare against multiple radiologists. This paper uses a deep-learning algorithm with PA and LAT radiographs to assess KL severity, finding radiologist-level agreement and reproducible scoring.

  • Problem

    KL grading is used to assess knee osteoarthritis severity, but human scoring varies, and prior approaches did not incorporate lateral views or compare with multiple radiologists.

    Source: Introduction · p. 2

  • Method

    The algorithm localizes knee joints and classifies KL severity using both PA and LAT radiographs.

    Source: Step 1: knee joint localization… · p. 6, Step 2: Knee OA severity… · p. 7

  • Results

    The algorithm’s assessment differed from radiologists about as much as radiologists differed from one another; its accuracy on the entire MOST test set was 71.90%.

    Source: Knee osteoarthritis diagnosis · p. 8, Discussion · p. 10

  • Takeaways & Limitations

    The algorithm could provide an accurate and reproducible measure of osteoarthritis severity for research and clinical decision-making.

    Source: Discussion · p. 11

    The small two-view improvement over PA-only applies specifically to this algorithm, and the study lacked radiologist comparisons using both views versus PA alone.

    Source: Discussion · p. 11

Abstract

from arXiv · show

A fully-automated deep learning algorithm matched performance of radiologists in assessment of knee osteoarthritis severity in radiographs using the Kellgren-Lawrence grading system. To develop an automated deep learning-based algorithm that jointly uses Posterior-Anterior (PA) and Lateral (LAT) views of knee radiographs to assess knee osteoarthritis severity according to the Kellgren-Lawrence grading system. We used a dataset of 9739 exams from 2802 patients from Multicenter Osteoarthritis Study (MOST). The dataset was divided into a training set of 2040 patients, a validation set of 259 patients and a test set of 503 patients. A novel deep learning-based method was utilized for assessment of knee OA in two steps: (1) localization of knee joints in the images, (2) classification according to the KL grading system. Our method used both PA and LAT views as the input to the model. The scores generated by the algorithm were compared to the grades provided in the MOST dataset for the entire test set as well as grades provided by 5 radiologists at our institution for a subset of the test set. The model obtained a multi-class accuracy of 71.90% on the entire test set when compared to the ratings provided in the MOST dataset. The quadratic weighted Kappa coefficient for this set was 0.9066. The average quadratic weighted Kappa between all pairs of radiologists from our institution who took a part of study was 0.748. The average quadratic-weighted Kappa between the algorithm and the radiologists at our institution was 0.769. The proposed model performed demonstrated equivalency of KL classification to MSK radiologists, but clearly superior reproducibility. Our model also agreed with radiologists at our institution to the same extent as the radiologists with each other. The algorithm could be used to provide reproducible assessment of knee osteoarthritis severity.

Purpose

The paper targets reproducible knee osteoarthritis severity assessment, where radiographic KL grading is used in care decisions but human scoring varies. It introduces a two-view algorithm and a reader study to compare its performance with radiologists.

Purpose

  • Radiographic severity assessment is included in proposed knee-replacement appropriateness criteria, making reproducible KL grading relevant to those decisions.The introduction notes that approximately one-third of TKAs in one US sample were classified as inappropriate using published criteria.Source: Introduction · p. 2
  • The KL scale runs from 0 to 4 and incorporates joint-space narrowing, osteophytes, and bony changes assessed on knee radiographs.The paper identifies inter-reader variability in current assessment processes.Source: Introduction · p. 2
  • Prior approaches did not incorporate the lateral radiographic view or compare algorithm performance with multiple radiologists.Source: Introduction · p. 3
  • The study presents an automatic KL assessment model incorporating lateral radiographs and evaluates it in a reader study with five radiologists.Source: Introduction · p. 3

Dataset

The study used paired PA and LAT knee radiographs from MOST, applying exclusions to form a final dataset of 9,739 exams from 2,802 patients. Patients were split into non-overlapping training, validation, and test sets.

Dataset

  • Knees were excluded for marked or missing data, missing JSN or osteophyte grades, or absent paired PA and LAT views; this removed less than 10% for missing grades.Source: Dataset · p. 4
  • The final dataset contained 9,739 exams from 2,802 patients, with 9,739 PA, 9,239 left LAT, and 9,264 right LAT images.Source: Dataset · p. 4
  • When multiple images for one view were available at a visit, one was selected randomly.Source: Dataset · p. 4
  • Patients were randomly assigned to non-overlapping training, validation, and test subsets, with dataset splits and KL-grade distributions presented in Tables 1 and 2.Source: Dataset · p. 4, Table 1 · p. 4, Table 2 · p. 4
  • MOST grade 1.9 cases were treated as grade 2 because grade 1.9 is not part of the standard KL system.Source: Dataset · p. 4

Knee joint annotations

The detection-model annotations came from manually marking knee-joint centers in randomly selected training-set knees. The annotated knees were divided into detection-model training, validation, and testing groups.

Knee joint annotations

Kellgren-Lawrence annotations from our institution

Five radiologists at the institution provided KL labels for a subset of the classification test set. Their readings used both PA and LAT views.

Kellgren-Lawrence annotations from our institution

Image preprocessing

The pipeline preprocesses radiographs, detects knee joints, and then classifies their osteoarthritis severity. Localization uses a Faster R-CNN-based region proposal network to identify knee regions in PA and LAT images.

Image preprocessing

  • Radiographs were rescaled to 0.2 mm per pixel, converted from 16-bit to 8-bit grayscale, and normalized using maximum intensity and pixel-intensity standard deviation.Source: Image preprocessing · p. 5

The algorithm for knee osteoarthritis severity estimation

Step 1: knee joint localization algorithm

  • A Faster R-CNN with a VGG16 backbone uses a region proposal network to localize knee regions in the X-ray images.Source: Step 1: knee joint localization… · p. 6
  • Localization models were trained separately for right and left knees in PA and LAT views, using 1000-pixel bounding boxes centered on labeled joints and selecting the most certain detection when multiple boxes were returned.Source: Step 1: knee joint localization… · p. 6

Step 2: Knee OA severity classification using Multi-input CNN

The classification model combines PA and LAT views in a multi-input CNN to predict knee osteoarthritis severity. It uses cropped knee patches, class-balanced sampling, augmentation, and validation-based model selection.

Step 2: Knee OA severity classification using Multi-input CNN

  • The multi-input CNN processes PA and LAT views in separate early branches, combines their feature maps, and predicts KL-grade confidence scores.Source: Step 2: Knee OA severity… · p. 7
  • The network receives 700 × 700-pixel PA and LAT patches centered using Faster R-CNN detections, then resized to 256 × 256 pixels.Source: Step 2: Knee OA severity… · p. 7
  • After ten epochs without augmentation, training used horizontal flips, rotations, translations, scaling, and shearing; Adam used a learning rate of 0.00001 with early stopping patience 20.Source: Step 2: Knee OA severity… · p. 7
  • Training batches sampled all classes with equal probability; ten models were trained, and the model with the best validation score was tested.Source: Step 2: Knee OA severity… · p. 7
  • Separate PA-only and LAT-only architectures were also developed, with the same training process as the multi-input model.Source: Step 2: Knee OA severity… · p. 7

Assessment of the proposed models in a multi-reader setting

The study evaluated agreement between the algorithm and five institutional radiologists, alongside agreement among the radiologists themselves. It used quadratic-weighted kappa to measure agreement while weighting misclassification errors differently.

Assessment of the proposed models in a multi-reader setting

Knee Joint detection

The Faster R-CNN detection models localized nearly all evaluated knees at the specified intersection-over-union threshold. Table 3 presents detection-test-set results.

Knee Joint detection

Knee osteoarthritis diagnosis

The multi-input model achieved 71.90% accuracy and a quadratic weighted Kappa of 0.907 against MOST expert annotations on the entire test set. Single-view models had lower reported accuracies.

Knee osteoarthritis diagnosis

  • 71.90% accuracy and 0.907 quadratic weighted Kappa were achieved by the multi-input model against MOST expert annotations on the entire test set.Figure 3 shows the model's unnormalized and normalized confusion matrices; Table 4 is identified as a comparison of classification results.Source: Knee osteoarthritis diagnosis · p. 8, Figure 3 · p. 9, Table 4
  • 70.85% accuracy for PA and 61.18% for LAT were reported for the single-view models.Source: Knee osteoarthritis diagnosis · p. 8
  • Figure 3 presents unnormalized and normalized confusion matrices for the multi-input model.Source: Figure 3 · p. 9

Classification performance in comparison to 5 readers at our institution

The algorithm’s agreement with institutional radiologists was similar to their agreement with one another. The discussion notes limits in interpreting KL grades and in assessing the added value of lateral views.

Classification performance in comparison to 5 readers at our institution

  • 0.769 was the average quadratic-weighted Kappa between the multi-input model and five institutional readers, versus 0.748 between institutional readers; Figure 4 presents the Kappa comparisons.The average quadratic-weighted Kappa between Duke Hospital and MOST readers was 0.759.Source: Classification performance in… · p. 9, Figure 4 · p. 10

Discussion

  • The authors conclude that algorithm–radiologist differences were virtually the same as differences between radiologists, while the algorithm provides scores free of inter-reader variability.They describe the same cases receiving the same score and all cases being read by the same algorithm.Source: Discussion · pp. 10–11
  • The greatest algorithm–reader differences occurred at lower KL grades, particularly grade 1; the authors hypothesize that greater subjectivity contributes to this pattern.They note that differences between radiologists were also higher for those grades.Source: Discussion · p. 11
  • High inter-reader variability in the gold standard makes algorithm assessment difficult because prediction–reader disagreements may reflect algorithm error, reader error, or task subjectivity.The reader study compared algorithm–radiologist differences with differences between radiologists; future work could assess prediction of knee-replacement outcomes.Source: Discussion · p. 11
  • The authors state that their algorithm outperformed prior algorithms under the evaluations reported in those studies, while noting those approaches used PA views and lacked comparison with multiple radiologists.The paper points to Table 5 for the cross-publication comparison.Source: Discussion · p. 11
  • The two-view model showed a slight improvement over the one-view model, but this study cannot establish the lateral view’s independent contribution.The authors note that their comparison was specific to this algorithm and lacked radiologist assessments using PA alone and PA plus LAT views.Source: Discussion · p. 11
  • The conclusion proposes the algorithm as an accurate and reproducible measure of osteoarthritis severity for research and clinical decision-making.Source: Discussion · p. 11

Appendix A Single input model architecture

The appendix identifies a single-input model architecture and lists supplementary single-view results and reader-agreement materials.

Appendix A Single input model architecture

  • Figure X is identified as the single-input model architecture; the supplied caption gives no further architectural details.Source: Figure X · p. 14

Appendix B Single view models’ results

Appendix D Confusion matrixes between Duke readers, MOST assignments and deep learning.

  • Figure Z2 is identified as confusion matrices comparing Duke readers, MOST assignments, and deep learning.Source: Figure Z2 · p. 17
Loading 2207.12521v1…