Source-linked AI summary
Automatic Grading of Individual Knee Osteoarthritis Features in Plain Radiographs using Deep Convolutional Neural Networks
Aleksei Tiulpin, Simo Saarakkala
TL;DR
Knee OA grading relies on subjective KL and OARSI assessments, while few methods automate individual OA features from plain radiographs. This study uses an ensemble deep-learning pipeline to predict KL and OARSI grades simultaneously, achieving strong agreement on independent testing and high performance for detecting radiographic OA.
Problem
Radiographic knee OA grading remains subjective, and few methods assess individual OA features from plain radiographs despite OARSI providing feature- and compartment-specific grading.
Method
The method uses transfer learning with an ensemble of two 50-layer residual convolutional networks to predict one KL grade and six OARSI grades from knee radiographs.
Results
The independent evaluation produced Cohen’s kappa values of 0.83 for KL and 0.79–0.94 across OARSI grades, with balanced accuracy ranging from 64.72% to 83.34%.
Takeaways & Limitations
The methodology may become a useful tool in clinical OA trials, while OARSI grades support more comprehensive quantification of OA-related changes between follow-up examinations.
Takeaways & Limitations
The ensemble is computationally heavy because of ensembling and could affect real-life use unless deployed on a GPU.
Abstract
from arXiv · showhide
Knee osteoarthritis (OA) is the most common musculoskeletal disease in the world. In primary healthcare, knee OA is diagnosed using clinical examination and radiographic assessment. Osteoarthritis Research Society International (OARSI) atlas of OA radiographic features allows to perform independent assessment of knee osteophytes, joint space narrowing and other knee features. This provides a fine-grained OA severity assessment of the knee, compared to the gold standard and most commonly used Kellgren-Lawrence (KL) composite score. However, both OARSI and KL grading systems suffer from moderate inter-rater agreement, and therefore, the use of computer-aided methods could help to improve the reliability of the process. In this study, we developed a robust, automatic method to simultaneously predict KL and OARSI grades in knee radiographs. Our method is based on Deep Learning and leverages an ensemble of deep residual networks with 50 layers, squeeze-excitation and ResNeXt blocks. Here, we used transfer learning from ImageNet with a fine-tuning on the whole Osteoarthritis Initiative (OAI) dataset. An independent testing of our model was performed on the whole Multicenter Osteoarthritis Study (MOST) dataset. Our multi-task method yielded Cohen's kappa coefficients of 0.82 for KL-grade and 0.79, 0.84, 0.94, 0.83, 0.84, 0.90 for femoral osteophytes, tibial osteophytes and joint space narrowing for lateral and medial compartments respectively. Furthermore, our method yielded area under the ROC curve of 0.98 and average precision of 0.98 for detecting the presence of radiographic OA (KL $\geq 2$), which is better than the current state-of-the-art.
Introduction
Knee OA assessment relies on subjective KL and OARSI radiographic grading, while prior automated approaches provided limited individual-feature analysis. This study presents a multi-task framework using independent OAI training and MOST testing to predict KL and OARSI grades from plain radiographs.
- Clinical assessment: OA is commonly diagnosed with radiography, but KL provides a subjective composite score rather than separate compartment-specific feature grades.OARSI separately grades femoral osteophytes, tibial osteophytes, and joint space narrowing in lateral and medial compartments, but remains reader-subjective.
- Prior work: Only a few studies had attempted to assess individual knee OA features from plain radiographs using deep learning.Deep learning learns image features directly from data and had already been applied successfully to automated KL grading and other OA analyses.
- Prior work: Prior osteophyte analysis reported an area under the ROC curve of 0.85 but used a relatively small test set and a clinically questionable binary split.The split grouped OARSI grades 0–1 versus 2–3 even though grade 1 already indicates osteophyte presence.
- Prior work: A preceding CNN study jointly predicted KL and OARSI grades, but combined MOST and OAI data and produced lower agreement than human inter-rater agreement.The present study addresses these limitations with separate datasets for training and testing.
- Study contribution: The study presents a deep-learning framework for simultaneous prediction of individual knee OA features and overall severity from plain radiographs.The contribution includes extensive evaluation of network architecture, transfer learning, ensembling, and joint KL–OARSI learning.
- Study contribution: The authors use OAI and MOST as independent training and testing datasets for automatic OARSI grading.This design is presented as a first use of two independent datasets for this assessment task.
Materials and Methods
The study used OAI radiographs for training and MOST radiographs for testing, with preprocessing, ImageNet transfer learning, and an ensemble of two multi-task convolutional networks predicting KL and six OARSI grades.
- Datasets: 19,704 knees from OAI formed the training set, while 11,743 knees from MOST formed the testing set after removing missing labels.Both datasets provided fixed-flexion bilateral PA radiographs; OAI included 4,796 participants and MOST included 3,026 participants.
- Preprocessing: Knee joint landmarks were localized with random forest regression voting, after which images were cropped, aligned, normalized, and resized to 310×310 pixels.The cropped regions measured 140 × 140 mm and were rescaled at 0.45 mm resolution.
- Model: The ensemble comprised two convolutional neural networks with ImageNet-pretrained convolutional parts and seven task-specific fully connected layers for KL and six OARSI outputs.Average pooling connected each convolutional block to the task-specific layers.
- Training and evaluation: Global Weighted Average Pooling and weighted sampling did not improve cross-validation scores, so the reported approach used average pooling without sampling-based score gains.The training procedure also used augmentation, weight decay, dropout, and Adam optimization.
- Training and evaluation: Five-fold subject-wise stratified cross-validation selected two models, whose predictions were averaged across five snapshots per model for final testing.The experiments compared transfer learning, training from scratch, joint KL/OARSI prediction, and OARSI-only prediction.
Results
The ensemble achieved strong agreement for KL and individual OARSI features on the test set, while experiments found that joint KL/OARSI training reduced automatic OARSI-grading performance.
- Ablation findings: Jointly training KL and OARSI grading tasks hurt automatic OARSI-grading performance in the cross-validation experiments.The study nevertheless evaluated joint prediction because KL provides a composite knee assessment used by practitioners.
- Test-set performance: 0.82 Cohen’s kappa was achieved for KL grading on the test set.The corresponding balanced accuracy was 66.68%.
- Test-set performance: 0.79, 0.84, 0.94, 0.84, 0.83, and 0.90 Cohen’s kappa were obtained for FO, TO, and JSN in lateral and medial compartments, respectively.The corresponding balanced accuracies were 63.58%, 68.85%, 78.55%, 65.49%, 72.02%, and 80.66%.
- Additional analyses: The evaluation included confusion-matrix analyses for OARSI and KL grades and detection of OA, osteophyte presence, and abnormal JSN by compartment.OARSI confusion matrices appeared in Figure 4, while the KL confusion matrix was provided in Supplementary Figure 1.
Discussion
The study presents a transfer-learning framework for simultaneous OARSI and KL grading, validated on an independent test set. It reports improved automatic grading performance, while acknowledging interpretability, computational, imaging, and feature-coverage limitations.
- Method: The method uses transfer learning and an ensemble of two 50-layer residual networks with squeeze-excitation and ResNeXt blocks for simultaneous OARSI and KL grading.The ensemble combines SE-ResNet-50 and SE-ResNeXt50-32x4d backbones.
- Validation: Independent testing on MOST provided the first reported independent-test evaluation of automatic OARSI grading from plain radiographs.The model was trained using OAI and tested using MOST.
- Experimental findings: Cross-validation showed that transfer learning helped OARSI grading, whereas joint KL–OARSI training reduced performance despite its clinical relevance.KL provides a composite knee assessment, while OARSI captures individual features for monitoring OA-related changes over time.
- Limitations: The authors identify absent attention-map analysis, computationally heavy ensembling, whole-knee image training, and incomplete OARSI feature coverage as limitations.The computational burden could affect real-life use unless the model is deployed on a GPU.
- Robustness analysis: Evaluation using only MOST’s first imaging follow-up produced results consistent with previous KL state-of-the-art performance and better than Antony’s joint-training results.The authors used this analysis to assess the potential impact of changing image appearance across follow-ups.
- Implications: The authors conclude that the methodology could support clinical OA trials and provide clinicians with more systematic quantitative information.This conclusion is explicitly stated despite the study’s limitations.
Supplementary data
The supplementary materials show KL grading and OARSI-related dataset distributions, alongside cross-validation reporting for the evaluated models. The model notation identifies architectures, compartments, and feature-specific grading tasks.
- The KL confusion matrix reports percentages for predicted versus graded Kellgren-Lawrence categories.
- Cross-validation results report balanced accuracy for each trained task on out-of-fold OAI samples.
- The evaluated architectures include SE-ResNet-50 and SE-ResNext50-32x4d, trained from scratch or with transfer learning, with and without the KL grade.
- The supplementary figures visualize lateral OARSI grade distributions in the MOST and OAI datasets.