PaperlayerView exact revision on arXiv

Source-linked AI summary

KneePreM: Towards 3D Knee MRI Foundation Models via Large-Scale Unlabeled Pretraining and Label-Efficient Fine-Tuning

Xinxin Wang, Liam Hazan, Jing Li, Simona Rabinovici-Cohen, Xiaojuan Li, Mingrui Yang

arXiv:2609.31461v1cs.CV

TL;DR

Unlabeled knee MRI scans are available but insufficiently leveraged, while annotation costs and three-dimensional knee structure make analysis challenging. KneePreM is a 3D self-supervised pretrained model whose encoder is reused for discriminative and dense tasks. It outperformed both initialization baselines in full-data classification and had the highest mean Dice at each reported segmentation data setting, with larger separations in reduced-label settings.

  • Problem

    Large volumes of unlabeled knee MRI scans are available but remain insufficiently leveraged, despite costly annotations and the challenges of automated knee MRI analysis.

    Source: Introduction · p. 2

  • Method

    KneePreM is a 3D self-supervised pretrained model whose encoder can be reused for discriminative and dense tasks.

    Source: Discussion · pp. 8–9

  • Results

    KneePreM outperformed random initialization and SuPreM in full-data classification and achieved the highest mean Dice at each reported segmentation data setting, with larger separations in reduced-label settings.

    Source: Discussion · pp. 8–9

  • Takeaways & Limitations

    Its advantage was particularly evident in reduced-label settings across the reported classification and segmentation evaluations.

    Source: Discussion · p. 9

    Pretraining used OAI data acquired with relatively standardized imaging protocols, which may limit generalizability to more heterogeneous settings.

    Source: Discussion · p. 10

Abstract

from arXiv · show

Background: Large volumes of unlabeled knee MRI scans are available across repositories but remain insufficiently leveraged. We developed KneePreM, a knee-specific 3D self-supervised model, and evaluated transfer and label efficiency for classification and segmentation. Methods: A 3D U-Net masked autoencoder was pretrained on 19,011 unlabeled Osteoarthritis Initiative (OAI) MRI series from 4,791 participants. Downstream fine-tuning used full and reduced training sets for fastMRI+ two-label classification (1,172 examinations), Arthroscopic Partial Meniscectomy (APM) eight-target classification (1,716 examinations), SKM-TEA segmentation (155 examinations), and APM segmentation (25 examinations). Baselines were random initialization and SuPreM. Deployment workflow was implemented with a Model Context Protocol interface. Evaluation metrics included balanced accuracy, F1 score, ROC AUC, PR AUC, and Dice score. Statistical analysis used bootstrap confidence intervals and paired bootstrap tests for classification and Wilcoxon signed-rank tests for segmentation. Results: KneePreM achieved higher full-data macro ROC AUC than both baselines for fastMRI+ and APM (all p < .001). For fastMRI+ classification, KneePreM achieved a ROC AUC of 0.722 using 50% of the training data, exceeding both full-data baselines. In APM classification, KneePreM reached a ROC AUC of 0.740 with 70% of the data, matching the full-data random baseline and outperforming SuPreM. For SKM-TEA segmentation, its 70%-data Dice of 0.838 exceeded the full-data random baseline (0.835) and both same-budget comparators. In APM segmentation, its 75%-data Dice of 0.746 exceeded the full-data random baseline (0.731) and both same-budget comparators. Conclusion: KneePreM improves transfer performance and label efficiency across knee MRI classification and segmentation tasks, particularly when labeled training data are limited.

Introduction

KneePreM addresses underused unlabeled knee MRI and the need to capture three-dimensional structure in a domain-specific pretrained model.

Introduction

  • Costly annotations, heterogeneous protocols, and inherently 3D knee morphology make automated analysis challenging, especially when labeled data are limited.Source: Introduction · p. 2
  • Because 2D models do not intrinsically capture through-plane continuity, compact 3D masked reconstruction is proposed to learn whole-knee context.Source: Introduction · p. 2
  • The study aims to develop KneePreM, a 3D self-supervised knee MRI model that leverages unlabeled scans while preserving essential spatial structures.Source: Introduction · p. 3

Materials and Methods

  • The study used public research datasets and a previously collected clinical knee MRI cohort, with OAI for pretraining and external cohorts for transfer.Source: Materials and Methods · p. 3

Pretraining Dataset

  • Pretraining used 19,011 OAI MRI series from 4,791 participants, including sagittal DESS and intermediate-weighted turbo spin-echo examinations.Source: Pretraining Dataset · p. 3

Downstream Classification Datasets

The transfer evaluation draws on classification and segmentation cohorts with varied sizes and imaging characteristics, using standardized volumes and task-specific preprocessing.

Downstream Classification Datasets

Downstream Segmentation Datasets

Image Processing and Augmentation

KneePreM Architecture and Self-Supervised Pretraining

KneePreM learns multiscale 3D features through masked reconstruction, then transfers its encoder to classification and segmentation under full and reduced-label settings.

KneePreM Architecture and Self-Supervised Pretraining

  • Masked autoencoding hid 64% of each 3D input in 4 × 4 × 4-voxel cuboids and reconstructed masked voxels from remaining context using summed voxelwise mean squared error.Source: KneePreM Architecture and… · p. 5

Downstream Fine-Tuning

  • Fine-tuning used task-specific classification heads or a 3D U-Net decoder for segmentation, with task losses and validation-selected checkpoints.Source: Downstream Fine-Tuning · p. 5

Reduced-Data Experiments

  • Label efficiency was tested by reducing training data while keeping validation and test sets fixed; fastMRI+ used 100%, 70%, 50%, and 30% training sizes.Source: Reduced-Data Experiments · p. 6

Baseline Methods

  • Random initialization, SuPreM, and KneePreM used the same task architecture and training setup, differing only in initial weights.Source: Baseline Methods · p. 6

Statistical Analysis

KneePreM exceeded both baselines in full-data classification ROC AUC on fastMRI+ and APM, with strong performance also reported at reduced training sizes.

Statistical Analysis

  • Classification confidence intervals and paired macro ROC AUC comparisons used bootstrap resampling, while segmentation comparisons used two-sided Wilcoxon signed-rank tests.Source: Statistical Analysis · p. 6

APM Compartment-Status Classification

fastMRI+ Classification

  • At 50% of fastMRI+ training data, macro ROC AUC was 0.722, exceeding the full-data random and SuPreM values of 0.714 and 0.686.Source: fastMRI+ Classification · p. 7

APM Compartment-Status Classification

  • At 70% of APM training data, macro ROC AUC was 0.740, approximating full-data random initialization at 0.741 and exceeding same-size random and SuPreM values of 0.708 and 0.633.Source: APM Compartment-Status Classification · p. 7
  • APM confusion matrices showed heterogeneous target prevalence and differences in true-positive or true-negative detections across compartments relative to random initialization.Source: APM Compartment-Status Classification · p. 7

SKM-TEA Segmentation

KneePreM achieved the highest mean Dice at every reported SKM-TEA training size. Its 70%-data result exceeded both same-budget baselines and the full-data random baseline.

SKM-TEA Segmentation

  • With all 86 training examinations, mean Dice was 0.842 for KneePreM, 0.840 for SuPreM, and 0.835 for random initialization.KneePreM had the highest mean Dice at every reported training size.Source: SKM-TEA Segmentation · p. 7
  • At 70%, the reported values were 0.838, 0.832, and 0.827; KneePreM exceeded both same-budget baselines and the full-data random baseline.Both same-budget comparisons had p < .001; the full-data random baseline was 0.835.Source: SKM-TEA Segmentation · p. 7

APM Segmentation

KneePreM had the highest APM mean Dice at every reported training size. Its 75%-data result exceeded the full-data random baseline and both same-budget comparators, although the paired comparison was limited by five folds.

APM Segmentation

  • With 16 training examinations per fold, mean Dice was 0.779 for KneePreM, 0.752 for SuPreM, and 0.731 for random initialization.KneePreM had the highest APM mean Dice at every training size.Source: APM Segmentation · p. 8
  • KneePreM’s largest structure-level advantages over random initialization involved lateral tibial cartilage and both menisci.At full data, its paired Dice improvement over random initialization averaged 0.048 across folds (median, 0.041; 95% CI, 0.016-0.080).Source: APM Segmentation · p. 8
  • At 75% data, mean Dice was 0.746 for KneePreM, 0.713 for SuPreM, and 0.672 for random initialization.The paired improvement over random initialization averaged 0.074 (median, 0.071; 95% CI, 0.017-0.130); the comparison was limited by five folds (p = .06).Source: APM Segmentation · p. 8
  • At 50% data, mean Dice was 0.684 for KneePreM, 0.664 for SuPreM, and 0.617 for random initialization.At 25% data, the corresponding means were 0.578, 0.576, and 0.503.Source: APM Segmentation · p. 8

Qualitative Segmentation Assessment

The section examines KneePreM’s transfer and label efficiency across knee MRI tasks, including qualitative segmentation comparisons with reference anatomies and baselines. It also discusses limitations in generalizability, resolution, external validation, and prospective workflow evaluation, alongside directions for improving robustness.

Qualitative Segmentation Assessment

  • In representative SKM-TEA cases, KneePreM recovered more LTC and LM extent, better preserved the anterior LM boundary, and more closely matched the MM reference than Random and SuPreM.Together, the cases illustrate improved delineation of highlighted cartilage and meniscal structures.Source: Qualitative Segmentation Assessment · p. 8
  • In representative APM cases, KneePreM better matched MM and LTC references, reduced FC-to-PC mislabeling and tibial-compartment confusion, and produced a more continuous FC boundary than the baselines.The examples collectively showed fewer missed structures, reduced spurious predictions, and improved boundary fidelity across cartilage and meniscus compartments.Source: Qualitative Segmentation Assessment · p. 8

Discussion

  • Across classification and segmentation, KneePreM outperformed both baselines in full-data classification and achieved the highest mean Dice at every segmentation data setting, with larger advantages under reduced labels.The Discussion reports improved fastMRI+ target ROC AUCs and strongest macro PR AUC, as well as highest mean Dice across SKM-TEA and APM segmentation settings.Source: Discussion · pp. 8–9
  • At 70% of fastMRI+ labels, KneePreM's macro ROC AUC exceeded both full-data baselines; at 30%, it remained near full-data SuPreM and beat same-size comparators in PR AUC.It also achieved the strongest macro PR AUC and improved both target ROC AUCs.Source: Discussion · pp. 8–9
  • KneePreM reconstructs raw knee MRI without task-specific labels and reuses one encoder for dense and discriminative tasks; SuPreM transferred effectively to segmentation but less favorably to classification.Masked autoencoding uses a reconstruction target available in every unlabeled volume and aligns with an encoder-decoder architecture.Source: Discussion · p. 9
  • A 3D U-Net preserves volumetric context across contiguous slices and supported large-scale pretraining on a single 80-GB GPU; the compact domain-specific volumetric model is computationally practical for specialist 3D pretraining.Source: Discussion · p. 10
  • Pretraining on OAI scans acquired with relatively standardized imaging protocols may limit generalizability to more heterogeneous imaging settings.Source: Discussion · p. 10
  • Downsampling made 3D masked autoencoding computationally feasible but may have failed to preserve some fine anatomical detail; external validation across institutions, scanners, vendors, field strengths, and pulse sequences remains necessary.Source: Discussion · p. 11
  • The agentic workflow was assessed primarily through model development and technical validation, not prospective evaluation in clinical practice.Source: Discussion · p. 11
  • Future work could assess multi-center, multi-vendor, and multi-sequence data for robustness and generalizability, while higher-resolution volumetric modeling may improve performance on subtle structural abnormalities.Source: Discussion · p. 11
  • Masked reconstruction of 19,011 unlabeled OAI knee MRI series enabled a compact 3D U-Net encoder to transfer across external classification and segmentation tasks.Source: Discussion · p. 11
  • Under matched architectures and training procedures, KneePreM outperformed random initialization and showed more consistent cross-task transfer than segmentation supervised pretraining, with clearest advantages in reduced-label settings.The findings support anatomy- and modality-specific volumetric pretraining as a practical strategy for improving label efficiency across knee MRI tasks, complementary to larger generalist or task-specific foundation models.Source: Discussion · p. 11

Data Availability

OAI, fastMRI+, and SKM-TEA are public datasets; the APM dataset contains protected clinical data and cannot be shared publicly for privacy and regulatory reasons.

Data Availability

Code Availability

Code for data processing, training, evaluation, and the agentic workflow is available in FuseMedML at the stated OAI example path.

Code Availability

Generative AI Declaration

OpenAI ChatGPT was used for language editing, manuscript organization, and formatting; the authors reviewed and edited the manuscript and take responsibility for its content.

Generative AI Declaration

Tables

The tables describe the pretraining and transfer cohorts, while the figures map KneePreM’s workflow and compare classification and segmentation evaluations. The supplied passages specify evaluation scopes and visual encodings, but provide few numerical outcome details.

Tables

  • Table 1 reports cohort and imaging characteristics, including OAI pretraining data and the fastMRI+, SKM-TEA, and APM transfer cohorts.OAI contributed 19,011 MRI series from 4,791 participants; fastMRI+ reports 1,172 examinations, and SKM-TEA includes 155 unique qDESS scans.Source: Table 1 · p. 15
  • Tables 2 and 3 report held-out test performance as macro estimates across two fastMRI+ labels and eight APM compartment-status targets, respectively.Both captions specify patient-level bootstrap 95% confidence intervals.Source: Table 2 · p. 16, Table 3 · p. 17
  • Tables 4 and 5 report overall and structure-level Dice by training-set size for SKM-TEA and APM clinical segmentation.Source: Table 4 · p. 18, Table 5 · p. 19

Figure Legends

  • Figure 1 depicts masked-volume reconstruction pretraining, encoder transfer and fine-tuning, classification heads, and a segmentation decoder.The segmentation decoder is randomly initialized.Source: Figure 1 · p. 20
  • Figure 2 compares fastMRI+ and APM macro ROC AUC by training-set size and with full-data bootstrap distributions and ROC curves.The caption identifies scratch as random initialization.Source: Figure 2 · p. 21
  • Figure 3 arranges full-data confusion matrices by task and model, with reference classes in rows and predicted classes in columns.The upper block covers meniscus tear and cartilage defect; the lower block covers eight APM statuses.Source: Figure 3 · p. 22
  • Figure 4 displays segmentation Dice across training sizes and distributions, including pairwise p values and KneePreM-minus-random effect summaries.Figure 5 provides qualitative slice comparisons across methods and data fractions.Source: Figure 4 · p. 23, Figure 5 · p. 25
  • In highlighted segmentation regions, KneePreM more closely matches the reference than the baseline methods.The qualitative examples include SKM-TEA and APM at full or reduced training-set sizes.Source: Figure Legends · p. 24, Figure 5 · p. 25

Related papers

Loading 2609.31461v1…