Source-linked AI summary
Repeatability of Multiparametric Prostate MRI Radiomics Features
Michael Schwier, Joost van Griethuysen, Mark G Vangel, Steve Pieper, Sharon Peled, Clare M Tempany, Hugo JWL Aerts, Ron Kikinis, Fiona M Fennessy, Andrey Fedorov
TL;DR
Radiomics features could serve as biomarkers, but their usefulness requires stable values across repeat scans, and repeatability evidence for prostate mpMRI remains limited. This study tests small prostate-tumor radiomics across imaging, preprocessing, extraction, and ROI configurations, finding that repeatability varies greatly with processing choices. The authors therefore recommend study-specific repeatability assessment and detailed reporting of configurations and implementations.
Problem
Repeatability evidence for MRI-based radiomics, particularly prostate cancer imaging, remains limited despite repeatability being necessary for reliable biomarker use.
Method
The study evaluates radiomics repeatability in prostate mpMRI test-retest data across normalization, pre-filtering, 2D/3D texture computation, discretization, ROI localization, and ICC measurement.
Results
Repeatability varies greatly and is highly sensitive to image type, preprocessing, and ROI configuration, although many features show higher repeatability than the Volume reference.
Takeaways & Limitations
Feature pre-selection should use repeatability analysis on representative study data when possible, with close attention to extraction configurations when prior evidence is used.
Takeaways & Limitations
The study is limited by a small sample, small tumor volumes, absent multi-reader consistency analysis, and no rigorous statistical modeling or testing of all preprocessing variations.
Abstract
from arXiv · showhide
In this study we assessed the repeatability of the values of radiomics features for small prostate tumors using test-retest Multiparametric Magnetic Resonance Imaging (mpMRI) images. The premise of radiomics is that quantitative image features can serve as biomarkers characterizing disease. For such biomarkers to be useful, repeatability is a basic requirement, meaning its value must remain stable between two scans, if the conditions remain stable. We investigated repeatability of radiomics features under various preprocessing and extraction configurations including various image normalization schemes, different image pre-filtering, 2D vs 3D texture computation, and different bin widths for image discretization. Image registration as means to re-identify regions of interest across time points was evaluated against human-expert segmented regions in both time points. Even though we found many radiomics features and preprocessing combinations with a high repeatability (Intraclass Correlation Coefficient (ICC) > 0.85), our results indicate that overall the repeatability is highly sensitive to the processing parameters (under certain configurations, it can be below 0.0). Image normalization, using a variety of approaches considered, did not result in consistent improvements in repeatability. There was also no consistent improvement of repeatability through the use of pre-filtering options, or by using image registration between timepoints to improve consistency of the region of interest localization. Based on these results we urge caution when interpreting radiomics features and advise paying close attention to the processing configuration details of reported results. Furthermore, we advocate reporting all processing details in radiomics studies and strongly recommend making the implementation available.
Introduction
Radiomics aims to use quantitative imaging features as disease biomarkers, but their clinical usefulness depends on repeatability across stable test-retest scans. Prostate mpMRI radiomics lacks consistent evidence on repeatability and processing choices, motivating systematic evaluation.
- Radiomics extracts quantitative imaging features intended to characterize disease and potentially support treatment-response prediction and patient management.
- Quantitative mpMRI analysis remains limited clinically despite mpMRI’s established role in image characterization, treatment planning, and response assessment.
- Repeatability requires a feature’s value to remain stable between scans when conditions remain stable, and it is necessary but not sufficient for predictive power.
- Repeatability can help pre-select features from hundreds of candidate feature sets and many parameter, filter, and preprocessing combinations.
- Prior repeatability studies largely focused on CT, while MRI presents challenges from relative signal intensities, acquisition artifacts, and lower spatial resolution.
- This study evaluates small prostate-tumor radiomics repeatability in mpMRI across normalization, pre-filtering, 2D versus 3D textures, discretization, and ROI localization.
Methods
The study used a prostate mpMRI test-retest dataset and extracted radiomics features under varied preprocessing and texture configurations. Repeatability was assessed with ICC while accounting for image quality, segmentation, and feature-extraction choices.
- Image Data and Segmentations: The dataset included fifteen men undergoing a second prostate MRI within two weeks, with T2w, SUB, and ADC images and expert-annotated tumor and prostate ROIs.
- Image Data and Segmentations: One subject was excluded from ADC analysis because one time point had substantially worse image quality, leaving fourteen subjects for ADC repeatability evaluation.
- Feature Extraction: Features were extracted with pyradiomics across First Order, Shape, GLCM, GLSZM, and GLRLM classes, with specified exclusions for correlated or unsuitable shape features.
- Preprocessing and Feature Extraction: The analysis varied image pre-filtering, normalization, texture dimensionality, and fixed-bin-width discretization while otherwise using pyradiomics settings.
- Measure of Repeatability: Repeatability was measured with ICC(1,1), relating within-subject repeated-scan variation to total population variability; BMS and WMS represent between- and within-subject mean squares.
- Measure of Repeatability: Because ICC is invariant to linear scaling and shifting, feature ICCs were compared with the study’s Volume ICC reference rather than fixed universal thresholds.
- Evaluation Approach: The evaluation began with a literature-informed subset of intensity and GLCM features, then summarized results across features or selected the top three per feature group.
- Evaluation Approach: The large extraction output prevented detailed exploration of every aspect in the paper, with additional analyses provided in supplementary material and source code.
Results
Repeatability varied substantially across preprocessing, discretization, feature-extraction, and ROI-localization configurations. Some features and configurations performed well, but no consistently superior normalization, pre-filtering, bin-width, or registration strategy emerged.
- Normalized vs Non-Normalized Images: Normalized ADC images produced several features matching or exceeding Volume repeatability, while no feature exceeded Whole Gland Volume ICC of 0.99.In the Tumor ROI, selected features reached at least the ADC Volume ICC of 0.7; in the Peripheral Zone, several reached approximately 0.91 or higher.
- Normalized vs Non-Normalized Images: SUB images yielded feature ICCs above Volume in the Tumor ROI and Peripheral Zone, but no feature exceeded Whole Gland Volume ICC of 0.94.Tumor ROI Entropy, Idm, and 10Percentile reached ICCs of 0.83, 0.91, and 0.87, respectively; Peripheral Zone Idm reached 0.85.
- Normalized vs Non-Normalized Images: T2w normalization generally reduced repeatability, whereas ADC normalization usually improved it and SUB normalization showed no clear trend.For T2w images, normalization lowered ICCs except for Whole Gland Correlation; for SUB images, effects depended on structure and feature.
- Influence of Different Bin-widths: Different bin widths caused relatively small ICC changes for most features, with bin widths 15 and 20 occupying middle rank distributions.At the lowest and highest bin widths, best and worst ranks appeared about equally often.
- Influence of Pre-filtering: Pre-filtering produced feature-dependent ICC variation, with JointEntropy consistently high across wavelet filters and 2D/3D extraction but no generally superior filter.Some filters improved Contrast above the reference threshold, while others worsened repeatability for different features; LoG with high sigma and exponential filtering with 3D extraction tended to perform poorly.
- ROI Localization: Registration-based ROI transfer improved ICCs for about half of features and reduced repeatability for the other half.The comparison used original ROIs versus ROIs transferred between time points by image registration.
Discussion
The discussion shows that radiomics-feature repeatability depends strongly on preprocessing, feature type, and ROI localization, without a universally superior configuration. Many features perform well in selected settings, but the study’s small, heterogeneous evaluation limits broad recommendations.
- Feature behavior: JointEntropy, Idm, Median, and Mean showed good repeatability in T2w and ADC images, whereas Kurtosis and Contrast sometimes approached ICC values near 0.Even well-performing features showed good repeatability only under specific preprocessing configurations.
- Discretization: Bin width influenced repeatability even within recommended limits, supporting evaluation of its effect in every new study.The observed influence was generally modest but still measurable.
- Pre-filtering: Pre-filtering produced substantial ICC variation across options, and no filter consistently improved repeatability for Tumor ROI T2w features.Peripheral Zone ADC features were a relative exception, showing greater stability.
- Preprocessing: Different preprocessing configurations produced the best repeatability for different features, preventing a small set of generally recommended settings.The results remained diverse even after narrowing the analysis to a specific image type and structure.
- Feature behavior: Some shape features exceeded Volume’s repeatability and were invariant to the tested preprocessing, but their correlation with Volume means they likely add limited information.These features capture less shape information than Volume.
- Interpretation and limitations: The study could not fully reproduce literature-reported repeatability, indicating that factors beyond the assessed preprocessing choices influence feature stability.The authors caution that the small study size and diverse results prevent confident general recommendations.
- ROI localization: Registration did not consistently improve repeatability and dramatically decreased it for some features, partly because small tumor ROIs make registration errors consequential.Expert segmentations also may not represent exactly the same tissue region across timepoints.
- Interpretation and limitations: The dataset had a small sample, small tumors, and no multi-reader consistency analysis, while rigorous statistical modeling was outside the study’s scope.The authors state that the study was intended as an overview of preprocessing effects rather than a thorough statistical analysis.
Conclusion
The study found that radiomics-feature repeatability varies greatly with image type, preprocessing, and ROI, without a universally stable feature–preprocessing combination. The authors therefore recommend study-specific repeatability analysis and detailed reporting of preprocessing and implementations.
- Repeatability varied greatly across image types, preprocessing configurations, and regions of interest in this prostate mpMRI dataset.
- Repeatability analysis should, whenever possible, be included in study-specific feature selection.
- Radiomics reports should provide preprocessing details, follow consensus feature definitions, and make implementations available when possible.
- The study did not identify a universally stable combination of radiomics feature and preprocessing configuration.
- Many features, especially top-three features within each feature group, showed higher repeatability than the Volume reference and may be candidates for prognostic features.
Figures
The figures illustrate that segmentation, registration, image normalization, bin width, filtering, and 2D versus 3D extraction can affect repeatability in configuration-dependent ways. No single normalization, filter, dimensionality, or bin-width choice consistently improves ICC across the evaluated settings.
- Segmentation and registration: Visually matched scans show changes in tumor and gland location or appearance despite stable disease and comparable acquisition parameters.
- Segmentation and registration: Manual segmentations can differ between timepoints, including inconsistent lesion annotation and boundary localization.
- Segmentation and registration: Registration does not necessarily align timepoint segmentations because both annotation inconsistencies and registration errors contribute to misalignment.
- Image quality: One ADC case was removed because its two scans had visibly different image quality, indicating an acquisition-protocol deviation.
- Normalization and bin width: Normalization improved ICCs for ADC, showed no clear trend for SUB, and was generally better omitted for T2w images.
- Normalization and bin width: For most features, changing bin width produced only modest ICC variation, with the majority of maximum differences around 0.2.
- Filtering and dimensionality: ICCs were widely distributed across tumor and peripheral-zone analyses, and no pre-filter consistently performed above the reference.
File Format Description
The feature files use CSV rows for image–mask combinations, with pyradiomics-derived feature columns followed by metadata describing extraction settings, image and mask identity, geometry, and software versions.
- Each CSV row contains all features extracted for one image and mask combination.
- Metadata records extraction settings, enabled image types, image and mask hashes, voxel spacing, bounding box, and software versions.
- Additional fields include connected-component count and voxel count for the specified mask label.
- Feature columns follow the pattern [pre-filter]_[feature group]_[feature name].
- The example wavelet-HH_glcm_JointEnergy identifies a pre-filter, feature group, and feature name.
Filename Pattern Description
Feature CSV filenames encode study settings, normalization, dimensionality, bias correction, registration, reference normalization, image type, and texture bin size. These codes allow the extraction configuration to be identified from the filename.
- The figure-generation notebook parses filename metadata and stores it with the resulting statistics.
- FullStudySettings indicates that the extraction used the study’s full settings.
- noNormalization indicates that default pyradiomics whole-image normalization was deactivated.
- 2d and 3D indicate whether texture features were computed in two or three dimensions.
- biasCorrected, TP2Registered, and MuscleRefNorm identify bias correction, registered T2w masks, and muscle-reference normalization, respectively.
- T2AX identifies T2w-image results, while bin10, bin15, bin20, and bin40 identify texture bin sizes.