QICOM: Open-Source Software for Reproducible Image Quality Evaluation, Blinded Assessment, and Radiomic Comparison

• Methods span image-quality and ROI measurements, structured reader scoring and diagnostic-performance analysis, and radiomic features that characterize processing-related image changes.ROI measures such as SNR and CNR require appropriate noise estimation when reconstruction or post-processing changes noise characteristics.Source: §1 · pp. 1–2
• QICOM combines quantitative image evaluation, blinded assessment, and paired radiomics, with saved configurations, projects, and response mappings preserving links between inputs, processing conditions, and results.The workflow addresses the need to keep measurements, radiomic features, and reader responses tied to the same images and settings.Source: §1 · p. 2
• The study evaluates reference agreement, responses to controlled perturbations and spatial restrictions, and whether saved inputs, settings, and outputs support re-execution of comparisons.The experiments assess computational agreement and workflow reproducibility; human perceptual validity and diagnostic performance require subsequent evaluation.Source: §1 · p. 2

• The Qt interface organizes quantitative image evaluation, paired-volume radiomics, and blinded response collection and aggregation into three workbenches.The quantitative workbench contains 37 descriptors covering intensity errors, structure, distributions, perceptual backends, and ROI statistics.Source: §2 · p. 2, Figure 1 · p. 3
• Utility Tools were added in later QICOM versions, but were not part of the experiments reported in this paper.The added tools include segmentation, mask comparison, conversion, resampling, image arithmetic, and DICOM header operations.Source: §2 · p. 3
• The numerical, I/O, session, radiomics, and reporting packages run without Qt, allowing the interface and headless experiments to call the same computational functions.Configurations record data range, ROI, geometry, discretization, and slice aggregation; quantitative analyses save input, configuration, and result hashes.Source: §2 · p. 3
• Native radiomics projects store arrays, masks, geometry, configuration, and feature tables, while blinded deliveries use presentation identifiers and a separate author-side mapping to recover source conditions.Analysis runs locally without network access; optional learned metrics use previously installed checkpoints with recorded identities.Source: §2 · p. 3

3.1 Reference agreement and controlled perturbation

• Analytical phantoms test identity pairs, offsets, gains, patterns, and realized noise, with independent SSIM and HFEN calculations and separate PIQ and LPIPS integration comparisons.For C = R + b without clipping, expected MSE is b^2 and signed bias is b.Source: §3.1 · p. 3
• Graded noise, blur, and offsets probe metric responses, while masking fixtures compare radial erosion with complete kernel support and spectral fixtures compare PW, NPW, and eye-filtered calculations.The spectral fixtures use flat and colored noise with a nonconstant eye filter.Source: §3.1 · p. 4
• Radiomics is evaluated against IBSI 1 digital-phantom and public CT reference worksheets, including CT configurations A–E and the optional MIRP feature families and aggregation modes.The original PyRadiomics subset is evaluated separately; acceptance uses workbook-specific truncation and absolute tolerances.Source: §3.1 · p. 4
• Qualitative experiments use synthetic responses and independent coefficient calculations, with fixtures for blinding, hidden repeats, transport, missingness, and crossed reader–case interval coverage.Each generated dataset is one simulation replicate.Source: §3.1 · p. 4

3.2 Filtering comparison across the three workflows

• The filtering demonstration applies in-plane Gaussian filters at 0.65 and 1.5 pixels to a noisy analytic float32 volume, then compares image metrics and radiomics using a fixed interior ROI.The workflow saves configuration and image files before computation; quantitative exports and native radiomics projects are reopened for repeated calculations, while two blinded deliveries carry scripted responses.Source: §3.2 · pp. 4–5

3.3 Experimental records

• Companion files retain inputs, settings, expected and observed values, row-level statuses, and source hashes, with experiment snapshots and dependencies documented separately.Figures are drawn from these records, and Figure 4 shows the CT input and its supplied mask.Source: §3.3 · p. 5

4.1 Agreement with numerical and external references

• Maximum absolute discrepancies were 1.67×10−8 for MSE, 4.63×10−7 for SSIM, and 1.80 × 10−7 for HFEN; direct PIQ and LPIPS comparisons met integration tolerances.Source: §4.1 · p. 5
• All 482 digital-phantom entries and 1,622 CT entries across configurations A–E met QICOM/MIRP reference tolerances; all 1,804 feature/aggregation comparisons and 300 preprocessing diagnostics passed.The five CT configurations analyze one public case.Source: Table 1 · p. 5, §4.1 · p. 5

4.2 Behavior under perturbation and spatial restriction

• MSE increased across tested noise, blur, and offset levels, while SSIM decreased with noise and blur and stayed closer to unity under offsets.Source: §4.2 · p. 7, Figure 5 · p. 7
• Complete kernel support made masked SSIM and HFEN invariant to tested excluded-pixel changes; with radial erosion, masked SSIM changed when those pixels changed.Source: §4.2 · p. 7
• PW and NPW matched for flat-spectrum noise but differed for colored spectra, while NPWE with a nonconstant eye filter differed from both in all three fixtures.Source: §4.2 · p. 7, Table 2 · p. 7

4.3 Simulated reader responses

• All 127 scripted response rows were recovered under their source conditions, and none of the 96 hidden-repeat pairs violated specified spacing.Source: §4.3 · pp. 7–8
• 95/100 datasets achieved empirical bootstrap interval coverage in both complete-data and 15% missingness settings.Table 3 reports 100 generated datasets per setting and approximate 95% Wilson intervals of 0.888–0.978 in both settings.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 A complete synthetic filtering comparison

• Both filters reduced MSE versus the noisy input, while increasing smoothing from 0.65 to 1.5 pixels increased SSIM, MSE, and HFEN; GLCM contrast shifted from +141.6% for noise alone to −55.9% after stronger filtering.Mean ROI intensity changed by less than 0.085% across the three conditions.Source: §4.4 · p. 8, Table 4 · p. 8

5.1 Filtering changes intensity, structure, and texture differently

• Stronger filtering raised SSIM while worsening MSE and HFEN, moving GLCM contrast further from the clean reference; reopened projects retain measurements for recalculation and blinded assessment.An SSIM-only choice would favor the stronger filter in this example; the filtering demonstration is retrospective.Source: §5.1 · p. 9

5.2 Spatial support and model definitions

• Radial erosion allowed pixels outside the ROI to affect local calculations, whereas restricting centers to complete kernel support removed the observed dependence.ROI definitions must specify both averaged pixels and neighboring pixels used for local scores.Source: §5.2 · p. 9
• No-reference scores depend on their declared reference model; the distributed NIQE model is photograph-trained, and RIQE was not part of these experiments.The later RIQE version was validated on CT, but its filtered-image preferences did not follow lesion signal and it is not a denoiser-selection criterion.Source: §5.2 · p. 9

5.3 Contribution relative to existing tools

• QICOM concentrates paired comparison and blinded delivery, while its adapters and persistent formats must evolve with underlying libraries; reference agreement, same-backend handling, and external-reference testing serve distinct purposes.The reported residuals measure computational agreement, not image-quality improvement.Source: §5.3 · p. 9
• The QICOM/MIRP mapping covers additional intensity, histogram, distance-zone, and texture aggregations, and CT comparisons exercise interpolation, resegmentation, and discretization.Backend identity is saved because implementations and aggregation conventions determine meaningful comparisons; exact spatial sums avoid sampling variability for Moran’s I and Geary’s C.Source: §5.3 · pp. 9–10
• Reference agreement and re-execution under controlled technical conditions do not establish repeatability across acquisitions or reproducibility across scanners or sites.Those questions require additional physical-phantom experiments with acquisition variability and uncertainty estimated at the corresponding experimental unit.Source: §5.3 · p. 10

5.4 Scope and limitations

• Evidence is bounded to synthetic images, one public CT case, scripted responses, and one recorded software environment; human perceptual validity, clinical utility, and cross-site performance remain untested.The NSS distances depend on their reference distribution, and their scales, composite-index normalization and weights, and the one-dimensional observer model lack human calibration; IBSI 2 filtering and vendor-specific encodings require separate evaluation.Source: §5.4 · p. 10

5.5 A subsequent observer study

• A subsequent reader study should fix the visual task, processing conditions, eligibility, display settings, training, and randomization before collection.The proposed feasibility numbers require a pilot before a confirmatory sample size can be calculated; analyses should account for crossed reader and case dependence and response completeness.Source: §5.5 · p. 10

6 Availability and reproducibility

• The QICOM repository was private at the time described and was to be made public before arXiv submission.Evaluated snapshots, dependency versions, source hashes, and experiment records are listed in Supplementary Section B.Source: §6 · p. 10
• The RIQE model added in later QICOM versions is archived under CC BY 4.0.The RIQE archive is separate from the source and experiment materials described elsewhere.Source: §6 · p. 11
• The experiment bundle is held locally and separately from the software repository, and external IBSI resources and learned checkpoints must be obtained separately.The bundle contains scripts, synthetic inputs, results, and both source archives.Source: §6 · p. 11

7 Conclusion

• Reopened filtering projects reproduced their measurements and recovered scripted responses under their source conditions.The proposed next step is a task-specific reader study with realistic acquisition variability.Source: §7 · p. 11

A.1 Quantitative phantom experiments

• For C = R + b without clipping, the expected offset references are MSE = b^2, MAE = RMSE = |b|, bias = b, and SD(C − R) = 0.PSNR is separately defined for b ≠ 0, while identity and undefined-input cases are handled separately.Source: §A.1 · p. 15
• Independent references cover histogram and order-statistic calculations, structured patterns, and local-window structural calculations matched to the tested SSIM conventions.The catalog also includes MS-SSIM, FSIM, GMSD, VIF, HaarPSI, NMI, and intensity Wasserstein-1.Source: §A.1 · p. 15
• Edge preservation is measured by Laplacian correlation, while HFEN is a normalized Laplacian-of-Gaussian error with default σ = 1.5 pixels or voxels.Kernel support, dimensionality, and boundaries are part of the tested convention.Source: §A.1 · p. 15
• Synthetic-array degradation tests varied Gaussian-noise standard deviations (0, 1, 2, 4, 8, 16), blur standard deviations (0, 0.5, 1, 1.5, 2, 3 pixels), and offsets (0, 2, 4, 8, 16, 24), producing 342 metric responses.Inputs were float32 with dynamic range 255. Figure 7 montage normalization was only for visualization; computations used saved numerical arrays.Source: §A.1 · p. 15, Figure 7 · p. 16

A.2 Expert diagnostics and image I/O

• Noise diagnostics compare NPS calculations with Parseval and finite-ensemble references, using specified detrending, windows, frequency grids, and spacing conventions.The experiments use 128 independent 64 × 64 noise arrays and test white and filtered noise.Source: §A.2 · pp. 16–17
• Synthetic NIfTI and DICOM files test selected intensity and geometry conversions.Source: §A.2 · p. 17
• Observer-model outputs are evaluated against integrals for their model labels, with PW and NPW using a shared frequency grid and NPWE adding an illustrative eye filter.The NPWE filter is E(f) = (f + 0.01) exp(−2f), and flat and colored noise spectra are evaluated with that and a unity filter.Source: §A.2 · p. 17
• Gaussian-edge diagnostics compare 10–90% width and MTF50 with Gaussian references; only axis-aligned, noiseless edges were tested, and nonzero edge angles are rejected.The tested edge standard deviations are 0.8, 1.5, and 2.5 mm at 0.1 mm spacing.Source: §A.2 · p. 17

A.3 Radiomic verification and derived quantities

• The mapped evaluation includes 482 digital-phantom rows and 406 rows each for CT A and B, plus 270 each for CT C, D, and E.The original PyRadiomics manifest maps 104 digital-phantom rows in a separate experiment.Source: §A.3 · p. 17
• MIRP supplies eleven feature families and 2D, 2.5D, and 3D aggregation modes; the five CT configurations operate on the same acquired case.Presets use MIRP preprocessing after QICOM NIfTI loading, whereas custom profiles preprocess in QICOM before extraction.Source: §A.3 · p. 17
• CT configurations vary sampling and interpolation: A retains acquired spacing, B uses in-plane 2 mm interpolation, and C–E use isotropic 2 mm sampling.Image interpolation is linear in B–D and cubic in E; mask interpolation is linear with inclusion threshold 0.5, and anti-aliasing is disabled.Source: §A.3 · p. 17, Table 6 · p. 18
• QICOM computes Moran’s I and Geary’s C by exact inverse-Euclidean-distance sums, replacing MIRP’s sampling approximation for those outputs.Small 2D and 3D fixtures compare the calculations with explicit all-pairs results, including anisotropic and sheared geometry.Source: §A.3 · p. 17
• The acceptance rule truncates the absolute error to three significant digits of the reference when |e| > 10^-12, then compares it with the row’s absolute tolerance.For |e| ≤ 10^-12, QICOM compares the untruncated error with the tolerance; each reference/tolerance pair is retained in the comparison CSV.Source: §A.3 · p. 18
• The first-order ROI fixture tests offsets and translations against values calculated from a linear intensity object and ROI coordinates.The analytic mean change for an x-translation is twice the signed displacement.Source: §A.3 · p. 18
• Scalar references test two-way ICC variants and Lin’s CCC, which separates association from location and scale agreement.The checks include positive, negative, zero, and near-zero denominators; Equation 3 also permits a defined Cb when ρ = 0.Source: §A.3 · p. 18

A.4 Technical evaluation of the qualitative workflow

• Independent coefficient references test applicable ordinal agreement and ranking calculations, while seeded resamples are checked against an independent calculation of the same resamples.The documented software also exposes several kappa and ICC measures, distinct from the ordinal-alpha and AC2 fixtures reported here.Source: §A.4 · p. 18
• Independent-sample summaries do not account for crossed reader/case dependence; the described crossed bootstrap addresses paired reader-by-case comparisons.Source: §A.4 · p. 19
• The delivery tests check saved order, hidden repeats, metadata separation, source-role recovery, missing-response denominators, and sentinel metadata through conversion.Four complete evaluator deliveries supplied 127 simulated answer records.Source: §A.4 · p. 19
• An assignment roster identifies wholly absent responses; without one, the denominator includes only received records.The missing-response fixture includes one answered, one skipped, one not assessable, and one entirely absent assignment.Source: §A.4 · p. 19
• The crossed reader–case bootstrap is assessed with 100 simulated datasets per setting, four readers, 12 cases, and 199 bootstrap replicates.Coverage is the fraction of intervals containing the population median difference of 0.5; settings are complete data and 15% missing completely at random.Source: §A.4 · p. 19

A.5 Workbench views

• Figure 8 shows the blinded evaluator view from a delivered package, and Figure 9 shows paired radiomics reopened from a native project.The captures use the saved synthetic filtering comparison and a monochrome publication palette.Source: Figure 8 · p. 20, Figure 9 · p. 20, §A.5 · p. 20

B Reproduction records

• The companion bundle contains two source archives with file-level SHA-256 manifests, using QICOM 1.1.0.dev0 and Python 3.12.3 on Linux x86_64.The original archive accompanies the quantitative, diagnostic, qualitative, PyRadiomics, and synthetic filtering experiments; the extension archive accompanies the IBSI 1 evaluation and GUI captures.Source: B Reproduction records · p. 21

Original application SHA-256

• The cited passage consists of the hexadecimal value b1c3cdbdfa96140f9822f8ad8542e0a9fba24a41e44a8fe25ab695afab34a922, without stating what the value identifies.Source: Original application SHA-256 · p. 21

D Later software version: workbenches and Utility Tools

• The later QICOM 1.1.0.dev0 capture shows added workbenches and Utility Tools, but these additions were not used in the study experiments.The capture uses a synthetic thorax phantom; its displayed values are not study results.Source: D Later software version: workbenches… · p. 23

D.1 The three workbenches in the later version

• The quantitative, qualitative, and radiomics workbenches guide users through distinct workflows, including blinded delivery and reopening saved radiomics projects.Blinded packages use neutral labels and randomized order; radiomics tables are stored as CSV and typed JSON, which QICOM reads back value by value.Source: §D.1 · pp. 23–26, Figure 14 · p. 26

D.2 Utility Tools

• Utility Tools share read-only sources, staged all-or-nothing output, geometry-aware image writing, and safeguards against copying source header data into derived images.They also support lossless JPEG and JPEG-LS decoding through GDCM, while baseline JPEG and JPEG 2000 remain decoded by Pillow.Source: §D.2 · p. 28

D.2.1 Semi-automatic segmentation

• Segmentation builds an editable label map by sampling an ROI to define intensity ranges, with optional refinements, previews, and undoable steps.Labels occupy unassigned pixels unless overwritten; the saved NIfTI label map has a JSON sidecar. The method separates intensity ranges, not organs, and preset ranges are not validated thresholds.Source: §D.2.1 · pp. 28–29

D.2.2 Mask operations and comparison

• Mask tools combine masks, fill gaps between drawn slices, and compare them using voxel, volume, overlap, and surface-distance measures.Comparisons require a shared grid without resampling; slice filling interpolates signed distance maps, and surface distance is undefined if either mask is empty.Source: §D.2.2 · p. 30

D.2.3 Image conversion

• Image conversion preserves physical values in TIFF and geometry-aware NIfTI, while PNG uses the displayed window; resampling supports target spacing or a reference grid.Reference-grid resampling places images by voxel-to-patient coordinates and assumes a shared frame of reference rather than registration; interpolation may be nearest-neighbour, linear, or cubic.Source: §D.2.3 · p. 31, §D.2.4 · p. 32

D.2.6 Controlled degradation

• Image arithmetic performs voxelwise operations on same-sized, same-spacing images, while controlled degradation applies seeded noise, blur, resolution loss, or JPEG compression.Degradation changes are recomputed from the original image, and a versioned JSON record stores the steps, seed, and source and result pixel hashes.Source: §D.2.5 · p. 33, §D.2.6 · p. 34

D.2.7 DICOM tag inspection and editing

• DICOM tools inspect and edit headers, compare matching elements across headers, and check folders for series organization and common instance, spacing, and orientation problems.Editing preserves originals and records actions and source/copy hashes; folder checks export findings as CSV and can copy files into an ordered tree.Source: §D.2.7 · p. 35, §D.2.8 · p. 36, §D.2.9 · p. 37

D.2.9 DICOM folder check and organization

• DICOM folder organization reads headers only and creates study-, series-, and modality-ordered copies without changing file contents.It does not follow links; the example shows a missing slice in one phantom series and a skipped text file.Source: §D.2.9 · p. 37, Figure 26 · p. 38

D.2.10 DICOM decompression

• Compressed DICOM can be rewritten as uncompressed Explicit VR Little Endian, one file per frame, for readers unable to decode it.The tool decodes pixels in the same order as QICOM’s loader, writes copies all together or not at all, and assigns new instance and series UIDs to multi-frame copies. Decompression does not reverse lossy compression; the output is marked accordingly when needed.Source: §D.2.10 · p. 38, Figure 27 · p. 39

D.2.11 Image de-identification

• De-identification creates mirrored copies of DICOM, NIfTI, and bitmap images, applying format-specific identity and metadata handling.DICOM receives the PS3.15 Basic Profile with consistent UID, date, and name replacement; NIfTI identifying header content and bitmap text metadata, EXIF, and XMP are removed. Burned-in text is not detected: DICOM files are refused under specified annotation conditions, and unverified files require explicit approval; original filenames remain unless the user selects filename replacement.Source: §D.2.11 · p. 39, Figure 28 · p. 40