Source-linked AI summary
Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions
Yang Nan, Javier Del Ser, Simon Walsh, Carola Schönlieb, Michael Roberts, Ian Selby, Kit Howard, John Owen, Jon Neville, Julien Guiot, Benoit Ernst, Ana Pastor, Angel Alberich-Bayarri, Marion I. Menzel, Sean Walsh, Wim Vos, Nina Flerin, Jean-Paul Charbonnier, Eva van Rikxoort, Avishek Chatterjee, Henry Woodruff, Philippe Lambin, Leonor Cerdá-Alberich, Luis Martí-Bonmatí, Francisco Herrera, Guang Yang
TL;DR
Multicentre data differences challenge robust digital-healthcare studies, and existing reviews provide limited multimodal evaluation guidance. This survey synthesises harmonisation approaches across modalities and proposes a taxonomy and checklist; its synthesis finds different strategies dominate in different modalities.
Problem
Existing harmonisation surveys largely cover one modality or application and rarely address evaluation metrics or research guidance, despite the need to integrate data affected by different acquisition schemes.
Source: §1 · p. 3Method
The survey reviews more than 96 papers across gene expression, radiomics, and pathology, organising computational harmonisation methods, evaluations, and applications in a three-fold taxonomy.
Source: §1 · p. 4Results
Distribution-based methods account for 79% of gene-expression and 59% of radiomics approaches, but only 5% of digital-pathology approaches.
Source: §7 · p. 31Takeaways & Limitations
A checklist of study steps is proposed to guide researchers in reporting data-harmonisation findings more effectively.
Source: §1 · p. 4Most synthesis methods were developed on existing multicentre datasets without evaluation on new datasets, while GAN-based models may be unstable, hallucinate changes, and require substantial training data.
Source: §7.2 · p. 34
Abstract
from arXiv · showhide
Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.
1. Introduction
Multicentre data variation complicates robust digital-healthcare modelling, while existing harmonisation reviews offer limited cross-modality coverage and research guidance. This survey synthesises multimodal strategies and proposes a taxonomy and checklist for reporting harmonisation studies.
1. Introduction
- Different acquisition schemes create cohort bias, and retrospective datasets often cannot be reacquired under standard protocols, motivating computational harmonisation.Standard acquisition protocols can reduce non-biological variance, but personalised healthcare may also require non-standard protocols.Source: §1 · p. 3
- Earlier harmonisation reviews focused on a single modality or application and rarely covered evaluation metrics or research guidance.The paper identifies a lack of a checklist summarising common practice and guiding methodology selection and development.Source: §1 · p. 3
- The survey covers gene expression, radiomics, and pathology, reviewing more than 96 qualified papers published over two decades.It presents the exploration as the most comprehensive to the authors’ knowledge.Source: §1 · p. 4
- Its contributions include a three-fold taxonomy of harmonisation methodology, evaluation, and applications, plus a checklist for future studies.The checklist is intended to help researchers report findings more effectively; the survey also critiques existing strategies and discusses limitations.Source: §1 · p. 4
and how?
The section defines computational harmonisation as reducing non-biological cohort differences across data sources. It describes why those differences matter and how acquisition-device variation can persist even under fixed protocols.
2.1 What?
- Data harmonisation combines data from different sources into one cohesive dataset by adjusting formats, terminology, and measuring units.Computational harmonisation uses computational strategies to reduce non-biological heterogeneity caused by acquisition schemes and devices; it concerns intra-modality datasets rather than cross-modality style transformation.Source: §2.1 · p. 5
2.2 Why?
- Acquisition-site differences can destabilise features and diagnosis; Zhao et al. reported radiomics-feature concordance correlation coefficients falling from 0.76 to 0.28 with segmentation inconsistency.Hardware, acquisition protocols, and laboratory preparations are identified as sources of non-biological variance that can weaken reproducibility.Source: §2.2 · p. 6
Heterogeneity of acquisition devices (inter-device variability)
- Even with a fixed acquisition protocol, radiomics features varied across scanner brands, with 16% to 85% reproducible across five scanners in Berenguer et al..Sunderland et al. also reported higher maximum SUV values on newer than older scanner brands.Source: Heterogeneity of acquisition devices… · p. 7
Heterogeneity of acquisition protocols (intra-device variability)
Protocol, laboratory, and multicentre differences can undermine reproducibility and model transfer. The section connects these challenges to harmonisation approaches that operate on samples or extracted features, with different development demands.
Heterogeneity of acquisition protocols (intra-device variability)
- Only 15.2% of radiomics features were reproducible between soft and sharp reconstruction kernels, illustrating protocol-related variability.Protocol differences include scanning parameters and reconstruction approaches; strict standardisation can conflict with case-specific image-quality adjustments.Source: Heterogeneity of acquisition protocols… · pp. 7–8
Heterogeneity of laboratory preparations (Preparation variability)
- Laboratory variation in preparation, assays, slicing, staining, and manual pathology operations affects gene expression, radiomics, and pathological data.These differences can alter pixel values and stain consistency, while platform biases make multi-centre gene-expression results difficult to integrate and compare.Source: Heterogeneity of laboratory… · p. 8
2.3 What for?
- Without harmonisation, multicentre information fusion may not yield reproducible large-scale or longitudinal results, including PET measurements collected across scanners.The passage notes that multi-scanner longitudinal PET may conceal the relationship between SUV and outcomes.Source: §2.3 · p. 8
2.4 How?
- Sample-wise harmonisation processes images, signals, or gene matrices before modelling; feature-wise harmonisation aligns extracted features but requires multiple feature-extraction models.Feature-wise development is difficult when each cohort has few samples; both approaches are described as reducing variance and improving analysis performance.Source: §2.4 · p. 9
3. Methods
The review used a multi-researcher Scopus search and screening process, then documented study characteristics for quality review.
3.1 Literature search and review
- Two researchers searched Scopus independently, with a third reviewer resolving agreement; 96 papers remained after 238 full-text screenings and 139 exclusions.Searches covered publications through July 10, 2021, using harmonisation, batch-effect, reproducibility, and image-standardisation terms.Source: §3.1 · p. 10
3.2 Inclusion and exclusion criteria
- Eligible studies were original peer-reviewed journal or international-conference research on computational harmonisation of digital healthcare data.The review excluded application-only, manual-harmonisation, review, and reproducibility-only studies.Source: §3.2 · pp. 10–11
3.3 Data collection
- The quality-review spreadsheet recorded each paper’s modality, methods, metrics, data scale and properties, applications, cohorts, and cases.The recorded fields also included title and publication year.Source: §3.3 · p. 11
4. Data harmonisation strategies for information fusion
The review organizes computational harmonisation strategies into four broad families, then describes methods that estimate or correct cohort differences through distinct representations and transformations.
4. Data harmonisation strategies for information fusion
- The taxonomy groups computational harmonisation into distribution-based methods, image processing, synthesis, and invariant feature learning.The review uses the taxonomy to introduce the basic ideas and relationships among approaches.Source: §4 · p. 12, Figure 4 · p. 12
4.1 Distribution based methods
- Distribution-based methods estimate cohort bias in a latent space, then map source data toward target data with correction vectors or alignment functions.The taxonomy presents these as one family of computational approaches.Source: §4.1 · p. 12
4.1.1 Location-scale methods (LS)
- Location-scale methods align cohort means and variances; ComBat estimates batch parameters using empirical Bayes shrinkage before applying location-scale correction.The review also describes extensions including negative-binomial ComBat-seq and reference-based M-ComBat.Source: §4.1.1 · pp. 12–13
4.1.2 Iterative clustering methods (IC)
- Iterative clustering methods repeatedly cluster samples and calculate cohort-bias correction vectors from cluster centroids.Harmony repeats PCA-based clustering and correction until convergence; XPN clusters genes and samples with similar expression.Source: §4.1.2 · p. 13
4.1.3 Nearest neighbours methods (NNM)
- Nearest-neighbour methods estimate bias from mutual-nearest sample pairs and apply correction vectors, with variants differing in how they identify or combine matches.MNN requires each participating batch to share at least one type with another; Scanorama merges batches using weighted correction vectors.Source: §4.1.3 · pp. 13–14
4.1.4 Remove unwanted variations (RUV)
- Remove-unwanted-variation methods assume cohort bias is independent of biological variation and estimate unwanted factors, often using negative controls, for removal.The review also describes SVD-based, surrogate-variable, and regression approaches within this family.Source: §4.1.4 · pp. 14–15
4.1.6 Spherical harmonics (SH)
- Spherical-harmonics methods map MRI data toward a target cohort’s spherical-harmonic domain using rotation-invariant features or spherical moments.SMM matches spherical mean and variance per diffusion-weighting b-value using a linear transform.Source: §4.1.6 · p. 15
4.1.7 Distribution alignment (DA)
- Distribution-alignment methods transform source-cohort distributions toward a reference using cumulative distribution functions or probability density functions.Examples include nonlinear voxel-intensity mapping to match target CDFs and density-based offset estimation for common genes.Source: §4.1.7 · pp. 15–16
4.2 Image processing
Image-processing approaches harmonise multicohort data through filtering, physical-size resampling, intensity or colour normalisation, and stain separation.
4.2 Image processing
- Image processing applies digital algorithms—including filtering, registration, resampling, and normalisation—to harmonise multicohort data.The review describes filtering, physical-size adjustment, colour standardisation, and stain separation as specific approaches.Source: §4.2 · p. 16
4.2.1 Image filtering (IF)
- A second-order Butterworth low-pass filter was reported to eliminate cohort bias between CT images with different voxel sizes.Local binary pattern filtering was reported to produce stable and reproducible radiomic features.Source: §4.2.1 · p. 16
4.2.2 Physical-size resampling (Resample)
- Bilinear resampling can reduce bias from differences in pixel, voxel, or pathology microns-per-pixel size by equalising physical sizes.Source: §4.2.2 · p. 16
4.2.3 Standardisation/normalisation (SN)
- Standardisation and normalisation shift image-space or histogram values; global colour normalisation may lose information, while local methods preserve region-of-interest intensity but require predefined regions.Histogram matching scales source pixel values to fit a specified target histogram; other approaches include category-based and illumination-and-spectrum normalisation.Source: §4.2.3 · pp. 16–17
4.2.4 Stain separation methods (SS)
- Stain-separation methods factor optical-density values into stain-depth and stain-colour-appearance matrices, then normalise or recombine stain channels across cohorts.The review describes colour deconvolution, sparse non-negative matrix factorisation, neural-network stain separation, adaptive deconvolution, and circular clustering approaches.Source: §4.2.4 · pp. 17–18
4.3 Synthesis
Synthesis treats cohorts as styles and generates data in a shared modality or domain, using either paired or unpaired training samples.
4.3 Synthesis
- Synthesis aims to reproduce a sample in a missing modality or domain by transferring cohort-specific styles to a common style.The review divides synthesis methods according to whether training uses paired or unpaired samples.Source: §4.3 · p. 18
4.3.1 Paired sample-to-sample synthesis (P-s2s)
- Paired sample-to-sample synthesis learns transfers from the same object acquired under different protocols, limiting its application to radiomic data.Examples include neural networks for diffusion MRI, CT slice thickness, and reconstruction-kernel transfer; MMD-ResNet requires retraining for inverse transformation.Source: §4.3.1 · pp. 18–19
4.3.2 Unpaired sample-to-sample synthesis (Up-s2s)
- Unpaired sample-to-sample methods use cohort-labelled data without paired training samples, including CycleGAN variants and conditional VAE-based cohort transfer.The review also describes scAlign’s duplex mapping and iMAP’s use of cohort-invariant features and mutual-nearest-neighbour pairs.Source: §4.3.2 · pp. 19–20
4.4 Invariant feature learning
Invariant feature learning aims to extract representations that exclude cohort variability and can support downstream tasks. The review describes dictionary-learning, autoencoder, and adversarial approaches to this goal.
4.4 Invariant feature learning
- Invariant feature learning seeks cohort-invariant representations for downstream segmentation, classification, or regression tasks.The approach assumes representations built across cohorts will exclude inter- and intra-cohort variability.Source: §4.4 · p. 20
4.4.1 Dictionary learning (DictL)
- Sparse dictionary learning decomposes inputs into components to identify cohort-invariant features and reconstruct raw data; UCRL learns stain-colour appearance mappings.UCRL estimates sparsity using SPCN and combines source stain-mixing coefficients with a reference colour-appearance matrix.Source: §4.4.1 · p. 20
4.4.2 Autoencoder based methods (AE)
- DESC and BERMUDA use autoencoders to produce invariant embeddings, with clustering used to initialize or organize cohort groups.DESC initializes cluster centres with Louvain; BERMUDA matches cohort clusters with MetaNeighbor before learning a latent representation.Source: §4.4.2 · p. 20
4.4.3 Adversarial learning methods (AdvL)
- Adversarial methods jointly learn scanner- or protocol-invariant features and preserve the main-task performance, using cohort identification alongside task learning.AD2AH, DUH, and scDGN optimize adversarial and task losses; NormAE adds a decoder and reconstruction loss.Source: §4.4.3 · pp. 20–21, Figure 5 · p. 21
5. Evaluation approaches of the data harmonisation strategies
The review groups harmonisation evaluation into several metric families and also describes cohort classification and visualization. The measures assess distribution mixing, feature reproducibility, value differences, or downstream task performance.
5. Evaluation approaches of the data harmonisation strategies
- Evaluation approaches are organized as distribution-, correlation-, value-, and task-based metrics, alongside cohort classification and visualization.The review also presents a taxonomy of harmonisation metrics.Source: §5 · p. 22, Figure 6 · p. 22
5.1 Distribution based evaluation
- Distribution metrics include clustering and location-scale measures; ARI compares clusters with labels, kBET tests mixing, and LISI measures local diversity.A low kBET rejection rate indicates good harmonisation; larger ARI indicates greater similarity to real labels.Source: §5.1 · p. 22, §5.1.1 · p. 23, §5.1.2 · p. 23
- Structural-similarity metrics assess image preservation, but can suffer uniform pooling, distortion, and instability on hard edges or low-variance regions.NMI assesses colour consistency using median intensity relative to the 95th percentile; its limitation is not accounting for ROI consistency within a biopsy set.Source: §5.1.4 · p. 23, §5.1.5 · pp. 23–24
- COV-family measures describe feature variability, while KL divergence compares distributions and is zero when their information quantities are identical.A lower multivariate COV indicates better reproducibility across cohorts.Source: §5.1.6 · p. 24, §5.1.7 · p. 24
5.2 Correlation based evaluation
- Correlation measures differ in scope: PCC measures linear correlation, CCC captures agreement and gap, while ICC assesses grouped, multi-cohort variability.The described one-way random ICC model assumes no systematic bias.Source: §5.2.1 · p. 25, §5.2.2 · p. 25, §5.2.3 · p. 25
- Paired hypothesis tests and reproducible-feature percentages are also used to assess change or reproducibility before and after harmonisation.The cited paired tests use p-value < 0.05 as a significance threshold.Source: §5.2.4 · pp. 25–26, §5.2.5 · p. 26
5.3 Value based evaluation
- Value-based evaluation compares harmonised data with reference values using MAE, RMSE, or PSNR; RMSE requires paired datasets.The review notes that paired datasets are a substantial burden for digital healthcare research.Source: §5.3 · p. 26, §5.3.1 · p. 26, §5.3.2 · p. 26
5.4 Main task based performance evaluation
- Task performance is compared before and after harmonisation, cohort classification tests whether cohorts remain identifiable, and visualization checks for mixed distributions.Main-task results may be affected by model random initialization; effective harmonisation makes cohort identification more difficult.Source: §5.4 · p. 26, §5.5 · p. 27, §5.6 · p. 27
6. Applications of computational data harmonisation
The review covers computational harmonisation in gene expression, radiomics, and pathology, while identifying time-series device and site variation as a largely unaddressed area. It also notes an application in metabolomics.
6. Applications of computational data harmonisation
- Applications span gene expression, radiomics, and pathology; device- and site-related harmonisation of time-series signals remains understudied.Prior EEG and ECG artifact-removal work mainly targets biological variance rather than device or site variation.Source: §6 · p. 28, Figure 7 · p. 28
6.1 Gene expression analysis
- Microarray and scRNA-seq data are harmonised because platform and experimental-environment differences contribute to cohort bias.The review cites use of computational harmonisation in both microarray and single-cell RNA sequencing studies.Source: §6.1 · p. 28
6.2 Radiomics analysis
- Radiomics studies address scanner variability across imaging modalities, with cited applications spanning MRI, CT, PET, and optical coherence tomography.The review describes MRI work focused mainly on brain and breast, and CT work on ear, liver, and lung.Source: §6.2 · pp. 28–29
6.3 Pathology analysis
- Pathology harmonisation mainly targets stain or colour variance across studies of multiple tissue types.Examples include uterus, breast, lymphoma, skin, liver, renal, and stomach tissue.Source: §6.3 · p. 29
6.4 Other modalities
- Computational methods have also been applied to remove nonlinear cohort bias in metabolomics data acquired by liquid chromatography–mass spectrometry.Source: §6.4 · p. 29
7. Meta-analysis
The meta-analysis maps study trends, modality-specific harmonisation strategies, and evaluation practices across gene expression, radiomics, and pathology. It also identifies methodological constraints and research directions, including reproducibility, generalisability, and reporting.
7. Meta-analysis
- Studies increased sharply after 2019; microarray work preceded 2015, while sRNA-seq, MRI reproducibility, and digital pathology gained recent attention.The review also presents publication trends by data properties and modalities.Source: Figure 8 · p. 30, §7 · p. 30
- Distribution-based approaches accounted for 79% of gene-expression and 59% of radiomics methods, but 5% of pathology methods; image processing chiefly served pathology.Invariant feature learning and synthesis were used across all three modalities recently, dominated by deep-learning algorithms.Source: Figure 9 · p. 31, §7 · p. 31
- Visualization was the most commonly used evaluation approach, followed by task-based evaluation; cohort-classification results alone do not establish that data are well harmonised.Evaluation metrics assess distributions, correlations, and values between source and target cohorts.Source: §7 · pp. 31–32, Figure 10 · p. 32
- Among radiomics studies, 69.7% did not report image size; more than half of gene-expression studies had >5,000 cases per cohort, while most radiomics cohorts had <200 scans.For pathological studies, 20% did not report image size, and most used large-scale images (ε > 500).Source: §7 · p. 33
7.2 Research directions
- Distribution-based methods often require refined feature vectors and prior ROI knowledge, can be computationally demanding for images or high-dimensional signals, and may require rerunning when data are added.Pairwise methods also complicate training across more than two cohorts; suggested directions include raw-data harmonisation, efficiency, robustness, and simultaneous multicentre processing.Source: §7.2 · pp. 33–34
- Image-processing methods can depend on field-specific techniques or empirical settings, while synthesis methods face reproducibility and generalisability concerns and invariant-feature methods do not themselves produce harmonised data.The review calls for more general image-processing approaches, synthesis evaluations on new datasets with greater stability and less training data, and methods to generate harmonised data from invariant features.Source: §7.2 · p. 34
- The authors propose XAI as a research direction for identifying visual artefacts that may imprint bias on model predictions, while noting that harmonisation may benefit XAI by reducing cohort bias.They also suggest local explanations may help detect out-of-distribution examples linked to equipment or protocol changes.Source: §7.2 · p. 34
- Without stepwise design and consistent definitions of reproducibility, studies are harder to reproduce and compare; more than half of radiomics studies did not report image scale.The review also notes that varying evaluation metrics hinder method comparison.Source: §7.2 · pp. 34–35
8. Checklist and guidance
The paper presents CHECDHA as a framework for reporting data harmonisation studies and offers condition-based guidance for choosing harmonisation strategies and evaluation metrics. The checklist spans study motivation, data, methods, evaluation, results, and discussion.
8.1 Checklist criteria
- The checklist aims to enhance reproducibility and methodological principles, and its criteria are organised around data, model, evaluation, result, and discussion.It was inspired by the Checklist for Artificial Intelligence in Medical Imaging (CLAIM).Source: §8 · p. 36, §8.1 · p. 36
- CHECDHA structures reporting across motivation, data, harmonisation strategy, evaluation, results, and discussion, with novelty, limitations, and future work addressed in discussion and conclusions.The workflow also calls for describing methodology implementation, assessing reproducibility, and comparing performance with previous approaches.Source: §8.1 · pp. 36–38
8.2 Guidance of data harmonisation strategies and metrics
- For strategy selection, distribution-based methods suit refined features or gene matrices; image-processing methods are recommended for high-dimensional images when a high-performance GPU is unavailable.Deep-learning methods can apply across modalities but require sufficient training samples; invariant feature learning is recommended when the main task can be integrated into training, since synthesis may introduce unrealistic artefacts.Source: §8.2 · p. 38, Figure 13 · p. 39
- Metric choice depends on available evidence: use main-task performance before and after harmonisation, distribution-based metrics without harmonised ground truth, or value- and correlation-based metrics when it is available.Visualisation of raw or harmonised data with t-SNE, UMAP, or PCA is another way to analyse results.Source: §8.2 · p. 39, Figure 14 · p. 40
9. Conclusion
The paper synthesises and categorises data harmonisation strategies and metrics, then presents CHECDHA criteria and selection guidance. It argues that standardised reporting can support reproducibility and that method limitations and future directions should inform further work.
9. Conclusion
- The paper categorises existing data harmonisation strategies and metrics by underlying theories and presents CHECDHA criteria to address unclear method and metric choices and the lack of standardised study design.The authors identify poor reproducibility among existing studies as a consequence of the absence of a standardised stepwise methodology.Source: §9 · p. 41
- CHECDHA is intended to standardise study reporting and advance reproducibility, alongside summaries of harmonisation approaches and metrics across three modalities and condition-based selection guidance.The paper also surveys method limitations and directions for future work.Source: §9 · p. 41
- The authors call for researchers to follow and expand the checklist, and describe data harmonisation as relevant to multicentre studies and computational biomedical research, including radiomic, genetic, and pathological studies.They state that harmonisation can be adapted to federated learning systems.Source: §9 · p. 41