PaperlayerView exact revision on arXiv

Source-linked AI summary

Domain Generalization for Medical Image Analysis: A Review

Jee Seok Yoon, Kwanseok Oh, Yooseung Shin, Maciej A. Mazurowski, Heung-Il Suk

arXiv:2310.08598v3eess.IVcs.AIcs.CV

TL;DR

Medical image models can lose reliability on unfamiliar or heterogeneous data distributions. This review organizes domain-generalization methods across the MedIA workflow and examines their strengths, weaknesses, and future directions. It concludes that domain generalization for MedIA holds promise for improving model robustness and reliability.

  • Problem

    Models that perform well on homogeneous data may be less reliable and robust on unseen or heterogeneous distributions.

    Source: §I · p. 1

  • Method

    The review categorizes domain-generalization methods at data, feature, model, and analysis levels and considers their roles across the MedIA workflow.

    Source: B. Our Contributions · p. 2, §III · p. 5

  • Results

    The review concludes that domain generalization for MedIA holds promise for improving model robustness and reliability.

    Source: §VI · p. 22

  • Takeaways & Limitations

    The review identifies a need to expand benchmark resources and highlights future research directions for this rapidly evolving field.

    Source: §V · p. 20, §VI · p. 22, C. Benchmark Datasets · p. 22

Abstract

from arXiv · show

Medical image analysis (MedIA) has become an essential tool in medicine and healthcare, aiding in disease diagnosis, prognosis, and treatment planning, and recent successes in deep learning (DL) have made significant contributions to its advances. However, deploying DL models for MedIA in real-world situations remains challenging due to their failure to generalize across the distributional gap between training and testing samples - a problem known as domain shift. Researchers have dedicated their efforts to developing various DL methods to adapt and perform robustly on unknown and out-of-distribution (OOD) data distributions. This article comprehensively reviews domain generalization (DG) studies specifically tailored for MedIA. We provide a holistic view of how DG techniques interact within the broader MedIA system, going beyond methodologies to consider the operational implications on the entire MedIA workflow. Specifically, we categorize DG methods into data-level, feature-level, model-level, and analysis-level methods. We show how those methods can be used in various stages of the MedIA workflow with DL equipped from data acquisition to model prediction and analysis. Furthermore, we critically analyze the strengths and weaknesses of various methods, unveiling future research opportunities.

I. INTRODUCTION

Deep learning has advanced medical image analysis, but performance can be compromised by unfamiliar data distributions. This review surveys domain generalization for MedIA across the workflow and formalizes the problem of learning from source domains for unseen targets.

I. INTRODUCTION

  • Models that perform well on homogeneous data may lose reliability and robustness on unseen or heterogeneous distributions arising from equipment, protocol, or patient variation.MedIA also faces high-dimensional images, limited diverse labeled data, and interpretability, safety, and privacy challenges.Source: §I · p. 1, A. Domain Generalization for Medical… · pp. 1–2

B. Our Contributions

  • Earlier MedIA reviews focused on particular domains or tasks, while broader domain-generalization surveys did not deeply examine implications for the full MedIA workflow.Source: B. Our Contributions · p. 2
  • The review organizes domain generalization methods into data-, feature-, model-, and analysis-level categories to show how they fit across the MedIA workflow.Its workflow perspective spans data preparation through analysis and considers operational implications beyond methodological hierarchy.Source: B. Our Contributions · p. 2, Figure 1 · p. 2

C. Scope of Review

  • The literature search yielded 1,621 papers using three Google Scholar strategies, prioritizing peer-reviewed studies while allowing selected relevant archive papers.Eligible papers had at least one medical-image experiment in a domain-generalization setting, even if they used another term.Source: C. Scope of Review · p. 2

A. Problem Definition

  • The review formalizes domain generalization as learning from M source domains with differing joint distributions to minimize prediction error on K unseen target domains inaccessible during training.Source: A. Problem Definition · p. 3

B. Settings of Domain Generalization

Domain-generalization settings vary in the number of source domains and in how source and target distributions differ. The review groups MedIA shifts by setting and describes common image, population, and label variations.

B. Settings of Domain Generalization

1) Settings for Source Domain

  • Multi-source DG uses multiple relevant domains (M > 1) to learn representations invariant to their marginal distributions, commonly by minimizing source-domain discrepancy.Source: 1) Settings for Source Domain · p. 3
  • Single-source DG uses homogeneous training data (M = 1), requires no domain labels, and is described as more challenging because it may not capture the diversity of other domains.Source: 1) Settings for Source Domain · p. 4

2) Settings for Target Domain

Medical-image DG settings distinguish target-domain types and shift mechanisms, while related learning tasks differ in their access to target data and the shifts they address. These distinctions clarify the scope of DG relative to adaptation, transfer, and other approaches.

2) Settings for Target Domain

  • Cross-site DG targets generalization across datasets from multiple healthcare institutions, with a site sometimes defined by vendor, scanner, or patient.The review describes cross-site DG as the most prevalent MedIA setting.Source: 2) Settings for Target Domain · p. 4
  • Cross-sequence DG spans temporal and protocol sequences, while cross-modal DG tests on a previously unseen imaging modality.MRI protocol examples include T1-weighted, T2-weighted, and FLAIR sequences.Source: 2) Settings for Target Domain · p. 4

3) Settings for Domain Shift

  • Covariate shift changes the input distribution while preserving the input–output relationship; concept shift changes that relationship.The review illustrates the distinction with scanner or population variation versus differences in diagnostic criteria or labels.Source: 3) Settings for Domain Shift · p. 4
  • Cross-site and cross-temporal DG may involve covariate shift, whereas cross-protocol and cross-modal DG may involve concept shift.The review illustrates the latter with differences between MRI and CT image features and their disease-label relationships.Source: 3) Settings for Domain Shift · p. 4

C. Related Machine Learning Tasks

  • MTL targets its trained task set and TL uses target data during training, unlike DG's focus on unseen distributions without target-domain access.The review also distinguishes harmonization, which primarily addresses non-biological cross-site heterogeneity without necessarily restricting target access.Source: C. Related Machine Learning Tasks · pp. 4–5
  • DA variants use labeled or unlabeled target data, while ZSL may use target-related auxiliary information and TTA tunes on test samples.Source-free DA lacks source data after pretraining; TTA can use a single or mini-batch of test data.Source: C. Related Machine Learning Tasks · p. 5
  • OOD generalization focuses on detecting concept shift, whereas DG considers both covariate and concept shift without target-domain access.The review frames target access as a key distinction between DG and related tasks.Source: C. Related Machine Learning Tasks · pp. 4–5

III. METHODS

The review organizes medical-image DG methods across four levels and connects them to stages of the MedIA workflow. It also describes data-level methods as a way to expand input diversity, with choices about the contributions of original and manipulated data.

III. METHODS

  • The review categorizes MedIA DG methods into data-, feature-, model-, and analysis-level approaches, including methods for extreme constraints.It uses a bottom-up organization and aims to connect the categories to the broader MedIA pipeline.Source: §III · p. 5
  • Analysis-level methods support understanding and interpreting model decisions, and the taxonomy maps DG approaches across the MedIA workflow.The workflow spans data acquisition, image reconstruction, feature extraction, downstream tasks, and analysis.Source: §III · p. 5
  • Similar techniques can serve different pipeline levels: augmentation manipulates images or features, while adversarial and contrastive methods have distinct level-specific aims.Data-level adversarial methods generate challenging examples, whereas model-level adversarial training seeks domain-invariant features; contrastive learning is used for feature learning or self-supervised pretraining.Source: §III · pp. 5–6, Table III · p. 6

A. Data-level Generalization

  • Data-level methods manipulate existing inputs or augment them with new samples to increase diversity, though some require task-specific architectures or algorithms.The review notes promising empirical improvements on both OOD and in-distribution samples.Source: A. Data-level Generalization · p. 6
  • The data-level objective combines original and manipulated data according to λ1 and λ2; setting λ1 to zero uses manipulated data exclusively.When both λ1 and λ2 are positive, augmentation is used alongside original data.Source: A. Data-level Generalization · p. 6

1) Data Manipulation

Data-level methods manipulate inputs or expand training data to address domain variation, ranging from image-processing transformations and surrogate representations to augmentation strategies. The review covers both general-purpose approaches and techniques tailored to particular imaging settings.

1) Data Manipulation

  • Data manipulation broadens training data through image-processing transformations, while traditional tools such as registration, resampling, and filtering are often preprocessing rather than standalone DG solutions.Deep learning methods also incorporate these techniques into end-to-end architectures.Source: 1) Data Manipulation · p. 6
  • Intensity and color methods normalize or transform image statistics, including histogram matching, stain-aware transformations, and harmonization to reduce site or protocol variation.The review also describes deep-learning harmonization, including autoencoders, normalization layers, and scanner-to-scanner translation.Source: 1) Data Manipulation · pp. 6–8
  • Surrogate methods replace original inputs with representations such as Fourier components, raw acquisition signals, or learned dictionaries to emphasize useful structure or avoid reconstruction variation.Frequency-based methods manipulate amplitude while seeking to preserve phase semantics; other examples operate on MRI k-space or CT sinograms.Source: 1) Data Manipulation · p. 8

2) Data Augmentation

  • Data augmentation expands and diversifies input-space training data to expose models to domain variation and help address limited medical-imaging sample sizes.The review distinguishes this from feature augmentation, which modifies learned representations rather than raw inputs.Source: 2) Data Augmentation · p. 8
  • The review focuses on augmentation methods designed or adapted for domain generalization, distinguishing them from broader medical-image augmentation surveys.Table IV organizes data-level methods by source and target settings, task, organ, and modality.Source: Table IV · p. 7, 2) Data Augmentation · p. 9
  • Randomization-based augmentation applies transformations such as flips, rotations, noise, or randomized styles, while adversarial augmentation samples uncertain-region examples to improve generalization.Generative approaches translate image styles to enhance robustness to unseen protocols; STRAP applies styles from non-medical images while preserving semantic content.Source: 2) Data Augmentation · p. 9

3) Problem-specific Data-level Methods

Problem-specific data-level methods address modality and tissue-specific variation, while feature-level methods seek representations that remain useful across domains. Feature alignment does so by normalizing features or reducing distributional differences.

3) Problem-specific Data-level Methods

  • Cross-modal generative methods synthesize data or latent representations across modalities, aiming to bridge distribution gaps when modalities differ or are unavailable.Examples include cross-modal synthesis and adversarial domain synthesis; SLAug adjusts class-specific image distributions for cross-modal and cross-sequence segmentation.Source: 3) Problem-specific Data-level Methods · p. 9
  • Histopathology methods normalize or mix stain colors across domains to address staining variation and artifacts in cell and cancer detection.One approach mixes target and source stain colors to create transformed test images; Stain Mix-Up combines stain matrices and density maps during training.Source: 3) Problem-specific Data-level Methods · pp. 9–10

B. Feature-level Generalization

1) Feature Alignment

  • Feature normalization centers, scales, decorrelates, or whitens features, while dissimilarity-based alignment minimizes distribution differences between mapped source-domain features.Methods use statistics such as means and variances or measures including Wasserstein distance and KL divergence; Wasserstein metrics were reported to better capture cross-site histopathology shift among evaluated metrics.Source: 1) Feature Alignment · p. 10, Table VI · p. 11

2) Disentanglement Methods

Disentanglement methods separate task-relevant from domain-specific variation so models can emphasize features useful to the task. The review distinguishes implicit strategies from explicit mechanisms using supervision or constraints.

2) Disentanglement Methods

  • Disentanglement represents samples using task-related and domain-specific factors, aiming to emphasize task-relevant information while reducing reliance on domain-specific features.The review divides these methods into implicit and explicit approaches.Source: 2) Disentanglement Methods · p. 11

III-B2a Implicit Feature Disentanglement

  • Implicit methods use statistical properties or indirect incentives, including variational inference, information-theoretic objectives, and contrastive learning, to separate factors of variation.Variational approaches can structure latent dimensions for domain-specific and domain-invariant factors; DIVA partitions latent variables for domain, class, and residual variation.Source: §III-B · pp. 11–12
  • Information-theoretic methods can minimize mutual information between task and domain representations, while contrastive approaches encourage similarity for task-matched samples across domains.MIDNet and MI-SegNet use mutual-information estimation; contrastive methods can use positive pairs sharing task-relevant characteristics despite different acquisition domains.Source: §III-B · pp. 11–12
  • Other implicit examples include variational causal models, ordinal-label-conditioned representations, domain-prior embeddings, and regularization methods such as frequency dropout.These approaches target different factors, including causal versus domain-aware features, class ordering, domain knowledge, or frequency-specific features.Source: §III-B · p. 13
  • Explicit disentanglement uses supervision or hard constraints to separate task-relevant and domain-specific features, with a regularization term controlling separation strength.The loss may use a domain label or auxiliary attribute alongside the regularizer.Source: §III-B · p. 13

3) Other Representation Learning Methods

Other representation-learning methods augment learned features or use kernels to expose patterns useful across domains. Feature augmentation operates beyond transformations of the raw input, while kernel methods map data into higher-dimensional spaces.

3) Other Representation Learning Methods

  • Feature augmentation transforms derived features rather than raw inputs, allowing richer transformations and potentially more effective use of domain knowledge.MaxStyle searches adversarial feature-style compositions, while FedFA estimates layerwise vicinity distributions and manipulates channel statistics.Source: 3) Other Representation Learning Methods · pp. 13–14
  • Kernel-based methods map inputs into higher-dimensional spaces to reveal domain-invariant structure and model nonlinear relationships, using approaches such as SVM variants, MMD, and TCA.Examples include a kernel classifier for cross-site brain-disease diagnosis and multi-TCA for EEG seizure detection.Source: 3) Other Representation Learning Methods · p. 14

C. Model-level Generalization

Model-level generalization modifies learning strategies, architectures, or optimization to improve cross-domain robustness. Learning-strategy methods include meta-learning, self-supervision, and adversarial learning.

C. Model-level Generalization

  • Model-level methods alter learning strategy, model framework, or optimization and adaptation to improve generalization across domains.The section groups these approaches into learning strategy, model framework, and other model-based methods.Source: C. Model-level Generalization · p. 14

1) Learning Strategy

  • Meta-learning simulates domain shift by splitting source domains into meta-training and meta-test sets, then learns an initialization or update rule for rapid adaptation.The formulation follows MAML: meta-learned parameters from meta-training initialize task-specific parameters optimized on meta-test data.Source: 1) Learning Strategy · p. 14
  • Self-supervised learning derives representations from unlabeled data through pretext tasks, while contrastive learning and style augmentation are examples used for domain generalization.Other approaches combine supervised pretraining on natural images with medical-image contrastive learning, or adapt pretrained models using test-case data fidelity or self-supervision.Source: 1) Learning Strategy · p. 16
  • Adversarial learning seeks domain-invariant features; medical examples include adversarial reconstruction, intensity transformations, and intensity-distribution changes that preserve image content.Intensity transformations can simulate MRI inhomogeneities and artifacts, while adversarial reconstruction aims to remove style information and retain classification features.Source: 1) Learning Strategy · pp. 16–17

2) Model Framework

Model frameworks use ensembles, distillation, and distributed learning, while other model-based approaches exploit data geometry or optimize worst-case performance. Analysis-level methods examine generalized models and face cross-domain interpretability challenges.

2) Model Framework

  • Model frameworks combine complementary predictions, distill teacher knowledge into compact students, or aggregate learning across institutions while protecting patient privacy.Examples include ensembles of calibrated networks or image-and-optical-flow inputs, gradient-filter regularization for distillation, and privacy-preserving federated updates.Source: 2) Model Framework · p. 17

3) Other Model-based DG

  • Geometric learning uses structures such as graphs, topologies, and manifolds, while distributionally robust optimization targets performance over worst-case distributions or subgroups.Examples include graph-based image relationships, topology-preserving segmentation, and subgroup optimization for skin-lesion classification.Source: 3) Other Model-based DG · pp. 17–18

D. Analysis-level Generalization

  • Analysis-level generalization examines model decisions, generalization, biases, and failure modes after training to support clinical validation and trust.Interpretations should remain consistent across domains with varying characteristics, while balancing model performance against interpretability.Source: D. Analysis-level Generalization · p. 18

1) Interpretable AI

Analysis-level approaches interpret decisions across domains, including through interpretable AI and causal modeling. The review also covers source-limited, open-set, and other under-explored domain-generalization settings.

1) Interpretable AI

  • Interpretable AI methods visualize or explain model decisions, but their transfer across medical domains and tasks remains challenging.Examples include saliency maps using synthetic causal interventions, CAM-based multimodal knee diagnosis, and Grad-CAM++ visualization for sEEG.Source: 1) Interpretable AI · pp. 18–19

2) Causalility

  • Causal approaches model invariant factors behind domain differences, with examples using Bayesian networks, structural causal models, and counterfactual harmonization.The reviewed applications include pneumonia detection, Alzheimer’s diagnosis across imaging sites, and single-source CT segmentation.Source: 2) Causalility · p. 19

IV. DG UNDER LIMITED SOURCE

  • Domain generalization under severe source-domain restrictions has received relatively little attention in MedIA, despite its extreme conditions.The review discusses these settings and their implications for medical imaging.Source: §IV · pp. 19–20

A. Single-source Domain Generalization

  • Single-source domain generalization assumes one training domain and often uses data- or feature-space augmentation to simulate broader domain variability.Examples include color and frequency transformations, generative models, feature augmentation, contrastive and adversarial learning, and pretrained-model distillation.Source: A. Single-source Domain Generalization · p. 20

B. Open-set Domain Generalization

  • Open-set domain generalization addresses concept shift alongside covariate shift, including settings where target domains contain unseen class names.Approaches discussed include label-shift detection, stable-feature generalization, and causal methods addressing spurious correlations or confounders.Source: B. Open-set Domain Generalization · p. 20

C. Other under-explored DGs

  • Unsupervised, source-free, and zero-shot domain generalization impose distinct data-access constraints, but these areas have only preliminary MedIA research.A pretrained foundation model with zero-shot prompting is suggested as a possible solution, though these topics fall beyond the survey’s literature scope.Source: C. Other under-explored DGs · p. 20

V. FUTURE DIRECTIONS

The review identifies gaps in source-limited generalization, medical foundation models, and reproducible benchmarks, and offers workflow-oriented strategy guidance. It concludes that domain generalization holds promise for improving MedIA model robustness and reliability.

A. Source-limited Domain Generalization

  • Source-limited domain generalization remains a research gap, especially when only one source is available and concept shift accompanies covariate shift.Limited data access can arise because medical data collection is resource-intensive and privacy concerns restrict access to multiple sources.Source: A. Source-limited Domain Generalization · pp. 20–21

B. Medical Foundation Model

  • Medical foundation models could support source-limited generalization, but heterogeneous medical data are difficult to collect at scale.The review notes promising results from Segment Anything Model for source-limited segmentation tasks.Source: B. Medical Foundation Model · p. 21

C. Benchmark Datasets

  • Standardized benchmarks are needed because custom datasets have low reproducibility and existing resources should cover diverse modalities, populations, protocols, and DG settings.The review calls for benchmarks spanning multi- and single-source, cross-site, cross-sequence, cross-modality, covariate-shift, and concept-shift evaluations.Source: C. Benchmark Datasets · pp. 21–22

D. Suggestions for Domain Generalization for MedIA

  • Existing empirical evaluations do not provide a comprehensive guide to task-specific strategies across the MedIA workflow, so the review critically analyzes methods and suggests workflow integration.The suggestions are presented as problem-specific strategies for incorporating domain generalization into MedIA.Source: D. Suggestions for Domain… · p. 22

VI. CONCLUSION

  • The review spans data acquisition, preprocessing, prediction, and analysis, and concludes that domain generalization holds promise for improving MedIA model robustness and reliability.It also highlights the need to expand benchmark resources and identifies future research directions.Source: §VI · p. 22

A. Benchmark Datasets

The section surveys established public benchmarks for medical-image domain generalization and identifies emerging datasets that could broaden evaluation across institutions and populations.

1) Existing benchmark datasets

  • The authors identify Camelyon17-WILDS, M&Ms, and MIDOG as the only publicly open MedIA domain-generalization benchmarks known to them.Camelyon17-WILDS covers inter-institutional pathology variation; M&Ms spans five sites and scanners, while MIDOG spans five hospitals.Source: A. Benchmark Datasets · p. 27, 1) Existing benchmark datasets · p. 27
  • Existing benchmarks target varied tasks and sources of heterogeneity, including lymph-node metastasis detection, cardiac segmentation, and mitotic-figure detection.The cited challenges include differences in staining, scanners, imaging protocols, equipment, and patient populations.Source: 1) Existing benchmark datasets · p. 27

2) Emerging benchmark dataset

  • BRATS adds pediatric and African cohorts since 2022, broadening age and ethnicity representation for domain-generalization research.The section also identifies RETOUCH as a cross-site dataset of 70 OCT volumes from three sites.Source: 2) Emerging benchmark dataset · p. 27

B. Publicly Open Datasets

The table lists datasets used in reviewed literature, spanning multiple medical-imaging tasks and modalities.

  • The table is a catalog of datasets used by the reviewed literature.Its listed entries include resources for classification, segmentation, detection, restoration, localization, and reconstruction.Source: Table XII · p. 27, B. Publicly Open Datasets · pp. 28, 31
  • Listed segmentation resources include brain MRI, prostate MRI, cardiac MRI, spinal CT, and abdominal CT datasets.Examples include WMH31, NCI-ISBI, EMIDEC, VerSe, and SABSCT.Source: B. Publicly Open Datasets · pp. 28–30
  • The catalog also includes datasets for detection, restoration, localization, and reconstruction tasks.Examples include INbreast, DRIVE, SurgicalActions, and MalariaScreener.Source: B. Publicly Open Datasets · p. 31

Related papers

Loading 2310.08598v3…