PaperlayerView exact revision on arXiv

Source-linked AI summary

Causality matters in medical imaging

Daniel C. Castro, Ian Walker, Ben Glocker

arXiv:1912.08142v1eess.IVcs.AIcs.CVcs.LG

TL;DR

Medical-imaging machine learning faces scarce annotations and data mismatch that can undermine clinical generalization. The paper uses causal diagrams and causal analysis to scrutinize data-generation assumptions and learning strategies, finding that causal structure affects whether semi-supervision is appropriate and may make it theoretically futile for semantic segmentation.

Abstract

from arXiv · show

This article discusses how the language of causality can shed new light on the major challenges in machine learning for medical imaging: 1) data scarcity, which is the limited availability of high-quality annotations, and 2) data mismatch, whereby a trained algorithm may fail to generalize in clinical practice. Looking at these challenges through the lens of causality allows decisions about data collection, annotation procedures, and learning strategies to be made (and scrutinized) more transparently. We discuss how causal relationships between images and annotations can not only have profound effects on the performance of predictive models, but may even dictate which learning strategies should be considered in the first place. For example, we conclude that semi-supervision may be unsuitable for image segmentation---one of the possibly surprising insights from our causal analysis, which is illustrated with representative real-world examples of computer-aided diagnosis (skin lesion classification in dermatology) and radiotherapy (automated contouring of tumours). We highlight that being aware of and accounting for the causal relationships in medical imaging data is important for the safe development of machine learning and essential for regulation and responsible reporting. To facilitate this we provide step-by-step recommendations for future studies.

Introduction

Medical imaging machine learning has achieved major predictive gains, but data scarcity and data mismatch still hinder translation into broader clinical practice. A causal analysis can expose these risks and guide data, annotation, and learning decisions.

  • Deep-learning methods have produced substantial accuracy improvements in computer-aided diagnosis and automated radiotherapy planning and monitoring.Source: Introduction · p. 1
  • Data mismatch occurs when models trained in laboratory settings fail to generalize to real-world clinical data.Retrospective study criteria, patient populations, and prospective multicentre settings can differ.Source: Introduction · pp. 1–2
  • Data scarcity reflects insufficient high-quality image–annotation pairs for training predictive models.The shortage may arise from small samples, costly expert annotations, or expensive confirmatory laboratory tests.Source: Introduction · pp. 1–2
  • Causal reasoning makes assumptions about data generation and collection explicit, allowing potential problems to be identified before deployment.This can inform data collection, annotation procedures, and machine-learning strategy selection.Source: Introduction · p. 2
  • The article uses causal analysis to examine whether common learning strategies are appropriate under specific medical-imaging data conditions.Its examples include diagnosis and automated tumour contouring.Source: Introduction · p. 2

Causality Matters

Medical-imaging prediction must be analysed in relation to data distributions, deployment conditions, and the causal direction between images and targets. These relationships determine how mismatch is understood and which learning strategies are applicable.

  • Medical-imaging predictive modelling estimates P(Y |X) from images X to annotations or other targets Y across tasks such as segmentation and disease classification.Source: Predictive analytics in medical imaging · p. 2
  • Development data contain annotated image–target pairs used to train and test models, whereas deployment data may differ and lack ground-truth annotations.Consequently, development-test performance may not represent post-deployment clinical performance.Source: Predictive analytics in medical imaging · p. 3
  • Data scarcity stems partly from the high cost of expert annotations and confirmatory laboratory tests, motivating methods such as semi-supervised learning and augmentation.Source: Challenges in medical imaging · p. 3

Causality Matters

Causality in medical imaging

  • Causal analysis distinguishes causal prediction, X →Y, from anticausal prediction, Y →X, according to whether images cause targets or targets cause image appearance.The direction affects the applicability of semi-supervised learning and the preference for generative or discriminative models.Source: Causality in medical imaging · p. 4
  • Histopathology-based melanoma diagnosis is anticausal because disease status is treated as a cause of lesion appearance.Source: Skin lesion classification example · p. 4

Skin lesion classification example.

The skin-lesion example shows that causal direction, selection mechanisms, and deployment conditions shape how imaging models should be interpreted and evaluated.

Tackling data scarcity via semi-supervision.

Semi-supervision uses unlabelled inputs, but its causal value depends on whether input distributions inform labels; augmentation instead expands joint image–label information and applies broadly.

Semi-supervision

  • Semi-supervised learning requires the input distribution to carry relevant information about the prediction task.Source: Tackling data scarcity via… · p. 6
  • For causal prediction, P(X) is uninformative about P(Y |X), making semi-supervision theoretically futile; semantic segmentation is therefore unlikely to fundamentally benefit from unlabelled data.Source: Tackling data scarcity via… · p. 6
  • Unlabelled data may still regularize causal-task models by lowering variance or reducing the labelled-data requirement, but these gains need empirical validation.Source: Tackling data scarcity via… · pp. 6–7
  • A comprehensive empirical study found properly tuned supervised and transfer-learning models often competitive with or better than semi-supervised counterparts, while target shift can hurt classification.Source: Tackling data scarcity via… · p. 7

Data augmentation

Data mismatch due to dataset shift.

Dataset shift describes differences between training and test distributions arising from changes in conditional relationships or marginal distributions. Causal diagrams, including unobserved anatomy, distinguish medically meaningful shift types and clarify when transport or correction is possible.

Data mismatch due to sample selection bias.

Sample selection bias occurs when training samples are selectively observed from the same population as the test cohort. Its consequences depend on whether selection is random, image-dependent, target-dependent, or jointly dependent on image and target.

Discussion

The discussion presents causal reasoning as a transparent framework for understanding medical-imaging data scarcity and mismatch. It offers practical diagramming recommendations while emphasizing that causal reasoning itself does not solve these problems and that broader causal research remains preliminary.

  • Causal analysis suggests that semantic segmentation may not fundamentally benefit from unannotated images through semi-supervision.Source: Discussion · p. 11
  • Measurement bias and missingness are identified as related topics for future causal analysis, including semi-supervised settings where labels are observed for only a subset of data.Source: Discussion · p. 12
  • Causal inference and causal discovery remain largely unexplored in medical imaging, with possible applications including imaging biomarkers and personalized counterfactual predictions.Source: Discussion · p. 12
  • Causal reasoning supports responsible reporting and aligns with regulatory and machine-learning initiatives concerning safe development, monitoring, fairness, accountability, and transparency.Source: Discussion · pp. 12–13
  • The article frames causal diagrams as a way to express assumptions clearly so other researchers can confirm or dispute them.Source: Discussion · p. 13
  • Table 4 recommendations and the generic scaffold diagram are intended to help analysts construct fuller causal accounts of medical-imaging datasets and identify biases.Source: Table 4 · p. 12, Discussion · p. 13

Fundamentals of causal reasoning.

Causal reasoning distinguishes prediction, intervention, and counterfactual analysis, while emphasizing that causal direction matters even for predictive medical-imaging tasks. Observational data alone may not identify that direction, making domain knowledge or experiments important.

  • Prediction infers unobserved variables from observed data, interventions study forced changes, and counterfactuals ask what would have happened under an alternative treatment.Source: Fundamentals of causal reasoning · pp. 17–18
  • A direct cause A →B changes the likelihood of B when A is forced to different values, with B generated by a mechanism depending on A and other factors.Source: Fundamentals of causal reasoning · p. 18
  • In the radiology example, the scan causes the referral decision because changing the scan changes the decision distribution, whereas changing the decision would not alter the scan.Source: Fundamentals of causal reasoning · p. 18
  • Observational image and referral data may not distinguish A →B from B →A; interventions on either variable could help identify the correct direction.Source: Fundamentals of causal reasoning · p. 18

Causal graphical models.

Causal graphical models represent variables and directed causal influences while supporting intervention reasoning beyond ordinary Bayesian-network factorization. Their joint distributions factor into conditional mechanisms associated with each variable’s parents.

  • A causal diagram is a directed acyclic graph whose nodes represent variables and whose edges represent postulated direct causal influences.Source: Causal graphical models · p. 18
  • The joint probability distribution factorizes into conditional mechanisms, with pa(Vi) denoting the variables whose arrows point toward Vi.Source: Causal graphical models · p. 18
  • Unlike Bayesian networks, causal models use independent mechanisms and support interventions that disconnect a changed node from its parents while leaving other mechanisms unaffected.Source: Causal graphical models · p. 19

Building a causal diagram.

Constructing a causal diagram requires defining non-overlapping variables, specifying causal links and selection mechanisms, and iteratively checking the graph against domain knowledge. The graph’s abstraction level should balance accuracy with clarity.

  • Begin by eliciting well-defined variables that correspond unambiguously to real or postulated entities without overlapping meanings.Source: Building a causal diagram · p. 19
  • Determine causal links carefully because every arrow asserts causation, while every absent arrow asserts that no direct causal effect exists.Source: Building a causal diagram · p. 19
  • Selection diagrams use domain indicators to represent mechanisms by which environments such as hospitals or training and testing populations differ.Source: Building a causal diagram · p. 19
  • Selection variables capture preferential subsampling, with incoming arrows representing the criteria that influenced dataset collection.Source: Building a causal diagram · p. 20
  • Diagram construction is iterative: verify graph assumptions against domain knowledge and choose enough detail for accuracy without sacrificing clarity.Source: Building a causal diagram · p. 20

Interpreting causal diagrams.

Causal diagrams reveal how paths create or block associations among variables. They distinguish mediation, confounding, and collider structures, which clarify indirect effects and sources of bias.

Related papers

Loading 1912.08142v1…