PaperlayerView exact revision on arXiv

Source-linked AI summary

Memorisation bias in medical AI

Moritz A. Knolle, Martin J. Menten, Laurin Lux, Mélanie Roschewitz, Emma A. M. Stanley, Georgios Kaissis, Daniel Rueckert, Ben Glocker

arXiv:2609.17223v1cs.LGcs.CY

TL;DR

The paper asks whether memorisation of anonymised patient records can affect future clinical predictions, beyond its established privacy implications. It develops a longitudinal framework using temporally separated records and compares models trained with versus without each patient’s historical data. The study finds persistent, asymmetric diagnostic effects, while patient-level differential privacy substantially reduces memorisation bias.

  • Problem

    The consequences of memorisation for patients who later encounter a model trained on their historical data remain largely unexplored, despite routine deployment on overlapping populations.

  • Method

    The authors temporally split longitudinal records, train model groups with or without each patient’s historical data, and compare predictions on unseen future records.

  • Results

    Memorisation bias altered future predictions across datasets and persisted for decades; sensitivity fell for de novo conditions, while sensitivity and specificity rose when health states were unchanged.

  • Takeaways & Limitations

    Medical AI memorisation should be considered a potential source of diagnostic harm concentrated on patients who contributed training data, not solely a privacy risk.

  • Takeaways & Limitations

    Diagnostic harm was simulated using post-hoc thresholds and models assessed without clinician involvement, so prospective clinical impact remains to be quantified.

Abstract

from arXiv · show

Medical AI models hold immense potential to improve patient outcomes, but they are also known to unintentionally memorise individual records from their training datasets. While such memorisation has been linked to targeted privacy attacks, its consequences for clinical deployment, where patients may be assessed by a model that saw their historical data during training, remain poorly understood. Here we show that predictions on a patient's unseen future data can change significantly if a model observed that same patient's anonymised historical data during training, a phenomenon we term "memorisation bias". We demonstrate that this bias exists across diverse data modalities and model architectures, and over prolonged time spans: in some cases, memorisation bias persists on future records acquired decades after the historical records used for training. Moreover, in simulated prospective deployment, memorisation bias has asymmetric effects on the diagnostic accuracy of returning data contributors. When a patient returned with a de novo condition absent from their historical records in the training dataset, diagnostic sensitivity decreased significantly compared to an otherwise identical model not trained on their historical data. Conversely, when their health state was unchanged, both sensitivity and specificity were significantly inflated. Our findings reveal a previously uncharacterised risk in medical AI that arises when a model is deployed on patients who contributed to its training data. This exposes a shortcoming of current model development practice: the de-identification measures designed to protect patients' privacy make it difficult to identify returning contributors and exclude them from the AI-assisted interpretation of their own future data. Mitigating memorisation risks may thus require changes to current model training and deployment protocols.

Main

The study introduces memorisation bias, where models trained on a patient’s historical records systematically shift predictions on that patient’s unseen future data. Across datasets and architectures, this bias can persist for decades and asymmetrically affect diagnostic accuracy, while patient-level differential privacy substantially reduces it.

  • Main: The study’s practical mitigation challenge is that anonymisation makes returning contributors difficult to identify and exclude from interpretation of their own future data.The authors therefore argue that current training and deployment protocols may require reassessment.
  • Main: Models trained on patients’ historical records significantly altered predictions on a substantial share of their unseen future records across investigated datasets.Effect sizes were small on average but large changes occurred for some records.
  • Main: Memorisation effects were stronger on historical training records than on future records, consistent with direct exposure producing larger prediction changes.
  • Main: Memorisation bias became rarer and weaker over time but persisted across remarkably long acquisition intervals, including future records acquired decades later.Its decay rate differed considerably between datasets.
  • Main: For returning patients with de novo conditions, memorisation significantly reduced diagnostic sensitivity, whereas unchanged health states showed artificially increased sensitivity and specificity.The asymmetric pattern implies a higher false-negative risk when contributors develop conditions absent from their historical training records.
  • Main: Record-level differential privacy reduced but did not eliminate memorisation, while patient-level differential privacy was much more effective at preventing future-record effects.The privacy reduction came with lower held-out diagnostic performance, and patient-level protection still depends on its independence assumptions.

Methods

The study uses three large clinical datasets spanning chest radiographs, electronic health records, and ECGs to train diagnostic models with dataset-specific architectures and preprocessing.

  • Methods: MIMIC-ECG models use a modified one-dimensional vision transformer for 15-class electrocardiographic classification.The ECG signals are filtered, resampled to 250 Hz, and normalised before training.
  • Methods: Chest radiographs, electronic health records, and ECGs provide the study’s three clinical data modalities.The datasets are MIMIC-CXR, MIMIC-IV-ED, and MIMIC-ECG, respectively.
  • Methods: MIMIC-CXR models use ImageNet-pretrained DenseNet-121 classifiers with augmentation to detect 14 thoracic conditions.The test set contains 57,529 records, and the model has around 7 million parameters.
  • Methods: MIMIC-IV-ED models use random forests with 64 clinical features to predict hospitalisation and critical outcomes.The models use 100 estimators and a minimum leaf size of 4.

MIMIC-IV-ED processing and model training details.

A random forest trained on historical records from a random 50% patient subset achieved a macro-average AUROC of 83.39 ± 0.08% across all classes.

  • 83.39 ± 0.08% macro-average AUROC was achieved across all classes by the random forest trained on historical records from a random 50% patient subset.

HEEDB processing and model training details.

HEEDB provides long-follow-up ECG data from Massachusetts General Hospital, processed into standardised signals for supervised multi-label classification.

  • HEEDB processing and model training details.: The Massachusetts General Hospital portion of HEEDB contains 10,457,413 ECGs from 1,802,104 patients.The dataset was selected because it provides long patient follow-up.
  • HEEDB processing and model training details.: Each 10-second ECG is resampled to 250 Hz, filtered, and normalised before classification.Preprocessing includes notch, bandpass, and median filtering.
  • HEEDB processing and model training details.: Models perform supervised multi-label classification across 36 ECG diagnostic classes.The class set includes normal and abnormal rhythms, conduction abnormalities, infarcts, and waveform changes.
  • HEEDB processing and model training details.: Training uses AdamW, exponential moving averages, weight decay, and cosine-decay learning rates with linear warm-up.Hyperparameters are selected by random search to maximise validation macro-average AUROC.

Patient-level temporal split strategy.

The study creates patient-level historical and future datasets, enriches future records for de novo conditions, and compares predictions from models trained with or without each patient’s historical data.

  • Patient-level temporal split strategy.: Patient timelines are split adaptively to enrich future datasets for de novo cases absent from patients’ historical records.Fixed calendar or timeline-fraction splits would produce too few such cases for reliably studying negative memorisation effects.
  • Patient-level temporal split strategy.: Prediction distributions are compared between models trained with and without each patient’s historical data using an energy test.Balanced subset assignment gives each patient 100 models per group, and the energy distance is zero only for identical underlying distributions.

Statistical testing of receiver operating point differences.

The analysis compares sensitivity and specificity between models trained with or without each patient’s historical data using fixed operating points and exact permutation-based inference.

  • Receiver operating point comparison: Sensitivity and specificity were compared at fixed operating points using class-specific thresholds selected by maximising Youden’s J statistic.Thresholds were based on mean predicted probabilities across models for the unseen test dataset and were independent of model-group assignments.
  • Test statistic: The test statistic aggregates prediction differences across records, corresponding to sensitivity differences for positive cases and specificity differences for negative cases.
  • Permutation test: Exact randomisation inference used 100,000 permutations of model labels against subset assignments under the null that historical-data inclusion does not alter predictions.Permuting entire assignment vectors preserved within-patient record correlations, so no additional clustering adjustment was required.
  • Inference and confidence intervals: Two-sided p values doubled the smaller one-sided tail, while confidence intervals were obtained by inverting the same permutation test at the corrected level.Adding 1 to the numerator and denominator prevents a zero p value from finite permutation draws.

Tabular data model comparison on MIMIC-IV-ED.

The MIMIC-IV-ED analysis compares memorisation effects across three tabular model architectures and also evaluates differential-privacy mitigation under record- and patient-level protection.

  • Model comparison: MIMIC-IV-ED was analysed with a random forest, L2-regularised logistic regression, and a fully connected tabular ResNet spanning different inductive biases.The tabular ResNet used five residual blocks of width 500 and 3.8 million parameters.
  • Privacy mitigation: Differential-privacy experiments used DP-Adam with reduced ViT-T-25 models and evaluated privacy budgets ε ∈ {1, 10, 100, 1000}.Patient-level protection retained only the most recent record per patient, reducing the training data to one record per patient.

Computational resources.

Reproducing the experiments requires substantial computational resources, dominated by differential-privacy mitigation runs and HEEDB model training.

  • Total resources: 5,200 models and approximately 10,000 NVIDIA A100 GPU-hours are required to reproduce all experiments.
  • Resource breakdown: Differential-privacy experiments account for approximately 8,200 GPU-hours, while non-private experiments account for 1,681 GPU-hours.These figures exclude hyperparameter search and data preprocessing.

Code availability.

The study’s code is available on GitHub.

  • Code availability: The experimental code is available at the authors’ GitHub repository.

Extended Data

Extended Data examines memorisation effects across datasets, acquisition intervals, health-state changes, and differential privacy protections. The figures compare prediction changes, diagnostic performance shifts, and effect-size distributions under patient-inclusion and control partitions.

  • Datasets and architectures: Across four datasets, figures compare diagnostic performance, energy-test statistics, acquisition intervals, and memorisation across logistic regression, random forest, and tabular ResNet architectures.The architecture analysis uses identical random-subset protocols, while performance is shown for training and unseen test sets.
  • Temporal persistence: Effect-size summaries track prediction changes against acquisition intervals using 99th percentile, 99.9th percentile, and maximum statistics, with random partitioning as the baseline.Effect size is the maximum absolute class-probability difference between models trained with and without the patient’s historical records.
  • Diagnostic effects: Unchanged health states show memorisation-induced increases in sensitivity and specificity across condition-specific comparisons.The figure contrasts models trained with versus without patients’ historical data; positive differences indicate inflated performance.
  • Statistical presentation: The diagnostic figures use mean sensitivity and specificity differences, confidence intervals, condition-specific Youden thresholds, and multiplicity-corrected permutation significance tests.The displayed comparisons are organized by condition, with sample counts reported separately for positive and negative cases.
  • Diagnostic effects: De novo-condition analyses report sensitivity and specificity differences for every available HEEDB condition, with reduced performance represented by estimates left of zero.Cases were matched to age- and sex-matched controls where possible, and significance used corrected permutation tests.
  • Privacy protection: Differential-privacy comparisons vary ε and distinguish record-level from patient-level protection for future and historical records in MIMIC-ECG and HEEDB.Effect sizes and energy statistics are compared against dashed random-partitioning baselines.
Loading 2609.17223v1…