Source-linked AI summary
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data?
Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, Byron C. Wallace
TL;DR
The paper asks whether BERT weights trained on non-deidentified EHR notes can reveal patient PHI, a concern motivated by model sharing and memorization risks. It tests probing, fill-in, and generation-oriented attacks on BERT trained with MIMIC-III-based data. Simple methods mostly fail to meaningfully recover PHI, but the results do not rule out more advanced attacks.
Problem
The paper asks whether publicly shared BERT weights trained on non-deidentified EHR can reveal patient names and associated conditions.
Method
The study runs probing, fill-in, name-insertion, template, and generation-oriented attacks against BERT weights trained on MIMIC-III-based clinical notes.
Results
Simple methods mostly fail to meaningfully expose PHI: patient-condition recovery does not exceed baseline condition frequencies, including with names inserted into every sentence.
Takeaways & Limitations
The findings suggest that extracting PHI from these BERT weights is not trivial, while more sophisticated attacks remain a promising direction for research.
Takeaways & Limitations
The study considers only relatively simple attacks and BERT; more advanced autoregressive or larger models may be more prone to memorization and leakage.
Abstract
from arXiv · showhide
Large Transformers pretrained over clinical notes from Electronic Health Records (EHR) have afforded substantial gains in performance on predictive clinical tasks. The cost of training such models (and the necessity of data access to do so) coupled with their utility motivates parameter sharing, i.e., the release of pretrained models such as ClinicalBERT. While most efforts have used deidentified EHR, many researchers have access to large sets of sensitive, non-deidentified EHR with which they might train a BERT model (or similar). Would it be safe to release the weights of such a model if they did? In this work, we design a battery of approaches intended to recover Personal Health Information (PHI) from a trained BERT. Specifically, we attempt to recover patient names and conditions with which they are associated. We find that simple probing methods are not able to meaningfully extract sensitive information from BERT trained over the MIMIC-III corpus of EHR. However, more sophisticated "attacks" may succeed in doing so: To facilitate such research, we make our experimental setup and baseline probing models available at https://github.com/elehman16/exposing_patient_data_release
1 Introduction
The paper examines whether sharing BERT weights trained on non-deidentified EHR notes could expose sensitive patient information. It evaluates initial extraction strategies and finds that simple methods recover limited associations but do not outperform baseline condition frequencies.
- Domain-specific BERT pretraining improves performance across biomedical and clinical predictive tasks, motivating the sharing of pretrained model parameters.
- Non-deidentified EHR creates privacy concerns because model parameters might leak Protected Health Information even when the underlying text is not shared.
- The study runs a battery of experiments to assess whether BERT weights trained with masked language modeling over EHR notes reveal sensitive information.
- Simple methods recover patient-condition associations better than chance but do not exceed performance achievable from baseline condition frequencies.
- These findings hold even when patient names are inserted into every sentence, while more sophisticated generation-based attacks produce mixed results.
2 Related Work
Prior work establishes privacy risks from unintended memorization and sensitive-information extraction, but leaves unclear whether contextualized Transformer representations trained on EHR notes expose PHI.
- Unintended memorization in models trained on non-deidentified data creates privacy concerns about sharing trained parameters.
- The paper positions its work as an attack study focused on Transformer encoders trained on EHR notes.
- Previous research found that static word embeddings trained with imperfectly deidentified EHR data can reveal sensitive information to some degree.
- It remains unclear whether the same finding applies to contextualized embeddings produced by large Transformer architectures.
- Existing probe-based extraction methods primarily target general-domain relational knowledge rather than privacy-sensitive clinical tasks.
3 Dataset
The experiments use selected MIMIC-III clinical notes, reintroduce synthetic patient names, and focus on patients whose names appear in the corpus. This design supports reproducibility but limits generalization beyond MIMIC-III.
- The study uses MIMIC-III English Physician, Nursing, Nursing/Others, and Discharge Summary notes, whose original PHI was largely removed through deidentification.
- Patient first and last names are replaced with names sampled from U.S. Census data to simulate PHI while retaining a releasable dataset.
- Only 27,906 of 46,520 patients have a name mentioned in at least one note, but they account for 82.6% of notes considered.
- Name mentions are sparse and noisy: 17,044 notes contain first-name mentions, 220,782 contain last-name mentions, and additional false positives occur.
- The authors note that results could differ for EHR datasets from other hospitals with different token and name distributions.
4 Enumerating Conditions
The paper frames condition recovery as predicting which conditions are associated with a queried patient name. It represents conditions using ICD-9 codes or extracted clinical concepts, then labels patients for recovery experiments.
- The primary task asks whether a model can recover conditions associated with a patient name appearing in the pretraining EHR.
- Conditions are enumerated either from ICD-9 codes attached to patient records or from condition strings extracted from free text.
- The ICD-9 representation contains 6,841 unique codes, descriptions averaging 7.03 BERT word-piece tokens, and 13.6 unique codes per patient record.
- MedCAT extracts Disease / Symptom entities from notes, normalizes them to UMLS concept identifiers and descriptions, and supplies an alternative condition representation.
- For each condition set, patients receive binary association labels and the experiments attempt to recover the positive conditions for individual patients.
- The evaluation favors the adversary by using only conditions associated with reidentified patients.
5 Model and Pretraining Setup
The study retrains BERT on EHR data using a ClinicalBERT-style setup, then evaluates variants designed to make patient-information recovery easier. It also compares contextualized BERT representations with static word-embedding approaches.
- Pretraining setup: BERT is retrained over selected EHR notes using a ClinicalBERT-style process with full-word masking and three duplicates of static masking.The study considers Base and Large BERT variants and adopts hyperparameters from prior ClinicalBERT work.
- Semi-synthetic variants: The Name Insertion Model prepends patient names to every sentence in corresponding notes to increase their representation during training.The inserted names ignore grammar and are intended to make sensitive information easier to recover.
- Semi-synthetic variants: The Template Only Model generates sentences linking each patient to every associated MedCAT condition using a fixed name-and-condition template.This over-represents names and patient-condition associations in the training data.
- Static embeddings: Static CBoW and skip-gram word2vec models are evaluated by averaging constituent word vectors for names and conditions, then computing cosine similarities.This follows a prior static-embedding approach to testing whether patient information can be retrieved.
6 Methods and Results
The paper evaluates whether BERT models pretrained on MIMIC-III can reveal patient names or patient-condition associations through masked-template scoring, probing, similarity analysis, and text generation. Simple methods largely fail to recover meaningful PHI, while generation produces mixed evidence that remains difficult to interpret.
- 6.1 Fill-in-the-Blank: Masked templates score candidate conditions for each patient name using BERT perplexity, with experiments covering ICD-9 and MedCAT condition descriptions.The generic template inserts a patient name and masks the condition; performance is evaluated with AUC and A@10.
- 6.1 Fill-in-the-Blank: The generic template performs better than chance but worse than ranking conditions by empirical frequency, including in semi-synthetic data with more frequent names.On one restricted condition set, AUC is 0.570 versus 0.794 for the frequency baseline.
- 6.2 Probing: CLS-token probes nearly match frequency-baseline AUCs, while removing patient names performs similarly or slightly better, suggesting reliance on condition frequencies.Independent per-condition probes are at best modestly better than chance except for the rarest conditions, and all A@10 metrics are approximately zero.
- 6.3 Differences in Cosine Similarities: Name-condition similarity analysis finds no leakage from contextualized embeddings, even after names are artificially inserted frequently into training data.The comparison includes BERT and word2vec embeddings; positive differences would indicate that names are closer to conditions patients have than to conditions they do not have.
- 6.4 Can we Recover Patient Names?: Direct name-recovery experiments yield near-chance AUCs, with standard BERT performing similarly to MIMIC-trained models despite not seeing MIMIC notes.A separate first-name masking experiment finds reasonable signal in the semi-synthetic Name Insertion and Template Only variants.
- 6.6 Text Generation: Text generation produces mixed evidence: the Name Insertion model generates more names and has higher A@100, but only 23.5% of Base-model outputs containing a name and condition match a patient’s condition.The authors cannot determine whether this reflects memorized patient-condition pairs or broader patient-agnostic condition distributions; NER errors can also inflate scores.
7 Limitations
The study’s conclusions are limited by its focus on simple attacks, BERT, MIMIC-III, and particular masking and tokenization choices. These boundaries leave open whether other models, datasets, or training configurations would show greater leakage.
- The experiments considered relatively simple token in-filling and probing attacks, while generation-based attacks remained a future direction.The authors note that BERT’s generation quality may limit the advanced approach.
- The study examined only BERT, so autoregressive or larger models may exhibit different memorization and leakage risks.The paper specifically identifies GPT-2, T5, and GPT-3 as models that could change the risk profile.
- The study used only MIMIC-III, whose infrequent names may make recovering patient-condition associations particularly difficult.A semi-synthetic Name Insertion variant did not produce qualitatively different results for most experiments.
- Results may change on EHR datasets from other hospitals with different token and name distributions.
- Different masking strategies or tokenizers may affect memorization and extraction performance.The paper gives dynamic masking as one example of an alternative training choice.
8 Conclusions
The study initially found that simple methods mostly could not meaningfully expose PHI from BERT weights trained on MIMIC-III, even after inserting names into every sentence. More advanced generation results were intriguing but inconclusive, and the released setup supports further research.
- Simple methods mostly failed to meaningfully expose PHI from BERT weights trained on MIMIC-III.This remained true even in a variant that prepended patient names to every sentence before pretraining.
- Advanced generation-based attacks produced intriguing but inconclusive initial results.
- The authors released their experimental setup and baseline probing models to facilitate further research.The passage provides the repository path for these resources.
Ethical Considerations
The paper frames model release over sensitive clinical data as a patient-privacy issue because PHI can be misused if confidentiality protections fail. It therefore examines whether releasing models trained on sensitive EHR could expose such information.
- HIPAA prohibits distributing PHI, reflecting the importance of keeping patient information private.The passage gives potential lending and credit decisions as examples of harmful misuse.
- The paper raises whether models pretrained over sensitive clinical data could create a practical privacy concern when released.
A.1 Training Our BERT Models
The models were trained on approximately 400M words using cloud TPUs, with training schedules determined by token, batch-size, and sequence-length counts. The reported schedules were 40 epochs for the ++ models and 29 epochs for standard models.
- The training corpus contained approximately 400M words, ignoring wordpieces.
- At batch size 128 and sequence length 128, training for 1M steps yielded 40 epochs for ++ models and 29 epochs for standard models.The epoch calculation uses the number of steps, batch size, sequence length, and total token count.
- The models were trained using cloud TPUs, while experiments ran on V100, Titan RTX, and Quadro RTX 8000 GPUs.
A.2 Condition Distribution
The appendix characterizes condition frequencies and reports supplementary probing, masking, pooling, and model-comparison results. Condition distributions are highly uneven, and performance varies with model choice, condition length, masking, and pooling strategy.
- Condition Distribution: 4 ICD-9 conditions are shared across more than 10,000 patients, compared with 32 MedCAT conditions.The distributions are measured across the patient population described in the appendix.
- Condition Distribution: Supplementary tables report AUC, accuracy at 10, and Spearman coefficients relative to condition frequency.The appendix also includes results for Base++, Large++, and Pubmed-Base models.
- Condition Distribution: Base++, Large++, and Pubmed-Base models perform worse in AUC than the Regular Large model.Appendix Figure A3 further examines how results change with condition length because MLM likelihoods are harder to compute for sequences of varying length.
- Condition Distribution: Large and Pubmed-Base models perform better when names are excluded, while the Templates Only model shows the largest difference between the two appendix tables.The authors suggest this pattern indicates memorization of patient-condition relationships in the Templates Only model.
- Condition Distribution: The individual-condition experiment samples 50 conditions from each of four frequency bins and trains a probe to distinguish patients with each condition from those without it.Unlike fill-in-the-blank experiments, it computes one AUC per condition.
- Condition Distribution: Mean pooling outperforms the alternative pooling mechanisms reported in the supplementary results.The comparison uses similarity between positive and negative conditions from ICD-9 codes.