Source-linked AI summary
Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F. Stewart, Jimeng Sun
TL;DR
Limited access to EHR data, driven partly by privacy concerns, motivates synthetic records for research. The paper introduces medGAN, combining an autoencoder with GANs to generate high-dimensional discrete EHR variables and minibatch averaging to address mode collapse. Across distribution, prediction, expert-review, and privacy evaluations, synthetic records showed close-to-real performance and limited observed privacy risks.
Problem
Privacy concerns and regulated access limit the secondary use and collaborative sharing of EHR data for research.
Method
medGAN combines a pre-trained autoencoder with a generative adversarial framework to generate high-dimensional multi-label discrete EHR variables.
Results
medGAN showed close-to-real performance across distribution statistics, classification, and medical expert review, with acceptable risks for presence and attribute disclosure.
Takeaways & Limitations
medGAN provides a synthetic-data approach for general multi-label discrete EHR records, including binary and count variables, in settings where EHR access is difficult.
Takeaways & Limitations
The study assumes that the discriminator and generator are implemented with feedforward neural networks.
Abstract
from arXiv · showhide
Access to electronic health record (EHR) data has motivated computational advances in medical research. However, various concerns, particularly over privacy, can limit access to and collaborative use of EHR data. Sharing synthetic EHR data could mitigate risk. In this paper, we propose a new approach, medical Generative Adversarial Network (medGAN), to generate realistic synthetic patient records. Based on input real patient records, medGAN can generate high-dimensional discrete variables (e.g., binary and count features) via a combination of an autoencoder and generative adversarial networks. We also propose minibatch averaging to efficiently avoid mode collapse, and increase the learning efficiency with batch normalization and shortcut connections. To demonstrate feasibility, we showed that medGAN generates synthetic patient records that achieve comparable performance to real data on many experiments including distribution statistics, predictive modeling tasks and a medical expert review. We also empirically observe a limited privacy risk in both identity and attribute disclosure using medGAN.
1. Introduction
EHR data are valuable for computational health but difficult to access because privacy concerns restrict research sharing. medGAN addresses this challenge by generating high-dimensional multi-label discrete EHR variables with an autoencoder–GAN combination and evaluating realism and privacy.
- Privacy concerns and institutional review processes limit researchers’ access to EHR data for secondary use.EHRs combine personal identifiers with sensitive medical information, and access is regulated and controlled by healthcare organizations.
- De-identification can remain vulnerable to linkage attacks that re-identify individuals from residual information.Synthetic data generation is presented as an alternative approach to de-identification.
- Generating longitudinal event sequences is desirable, but this work focuses on the important challenge of high-dimensional discrete variables.
- medGAN combines an autoencoder with GAN training to generate high-dimensional multi-label discrete EHR variables, including binary and count features.The model represents events such as diagnoses and medications.
- medGAN is evaluated through distribution statistics, classification performance, medical expert review, and empirical privacy-risk assessments.The reported privacy assessments cover presence disclosure and attribute disclosure.
2. Related work
Prior synthetic-EHR approaches were often disease-specific, domain-dependent, or limited to binary and low-dimensional data. medGAN instead targets general EHR records with high-dimensional multi-label binary and count variables.
- De-identification is widely used to protect patient privacy when sharing EHR data, but it does not guarantee that risk is eliminated.
- Existing healthcare synthetic-data methods often focus on specific diseases and rely heavily on domain knowledge and actual data.
- A perturbed Gibbs-sampler generator handled only binary variables and was evaluated on a small, low-dimensional dataset.
- Unlike SeqGAN’s one-hot word generation, medGAN targets multi-label binary and count variables in patient records.SeqGAN uses reinforcement learning and Monte Carlo search to generate word sequences.
- medGAN generates distributed patient-record representations with a GAN and decodes them into simulated records using an autoencoder.
3. Method
medGAN models EHR records as high-dimensional binary or count vectors and combines an autoencoder with GAN training to generate discrete synthetic records. Minibatch averaging, batch normalization, and shortcut connections address diversity and training stability.
- Data representation: EHR records are represented as fixed-size vectors of discrete diagnosis, medication, procedure, demographic, binary, or count variables.Count dimensions record occurrences, whereas binary dimensions indicate absence or occurrence.
- GAN framework: GAN training uses a generator that maps random prior samples to synthetic data and a discriminator that distinguishes synthetic from real records.The generator and discriminator iteratively optimize their respective parameters in an adversarial game.
- Enhanced generator training: Batch normalization and shortcut connections are used in the generator, while the discriminator remains a simple feedforward network because stronger techniques consistently overpowered the generator.The generator uses ReLU activations except for a tanh output layer; the discriminator uses ReLU and a sigmoid output layer.
- Autoencoder integration: medGAN uses an autoencoder so the generator produces a distributed representation that a pretrained decoder converts into discrete patient records.The decoder is fine-tuned while optimizing the generator, allowing discriminator gradients to flow through the decoder.
- Minibatch averaging: Minibatch averaging gives the discriminator the average of real or fake minibatch samples, helping expose mode collapse and encourage diverse outputs.For binary variables, minibatch averages estimate Bernoulli success probabilities; for count variables, they provide distribution-mean statistics.
4. Experiments
The experiments evaluate medGAN and baselines on binary and count EHR variables using distribution, prediction, expert-review, and privacy assessments. medGAN generally approaches real-data quality, while privacy attacks show limited effectiveness.
- Experimental setup: The study evaluates three EHR datasets, comparing medGAN variants with random noise, independent sampling, DBM, and VAE.Dataset A and B support binary-variable experiments, while dataset C supports count-variable experiments.
- Dimension-wise probability: Dimension-wise probability performance improves across increasingly advanced medGAN variants, with full medGAN performing best.Minibatch averaging significantly improves performance, while minibatch discrimination does not; batch normalization and shortcut connections further help.
- Dimension-wise probability: medGAN captures independent code distributions, while DBM performs comparably and VAE is weaker for low-probability codes.Independent sampling performs strongly on this dimension-wise task by construction.
- Dimension-wise prediction: Full medGAN again performs best among its variants for dimension-wise prediction, which measures inter-dimensional relationships.Independent sampling cannot capture these relationships; DBM is comparable or slightly better, and VAE remains weak for low-occurrence codes.
- Qualitative evaluation: Medical experts generally find medGAN records indistinguishable from real records, aside from several outliers.Outliers either lacked appropriate medication codes or combined male- and female-related codes, revealing a current limitation.
5. Conclusion
The paper presents medGAN for learning distributions of real-world multi-label discrete EHR data and reports strong results across binary and count variables. Its privacy evaluation finds very limited attribute-disclosure risk, while future work targets sequential and additional medical modalities.
- Conclusion: medGAN uses a generative adversarial framework to learn distributions of real-world multi-label discrete EHR data.The paper evaluates the approach on real datasets for both binary and count variables.
- Conclusion: The privacy evaluation demonstrates very limited risk of attribute disclosure with medGAN.Planned extensions include sequential generation and modalities such as laboratory measures, demographics, and free-text notes.
Appendix A. medGAN training algorithm
Appendix A describes medGAN’s optimization and implementation configurations, including repeated discriminator updates, matched model capacity, and architecture-specific training procedures.
- Training algorithm: The discriminator parameters are updated k times per iteration, while the generator and decoder are updated once; experiments set k = 2.The repeated discriminator updates are intended to ensure discriminator optimality, although larger k has not shown clear improvement.
- Model configuration: All models are tuned across hidden layers, minibatch size, learning rate, and training epochs while matching parameters used for synthetic-data generation.The reported configuration for each model is its best-performing tested setup.
- medGAN architecture: medGAN compresses records to 128 dimensions, uses a two-hidden-layer generator, and includes batch normalization and shortcut connections.Its encoder and decoder are single-layer feedforward networks, while the discriminator has two hidden layers.
- DBM configuration: DBM uses four 128-dimensional layers and persistent contrastive-divergence training with Gibbs sampling to generate binary samples.Synthetic samples are generated through Gibbs sampling followed by propagation to the input layer.
- VAE configuration: VAE uses three-hidden-layer encoder and decoder networks with a 128-dimensional latent representation and Adam training.The reported setup trains for 1,000 iterations with minibatches of 1,000 records.
Appendix C. Quantitative evaluation results for binary dataset B
For binary dataset B, medGAN’s full model generally outperformed its variants and showed competitive dimension-wise probability and predictive performance, though all models were weaker than on dataset A.
- Dimension-wise probability: The full medGAN version consistently outperformed its alternative versions on dimension-wise probability evaluation.The advantage of minibatch averaging was especially pronounced for dataset B.
- Dimension-wise probability: DBM slightly outperformed medGAN for low-probability codes but slightly underperformed for high-probability codes.
- Overall comparison: All models showed weaker probability and predictive performance for dataset B than for dataset A.The probability difference was attributed to smaller data volume and sparser code distribution.
- Dimension-wise prediction: The full medGAN version outperformed other versions on dimension-wise predictive performance.DBM slightly outperformed medGAN, especially for highly predictable codes.
- Dimension-wise prediction: Independent sampling failed to make meaningful predictions, while VAE was weak at predicting low-probability codes.
Appendix D. Quantitative results for count variables
Count-variable evaluation used a fixed 18-month heart-failure dataset and adapted the models and metrics to count-valued records.
- Dataset and setup: Count-variable evaluation used dataset C, comprising 30,738 patients with records observed for exactly 18 months.Variation in hospital visits during this period provided a test case for count variables.
- Dataset and setup: For count generation, VAE and medGAN used ReLU activations except at the discriminator output.The independent-sampling baseline used a Gaussian-kernel density estimator with bandwidth=0.75.
- Evaluation metrics: Dimension-wise average count replaced dimension-wise probability in the count-variable evaluation.Prediction used binary labels with logistic regression trained on count features, while retaining F1-score as the metric.
D.0.1 DIMENSIONS-WISE AVERAGE COUNT
On count variables, medGAN closely matched real count distributions and outperformed baseline models in average-count and dimension-wise prediction evaluations.
- Dimension-wise average count: VAE produced discontinuous average-count behavior because its synthetic samples had extremely low variance.VAE samples had, on average, nine orders of magnitude smaller standard deviation than medGAN samples.
- Evaluation figures: Figure 9 compares each model’s dimension-wise average count with the training dataset, while Figure 10 compares training- and synthetic-data prediction F1-scores.
- Dimension-wise average count: MedGAN performed well on average-count evaluation after simply replacing the activation functions.
- Dimension-wise average count: MedGAN’s synthetic histograms closely matched the real distributions for the five most frequent codes.This indicates that medGAN learned the actual count distributions rather than only matching their means.
- Dimension-wise prediction: MedGAN showed superior dimension-wise prediction performance compared with baseline models for count variables.The authors describe this as a smooth transition from binary to count variables using an activation-function replacement.
Appendix E. Dataset construction for heart failure studies
The heart-failure study defined cases using qualifying diagnosis evidence and repeated encounters, then matched controls by demographic and temporal criteria.
- Case definition: Heart-failure cases were patients aged 40 to 85 at diagnosis who met qualifying ICD-9 and encounter-based criteria.Diagnosis required qualifying codes and at least three clinical encounters within 12 months, with a specified date-assignment rule.
- Control selection: Up to ten controls were selected per case using sex, age, and location matching, yielding an overall ratio of nine controls per case.Controls also had to lack the operational heart-failure diagnosis before the case-specific cutoff.
Appendix F. Presence disclosure
The presence-disclosure evaluation tests whether synthetic records reveal patients used to train medGAN. Results indicate attacker knowledge and synthetic dataset size generally provide limited disclosure power, with a notable exception for small known patient groups.
- Evaluation design: The experiment compares training and test records using Hamming-distance matches to assess sensitivity and precision of presence disclosure.Records sampled from both sets enable true-positive and false-positive assessment; the number of known patients and distance threshold are varied.
- Impact of attacker knowledge: At a Hamming-distance threshold of 0, attackers discover only 10% of known patients used to train medGAN.Sensitivity measures the fraction of known patients correctly identified as training records.
- Impact of attacker knowledge: Precision is mostly 50%, but rises to approximately 80% when attackers focus on fewer than 100 known patients.The reported precision suggests attacker knowledge is generally unhelpful, while small targeted groups form an exception.
- Impact of synthetic dataset size: With 100 known real patients, changing the number of synthetic records has almost no impact on presence disclosure.Figures 11c and 11d evaluate sensitivity and precision while varying synthetic dataset size.