PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentially Private Synthetic Medical Data Generation using Convolutional GANs

Amirsina Torfi, Edward A. Fox, Chandan K. Reddy

arXiv:2012.11774v1cs.LGcs.AI

TL;DR

Medical synthetic-data generation must preserve useful structure while protecting sensitive patient information, especially across mixed data types, temporal patterns, and feature correlations. The paper proposes RDP-CGAN, combining Rényi differential privacy with convolutional autoencoders and GANs. It reports higher-quality synthetic data under the same privacy budget and supports both supervised and unsupervised evaluation settings.

  • Problem

    Medical synthetic-data generation must jointly address privacy, mixed discrete-continuous data, quality evaluation, temporal information, and correlated features.

  • Method

    RDP-CGAN combines Rényi differential privacy with convolutional autoencoders and convolutional GANs to represent mixed data and capture temporal and correlated information.

  • Results

    The model generates higher-quality synthetic data under the same privacy budget and provides higher privacy under the same synthetic-data quality.

  • Takeaways & Limitations

    Convolutional architectures help the framework preserve temporal and correlated information while generating synthetic medical data across discrete, continuous, and mixed variables.

Abstract

from arXiv · show

Deep learning models have demonstrated superior performance in several application problems, such as image classification and speech processing. However, creating a deep learning model using health record data requires addressing certain privacy challenges that bring unique concerns to researchers working in this domain. One effective way to handle such private data issues is to generate realistic synthetic data that can provide practically acceptable data quality and correspondingly the model performance. To tackle this challenge, we develop a differentially private framework for synthetic data generation using Rényi differential privacy. Our approach builds on convolutional autoencoders and convolutional generative adversarial networks to preserve some of the critical characteristics of the generated synthetic data. In addition, our model can also capture the temporal information and feature correlations that might be present in the original data. We demonstrate that our model outperforms existing state-of-the-art models under the same privacy budget using several publicly available benchmark medical datasets in both supervised and unsupervised settings.

I. INTRODUCTION

Medical synthetic-data generation must balance privacy with the need to preserve data quality, mixed data types, temporal information, and feature correlations. The proposed framework combines Rényi differential privacy with convolutional models to address these challenges and reports higher-quality synthetic data under the same privacy budget.

  • Healthcare deep learning is constrained by stringent privacy requirements and the sensitivity of large medical datasets.
  • Synthetic data can support public sharing and collaborative research, but naive GAN use does not guarantee privacy.
  • Medical synthetic-data research must address privacy, discrete or mixed data, quality evaluation, and temporal or correlated features.
  • RDP-CGAN uses Rényi differential privacy to track privacy loss with tighter bounds and improve performance under a fixed privacy budget.
  • Convolutional autoencoders provide compact representations for discrete and mixed data, while one-dimensional convolutions capture temporal and correlation dependencies.
  • The framework evaluates synthetic data statistically and through labeled-data generation, and reports higher quality under the same privacy budget.

II. RELATED WORKS

Prior synthetic medical-data methods addressed generation, correlations, temporal information, or privacy separately, motivating a framework that combines privacy preservation with broader data modeling capabilities.

  • Differentially private synthetic-data research commonly trains neural networks with gradient clipping and noise addition, extending a general privacy mechanism.The paper positions its approach as an extension using Rényi differential privacy and a privacy accountant.
  • Earlier medical-data methods had important gaps: MedGAN lacked privacy, Synthea used limited modeling assumptions, and CorGAN, TableGAN, and CTGAN did not guarantee privacy.These limitations create practical privacy concerns for synthetic medical data generation.
  • Existing privacy-preserving approaches included an auxiliary-classifier GAN requiring labeled data, PATE-GAN, and DPGAN, whereas this work targets labeled and unlabeled real data.The comparison highlights differing data and privacy assumptions across methods.
  • RDP-CGAN is compared with DPGAN and PATE-GAN because both provide differential privacy for synthetic-data generation.The paper identifies tighter privacy bounds from RDP as an advantage over the compared mechanisms.

III. PRELIMINARIES

The paper introduces autoencoders and differential-privacy concepts before defining Rényi differential privacy and its composition and conversion properties for the proposed framework.

  • Autoencoders encode inputs into a latent space and decode them for reconstruction, using BCE for binary inputs and MSE for continuous inputs.The model uses autoencoders to capture low-dimensional representations of discrete and continuous variables.
  • Differential privacy bounds privacy loss between neighboring datasets, with (ϵ, δ) representing a privacy budget whose smaller values indicate greater privacy confidence.Neighboring datasets differ by one record, such as a patient record.
  • The paper notes that advanced composition supports repeated private mechanisms but can cause parameter growth and multiple possible privacy values.Strong composition provides a tighter upper bound than basic composition for composed (ϵ, δ)-DP steps.
  • Rényi divergence generalizes Kullback–Leibler divergence, equals it at α = 1, and connects to ϵ-DP at α = ∞.At α = ∞, the divergence is the logarithm of the maximum probability ratio.
  • Rényi differential privacy composes additively at a fixed order, and an (α, ϵ)-RDP mechanism converts to (ϵ + log(1/δ)/(α−1), δ)-DP.These properties form the privacy-accounting basis for composing the autoencoder and convolutional GAN.

IV. PRIVACY-PRESERVING FRAMEWORK

The proposed RDP-CGAN combines a convolutional autoencoder with a convolutional GAN under Rényi differential privacy to generate synthetic data while modeling structure in medical records.

  • The framework uses RDP to train a privacy-preserving GAN and an autoencoder that creates a continuous feature space for data that GANs model poorly directly.The autoencoder supports the GAN while the GAN targets high-fidelity synthetic data generation.
  • One-dimensional convolutional networks are used in both autoencoder and GAN components to capture local feature correlations and temporal information.The design addresses limitations of multilayer perceptrons in modeling disease progression and related real-world structure.
  • The convolutional autoencoder is built from convolutional encoder and decoder layers, while the discriminator and generator use architectures similar to the encoder and decoder.This architecture is depicted in the proposed framework.
  • During RDP-CGAN training, generated fake data is decoded by a pretrained autoencoder before being supplied to the discriminator.The decoded representation changes the regular GAN training path.

A. Convolutional Autoencoder

The convolutional autoencoder creates a compact continuous representation that preserves neighboring-feature correlations and supports mixed discrete and continuous medical data under private training.

  • The 1D convolutional autoencoder captures neighboring-feature correlations, creates a compact feature space, and transforms discrete records into a continuous space.It is also intended to model discrete and continuous phenomena simultaneously.
  • The autoencoder is pretrained for a number of steps determined by the privacy budget, so the training duration changes with the desired ϵ.The paper also clips and noises both encoder and decoder training to preserve privacy.
  • Private autoencoder training divides minibatches into micro-batches, clips per-micro-batch gradients, adds Gaussian noise, aggregates them, and updates parameters.Clipping bounds sensitivity to individuals, while noise is added independently to micro-batch gradients.

B. Convolutional GAN

The convolutional GAN uses Wasserstein GAN training to improve robustness against mode collapse while enforcing differential privacy only on the discriminator, which accesses real data.

  • Differential privacy is enforced only on the discriminator because it is the sole component that directly accesses real data.
  • Wasserstein GAN training is used because it approximates Earth-Mover distance and is reported to be robust to mode collapse.
  • The Earth-Mover distance measures the minimum cost of transforming the generated distribution into the real distribution.
  • Because the direct Wasserstein formulation is intractable, the method uses the Kantorovich-Rubinstein dual optimization over 1-Lipschitz functions.
  • The generator can remain non-private because differential privacy of the discriminator is preserved under the generator’s post-processing mapping.

C. Architecture Details

The architecture combines convolutional discriminator and autoencoder components with transposed-convolution generators and adapts convolutional dimensions to dataset-specific input sizes.

  • The discriminator uses five convolutional layers and a final dense scalar-output layer without activation for decision making.
  • The generator uses one-dimensional transposed convolutions, while the autoencoder decoder reverses the encoder’s transposed-convolution structure.
  • PReLU activations are used throughout most encoder, decoder, discriminator, and generator layers, with specified final-layer exceptions.
  • The decoder’s final Sigmoid bounds reconstructed outputs to [0, 1] to match the input data’s discrete range.
  • Input-dependent convolution-kernel dimensions and strides are adjusted across datasets, while the generator retains a fixed noise dimension.

D. Privacy Loss

Privacy loss is tracked with Rényi differential privacy, composed across training steps and the autoencoder–GAN system, then converted to an (ε, δ)-DP guarantee using the tightest tested order.

  • The framework uses Rényi differential privacy accounting because it is computationally more precise than regular differential privacy for privacy-loss calculation.
  • The sampled Gaussian mechanism provides an (α, ε)-RDP bound for each private training step under unit ℓ2-sensitivity.
  • The total training-process privacy loss is obtained by multiplying the per-step ε by the number of steps for a fixed α.
  • Multiple α values are tested, and the minimum resulting ε is used with its associated α to compute the final (ε, δ)-DP guarantee.
  • The autoencoder and GAN compose to an (α, ε_ae + ε_gan)-RDP guarantee when their privacy orders are aligned.
  • The procedure aligns privacy orders, converts the smaller-order system’s bound to the selected total order, and combines the resulting budgets.

V. EXPERIMENTS

The experiments train on a training split, generate a synthetic dataset of matching size, and evaluate the framework under standardized optimization settings while comparing it with other methods.

  • The experiments first specify the setup and then report results across multiple experiments with comparisons against methods from the literature.
  • The dataset is split into training and test sets, and the trained model generates synthetic samples with size equal to the training set.
  • Both the convolutional autoencoder and convolutional GAN use Adam with learning rate 0.005 and mini-batches of 64.
  • Batch normalization is applied to both generator and discriminator training, and experiments run on one GeForce RTX 2080 GPU.

B. Datasets

The study evaluates synthetic medical data across multiple benchmark datasets, using distributional and predictive measures to compare methods under privacy settings. Its approach represents electronic health records as high-dimensional binary data and uses convolutional architectures to capture feature correlations.

  • Baselines: Benchmark comparisons exclude non-private models when experiments evaluate privacy-preserving settings.The comparison models therefore depend on the privacy conditions of each experiment.
  • Dataset construction: MIMIC-III records are represented as binary vectors over 1,071 unique ICD-9 codes.A variable is set to one when the corresponding diagnosis or procedure appears in a patient record.
  • Evaluation: The evaluation combines Maximum Mean Discrepancy for unsupervised distribution similarity with dimension-wise prediction using F1-score.Dimension-wise prediction uses top-10 and top-50 frequent features and averages results across Random Forest, XGBoost, and Decision Tree classifiers.
  • Evaluation: Without privacy enforcement, the proposed method outperforms other methods in dimension-wise prediction, with stronger relative performance as more top features are included.The authors attribute this to convolutional layers capturing correlations among features.

E. Supervised Synthetic Data Generation

The supervised evaluation trains classifiers on synthetic data and tests them on real data across medical classification datasets. The proposed model generally produces higher-quality synthetic data than competing privacy-preserving methods, especially for mixed continuous-discrete data.

  • 1) The effect of architecture: Table II compares methods using F1-score for dimension-wise prediction, where classifiers are trained on synthetic data except in the Real Data column.Results closer to the Real Data column indicate better synthetic-data quality.
  • Evaluation protocol: Synthetic-data quality is evaluated by training classifiers on generated data and testing them on real data, with closer performance to real-data training indicating higher quality.The generated data must preserve the real class distribution.
  • 1) The effect of architecture: Under no privacy constraints, the proposed architecture outperforms other methods on challenging datasets containing mixtures of continuous and discrete variables.The authors link this performance to feature-correlation capture and convolutional autoencoder representations.
  • Privacy-preserving evaluation: In most experiments, the proposed model generates higher-quality synthetic data for classification tasks than other models under the same privacy budget.The privacy comparison examines how accuracy changes at matched (ϵ, δ) levels.

3) The effect of privacy budget:

RDP-CGAN shows a privacy–quality trade-off, with stronger performance at lower privacy budgets and across benchmark datasets. Ablation results further indicate that convolutional architectures are important, while autoencoders help mainly when discrete variables are present.

  • The effect of privacy budget:: RDP-CGAN consistently outperforms other benchmarks across datasets and is particularly effective at lower privacy budgets.For the Kaggle Cervical Cancer dataset, it achieves significantly higher AUPRC than PATE-GA at ϵ = 100, 10, 1, and 0.1.
  • The effect of privacy budget:: The privacy-budget analysis measures synthetic-data quality with AUPRC, where higher values indicate better quality; the no-privacy baseline uses ϵ = ∞.The comparison is framed as a trade-off between privacy budget and synthetic-data quality.
  • Ablation study:: Ablation results demonstrate the importance of convolutional architectures in the autoencoder and GAN components.The study compares variants without convolutional autoencoders, autoencoders, convolutional generators, convolutional discriminators, and convolutional GAN architecture.
  • Ablation study:: For datasets containing only continuous values, autoencoders downgraded performance, whereas they were useful with discrete or mixed continuous-discrete data.The reported continuous-only datasets are UCI Epileptic Seizure, PTB Diagnostic ECG, and MIT-BIH Arrhythmia.
  • Ablation study:: The proposed framework uses Rényi Differential Privacy and convolutional neural networks to capture temporal information and feature correlations.The model combines convolutional autoencoders and GAN-based generation within a differentially private framework.
Loading 2012.11774v1…