Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale

Isotta Landi; Benjamin S. Glicksberg; Hao-Chih Lee; Sarah Cherng; Giulia Landi; Matteo Danieletto; Joel T. Dudley; Cesare Furlanello; Riccardo Miotto

Source-linked AI summary

Deep Representation Learning of Electronic Health Records to Unlock Patient Stratification at Scale

Isotta Landi, Benjamin S. Glicksberg, Hao-Chih Lee, Sarah Cherng, Giulia Landi, Matteo Danieletto, Joel T. Dudley, Cesare Furlanello, Riccardo Miotto

arXiv:2003.06516v2Mar 14, 2020q-bio.QMcs.LGstat.ML

TL;DR

EHR-based patient stratification is difficult because heterogeneous patient data are challenging to summarize and represent at scale. The paper introduces ConvAE, an unsupervised deep-learning framework for learning patient representations from heterogeneous EHRs. ConvAE outperformed several baselines in clustering complex conditions and identified clinically meaningful disease subtypes, while missing molecular or quantitative clinical information constrained some analyses.

Problem
Heterogeneous EHRs are difficult to summarize and represent for scalable patient stratification, limiting the discovery of disease subtypes.
Method
ConvAE uses unsupervised deep learning to derive patient representations from heterogeneous, domain-free EHR collections for clustering-based stratification.
Results
ConvAE significantly outperformed several baselines in clustering patients with different complex conditions and identified clinically meaningful disease subtypes.
Takeaways & Limitations
The framework supports scalable analysis of clinical heterogeneity across complex disorders using representations learned from broad EHR data.

Abstract

from arXiv · show

Deriving disease subtypes from electronic health records (EHRs) can guide next-generation personalized medicine. However, challenges in summarizing and representing patient data prevent widespread practice of scalable EHR-based stratification analysis. Here we present an unsupervised framework based on deep learning to process heterogeneous EHRs and derive patient representations that can efficiently and effectively enable patient stratification at scale. We considered EHRs of 1,608,741 patients from a diverse hospital cohort comprising of a total of 57,464 clinical concepts. We introduce a representation learning model based on word embeddings, convolutional neural networks, and autoencoders (i.e., ConvAE) to transform patient trajectories into low-dimensional latent vectors. We evaluated these representations as broadly enabling patient stratification by applying hierarchical clustering to different multi-disease and disease-specific patient cohorts. ConvAE significantly outperformed several baselines in a clustering task to identify patients with different complex conditions, with 2.61 entropy and 0.31 purity average scores. When applied to stratify patients within a certain condition, ConvAE led to various clinically relevant subtypes for different disorders, including type 2 diabetes, Parkinson's disease and Alzheimer's disease, largely related to comorbidities, disease progression, and symptom severity. With these results, we demonstrate that ConvAE can generate patient representations that lead to clinically meaningful insights. This scalable framework can help better understand varying etiologies in heterogeneous sub-populations and unlock patterns for EHR-based research in the realm of personalized medicine.

Introduction

Patient heterogeneity and incomplete, heterogeneous EHR data make scalable disease subtyping difficult, despite the potential of longitudinal records to refine conditions into clinically meaningful subtypes. The paper proposes ConvAE, an unsupervised framework that learns patient representations from large, domain-free EHRs without manual feature engineering.

Motivation: Patients with complex disorders can differ in clinical measurements, comorbidities, and treatment responses despite sharing a diagnosis.
Motivation: Longitudinal EHRs provide heterogeneous data that can help examine disease complexity and refine diseases into subtypes.
Prior work: Prior stratification studies commonly used curated, small disease-specific cohorts and manually selected features, limiting scalability and generalizability.
Challenges: Large-scale EHR modeling is challenged by high dimensionality, heterogeneity, sparseness, random errors, and systematic biases.
Approach: ConvAE combines medical-concept embeddings, convolutional neural networks, and autoencoders to learn low-dimensional patient representations from heterogeneous EHRs.
Evaluation: Representations learned from about 1.6M Mount Sinai patients improved clustering over commonly used baselines and supported clinically meaningful subtypes across complex disorders.
Approach: ConvAE processes complete EHR sequences without manual feature engineering or explicit event labeling, regardless of patient-history length.

Results

ConvAE representations improved multi-disease clustering over baseline methods and produced disease-specific subgroups associated with progression, severity, symptoms, and comorbidities. These patterns included distinct subtypes in type 2 diabetes, Parkinson’s disease, and Alzheimer’s disease, although EHR limitations constrained some interpretations.

Multi-disease clustering: ConvAE significantly outperformed healthcare representation-learning baselines, including Deep Patient, on entropy and purity scores.The comparison used hierarchical clustering and 2-fold cross-validation, with p < 0.001 after Bonferroni correction.
Multi-disease clustering: The one-CNN-layer ConvAE configuration yielded the best overall performance and clusters linked to the largest number of distinct clinical diagnoses.
Multi-disease clustering: ConvAE produced visible clusters for ADHD, T2D, PC, and PD; ADHD patients were most separated, with 80% purity.
Disease-specific subtyping: T2D patients formed three subgroups corresponding to mild symptoms, microvascular complications, and severe cardiovascular problems.The microvascular subgroup included nephropathy, neuropathy, and peripheral artery disease, while the cardiovascular subgroup included coronary disease indicators and related treatments.
Disease-specific subtyping: PD patients divided into motor-symptom and non-motor/independent-symptom groups, with the latter associated with coronary artery disease and a longer, not necessarily more severe, course.
Limitations: EHRs did not quantitatively capture PD symptom severity, preventing replication of findings based on that measure.
Disease-specific subtyping: AD patients separated into three subgroups marked by onset, progression, and severity, including early-onset, late-onset slower progression, and typical-onset mild-to-moderate dementia.

Discussion

The study presents ConvAE as a scalable framework for learning general patient representations and identifying clinically meaningful subtypes across complex disorders. Its analyses link observed heterogeneity primarily to comorbidities, disease progression, and symptom severity, while noting data and cohort limitations.

The framework was designed to disentangle heterogeneity in complex disorders and support large-scale EHR analyses using data-driven clinical patterns.
ConvAE significantly outperformed several baselines in clustering patients with different complex conditions and identified clinically meaningful disease subtypes.
T2D subgroups reflected cardiovascular and microvascular comorbidities alongside symptom severity, including newly diagnosed patients with milder symptoms.
PD patients differed by disease duration and motor or non-motor symptoms, while AD profiles distinguished onset, progression, severity, and neuropsychiatric or cerebrovascular features.
MM, prostate cancer, and breast cancer subgroups were characterized mainly by distinct comorbidities or disease progression patterns.
The authors caution that EHR noise, hospital-specific bias, false-positive cohort identification, limited disorder coverage, and frequency-based concept analysis constrain interpretation and generalizability.

Methods

The framework processes longitudinal EHR sequences through preprocessing, unsupervised ConvAE representation learning, and clustering of disease-specific cohorts. The data comprise de-identified Mount Sinai records, with filtering and sequence preparation used to reduce noise and create fixed-size model inputs.

The framework consists of data preprocessing, unsupervised ConvAE representation learning, and clustering analysis of disease-specific cohorts.
The dataset contains de-identified Mount Sinai EHRs spanning 1980–2016, with structured, semi-structured, and unstructured records and up to 12 years of follow-up.
Clinical data include demographics, ICD-9 diagnoses, RxNorm medications, CPT-4 procedures, vital signs, laboratory tests, and concepts extracted from clinical notes.
Patients were randomly partitioned for 2-fold cross-validation, with 30,000 training patients used for hyperparameter tuning in each split.
Preprocessing filters concepts by frequency, removes redundant concepts within fixed time windows, shuffles concepts within windows, and splits long histories into fixed-length subsequences.

The ConvAE architecture

ConvAE combines medical-concept embeddings, CNNs, and autoencoders to transform EHR subsequences into low-dimensional patient representations. Its convolutional design treats embedding dimensions as channels to learn dimension-specific features before encoding patient histories.

Architecture: ConvAE combines embeddings, CNNs, and autoencoders to derive vector-based patient representations from EHR subsequences.The model assigns medical concepts embedding vectors, extracts temporal patterns, and learns encoded representations through autoencoders.
Input representation: Medical concepts are represented as an L × N matrix whose rows preserve visit order and whose columns encode semantic dimensions.Each row corresponds to a temporally ordered concept embedding, retaining longitudinal information across visits.
Convolutional module: CNN filters learn local temporal features, followed by ReLU activation, max pooling, and concatenation into a vector of dimension L · f.The convolution uses padded inputs and filters spanning temporal windows while preserving the output dimension.
Convolutional module: ConvAE reshapes embedding matrices as RGB-like inputs with embedding dimensions treated as channels, enabling independent filter weights across dimensions.This differs from one-sided temporal convolution and allows filters to activate on salient features in each embedding dimension.
Autoencoding: Dense autoencoder layers estimate the input subsequences from hidden H-dimensional vectors, producing encodings that can be averaged into patient-level representations.Each patient sequence becomes a sequence of subsequence encodings, which are component-wise averaged for a unique vector representation.
Training objective: ConvAE is trained unsupervised by reconstructing one-hot medical-term subsequences with cross-entropy loss, requiring no supervised training samples.The output is reshaped to |V | × L, and reconstruction errors provide the training objective.

Clustering analysis for patient stratification

The study applies ConvAE representations to disease-cohort clustering without disease-specific feature engineering. Hierarchical clustering and frequency-based concept analysis are used to identify and characterize heterogeneous patient subgroups.

Cohort stratification: ConvAE representations support patient stratification in preselected disease cohorts without additional feature engineering or manual adjustments.Disease cohorts are selected using diagnosis codes or phenotyping algorithms before clustering their representations.
Interpretation: Systematic subgroup analysis can automatically identify medical concepts that significantly and uniquely define disease subtypes.The approach links cluster membership to interpretable clinical concepts through frequency ranking and statistical testing.
Clustering procedure: The analysis uses SNOMED–CT diagnosis to define cohorts and hierarchical clustering with Ward’s method and Euclidean distance to derive subgroups.The Elbow Method selects the smallest number of clusters that minimizes the increase in explained variance.
Subgroup characterization: Subgroup characterization ranks medical concepts first by in-group frequency and then by frequency across the complete disease cohort.This ranking prioritizes concepts prevalent within a subgroup while retaining their overall cohort prevalence.
Subgroup characterization: Pairwise chi-squared tests assess whether present-versus-absent concept distributions differ significantly across detected subgroups.The tests are applied to the most frequent concepts identified during subgroup analysis.

Implementation details

The implementation preprocesses longitudinal EHR histories into overlapping concept subsequences, initializes medical-concept embeddings, and trains a compact ConvAE model. The reported configuration uses GPU-oriented subsequence construction and empirically tuned neural-network components.

Preprocessing: A filtering score threshold of 10^-6 was used to discard low-frequency terms before modeling.Examples of discarded concepts included clotrimazole and torsemide.
Preprocessing: Patients with fewer than 3 medical concepts were discarded, and filtering reduced the vocabulary to 32,799 terms.The filtering removed 24,665 medical terms while retaining frequent concepts considered clinically informative.
Preprocessing: Histories were divided into half-overlapping 15-day windows and length-32 subsequences, yielding approximately 3M training subsequences.The subsequence length was chosen to support efficient GPU training of the autoencoder.
Embeddings: Skip-gram word2vec initialized 100-dimensional embeddings for 31,659 vocabulary concepts, which were further refined during ConvAE training.Training subsequences were treated as sentences and medical concepts as words.
Model configuration: The reported CNN used 50 filters with kernel size 5, while the autoencoder used four hidden layers and dropout regularization.The autoencoder used hidden sizes 200, 100, 200, and |V | × 32, with dropout p = 0.5 in its first two layers.
Optimization: Training used cross-entropy loss, Adam, learning rate 10^-5, weight decay 10^-5, five epochs, batch size 128, and 100-dimensional patient representations.The implementation used Python, scikit-learn, PyTorch, and an Nvidia Titan V GPU.
Architecture evaluation: The study evaluated one-layer, two-layer, and multikernel CNN variants, with multikernel filters using kernel dimensions 3, 5, and 7.The multikernel configuration trains distinct CNNs in parallel and concatenates their final outputs.

Baselines

ConvAE is compared with sparse count, SVD-reduced count, SVD-reduced TFIDF, and Deep Patient representations. These baselines produce vector encodings of size 100 using count weighting, matrix decomposition, or stacked denoising autoencoders.

Compared methods: The comparison includes RawCount, SVD-RawCount, SVD-TFIDF, and Deep Patient, with all baselines producing 100-dimensional patient encodings.The listed methods provide representation-learning or dimensionality-reduction alternatives to ConvAE.
Count-based baselines: RawCount represents each patient as a sparse vocabulary-length integer vector containing clinical-concept frequencies.Each vector element counts occurrences of the corresponding concept in the patient’s longitudinal history.
Count-based baselines: SVD-RawCount applies truncated singular value decomposition to raw count encodings to obtain dense lower-dimensional representations.The representation is defined by the largest singular values of the raw count matrix.
TFIDF baseline: SVD-TFIDF weights patient sequences using term frequency–inverse document frequency and then applies truncated SVD.Patient sequences serve as documents, the dataset as the corpus, and each patient initially receives one TFIDF weight per concept.
Deep Patient: Deep Patient transforms the raw count matrix with stacked denoising autoencoders using 5% corruption noise and five training epochs.The implementation follows the details reported by Miotto and colleagues and uses batch size 32.

Multi-disease clustering analysis

The study evaluated representation-learning approaches by clustering patients across eight complex disorders to assess whether clusters matched disease identities. Entropy and purity quantified cluster alignment, with repeated balanced subsampling used across test sets.

Cohort and task: Eight complex disorders were used to test whether patient representations could disentangle patients with different conditions.The diseases included type 2 diabetes, multiple myeloma, Parkinson’s disease, Alzheimer’s disease, Crohn’s disease, prostate cancer, breast cancer, and ADHD.
Clustering procedure: Hierarchical clustering used k = 8 clusters, matching the number of diseases, to evaluate whether patients with the same condition grouped together.
Clustering procedure: 100 repeated random subsampling experiments balanced highly populated disease cohorts and produced 100 clusterings per test set.Highly populated diseases were subsampled to 5,000 patients before repeating the clustering process.
Evaluation metrics: Entropy and purity scores measured how closely clusters matched the different diseases.Entropy was computed from disease probabilities within clusters, while purity captured the most represented disease in each cluster.
Evaluation metrics: Perfect clustering corresponds to averaged entropy of 0 and purity of 1.

Disease subtyping analysis

The study used ConvAE patient representations to discover subgroups within selected complex disorders. Clustering choices were designed to reduce age confounding and sequence noise, followed by qualitative and external clinical review.

Disease-specific clustering: ConvAE representations were clustered for patients with type 2 diabetes, Parkinson’s disease, Alzheimer’s disease, multiple myeloma, prostate cancer, and breast cancer.
Cohort selection: The selected disorders were late-onset, age-related conditions chosen to reduce confounding from age effects.Crohn’s disease and ADHD were excluded because their common onset ages were less defined.
Representation preprocessing: Patient subsequence representations were averaged from the first diagnosis forward, and sequences shorter than three concepts were dropped.
Clustering procedure: The number of clusters ranged from 2 to 15, with the Elbow Method selecting the smallest number that minimized the increase in explained variance.
Subtype validation: Subtype interpretation combined medical-concept characterization with checks against medical literature and support from a practicing clinician.

Data availability

The Mount Sinai Health System data are not publicly available because they were used under license. Access may be available from the authors upon reasonable request with Mount Sinai permission.

Access restrictions: The study data are restricted under license and are not publicly available.
Access restrictions: The data may be available from the authors upon reasonable request and with permission from Mount Sinai Health System.

Supplementary Material

Disease-specific clustering produced clinically interpretable subgroups across multiple cancers and neurological or metabolic disorders. The supplementary analyses also showed some split-dependent variation and identified limits related to breast-cancer treatment and molecular information.

Multiple myeloma: Five multiple-myeloma subgroups corresponded to pulmonary, bone, gastrointestinal, kidney, and peripheral-neuropathy manifestations.
Multiple myeloma: Multiple-myeloma subgroup I included pulmonary manifestations associated with amyloidosis, while subgroup II was characterized by bone and cartilage disorders.Pleural effusion is described as a pulmonary manifestation of amyloidosis, a comorbidity found in 10–15% of multiple-myeloma patients.
Multiple myeloma: Renal testing and kidney-disease diagnoses characterized multiple-myeloma subgroup IV, whereas chemotherapy-related concepts distinguished subgroups III and V.
Prostate cancer: Two prostate-cancer subgroups reflected diverging disease courses, including a subgroup associated with recurrent disease and post-prostatectomy or radiotherapy-related concepts.
Breast cancer: Breast-cancer clustering separated an advanced or metastatic subgroup from a younger, screening-related subgroup associated with earlier-stage diagnosis.
Breast cancer: Breast-cancer subtype interpretation was limited by nonspecific chemotherapy concepts and the absence of patients’ hormonal profiles.
Split sensitivity: A second data split yielded four rather than five multiple-myeloma subgroups and three prostate-cancer subgroups, while other diseases showed similar results.