Source-linked AI summary
Deepr: A Convolutional Net for Medical Records
Phuoc Nguyen, Truyen Tran, Nilmini Wickramasinghe, Svetha Venkatesh
TL;DR
EMR risk prediction is hindered by costly feature engineering and the difficulty of detecting clinical motifs in irregular episodic records. Deepr sequences records with coded time gaps and transfers, then uses an end-to-end CNN to learn motifs and predict risk. On hospital data, it achieved superior accuracy to bag-of-words while exposing disease, intervention, and motif structure.
Problem
Irregular, episodic EMRs make manual feature engineering and detection of predictive clinical motifs difficult for future-risk prediction.
Method
Deepr transforms EMRs into sequences of discrete events separated by coded time gaps and transfers, then uses embeddings, convolution, pooling, and classification end to end.
Results
Deepr showed superior accuracy to bag-of-words for six-month unplanned readmission prediction and learned predictive motifs and disease/intervention structure.
Takeaways & Limitations
Deepr provides inspectable motif-based predictions while learning features directly from raw medical records rather than relying on manual extraction.
Takeaways & Limitations
Deepr’s evaluation was limited to unplanned readmission, and its max-pooling captures long-term dependencies simplistically.
Abstract
from arXiv · showhide
Feature engineering remains a major bottleneck when creating predictive systems from electronic medical records. At present, an important missing element is detecting predictive regular clinical motifs from irregular episodic records. We present Deepr (short for Deep record), a new end-to-end deep learning system that learns to extract features from medical records and predicts future risk automatically. Deepr transforms a record into a sequence of discrete elements separated by coded time gaps and hospital transfers. On top of the sequence is a convolutional neural net that detects and combines predictive local clinical motifs to stratify the risk. Deepr permits transparent inspection and visualization of its inner working. We validate Deepr on hospital data to predict unplanned readmission after discharge. Deepr achieves superior accuracy compared to traditional techniques, detects meaningful clinical motifs, and uncovers the underlying structure of the disease and intervention space.
I. INTRODUCTION
Deepr addresses the difficulty of representing irregular, variable-length EMRs for future-risk prediction without manual feature engineering. It sequences records with coded temporal structure and uses a CNN to learn clinical motifs, achieving superior accuracy to bag-of-words on six-month readmission prediction.
- Motivation: EMR feature extraction is difficult because records are irregular, episodic, variable in length, and locally structured by disease and care patterns.Visit timing is largely random, while diseases may form clusters and progression may follow underlying biological processes.
- Motivation: Existing feature engineering is effort intensive and nonadaptive, while bag-of-words breaks collocations and ignores temporal structure.These limitations make automated bag-of-words representations inadequate for the stated EMR challenges.
- Approach: Deepr transforms irregular-time EMRs into sentences of coded events, time gaps, and transfers, then learns representations end to end with a convolutional architecture.The model embeds words, detects local motifs, pools them into a record representation, and predicts future risk.
- Evaluation: On 300K hospital patients, Deepr was evaluated for unplanned readmission within six months after discharge and showed superior accuracy to bag-of-words.The evaluation also examined motif learning and the structure of diseases and interventions.
- Contributions: Deepr learns clinical motifs, predicts future risk, uncovers disease-and-treatment structure, and supports inspection of motif-based explanations.These capabilities are presented as contributions of the architecture.
II. BACKGROUND
The background frames EMR prediction as a representation problem involving coded longitudinal visits, disease progression, and temporal dependencies. Existing approaches use manual or simplistic features and often fail to model variable length, long-term dependencies, or irregular timing, while deep learning offers end-to-end learning from raw data.
- Medical records: EMRs contain patient demographics and time-ordered hospital visits with diagnoses, procedures, medications, laboratory tests, and narratives.Diagnoses, procedures, and medications are represented as discrete entities, including ICD-10 diagnosis codes.
- Existing methods: Existing EMR prediction methods commonly rely on manual feature engineering or simplistic extraction.These approaches are described as inadequate for several structural properties of EMRs.
- Existing methods: Many existing methods ignore long-term dependencies, inadequately capture variable-length records, or fail to model temporal irregularity.The background contrasts these limitations with methods intended to capture disease progression.
- Deep learning: Deep learning aims to build end-to-end systems that learn from raw data without manual feature engineering.The background identifies feedforward, recurrent, and convolutional networks as major deep neural architectures.
III. Deepr: A DEEP NET FOR MEDICAL RECORDS
Deepr represents an EMR as a sequence of discrete clinical events and temporal markers, then applies a CNN to detect local motifs and aggregate them into a record-level risk prediction. Its supervised CNN can identify predictive clinical patterns, with word2vec offered as a possible pretraining enhancement.
- Representation: Deepr sequences diagnoses, procedures, derived time intervals, and hospital transfers as discrete words in an EMR sentence.This representation preserves event order at the record level while encoding temporal and transfer information.
- Architecture: Word embeddings map the discrete EMR words into continuous vectors before convolution.The embedding layer is learnable and produces one vector for each word in the sequence.
- Architecture: A sliding-window CNN detects local clinical motifs, and max-pooling converts their responses into an EMR-level feature vector for classification.The classifier predicts the future risk from the pooled representation.
- Interpretability: Clinical motifs include comorbidity, disease progression, disease-treatment patterns, and collocating treatments.Because the CNN is supervised, word2vec pretraining is proposed to pre-detect motifs when labels are limited.
B. Sequencing EMR
Deepr converts time-stamped EMR episodes into a word sequence whose phrases contain clinical events and whose separators encode transfers and discretized time gaps. Embedding, convolution, max-pooling, and classification then transform this sequence into a global representation and predicted outcome.
- Sequencing EMR: Each EMR episode is represented as a phrase containing discrete diagnoses and treatments, with within-phrase ordering supplied by the system or randomized.The method does not assume perfectly timed individual events within an episode.
- Sequencing EMR: Hospital transfers are encoded by a special TRANSFER word separating phrases associated with different care providers.An admission is treated as a phrase, while an episode may contain multiple transfer-separated phrases.
- Sequencing EMR: Time gaps between consecutive episodes are discretized into five month-based intervals: (0-1], (1-3], (3-6], (6-12], and 12+.Each interval receives a unique identifier treated as a special word.
- Embedding: Word embedding maps each discrete word w to a learnable vector E(w), producing the sequence vector x_t = E(w_t).Dense embeddings avoid the high dimensionality associated with one-hot coding.
- Convolution: Convolution reads sliding windows of size 2d + 1 and applies learnable filters and ReLU to produce local motif responses.ReLU enhances strong signals and eliminates weak ones.
- Pooling and classification: Max-pooling aggregates local filter responses into a global sentence-level vector, which the classifier uses to predict the outcome.The classifier must permit gradients to propagate to lower layers.
D. Training
Deepr trains a supervised model with trainable embeddings and convolutional parameters, using regularization and a classifier-dependent loss. Its embedding matrix can also be pretrained on unlabeled data with word2vec.
- Deepr jointly trains the embedding matrix, biases, convolution kernels, and classifier-specific parameters.Because the parameter count can be large, training uses regularizers such as ℓ2 weight shrinkage or dropout.
- For binary outcomes such as readmission, Deepr commonly trains a logistic classifier with cross-entropy loss.
- Word2vec can pretrain Deepr’s embedding matrix without labels, allowing use of large unlabeled datasets.
E. Model Inspection and Visualization
Deepr supports inspection of learned motifs, word relationships, and patient similarity through responses, embeddings, and dimensionality-reduced visualizations. These tools connect learned representations to clinical patterns and future-risk similarity.
- Motif inspection: Deepr identifies strong motif responses at sequence positions and retains frequent motifs representative of each outcome class.For a size-3 motif, the response is computed from the dot product between the subsequence and its convolution kernel.
- Embedding analysis: Word similarity is computed from embedded words using cosine similarity.
- Patient visualization: Patient vectors support retrieval of patients with similar histories and similar future-risk likelihoods.This similarity is learned rather than based on a heuristic combination of diseases and interventions.
- Disease/intervention visualization: Disease and intervention embeddings can be projected into two dimensions with PCA or t-SNE for visualization.
IV. IMPLEMENTATION
The implementation applies Deepr to ICD-coded hospital records, constructing episodes and words from admissions, transfers, diagnoses, and procedures. Training uses selected hyperparameters after development-set search, with controls for code granularity, sequence order, and length.
- Data representation: Deepr is implemented for EMR systems with ICD-10-coded diseases, while other coding versions require minimal changes.
- Data: The hospital dataset contains 590,546 admission records from 300,000 unique patients collected between July 2011 and December 2015.Diagnosis codes follow ICD-10-AM3 and procedures follow ACHI under the Australian Coding Standard.
- Data split: The evaluation identifies 4,993 patients with unplanned readmission within six months and randomly selects matched controls, partitioning cases into training, tuning, and test sets.Each risk/control group uses 830 patients for tuning, 830 for testing, and the remainder for training.
- Episode definition: Deepr links admissions into one episode when they are less than 12 hours apart, or 12–24 hours apart with documented transfer.
- Preprocessing: Implementation choices include level-3 diagnosis codes, randomized word order, and retaining the last min(100, len(sentence)) words.The sentence-length cap corresponds to up to 10 visits and covers more than 95% of patients.
- Training configuration: Grid search selects hyperparameters using development-set accuracy, with the reported Deepr configuration using m = 100, d = 1, motif sizes 3, 4, and 5, and 10 epochs.The configuration also uses 100 motifs per size, mini-batches of 64, and ℓ2 regularization λ = 1.0.
C. Baselines
Deepr is evaluated against a regularized logistic-regression bag-of-words baseline for six-month unplanned-readmission prediction. Time-gap coding helps the baseline, while Deepr’s learned motifs produce more grouped patient representations and a simpler decision boundary.
- Baseline: BoW+LR uses regularized logistic regression, with the development-set search selecting C = 0.1.
- Risk prediction: Time-gap coding improves BoW-based prediction but does not affect Deepr’s accuracy.The comparison concerns prediction of unplanned readmission within six months after a random index discharge.
- Risk prediction: Deepr remains superior to the bag-of-words representation and learns predictive clinical motifs for six-month readmission prediction.
- Decision geometry: On the unseen test set, Deepr groups similar patients and creates a more linear decision boundary, whereas BoW+LR scatters patients and yields a more complicated boundary.
B. Disease/Procedure Semantics
Deepr’s word embeddings reveal disease and procedure relationships that partly align with ICD10 structure while retaining complex dependencies. The learned semantic positions remain similar when initialized with word2vec.
- Deepr discovers disease clusters that partly correspond to nodes in the ICD10 hierarchy.
- Most conditions are not totally separate, indicating complex dependencies across the disease space.
- The main disease-space region includes heart, blood, metabolic, respiratory, nervous-system, and mental-health conditions.
- Word2vec initialization did not significantly change relative word positions, suggesting Deepr captures semantic relationships between words.
C. Filter Responses and Motifs
Deepr uses convolutional filter responses to identify local clinical motifs associated with risk classes. The strongest and most frequent motifs include diagnosis–procedure patterns and disease or treatment combinations across several clinical domains.
- Local filter responses are computed for each sentence, then strong and frequent responses are collected into clinical motifs.
- The subsequences Z85.1163.1910 and 1066.1067.I21 respond strongly for the positive class and contribute to classification.They represent cancer history with biopsy and cerebral anesthesia, and heart attack with kidney-related procedures, respectively.
- The largest-weight, most frequent motifs include toxic-substance removal co-occurring with dialysis and readmission within 1 month.
- Other motifs capture type-I diabetes education, type-II diabetes readmission within 1-3 months, and diabetes-related complications.Reported complications include heart failure, vitamin D deficiency, and kidney failure.
- Additional filters identify circulatory-system disease and treatment patterns alongside pregnancy- and birth-related motifs.
VI. DISCUSSION
Deepr extends predictive medical-record analysis beyond manual feature extraction by uncovering disease, intervention, and risk patterns. Its broader applicability is promising, but the evaluation and current representation remain bounded by important limitations.
- Significance: Deepr uncovers relationships among diseases and interventions, including comorbidity, care patterns, and disease progression motifs.These motifs provide a basis for examining interactions between diseases and between diseases and care.
- Significance: Deepr supports future-risk prediction, similar-patient retrieval by future risk, and motif-based explanations for predicted outcomes.The paper connects these capabilities to treatment selection, risk alerts, and prospective healthcare.
- Significance: Deepr can enable targeted monitoring, treatments, care packaging, resource allocation, and more personalized care using existing EMR systems.The authors also describe transparent, evidence-based reasoning as a possible aid to clinicians.
- Comparison to recent work: Unlike DeepCare, Deepr is reported not to suffer reduced effectiveness on short records with only one or two admissions.The comparison places Deepr among alternative architectures for routine medical records.
- Comparison to recent work: Deepr differs from embedding methods that ignore time gaps or model local collocations without explicitly representing motifs with precise relative positions.This comparison concerns word2vec, Med2Vec, and related medical-concept embedding approaches.
- Limitations and future work: The current model captures long-term dependencies through simplistic max-pooling and uses only recorded diagnoses and interventions, excluding clinical narrative.Evaluation was limited to unplanned readmission, although the framework is described as applicable to other future risks.