Source-linked AI summary
Deep Representation Learning of Patient Data from Electronic Health Records (EHR): A Systematic Review
Yuqi Si, Jingcheng Du, Zhao Li, Xiaoqian Jiang, Timothy Miller, Fei Wang, W. Jim Zheng, Kirk Roberts
TL;DR
Patient representation learning addresses how to convert complex EHR data into meaningful patient-level representations for analysis. This study systematically reviews deep-learning methods, screening 363 records and including 49 papers. The review identifies a common raw-data-to-representation-to-clinical-prediction workflow and concludes that patient representation learning is a promising direction for EHR analysis.
Problem
Complex EHR data require effective transformation into machine-learning representations, motivating systematic study of patient representation learning methods.
Method
The study conducts a systematic methodological review of deep-learning patient representation research from EHRs.
Results
The review found a common workflow in which raw patient data are transformed by deep-learning models and evaluated through clinical prediction tasks.
Takeaways & Limitations
Deep patient representation learning is presented as a promising direction for acquiring powerful, robust representations of EHR data.
Takeaways & Limitations
The review may have incompletely retrieved eligible studies, including studies on arXiv or bioRxiv, creating potential selective publication bias.
Abstract
from arXiv · showhide
Patient representation learning refers to learning a dense mathematical representation of a patient that encodes meaningful information from Electronic Health Records (EHRs). This is generally performed using advanced deep learning methods. This study presents a systematic review of this field and provides both qualitative and quantitative analyses from a methodological perspective. We identified studies developing patient representations from EHRs with deep learning methods from MEDLINE, EMBASE, Scopus, the Association for Computing Machinery (ACM) Digital Library, and Institute of Electrical and Electronics Engineers (IEEE) Xplore Digital Library. After screening 363 articles, 49 papers were included for a comprehensive data collection. We noticed a typical workflow starting with feeding raw data, applying deep learning models, and ending with clinical outcome predictions as evaluations of the learned representations. Specifically, learning representations from structured EHR data was dominant (37 out of 49 studies). Recurrent Neural Networks were widely applied as the deep learning architecture (LSTM: 13 studies, GRU: 11 studies). Disease prediction was the most common application and evaluation (31 studies). Benchmark datasets were mostly unavailable (28 studies) due to privacy concerns of EHR data, and code availability was assured in 20 studies. We show the importance and feasibility of learning comprehensive representations of patient EHR data through a systematic review. Advances in patient representation learning techniques will be essential for powering patient-level EHR analyses. Future work will still be devoted to leveraging the richness and potential of available EHR data. Knowledge distillation and advanced learning techniques will be exploited to assist the capability of learning patient representation further.
1. Introduction
EHR data are rich but difficult to use directly because they are heterogeneous, temporal, sparse, incomplete, and high-dimensional. The review examines deep patient representation learning methods that transform such data into features for downstream clinical prediction.
- EHR data combine uncurated, poor-quality, high-dimensional, sparse, heterogeneous, temporal, incomplete, and large-scale characteristics.
- Patient representation learning transforms raw EHR data into meaningful mathematical features that can support machine-learning prediction models.
- The review investigates patient representation learning from EHRs and collects methodological information from 49 papers.
- The typical workflow embeds raw structured or unstructured patient data, applies deep learning, and evaluates the learned representation on clinical tasks.
- Five patient-representation forms are described: vector-based, temporal matrix-based, graph-based, sequence-based, and tensor-based.
- Tensor decomposition can capture complex interactions among clinical events, including phenotypes, comorbidities, and medications, that flattened EHR data may not reveal.
3. Materials and Methods
The review defines eligibility criteria, searches five bibliographic databases, screens records in stages, and extracts methodological and reproducibility variables from included studies. From 363 initially assessed records, 49 articles met the final inclusion criteria.
- Eligible studies developed deep-learning or neural-network patient representations from longitudinal EHRs and evaluated them on downstream clinical predictions.
- The search covered PubMed, EMBASE, Scopus, ACM, and IEEE Xplore using terms combining deep learning, patient representation, and EHRs.
- 363 records were initially assessed, including 349 database records and 14 snowballed records.
- 54 duplicates were removed, leaving 309 records for title and abstract screening.
- 61 papers were excluded during full-text screening, leaving 49 included articles.
- Data extraction covered publication, representation, patient-data, EHR-resource, preprocessing, clinical-outcome, evaluation, interpretability, objective-function, computational, and code-reproducibility variables.
4. Results
Across 49 reviewed studies, patient representation learning grew rapidly and was dominated by structured EHR codes, recurrent architectures, and disease-prediction evaluations. Methodological patterns also included mixed learning scenarios, limited multimodal integration, reliance on private datasets, and attention to temporal evaluation design and reproducibility.
- Patient Data Types: 37 studies used structured codes, 6 used unstructured notes, and 6 jointly used structured codes and notes.The multimodal studies generally did not fully combine raw data from both modalities in one model.
- Preprocessing and Architecture: Among structured-code studies, 27 generated one-hot input vectors; sequential clinical-event order was modeled in 27 studies and omitted in 22.Attention mechanisms appeared in 13 studies, while hierarchical knowledge graphs appeared in 6.
- Learning Models and Architectures: LSTM was the most common architecture in 13 studies, followed by CNNs and GRUs in 11 studies each.LSTM studies included structured, unstructured, and combined data, whereas all GRU studies used structured EHR data.
- Clinical Outcomes and Applications: Disease prediction was the most common clinical task at 31 studies, followed by mortality prediction at 12 studies, while patient subtyping appeared in 4 studies.Most studies used two or more clinical outcomes to evaluate learned representations.
- Clinical Outcomes and Applications: Observation and prediction-period designs were highly correlated with the clinical outcomes being evaluated.The review identifies this relationship as an important consideration when selecting temporal windows for prediction tasks.
5.A. Growing Importance
The review documents growing interest in deep learning-based patient representations from EHRs and identifies their importance and feasibility for patient-level analyses.
- 5.A. Growing Importance: The review provides an overview of research into EHR patient representation learning and reports its growing development.The review included 49 studies identified after assessing 363 articles.
- 5.A. Growing Importance: Disease prediction was the most common clinical application, followed by mortality prediction in 12 studies.
- 5.A. Growing Importance: Learning patient representations can use large amounts of EHR data and may be more robust to small changes in data distributions.The review notes that this possible robustness requires further investigation.
- 5.A. Growing Importance: Capturing EHR characteristics such as temporal ordering and correlations between clinical events is identified as important for improving representations.The review describes hierarchical representations, longitudinal relationships, domain knowledge, and multiple modalities as complementary strategies.
5.B. Future Directions
Future work is directed toward advanced learning techniques and richer modeling of EHR structure to improve patient representations and address data scarcity.
- 5.B. Future Directions: Transformer-based approaches, including BHERT, are being explored for learning structured and transferable representations from EHR data.BHERT is described as scalable across a wide range of downstream applications.
- 5.B. Future Directions: Transfer learning, multi-task learning, and meta-learning may help address data scarcity and label insufficiency.The review discusses these techniques as directions for future patient representation learning.
- 5.B. Future Directions: Multi-task learning has been identified as a feasible approach for integrating related tasks to learn meaningful patient representations.
- 5.B. Future Directions: Future techniques are expected to incorporate knowledge bases and improve the clinical relevance of patient representation problems and solutions.
5.B.2. Data Sources
The review finds that EHR patient representation research largely relies on limited, heterogeneous data sources, while multimodal and privacy-preserving approaches remain important future directions.
- 5.B.2. Data Sources: Most current work focuses on a single EHR data type, although integrating multiple sources could expand features for fine-grained patient representations.
- 5.B.2. Data Sources: Only 12 of 49 studies used unstructured data, indicating that varied EHR information remains underused.
- 5.B.2. Data Sources: Multimodal representation learning is proposed to narrow heterogeneity among different EHR modalities.HORDE is described as dynamically embedding different EHR data sources.
- 5.B.2. Data Sources: Privacy creates a trade-off between patient-data access and research, making multi-institutional learning important for scalability and generalizability.
- 5.B.2. Data Sources: Distributed and federated learning enable collaborative training while keeping training data at the original institutions.The review identifies pilot studies applying these approaches to patient representation learning.
5.C. Limitations
The review is limited by possible incomplete retrieval and publication bias, weak reproducibility, and uncertainty about which representation methods work best.
- 5.C. Limitations: Incomplete retrieval of eligible studies and exclusion of arXiv or bioRxiv preprints may introduce publication-selection bias.
- 5.C. Limitations: The review does not answer all technical questions, partly because generic benchmarking pipelines remain scarce even for MIMIC-III.
- 5.C. Limitations: Many studies claim state-of-the-art results, but few can be verified externally because of limited transparency and reproducibility.The review identifies this as a potential barrier to future model development.
- 5.C. Limitations: It remains unclear which additional components or representation approaches best represent EHR data.The authors recommend more comprehensive comparative studies using shared clinical benchmark datasets.
- 5.C. Limitations: Comparative evidence indicates that greater deep-learning complexity without medical knowledge does not necessarily improve performance.The review cites contradictory observations from representation-learning comparisons in COPD readmission prediction.
6. Conclusion
The review identifies deep patient representation learning as a promising approach for modeling EHR data and acquiring powerful, robust, and precise representations. It synthesizes current research scenarios and anticipates continued development using advanced learning techniques.
- Deep representation learning has produced diverse innovations for modeling EHR patient data.
- Deep patient representation learning is presented as a promising direction for acquiring powerful, robust, and precise representations.
- Advanced learning techniques alongside model architectures address issues related to patient data.
- The review discusses current research scenarios and expects continued development of methods for learning meaningful patient representations.
- Patient representations are expected to play an increasingly important role in clinical prediction tasks.
Author Contributions Statement
The authors divided responsibilities across review conception, screening, data extraction, analysis, manuscript drafting, study design, editing, leadership, and final approval.
- YS and KR conceived the review, while YS, JD, ZL, and KR completed the initial screening.
- YS and ZL performed data extraction, and YS conducted the data analysis and drafted the initial manuscript.
- XJ, TM, FW, and WJZ contributed to study design and manuscript editing.
- KR provided overall leadership and revised the manuscript; all authors approved the final version.
Supplemental Tables
The supplemental tables catalogue methods, contributions, clinical tasks, datasets, and data types across the included papers. They show broad variation in models, prediction targets, and local hospital data resources, while some datasets are insufficiently described.
- The tables pair clinical prediction targets with diverse architectures, including CNNs, GRUs, LSTMs, autoencoders, and feed-forward networks.
- Disease prediction, mortality, readmission, length-of-stay, intervention, and patient-subtype tasks recur across the included studies.
- Specific methods include attention mechanisms, hierarchical or time-aware recurrent models, multi-task learning, similarity metric learning, and code or text embeddings.
- The tables include structured, unstructured, combined, claims, and local hospital EHR resources from multiple institutions.
- Reported local datasets range from 7,578 to 1.2 million patients across the listed resources.
- Some studies do not provide enough description of their datasets.