PaperlayerView exact revision on arXiv

Source-linked AI summary

Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction

Edward Choi, Andy Schuetz, Walter F. Stewart, Jimeng Sun

arXiv:1602.03686v2cs.LGcs.NE

TL;DR

The paper addresses how to represent heterogeneous EHR data and whether temporal occurrence can support useful constructed features. It learns medical-concept vectors from longitudinal EHR data and reports improved heart-failure prediction, with up to 23% improvement in AUC across classifiers.

  • Problem

    The paper addresses how to represent heterogeneous EHR data for analytics and whether temporal occurrence can support useful constructed features.

    Source: Introduction · p. 3

  • Method

    The paper learns medical-concept representations from longitudinal EHR data and maps heterogeneous data into a low-dimensional space accounting for temporal clustering.

    Source: Introduction · p. 4

  • Results

    Up to 23% improvement in AUC was reported across classifiers for heart-failure prediction.

    Source: Introduction · p. 4

  • Takeaways & Limitations

    The conclusion reports that medical-concept vectors improved many classifiers on the heart-failure prediction task.

    Source: CONCLUSION · p. 22

    The authors acknowledge limitations and future work, without naming a specific limitation in the conclusion.

    Source: CONCLUSION · p. 22

Abstract

from arXiv · show

Objective: To transform heterogeneous clinical data from electronic health records into clinically meaningful constructed features using data driven method that rely, in part, on temporal relations among data. Materials and Methods: The clinically meaningful representations of medical concepts and patients are the key for health analytic applications. Most of existing approaches directly construct features mapped to raw data (e.g., ICD or CPT codes), or utilize some ontology mapping such as SNOMED codes. However, none of the existing approaches leverage EHR data directly for learning such concept representation. We propose a new way to represent heterogeneous medical concepts (e.g., diagnoses, medications and procedures) based on co-occurrence patterns in longitudinal electronic health records. The intuition behind the method is to map medical concepts that are co-occuring closely in time to similar concept vectors so that their distance will be small. We also derive a simple method to construct patient vectors from the related medical concept vectors. Results: For qualitative evaluation, we study similar medical concepts across diagnosis, medication and procedure. In quantitative evaluation, our proposed representation significantly improves the predictive modeling performance for onset of heart failure (HF), where classification methods (e.g. logistic regression, neural network, support vector machine and K-nearest neighbors) achieve up to 23% improvement in area under the ROC curve (AUC) using this proposed representation. Conclusion: We proposed an effective method for patient and medical concept representation learning. The resulting representation can map relevant concepts together and also improves predictive modeling performance.

Introduction

The introduction identifies a gap in representing relations in longitudinal EHR data and proposes learned medical-concept vectors. It reports improved heart-failure prediction and describes adapting Skip-gram to medical concepts.

Introduction

  • Existing ontologies do not extract meaningful relations inherent to longitudinal patient data, motivating scalable features grounded in those records.Source: Introduction · p. 3
  • The proposed approach learns low-dimensional vectors accounting for temporal clustering and derives patient representations from medical-concept vectors.The introduction describes identifying co-occurring clusters and forming higher-order feature sets organized by prevalence.Source: Introduction · p. 4
  • Up to 23% improvement in AUC was reported across classifiers for heart-failure prediction.Reported AUCs: logistic regression 0.766 to 0.791; SVM 0.736 to 0.791; neural network 0.779 to 0.814; KNN 0.637 to 0.785.Source: Introduction · p. 4

Representation Learning in Natural Language Processing

Representation Learning in the Clinical Field

The study distinguishes itself by learning medical-concept representations, constructing patient representations, and applying them to a real-world prediction problem, where it reports improved performance.

Representation Learning in the Clinical Field

  • The study applies learned medical-concept and patient representations to a real-world prediction problem and reports improved performance.This contrasts with prior work on medical text and structured records that focused on learning concept representations.Source: Representation Learning in the… · pp. 5–6

MATERIALS AND METHODS

  • Patient vectors are constructed by aggregating medical-concept vectors, then used with logistic regression, SVM, MLP, or KNN for heart-failure prediction.At prediction time, the patient vector is passed to the trained model to generate a risk score.Source: MATERIALS AND METHODS · pp. 6–7

Medical Concept Representation Learning

  • Skip-gram learns concept vectors by predicting neighboring concepts in temporally ordered medical records using a sliding context window.The formulation ties target and neighboring-concept vectors to the same values, which the authors say allows faster training.Source: Medical Concept Representation Learning · pp. 8–9

Patient Representation Construction

Patient vectors are formed by summing learned medical-concept vectors. The section also describes the EHR cohort and shows diagnosis vectors grouping by category while placing some clinically related diagnoses near one another.

Patient Representation Construction

Population and Source of Data

  • The Sutter-PAMF primary-care EHR dataset comprised 265,336 patients and 555,609 unique clinical events, with cases and controls identified from May 16, 2000, through May 23, 2013.Records included demographics, smoking and alcohol consumption, and clinical and laboratory values.Source: Population and Source of Data · p. 11

Configuration for Medical Concept Representation Learning

  • The representation-learning corpus included encounter, medication-order, procedure-order, and problem-list records, yielding 38,599 unique medical concepts.The reported concept counts were 11,460 diagnoses, 17,769 medications, and 9,370 procedures.Source: Configuration for Medical Concept… · p. 11
  • Diagnosis vectors generally grouped by ICD-9 category, while related diagnoses from different groups could lie close together.The 2D t-SNE visualization includes distinct malignant and benign skin-neoplasm groups and nearby iridocyclitis and herpes-zoster eye infections.Source: Figure 5 · p. 12, Configuration for Medical Concept… · p. 13

Evaluation of Medical Concept Vector Additions

The section demonstrates medical-concept vector addition through selected examples and nearest-vector searches across diagnoses, medications, and procedures.

Evaluation of Medical Concept Vector Additions

Setup for Heart Failure Prediction Evaluation

The evaluation tests whether medical-concept vectors improve heart-failure prediction against one-hot inputs, using incident cases and matched controls from primary-care records.

Setup for Heart Failure Prediction Evaluation

  • The study frames heart-failure prediction as relevant because onset is associated with disability, high healthcare costs, and mortality.The passage reports roughly 50% mortality risk within five years of diagnosis and notes difficulty detecting HF before diagnosis.Source: Setup for Heart Failure Prediction… · p. 16
  • The comparison trained logistic regression, MLP, SVM, and KNN with both one-hot vectors and medical-concept vectors.Source: Setup for Heart Failure Prediction… · p. 16
  • Incident HF cases required qualifying codes, at least three coded encounters within 12 months, and age 50 to under 85 at diagnosis.Controls were primary-care patients matched on clinic, sex, and age, with no HF diagnosis in the preceding 12 months.Source: Setup for Heart Failure Prediction… · pp. 16–17
  • Models used records from the 18 months before each case’s HF diagnosis date and were assessed by six-fold cross-validation using held-out AUC.The identified cohort comprised 3,884 incident cases and 28,903 controls.Source: Setup for Heart Failure Prediction… · pp. 17–18

Evaluation of Heart Failure Prediction

Medical-concept vectors improve heart-failure prediction across the evaluated classifiers compared with one-hot inputs. The reported findings also associate vector representations with reduced sparsity and shorter model-training time.

Evaluation of Heart Failure Prediction

Future Work

The authors identify richer patient information and expert medical knowledge as directions for improving the representation, and propose extending the approach to other medical problems.

Future Work

  • Adding laboratory results and patient demographic information is proposed as a way to represent patients’ medical states more accurately.The authors describe these as deeper medical information that could make the representations more effective.Source: Future Work · p. 21
  • Combining well-established expert medical knowledge, such as specific features, remains a possible extension despite results obtained using medical records alone.The authors say the current results do not mean expert knowledge cannot provide benefit.Source: Future Work · p. 21
  • The authors propose applying the approach to medical problems beyond early heart-failure detection.They describe the approach as general enough for other disease-prediction problems and other medical applications.Source: Future Work · p. 21

CONCLUSION

The paper presents real-valued vectors for heterogeneous medical concepts and an efficient way to construct patient representations. The authors report clinically compatible concept relationships and improved performance across classifiers for heart-failure prediction.

CONCLUSION

  • The approach represents heterogeneous medical concepts as real-valued vectors and constructs patient representations using a deep-learning method.The conclusion characterizes patient representation construction as efficient.Source: CONCLUSION · p. 21
  • Qualitative evaluation found that trained medical concept vectors captured insights compatible with medical knowledge and experience.The conclusion separately reports the heart-failure prediction gains across classifiers.Source: CONCLUSION · pp. 21–22
  • Medical concept vectors improved the performance of many classifiers on heart-failure prediction, supporting the method’s quantitative effectiveness.The conclusion also notes limitations and future work involving deeper medical information, expert knowledge, and broader applications.Source: CONCLUSION · p. 22

SUPPLEMENTARY

The supplementary table is titled as a list of qualifying ICD-9 codes for heart failure.

SUPPLEMENTARY

  • Table 4 is identified as listing qualifying ICD-9 codes for heart failure.The supplied passage gives the table title but no code values.Source: Table 4 · p. 26

Red and Blue Box of Figure 5

Figures 7 and 8 provide detailed versions of the red and blue boxes, respectively, from Figure 5; accompanying tables list the ICD-9 codes and descriptions shown in each figure.

Red and Blue Box of Figure 5

  • Figure 7 is the detailed version of Figure 5’s red box, and Table 5 lists its ICD-9 codes and descriptions.The supplied passages identify the figure and table scope but provide no code values.Source: Figure 7 · p. 27, Table 5 · p. 27
  • Figure 8 is the detailed version of Figure 5’s blue box, and Table 6 lists its ICD-9 codes and descriptions.The supplied passages identify the figure and table scope but provide no code values.Source: Figure 8 · p. 28, Table 6 · p. 28

6-fold Cross Validation Scheme

For HF prediction, the scheme is presented as 6-fold cross-validation, while the prose describes a 7-chunk cohort with two chunks taking turns as validation and test sets.

6-fold Cross Validation Scheme

Hyper-parameters used for training models

The authors report selecting hyper-parameter settings that produced the best performance and list settings for logistic regression and SVM under one-hot and medical concept vectors.

Hyper-parameters used for training models

  • The authors say the reported hyper-parameter setting was selected after experimentation because it produced the best performance.Source: Hyper-parameters used for training… · p. 30
  • Logistic regression, MLP, and GRU were trained with Theano 0.7 and CUDA 7, while SVM and KNN used Scikit-Learn classifiers.SVM used Linear SVC; KNN used KNeighborsClassifier.Source: Hyper-parameters used for training… · p. 30
  • The table lists separate L2 regularization settings for logistic regression and SVM using one-hot versus medical concept vectors.Logistic regression uses 0.1 and 0.01, respectively, with a 100-epoch maximum; SVM uses 0.000001 and 0.001, respectively, with Dual set to False.Source: Table 7 · p. 30

procedures

This section summarizes patient-record entries for procedures and services across preventive care, diagnostic testing, specialty referrals, and treatments in varied clinical areas. Some listings also include diagnoses and medications; they do not establish outcomes.

procedures

  • Other listed procedures included nail debridement, trigger-point and joint injections, and destruction or biopsy of skin lesions.Source: procedures · pp. 31, 34
  • Procedures for specialty and sensory care included nerve-conduction studies, EMG, ophthalmic imaging and visual-field exams, hearing tests, and ear-wax removal.The records also include neurology, ophthalmology, and ENT referrals.Source: procedures · pp. 31, 33, 34
  • Preventive and diagnostic procedures included colonoscopy with biopsy or polyp removal, mammography, Pap testing, diabetic eye and foot exams, and vaccinations.Source: procedures · p. 32
  • Cardiac procedures included ECGs, stress echocardiography, echocardiography, and chest imaging.The records also list pulse oximetry and inhalation treatment for respiratory obstruction.Source: procedures · pp. 31–33
  • Cardiac workups list ECG, Holter monitoring, stress testing, and echocardiography alongside cardiac diagnoses including heart failure and atrial fibrillation.Other listed cardiac procedures include chest radiography and prothrombin-time testing.Source: procedures · pp. 35, 37, 38
  • Preventive care entries include influenza and Tdap vaccination, mammography, cervical screening, and routine gynecological examination.Other listed screenings include colon cancer screening and diabetic eye examination.Source: procedures · pp. 35, 37
  • Respiratory-related entries include pulse oximetry, chest radiographs, and inhalation treatment, alongside asthma, bronchitis, and upper-respiratory diagnoses.Medication entries include bronchodilators, antibiotics, and cough medicines.Source: procedures · pp. 35, 37
  • Sleep-related entries include referral to sleep disorders, polysomnography, sleep studies, and CPAP equipment, alongside obstructive sleep apnea.Source: procedures · p. 37
  • Other entries include mammography and bone-density screening, eye examinations and refraction, vaccination, and routine office visits.Source: procedures · pp. 38–39
  • Dermatology entries include skin biopsy and destruction of benign or premalignant lesions, with dermatology referrals.Source: procedures · pp. 38, 42
  • Colonoscopy entries include procedures with biopsy or snare removal, alongside gastroenterology referrals and diagnostic colonoscopy.Source: procedures · pp. 38–39
  • Musculoskeletal entries include lumbar imaging and MRI, physical medicine and therapy referrals, and knee imaging or joint injections.Source: procedures · pp. 40–41

procedures

  • The listings include preventive examinations, cancer screenings, and vaccinations alongside associated screening and immunization services.Examples include mammography and colonoscopy screening, tetanus-diphtheria and pneumococcal vaccines, and immunization administration.Source: procedures · pp. 42–43
  • Musculoskeletal and rheumatologic entries pair pain, arthritis, and spinal diagnoses with medications, imaging, referrals, injections, and physical therapy.Examples include rheumatoid arthritis with methotrexate and prednisone, and lumbar pain with MRI and physical therapy services.Source: procedures · pp. 43–44
  • Cardiovascular entries include atrial fibrillation, congestive heart failure, and coronary atherosclerosis, with warfarin and prothrombin-time services also listed.The medication list includes warfarin and the procedure list includes prothrombin time and ECG services.Source: procedures · p. 44
Loading 1602.03686v2…