Source-linked AI summary
RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records
Bum Chul Kwon, Min-Je Choi, Joanne Taery Kim, Edward Choi, Young Bin Kim, Soonwook Kwon, Jimeng Sun, Jaegul Choo
TL;DR
RNNs can predict patient states from EMRs, but their predictions are difficult to interpret and steer with clinical expertise. This paper introduces RetainVis, which couples interactive visualizations with the interpretable RetainEX model; the study reports insights into medical-code contributions in heart-failure and cataract risk prediction.
Problem
RNN-based EMR predictors lack established ways for experts to understand individual predictions and steer models using domain expertise.
Method
RetainVis couples visualizations with RetainEX, an interpretable and interactive RNN model improved from RETAIN using temporal information and user inputs.
Results
The study reports effective use of RetainVis to inspect how individual medical codes contribute to risk predictions for heart failure and cataract.
Takeaways & Limitations
The work provides a guideline for designing interpretable and interactive visual analytics tools with RNNs.
Takeaways & Limitations
The system has scalability limits from computationally expensive deep learning, screen real estate, overplotting, and long medical histories.
Abstract
from arXiv · showhide
We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is often challenging for users to understand why the model makes a particular prediction. Such black-box nature of RNNs can impede its wide adoption in clinical practice. Furthermore, we have no established methods to interactively leverage users' domain expertise and prior knowledge as inputs for steering the model. Therefore, our design study aims to provide a visual analytics solution to increase interpretability and interactivity of RNNs via a joint effort of medical experts, artificial intelligence scientists, and visual analytics researchers. Following the iterative design process between the experts, we design, implement, and evaluate a visual analytics tool called RetainVis, which couples a newly improved, interpretable and interactive RNN-based model called RetainEX and visualizations for users' exploration of EMR data in the context of prediction tasks. Our study shows the effective use of RetainVis for gaining insights into how individual medical codes contribute to making risk predictions, using EMRs of patients with heart failure and cataract symptoms. Our study also demonstrates how we made substantial changes to the state-of-the-art RNN model called RETAIN in order to make use of temporal information and increase interactivity. This study will provide a useful guideline for researchers that aim to design an interpretable and interactive visual analytics tool for RNNs.
1 INTRODUCTION
RNNs perform well on EMR prediction but remain difficult to interpret and steer with clinical expertise. RetainVis addresses these gaps by coupling the RetainEX model with interactive visual analytics for risk prediction.
- RNNs are increasingly used on EMRs to predict patients’ current and future states from diagnoses, medications, and other events.
- Clinical adoption is challenged because experts need to understand individual predictions and provide guidance to reduce costly Type II errors.
- No established method allowed users to interactively leverage domain expertise and prior knowledge to steer RNN models.
- The study designed, implemented, and evaluated RetainVis with RetainEX through iterative collaboration among medical, artificial intelligence, and visual analytics experts.The tool targets future diagnosis-risk prediction for heart failure and cataract using prior medical visits.
- RetainVis enabled insight into how individual medical codes contribute to risk predictions and supported exploration of patient histories.The study used real EMRs from patients with heart failure and cataract.
- The contributions include RetainEX, the integrated RetainVis system, and quantitative experiments and a real-EMR case study.
2 RELATED WORK
Prior work established RNN applications and attention-based interpretation but left gaps in direct output interpretation and user interaction for RNNs. RetainEX and RetainVis combine attention, visualization, and interaction to address these gaps.
- Deep learning has been applied to EMR diagnosis, risk prediction, representation learning, and phenotyping, often outperforming rule-based and conventional machine-learning baselines.
- Many medical deep-learning studies fail to provide the interpretability clinicians need to relate predictions to patients’ past visits.
- RETAIN decomposes prediction scores to individual medical codes, whereas some alternatives provide interpretability only at the visit level.
- Visualization research has developed methods for neural-network activations and structures, but RNNs have received less attention because of their intertwined structure.
- Interactive machine-learning platforms support what-if analyses by allowing users to add, edit, or remove data and recompute outputs.
- Few studies apply user interaction to deep-learning tasks, and RetainVis is presented as one of the first such systems for RNN-based tasks.
- RetainEX and RetainVis use attention and visualization methods to increase interpretability of RNN-based models.
3 USERS, DATA, AND TASKS
The study targets clinical users who analyze longitudinal EMRs and need interpretable diagnosis-risk predictions. User tasks span cohort exploration, contribution analysis, what-if testing, and model steering.
- Target users include physicians, health professionals, and medical researchers who analyze diagnoses, prescriptions, medical events, and patient states.
- The HIRA-NPS data comprise approximately 1.4 million patients and related prescription information collected during 2014–2015.
- Predicting Diagnosis Risk: Diagnosis-risk prediction observes prior visits before target diagnosis and predicts whether illness occurs during a later visit, presumably within six months.The task is formulated as binary classification over sequential data.
- The design study iteratively identified user tasks through discussions among visual analytics, deep-learning, and medical experts.
- Users explore demographics and medical histories, select cohorts, and summarize visits, medical codes, and prediction scores.
- Users investigate which visits and medical codes contribute to predictions and compare their contribution scores.
- Users conduct what-if analyses by editing records and can provide feedback when predictions or contribution scores conflict with prior knowledge.
- RETAIN was selected as the interpretable backbone but required temporal information and user steering capabilities, motivating RetainEX and RetainVis.
4 MODEL DESCRIPTION
RetainEX extends RETAIN with extra time dimensions and embedding matrices to support interactive, interpretable prediction over sequential EMR visits. The model maps visit sequences to near-future disease-risk scores.
- RetainEX is described as RETAIN with extra time dimensions and embedding matrices, adding features for greater interactivity.
- A patient’s EMR is represented as a time-ordered sequence of vectors, with each visit containing medical codes for diagnoses, treatments, surgeries, or medicines.
- In the learning-to-diagnose task, the model observes patient visits and returns the probability of near-future diagnosis with a target disease.
A B C
RetainEX extends RETAIN with temporal information, bidirectional RNNs, and separate embeddings while exposing visit- and code-level contributions. Its interactions support what-if edits and user steering of predictions.
- Prediction and interpretation: The model converts attention-weighted visit representations into a context vector, contribution score, and diagnosis-risk probability between 0 and 1.A value closer to 1 indicates higher diagnosis risk.
- Model architecture: RetainEX appends time-interval representations to visit embeddings and processes them with two bidirectional RNNs that produce scalar α and vector β attention weights.The model represents intervals using the interval, reciprocal, and logarithmic reciprocal values.
- Prediction and interpretation: RetainEX provides code-level contribution scores and aggregates them into visit-level scores for interpreting predictions.Visit-level scores sum the contribution scores of codes occurring in each visit.
- Interaction: Users can interactively add or remove codes, modify visit periods, and adjust individual code contributions to test hypotheses and steer the model.What-if edits recompute predictions, while contribution feedback updates learned parameters.
- Model architecture: Separate embedding vectors are used for attention computation and final prediction, allowing users to modify code contributions without changing attentions at other visits.This separates the embeddings used to compute α and β from those multiplied by the attention values to form the context vector.
5 RETAINVIS: VISUAL ANALYTICS WITH RETAINEX
RetainVis couples RetainEX with coordinated visualizations for patient overviews, temporal contribution summaries, individual-patient inspection, and interactive editing. These views support cohort construction, prediction interpretation, and what-if analysis.
- Overview: Overview uses a t-SNE scatter plot and coordinated charts to summarize patients by medical codes, contribution scores, demographics, and predicted risks.Users can map attributes to scatter-plot position and color, and filter patients across views.
- Overview: The Overview code chart displays only the top three contributors and their mean scores, limiting the number of measures shown simultaneously.The authors identify narrower horizontal bars as a possible future improvement.
- Patient Summary: Patient Summary aligns selected patients to their final visit and visualizes mean and standard-deviation contribution scores for nine codes over time.Area thickness represents variance, while vertical spikes represent mean contribution scores.
- Patient inspection: Patient List and Patient Details encode visit-level contributions and prediction risks so users can compare patients and inspect how risks progress over time.Patient Details links temporal code contributions with diagnostic progression risks.
- Patient Editor: Patient Editor supports what-if analysis by moving visits along the time axis, adding or removing codes, and requesting increased contributions for selected codes.After edits, RetainVis returns updated risks and contribution scores overlaid on the original records.
6 EXPERIMENTS
The experiments evaluate RetainEX on heart-failure and cataract prediction using HIRA-NPS data, comparing it with GRU, RETAIN, and a no-time variant. RetainEX outperforms the baselines while its contribution patterns align with medical knowledge.
- Experimental setup: The study evaluates binary prediction of future heart-failure and cataract diagnoses from patients’ first six months of medical records.The experiments use the HIRA-NPS dataset and separate case-control batches for the two conditions.
- Experimental setup: Models are assessed using Area under the ROC Curve (AUC) and Average Precision (AP), metrics selected for imbalanced positive and negative labels.Both metrics measure how successfully positive cases are ranked above negative cases.
- Quantitative analysis: RetainEX outperforms GRU, original RETAIN, and RetainEX without time intervals across all reported settings, with only a small increase in training time.The authors attribute gains to separate embeddings and added temporal information.
- Qualitative analysis: High contribution scores for several heart-failure codes agree with medical knowledge, including hypertensive disease, cardiovascular prescriptions, obesity, and thyroid disorders.The study reports these codes as associated with heart failure, its prevention, treatment, or risk.
7 CASE STUDY: PATIENTS WITH HEART FAILURE
RetainVis enabled Jane to explore heart-failure risk predictions from cohort-level groupings to individual patient histories. The analysis surfaced distinct patient subgroups, medically meaningful contributors, and cases requiring performance investigation.
- Study setup: Jane analyzed 3,724 patients diagnosed with heart failure after training RetainEX on 40,964 patients.The training set used a 1:10 case-to-control ratio, and the overview represented contributions from 1,400 medical codes.
- Cohort exploration: A lasso-selected cohort of 564 patients had an average prediction score of .97, indicating that RetainEX explained the patients well.The selection contained 297 females (F = 297).
- Cohort exploration: The selected cohort separated into three subgroups, including 201 patients associated with hypertension and Bisoprolo and 230 influenced by Sasang constitutional medicine.Sasang typology is a traditional Korean personalized medicine that clusters patients into four groups based on phenotypic characteristics.
- Cohort exploration: A third subgroup averaged 74.7 years, compared with 66.7 years in the other two groups, and was associated with hypertension and diseases of the oesophagus.The group also had high bilirubin contribution scores and was conjectured to contain many severe diseases with high prediction scores.
- Individual investigation: Patient-level inspection found a patient with 150 visits in six months whose top contributors included cerebrovascular disease, glimepiride, pravastatin, and hydrochlorothiazide.Cerebrovascular disease appeared in almost every visit, while the medications were associated with diabetes, high cholesterol prevention, and blood-pressure treatment.
- Performance investigation: Jane also sorted patients by prediction score and identified three patients not predicted as heart-failure cases, with scores below .5, for failure analysis.She considered whether data quality might explain these cases.
8 DISCUSSION
The discussion examines tradeoffs between interpretability, interactivity, performance, simplicity, safety, and scalability in RetainVis. It reports how user-guided retraining can improve predictions while identifying clinical, task-specific, visualization, and implementation boundaries.
- Interpretability and Model Performance: Improving interpretability through linear combinations can reduce the computational freedom of RNNs, making the tradeoff with performance important for tool design.The authors suggest using target user tasks to guide this tradeoff.
- Interpretability and Model Performance: Experts favored visualizations that simplify EMR data and pinpoint attention points, while retaining interpretability when users request justification.Complex information may burden clinicians and hinder decision making.
- Interpretability and Model Performance: RetainVis supports researchers’ exploratory what-if analyses, while a simplified version could assist clinicians by highlighting significant and anomalous events.The authors note that presented events may reveal insights that might otherwise be overlooked.
- Towards Interactivity: The model adds visit-interval features and code-contribution retraining, enabling users to adjust individual code contributions using domain knowledge.These functions were also reported to have an auxiliary effect on quantitative performance.
- Towards Interactivity: 0.812 to 0.814: mean diagnostic risk prediction scores increased after retraining one or two selected samples, without affecting attention scores for other samples.Medical experts corrected over- and under-represented code contributions for a case patient and a control patient.
- Towards Interactivity: Feature-level interpretations serve as interaction handles, allowing domain experts to teach the model through contribution updates rather than direct parameter editing.The interaction uses direct manipulation and menu selection on feature-level data representations.
- Issues in Visualization and AI for Health: False predictions remain a major concern because doctors retain responsibility for patient outcomes, and the authors call for more convincing performance metrics.The discussion specifically highlights the risk of missed serious conditions through Type-II errors.
- Issues in Visualization and AI for Health: High AUC or AP does not guarantee clear separation of safe and suspected patients; the F-1-maximizing threshold was near 0.2 rather than the idealized 0.5.The authors connect this issue to the risk that high scores may not prevent serious medical mistakes.
9 CONCLUSION
RetainVis integrates RetainEX with EMR visualizations to improve interpretability and interactivity while maintaining performance against RETAIN. The system supports exploration of real-world EMRs and helps users gain insights and generate hypotheses.
- RetainVis incorporates RetainEX into electronic medical datasets.
- The iterative design process improved interpretability and interactivity while maintaining performance against RETAIN.
- RetainVis helps users explore real-world EMRs, gain insights, and generate new hypotheses.
- The authors aim to extend the approach to more diverse medical records, including medical tests, sensor data, and personal-device data.
Appendices
The appendix explains recurrent neural networks as sequence models that produce hidden states and predictions, and describes training, bidirectional processing, and model variants.
- A.1 Recurrent neural networks: RNNs process sequential vectors and produce corresponding hidden-state vectors for each input.
- A.1 Recurrent neural networks: Each hidden state is computed from the current input, the previous hidden state, learnable parameters, and a nonlinear transformation.
- A.1 Recurrent neural networks: For binary prediction, a scalar output is passed through a sigmoid function to obtain a value between 0 and 1.
- A.1 Recurrent neural networks: The model is trained by repeatedly updating learnable parameters with gradient descent to reduce prediction loss.
- A.1 Recurrent neural networks: LSTM and GRU variants retain the characteristics of original RNNs while providing better performance; RetainEX is built on a GRU.
- A.2 Computational backgrounds of bidirectional RNNs: Bidirectional RNNs add backward processing, and concatenate forward and backward hidden states for each timestep.
B QUALITATIVE RESULTS
This appendix section presents experiment-result tables for qualitative analysis, including tables of contribution scores averaged across patients and across occurrences.
- The section provides two tables showing experiment results.
- Table 2 reports the top-five contribution scores averaged over the total number of patients.
- Table 3 reports the top-five contribution scores averaged over the total number of occurrences.
B.1 Large-scale diagram of RetainEX
The RetainEX diagram documents the model architecture and its processing pipeline from binary medical-event vectors to a probability score.
- The large-scale RetainEX architecture is provided as a diagram on the following page.
- Separate embeddings transform binary vectors into representations, with time intervals appended to the attention-related embeddings.
- Two bidirectional RNNs produce scalar visit weights α and vector medical-code weights β.
- The weighted representations are multiplied across timesteps, summed into vector o, and transformed into probability score ŷ.