PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Document Embeddings for Intensive Care Patient Mortality Prediction

Paulina Grnarova, Florian Schmidt, Stephanie L. Hyland, Carsten Eickhoff

arXiv:1612.00467v1cs.CL

TL;DR

Clinical-note text remains insufficiently studied for mortality prediction, despite the importance of predicting mortality for care decisions. The paper uses a two-level CNN to represent clinical notes and reports better performance than existing bag-of-words and doc2vec approaches across three MIMIC-III tasks.

  • Problem

    Clinical mortality prediction has largely relied on time-series and demographic data, leaving the unstructured text in clinical notes comparatively insufficiently studied.

    Source: §1 · p. 1

  • Method

    A two-level CNN encodes each sentence and combines sentence vectors into a patient representation for mortality prediction.

    Source: §2 · p. 2

  • Results

    On all three MIMIC-III ICU mortality tasks, the model outperforms existing bag-of-words approaches and doc2vec, with neural models particularly stronger than LDA on the harder tasks.

    Source: §4 · p. 4, §3.4 · p. 4

  • Takeaways & Limitations

    The findings support accounting for word and phrase compositionality when identifying important text patterns for mortality prediction.

    Source: §4 · p. 4

    Some neutral sentences can be ranked too highly because the model does not handle sentences that fail to distinguish the mortality classes.

    Source: §3.4 · p. 4

Abstract

from arXiv · show

We present an automatic mortality prediction scheme based on the unstructured textual content of clinical notes. Proposing a convolutional document embedding approach, our empirical investigation using the MIMIC-III intensive care database shows significant performance gains compared to previously employed methods such as latent topic distributions or generic doc2vec embeddings. These improvements are especially pronounced for the difficult problem of post-discharge mortality prediction.

1 Introduction

Mortality prediction matters for clinical decisions such as allocating scarce resources and planning post-discharge care. The paper addresses limited use of clinical-note text by proposing a CNN that represents phrases and documents, not just individual terms.

  • Clinical mortality prediction can inform scarce-resource allocation and the frequency and intensity of post-discharge care.Source: §1 · p. 1
  • Most competitive mortality-prediction methods rely on time series and demographics, while note-text approaches remain insufficiently studied and use term-wise or topic representations.Source: §1 · p. 1
  • Term-only representations may miss phrase compositionality, including negation, and long-range dependencies in clinical text.Source: §1 · p. 1
  • The paper proposes a CNN that represents terms, phrases, and documents to preserve subtleties of natural language.Source: §1 · p. 1

2 Model

The model uses CNNs to encode each sentence and combine sentence representations into a patient vector for mortality prediction. It also adds sentence-level prediction loss and can incorporate note-category information.

  • The model addresses a limitation of doc2vec-style feed-forward representations, which cannot directly recognize complex multi-word or multi-sentence patterns such as negation.Source: §2 · p. 2
  • Two CNN layers map sentences to sentence vectors, then combine them into one patient representation used to estimate mortality probability.Source: §2 · p. 2

Target replication

  • Sentence-level target replication adds a mortality prediction and cross-entropy term for each sentence to the objective.The sentence-level error regularizes the first CNN toward sentence representations tailored to mortality prediction; λ sets regularization strength.Source: §2 · p. 2

Incorporating note information

  • Concatenating each sentence vector with its note-category vector lets the classifier use one of 14 categories, such as nursing or physician.Source: §2 · pp. 2–3

3 Experiments

Experiments use MIMIC-III clinical notes to assess ICU mortality prediction across three time horizons and compare the two-level CNN with topic-model and doc2vec baselines. The CNN outperforms doc2vec across all tasks, with neural models particularly stronger than the LDA baseline on the harder tasks.

3 Experiments

  • The evaluation predicts death during hospitalization, within 30 days after discharge, or within one year after discharge, using AUC.Source: §3 · p. 3

3.1 Data

  • The filtered dataset contains 31,244 adult patients with one admission and 812,158 notes; discharge summaries and post-discharge notes are excluded.Source: §3.1 · p. 3

3.2 Baselines

  • The LDA baseline uses 50 topics and per-note topic distributions with linear SVMs; the DBOW doc2vec baseline also uses separate linear SVMs for each task.Source: §3.2 · p. 3

3.3 Parameters and Pretraining

  • The CNN uses 50-dimensional pretrained word vectors, 50 word-level filters of sizes 3–5, and 50 sentence-level filters of size 3.Category vectors have 10 dimensions; the patient representation has 50 dimensions, and the final layer uses L2 regularization and dropout with keep probability 0.8.Source: §3.3 · p. 3

3.4 Results

  • The two-level CNN significantly outperforms doc2vec on all three MIMIC-III mortality tasks, while both neural models surpass LDA especially on the two harder tasks.Source: §3.4 · p. 4, Table 1 · p. 4
  • The model’s mortality-task-trained sentence representations support retrieving informative sentences, but neutral sentences can be ranked too highly.The authors identify a more advanced attention mechanism as future work to address this issue.Source: §3.4 · p. 4

4 Conclusion

On MIMIC-III, the two-layer CNN outperforms BOW and doc2vec on all three ICU mortality prediction tasks. The authors conclude that word and phrase compositionality is crucial for identifying important text patterns.

4 Conclusion

  • On MIMIC-III, the model outperforms existing BOW approaches and doc2vec on all three ICU mortality prediction tasks.Source: §4 · p. 4
  • The study developed a two-layer convolutional neural network for ICU mortality prediction.Source: §4 · p. 4
  • The authors conclude that accounting for word and phrase compositionality is crucial for identifying important text patterns.They suggest these findings may inform clinical machine learning research aimed at reducing patient mortality.Source: §4 · p. 4

Related papers

Loading 1612.00467v1…