Source-linked AI summary
Medical Text Classification using Convolutional Neural Networks
Mark Hughes, Irene Li, Spyros Kotoulas, Toyotaro Suzumura
TL;DR
Clinical text classification needs richer context-sensitive representations than dictionary-based approaches provide. The paper applies CNNs to Word2vec-derived sentence representations and reports at least 15% higher accuracy than several alternative methods. The authors conclude that learned semantic representations can support sentence-level medical classification and potentially other text tasks.
Problem
Dictionary-based clinical-text representations are limited when modeling contextual relations and implicit social determinants of health.
Method
A CNN classifies medical sentences using Word2vec representations trained on domain-specific medical text.
Results
15% higher accuracy is reported for the CNN-based approach than for Sentence Embeddings, Mean Word Embeddings, and Word Embeddings with BOW.
Takeaways & Limitations
Learned CNN representations support semantic classification at the sentence level and may also be used for text comparison, retrieval, or larger text units.
Takeaways & Limitations
The authors identify the need to test the technique at larger scale and with a more fine-grained set of clinical classifications.
Abstract
from arXiv · showhide
We present an approach to automatically classify clinical text at a sentence level. We are using deep convolutional neural networks to represent complex features. We train the network on a dataset providing a broad categorization of health information. Through a detailed evaluation, we demonstrate that our method outperforms several approaches widely used in natural language processing tasks by about 15%.
1. Introduction
The paper addresses limitations in dictionary-based representations of clinical text by applying CNNs to learn context-based sentence representations. Compared with several alternatives, the CNN approach achieves at least 15% higher classification accuracy.
- Clinical notes require efficient interpretation because complex cases can cause information overload, delays, or missing information.
- Dictionary-based approaches struggle with contextual relations and implicit social determinants of health.The example involving a patient’s mother requires resolving who is described and recognizing social exclusion without an explicit associated word.
- The work aims to build automatically generated, context-based, rich representations of health-related information.
- The authors apply CNNs to sentence-level categorization using emergent semantics extracted from medical text.
- 15% higher accuracy is reported for the CNN-based approach than for Sentence Embeddings, Mean Word Embeddings, and Word Embeddings with BOW.
2. Related Works
The paper situates medical text classification within NLP and introduces sentence-level medical classification with convolutional neural networks as its stated first use of this technology for that task.
- Health-related text classification is treated as a specialized text-classification problem with prior applications of SVMs, LDA, and neural embeddings.
- Word2vec and Doc2vec provide dense representations for words and paragraphs, respectively, while CNNs have shown competitive results for high-level feature learning.
- The authors describe their work as the first approach using this technology for sentence-level classification of medical text.
3. Methods
The method combines domain-specific Word2vec representations with a CNN classifier trained on balanced, professionally categorized medical sentences. Sentences are converted to fixed-size matrices so they can be processed by the network.
- The case study trains Word2vec on a large medical corpus and the sentence classifier on a smaller pre-categorized corpus.
- The Word2vec model uses 15k PubMed clinical research papers spanning a wide range of medical subjects.
- The training dataset contains 26 medical categories with 4000 randomly selected sentences per category, balanced across categories.
- Each sentence becomes a word-level matrix whose rows are Word2vec sentence vectors, with a fixed maximum length of 50 words.Shorter sentences receive repeated special stop words, while longer sentences are truncated to their first 50 words.
- The best CNN configuration uses two pairs of convolutional layers, each followed by max pooling, with 256 filters of size 5.
4. Evaluation
The evaluation compares CNN classification with sentence embeddings, mean word embeddings, and BOW-based word embeddings. The CNN achieves the highest accuracy by a wide margin.
- The evaluation compares the CNN with Sentence Embeddings, Mean Word Embeddings, and Word Embeddings with BOW.
- Doc2vec sentence embeddings are inferred for test sentences and classified with Logistic Regression.
- Mean Word Embeddings represent each sentence by the pair-wise mean of its word embeddings, with zero-filling or elimination for out-of-vocabulary words.
- The BOW approach clusters Word2vec features into a vocabulary of 1000 and uses soft assignment with K=50 to reduce sparsity.
- The CNN-based approach has the highest accuracy by a wide margin, while the first three methods perform worse and BOW performs better among the alternatives.The authors suggest that CNNs capture more complex features than the other shallow approaches.
5. Conclusions and Future Work
The paper presents CNNs for sentence-level semantic classification of medical documents and argues that the learned representations may support broader text tasks and larger-scale clinical applications. Future work targets finer-grained classifications, larger datasets, domain adaptation, and patient-level representations.
- Conclusions: CNNs represent the semantics of clinical text for sentence-level classification.The paper contrasts multi-layer convolutional networks with shallow methods for learning sentence features during training.
- Conclusions: The learned semantic representations could also support text comparison and retrieval tasks.The paper frames these as alternative uses after the representations have been learned.
- Future Work: The approach could be scaled with limited effort from sentence representations to paragraph- or document-level representations.This proposed scaling extends the representation granularity beyond the sentence level.
- Future Work: Future work will test the technique at larger scale with a more fine-grained set of clinical classifications.The planned larger data sources include PubMed, relevant Wikipedia topics, medical books, and journals.
- Future Work: The authors also plan to explore domain adaptation and dense patient representations combining clinical notes with structured data.The proposed patient vector would capture medical conditions and treatments from unstructured notes, possibly alongside structured information.