Source-linked AI summary
Clinical Text Classification with Rule-based Features and Knowledge-guided Convolutional Neural Networks
Liang Yao, Chengsheng Mao, Yuan Luo
TL;DR
Clinical text classification studies often engineer features from rules or knowledge sources without fully exploiting automatic feature learning. This paper combines rule-based trigger phrases with a knowledge-guided CNN using word and UMLS entity embeddings, and reports that the method outperforms state-of-the-art methods on the i2b2 obesity challenge.
Problem
Existing clinical text classification studies often rely on rules or knowledge sources, while relatively few exploit deep learning’s feature-learning capability for long clinical text with knowledge sources.
Method
The method identifies trigger phrases, uses rules for classes with very few examples, and trains a CNN with word embeddings and UMLS CUI entity embeddings.
Results
The method outperforms state-of-the-art methods on the 2008 i2b2 obesity challenge.
Takeaways & Limitations
CNNs learn effective hidden features, and CUI embeddings are helpful for building clinical text representations in this evaluation.
Takeaways & Limitations
Performance is constrained by imbalanced classes and ineffective negative or uncertain trigger phrases, while improved entity recognition and sense disambiguation remain future directions.
Abstract
from arXiv · showhide
Clinical text classification is an important problem in medical natural language processing. Existing studies have conventionally focused on rules or knowledge sources-based feature engineering, but only a few have exploited effective feature learning capability of deep learning methods. In this study, we propose a novel approach which combines rule-based features and knowledge-guided deep learning techniques for effective disease classification. Critical Steps of our method include identifying trigger phrases, predicting classes with very few examples using trigger phrases and training a convolutional neural network with word embeddings and Unified Medical Language System (UMLS) entity embeddings. We evaluated our method on the 2008 Integrating Informatics with Biology and the Bedside (i2b2) obesity challenge. The results show that our method outperforms the state of the art methods.
1 Introduction
Clinical text classification supports organization and retrieval of clinical records, but existing studies often rely on engineered rules or knowledge features without automatically learning effective features. This study combines rule-based features with knowledge-guided deep learning for disease classification and reports improvement over state-of-the-art methods.
- Clinical text classification supports clinical-record organization, retrieval, cohort identification, and clinical decision support.
- Existing studies commonly use rules or knowledge sources for feature engineering, but often cannot learn effective features automatically.
- The proposed method combines rule-based features with knowledge-guided deep learning for disease classification.
- The method identifies trigger phrases, uses them for classes with very few examples, and trains a CNN with word and UMLS CUI entity embeddings.
- On the 2008 i2b2 obesity challenge, the method outperforms state-of-the-art methods.
2 Related Work
Clinical text classification has used rules, biomedical knowledge, and other information sources, while deep learning has demonstrated strong feature-learning ability. The paper targets the limited combination of deep learning, knowledge sources, and long clinical-text classification.
- Clinical Text Classification: Most top systems in the i2b2 obesity challenge were rule-based, including all four highest-ranked systems.
- Clinical Text Classification: Biomedical approaches commonly map narrative text to UMLS concepts and use CUIs or related concepts as classifier features.
- Clinical Text Classification: Other clinical classification studies use regular-expression discovery, semi-supervised learning, or active learning.
- Clinical Text Classification: These approaches seldom use effective feature learning, despite deep learning methods showing powerful feature-learning capabilities.
- Deep Learning for Clinical Data Mining: Although deep learning has been studied in clinical data mining, most prior work neither classifies entire clinical notes nor uses knowledge sources.
3 Obesity Challenge
The i2b2 2008 obesity challenge evaluates disease-status classification for obesity and 15 comorbidities. Its textual and intuitive tasks differ in evidence requirements and contain severe class imbalance, making Macro F1 the primary evaluation metric.
- The challenge evaluates disease-status classification for obesity and 15 common comorbidities.
- Documents receive Present, Absent, Questionable, or Unmentioned labels for each disease, with Macro F1 as the primary metric.
- The textual task identifies explicit disease evidence, whereas the intuitive task predicts disease status when evidence is not explicitly mentioned.
- Class distributions are highly uneven, with few negative or questionable examples and some classes lacking training examples.
4 Method
The method combines Solt’s rule-based trigger phrases with a knowledge-guided CNN, using rules for sparse classes and neural prediction for more populated classes. The CNN integrates word embeddings and selected UMLS CUI embeddings before classification.
- Method: The method has three steps: identify trigger phrases, predict sparse classes with rules, and train a knowledge-guided CNN for more populated classes.
- Trigger Phrase Identification: Trigger phrases use disease names, alternative names, and negative or uncertain words after shared preprocessing with Solt’s system.
- Sparse-Class Prediction: Because some classes have few or no training examples, the method removes sparse classes from CNN training and uses Solt’s rules for them.
- Knowledge-guided CNN: The CNN trains on positive trigger phrases and UMLS CUIs, while test records labeled Q or N by Solt’s system retain those labels.
- Knowledge-guided CNN: The CNN uses 200-dimensional clinical word embeddings, the best-performing size among the tested dimensions.
- Knowledge-guided CNN: Selected disease-relevant UMLS semantic types provide entity inputs that enrich the CNN representation.
- Knowledge-guided CNN: The architecture convolves word and entity embeddings separately, max-pools their features, concatenates them, and predicts labels through fully connected layers and softmax.
5 Results
The method generally outperformed the compared rule-based and conventional classifiers, with gains associated especially with CUI embeddings, while performance remained constrained by trigger-phrase and entity-recognition issues.
- CUI embeddings improved scores across diseases and raised overall Macro F1 and Micro F1 above Solt’s system in both tasks.The authors attribute this to disambiguated CUIs carrying disease-related semantic information.
- Word embeddings alone improved intuitive-task performance but did not improve the textual task, where explicit positive trigger phrases already supplied sufficient information.
- Ten runs produced overall Macro F1 and Micro F1 scores significantly higher than Solt’s paper and Perl implementation (p value < 0.05).
- Errors often arose from incomplete or misleading positive trigger phrases, while imbalanced diseases produced high Micro F1 but low Macro F1.The authors suggest enriching regular expressions to identify trigger phrases more accurately.
- Filtering CUIs by relevant semantic types moderately improved performance over using all CUIs, although MetaMap could introduce noisy or unrelated CUIs.
- The method’s overall Macro F1 and Micro F1 scores were significantly higher than Logistic Regression and SVM (p value < 0.05).The comparison used the same training set, while Solt’s system labeled very few-example classes.
6 Conclusion
The paper presents a clinical text classification method that combines rule-based trigger phrases with a knowledge-guided CNN using word and UMLS CUI embeddings. On the i2b2 obesity challenge, it outperformed state-of-the-art methods, while identifying domain knowledge integration as promising and broader evaluation as future work.
- The proposed method combines rules for trigger phrases, few-example class prediction, and a knowledge-guided CNN with word and UMLS CUI embeddings.
- Evaluation on the i2b2 obesity challenge showed that the method outperformed the challenge’s state-of-the-art methods.
- The authors conclude that CNNs can learn effective hidden features and that CUI embeddings help construct clinical text representations.
- The authors identify integrating domain knowledge into CNN models as promising and plan evaluation on more clinical-record datasets.