Source-linked AI summary
MiNER: Fine-Tuned Biomedical Natural Language Processing for Malaria Disease Entity Recognition in Clinical Texts
V. S. Anoop, Devika N
TL;DR
The paper addresses the challenge of extracting clinically significant entities from the large, unstructured malaria literature. It fine-tunes BioBERT on manually annotated malaria abstracts and compares resulting representations and classifiers, reporting that BioBERT embeddings outperform baselines for biomedical NER. The study also releases labeled resources, while acknowledging limits from malaria-specific data and annotation coverage.
Problem
The vast, unstructured malaria literature is difficult to manually curate and extract, while biomedical NER has limited domain-specific labeled data.
Method
The approach collects and manually annotates malaria PubMed abstracts, uses BioBERT to encode biomedical text, and trains machine-learning classifiers for entity recognition.
Results
BioBERT embeddings outperformed several baselines for biomedical named entity recognition.
Takeaways & Limitations
The paper publishes labeled data and code to support further development of malaria biomedical NER models.
Takeaways & Limitations
The malaria-specific corpus may limit generalizability to other biomedical domains, and performance depends on annotated training-data quality and coverage.
Abstract
from arXiv · showhide
Malaria remains a significant global health burden, necessitating continuous research efforts to understand its complex molecular mechanisms, epidemiology, and potential therapeutic interventions. Extracting essential biomedical information from the vast and constantly growing malaria literature is a challenging task that demands innovative approaches. Recently, pre-trained language models have revolutionized natural language processing tasks, demonstrating remarkable capabilities in various domains. This paper proposes a fine-tuned pre-trained biomedical language model for biomedical information extraction from scientific literature on malaria disease. The proposed methodology selects and preprocesses a large corpus of scientific articles on malaria, and then annotates them with entities of clinical significance. It then leverages BioBERT, a state-of-the-art pre-trained language model, to encode the textual data into context-aware representations. We fine-tune the model using domain-specific annotations and supervised learning to enhance its ability to extract relevant biomedical named entities. Extensive experiments and comparisons with different encoding and machine learning algorithms show that the proposed approach significantly outperforms them in precision, recall, and accuracy. We also publish our human-labeled dataset for entity and relation extraction to enable other health informatics researchers to train advanced models for malaria information extraction.
1 Introduction
Malaria literature contains valuable biomedical knowledge, but its scale and unstructured nature make manual extraction laborious, time-consuming, and error-prone. This paper proposes domain-specific language-model methods to extract clinically significant entities and publishes labeled resources for further research.
- Pre-trained language models have transformed information extraction by providing contextual understanding of natural language.
- Malaria remains a major global-health threat, while its complex literature requires substantial synthesis for prevention, diagnosis, and treatment research.
- The growing volume of malaria papers, trials, case studies, and reports makes extracting valuable information difficult.
- The proposed approach collects and manually labels malaria literature, then uses BioBERT contextual embeddings with machine-learning algorithms to identify clinical entities.
- The paper experimentally compares the approach with state-of-the-art machine-learning methods and publishes labeled data and code for advanced NER research.
2 Related Studies
Biomedical named entity recognition has progressed from feature-engineered machine-learning methods toward deep and transformer-based models. Domain-specific pre-training, particularly BioBERT, addresses contextual challenges but remains constrained by limited biomedical labeled data and specialized-domain coverage.
- Biomedical NER requires assigning labels to sequence tokens, including multi-word entities, and depends on sufficient labeled data.
- Traditional biomedical NER relies on time-consuming feature engineering, while deep learning reduces this dependence and improves contextual representation.
- CNNs, LSTMs, transformer models, and hybrid attention-based systems have been applied to biomedical entity extraction.
- Long-range dependencies challenged recurrent approaches, motivating pretrained models that capture domain-relevant contextual information.
- BioBERT is a biomedical-domain pretrained model that has shown improvements in text-mining tasks, although biomedical domain-specific pretrained models remain limited.
3 Materials and Methods
The methodology constructs and annotates malaria abstracts, encodes text with several feature representations, and trains conventional classifiers for named entity classification. Evaluation uses accuracy, precision, recall, and F1-score.
- Materials and Methods: The methodology converts labeled, preprocessed text into numerical vectors before training machine-learning classifiers to identify named entities.
- Feature Encoding: TF-IDF combines term frequency with inverse document frequency, whereas Count Vectors represent documents by word-occurrence counts while disregarding word order and context.
- Feature Encoding: The evaluated encodings include TF-IDF, Count Vectors, GloVe, Word2Vec, BERT, and BioBERT.
- Classification: The classifiers include SVM, Logistic Regression, Random Forest, and Naive Bayes, whose latter assumes conditional independence among features given the class.
- Evaluation: Performance is evaluated using accuracy, precision, recall, and F1-score.
4 Proposed Approach
The proposed malaria named entity recognition approach uses a PubMed-based, manually annotated corpus and evaluates embedding–classifier combinations to select the best-performing pair.
- 4 Proposed Approach: PubMed malaria abstracts are manually annotated with domain-specific entity labels for biomedical named entity recognition.The labels include Disease, Organism, Medication, Protein, Gene, Anatomical Structures, Chemical Structures, and Other(O).
- 4 Proposed Approach: The workflow is organized as an algorithm for the proposed named entity recognition approach.
- 4 Proposed Approach: The procedure outputs the best embedding-classifier pair with evaluation metrics.
1 Step 1: Preprocessing
Preprocessing prepares the labeled corpus for modeling by normalizing and tokenizing text, aligning tokens with BIO labels, and creating train, development, and test splits.
- 1 Step 1: Preprocessing: Text is normalized and tokenized before model training.
- 1 Step 1: Preprocessing: Tokens are aligned with BIO labels to preserve entity annotations.
- 1 Step 1: Preprocessing: The data is split into training, development, and test sets.
5 Step 2: Feature Extraction
Feature extraction compares six embedding and representation methods, transforming each input text into the feature vectors used by downstream classifiers.
- 5 Step 2: Feature Extraction: The embedding and feature set contains TF-IDF, CountVector, Word2Vec, GloVe, BERT, and BioBERT.
- 5 Step 2: Feature Extraction: Each input x(i) is transformed into a feature vector f(i) using the selected embedding method E.
9 Step 3: Model Training
Model training compares four classifiers on the extracted features, validates them with k-fold cross-validation, and records entity-level evaluation results for each embedding-classifier combination.
- 9 Step 3: Model Training: The classifier set comprises SVM, LR, RF, and NB.
- 9 Step 3: Model Training: Each classifier is trained on the extracted features with their corresponding labels.
- 9 Step 3: Model Training: Model performance is assessed using k-fold cross-validation.
- 9 Step 3: Model Training: Entity-level Precision, Recall, and F1-score are computed for evaluation.
- 9 Step 3: Model Training: Results are stored as embedding, classifier, and F1-score tuples.
17 Step 4: Final Evaluation
The final evaluation retrains the models on the combined training and development data, evaluates them on a held-out test set, and reports entity-level Precision, Recall, and F1. The system uses IOBES annotations and numerical feature encodings, including BERT and BioBERT embeddings, before machine-learning evaluation.
- The models are retrained on the train+dev set and evaluated on a held-out test set.
- Entity-level Precision, Recall, and F1 are reported as the evaluation measures.
- Annotated data uses IOBES tagging, distinguishing entity beginnings, interiors, ends, single-word entities, and irrelevant words.
- The pipeline converts preprocessed data into numerical feature representations before using the resulting vectors as model inputs.
- BERT and BioBERT embeddings convert split input words into numerical representations within the NER architecture.
- The authors publish the labeled dataset and associated code for researchers to train advanced biomedical NLP models.
5 Results and Discussions
The experiments compare classical classifiers across feature encodings using precision, recall, accuracy, and F-measure. BioBERT embeddings improve F-measure for SVM, Logistic Regression, and Random Forest, while malaria-specific data and surface-level modeling constrain generalizability and entity disambiguation.
- Experimental setup: The study reports precision, recall, accuracy, and F-measure for SVM, Logistic Regression, Random Forest, and Naive Bayes across multiple feature encodings.Evaluated encodings include TF-IDF, CountVector, Word2Vec, GloVe, BERT, and BioBERT.
- TF-IDF and CountVector: 97.41% accuracy was obtained by Random Forest with TF-IDF, alongside 95.20% precision, 97.45% recall, and 96.31% f-measure.
- Word2Vec and GloVe: 97.85% accuracy and 98.72% recall were obtained by Random Forest with Word2Vec, with 89.70% precision and 93.99% f-measure.
- BERT and BioBERT: For BERT embeddings, Random Forest achieved 98.45% precision, 88.75% recall, 92.35% accuracy, and 93.35% f-measure.
- BERT and BioBERT: BioBERT produced higher f-measure values for SVM, Logistic Regression, and Random Forest, whereas Naive Bayes achieved 87.84% f-measure.The text attributes the Naive Bayes result to its feature-independence assumption not holding for sequence understanding.
- Limitations: The malaria-only corpus may limit transferability to other biomedical domains, and limited manually annotated malaria-specific data may hinder recognition of rare or domain-specific entities.The model also relies primarily on surface-level textual features, creating challenges with similar entities, abbreviations, and synonyms.
6 Conclusions and Future Work
The conclusion presents BioBERT embeddings as outperforming several baselines for biomedical named entity recognition. It identifies more labeled data, deep learning approaches, and newer open-source biomedical models as future directions.
- BioBERT embeddings outperformed several baselines for biomedical named entity recognition in the reported comparison.
- Future work: The authors propose extending the methodology with deep learning approaches and more labeled data.
- Future work: Using newly published open-source pretrained biomedical models to generate embeddings is identified as an interesting future direction.