Source-linked AI summary
BanglaMed-QA: A Question Answering System for Healthcare Support in Bangla
Rowzatul Zannat, Abdullah Al Shafi, K. M. Azharul Hasan, Atia Shahnaz Ipa
TL;DR
Bangla medical QA remains constrained by limited resources and annotated datasets. BanglaMed-QA combines a structured knowledge base with linguistic processing, supervised categorization, and ensemble similarity matching, achieving strong automated and human-evaluated performance.
Problem
Reliable, understandable medical information and annotated Bangla resources remain limited, motivating healthcare question answering for Bangla.
Method
The system uses POS-based anaphora resolution, synonym-based question rephrasing, supervised question categorization, and combined similarity methods with voting strategies.
Results
0.9 out of 1.0 was the average human evaluation rating, while SVM showed the best overall question-categorization performance.
Takeaways & Limitations
BanglaMed-QA provides a practical Bangla medical QA system intended to improve answer reliability and support further Bangla NLP applications.
Abstract
from arXiv · showhide
Medical question answering (QA) systems have become crucial tools for providing reliable health information. But they remain very unexplored for low-resource languages like Bangla due to limited datasets and systems tailored to these languages. To address this, we introduce BanglaMed-QA, a robust QA system specifically designed for the Bangla medical domain. The process begins with building a structured medical knowledge base that includes 4,493 QA pairs in 9 categories under 506 diseases. To improve semantic comprehension, domain-specific root word dictionaries and synonym sets are proposed, in addition to part-of-speech tagging for anaphora resolution. We adopt supervised machine learning models in which SVM is found to be the best model to categorize questions. Multiple similarity metrics, including cosine, Jaccard, BM25, and Levenshtein, are applied with soft and hard voting methods for query matching. The performance of the QA system has been evaluated in two aspects, with a 95% F1 score in an automated evaluation and an average human satisfaction rating of 0.9 out of 1.0. This validates the real-world application of BanglaMed-QA in closing the healthcare information gap for Bangla speakers.
1 Introduction
BanglaMed-QA addresses limited access to reliable medical information in Bangla by developing a healthcare QA system with language-specific processing and similarity-based matching.
- Motivation: Reliable and understandable medical information remains difficult to access, especially for people using low-resource languages such as Bangla.The paper positions medical QA systems as tools for delivering precise answers instead of requiring users to scan lengthy documents.
- Contributions: The authors construct a structured Bangla medical knowledge base to support healthcare question answering.The contribution responds to the stated scarcity of Bangla medical corpora.
- Contributions: POS tagging supports anaphora resolution by resolving pronouns to corresponding nouns in medical questions.This module is designed specifically for the Bangla medical QA setting.
- Contributions: Supervised machine learning models classify questions to speed similarity matching.The paper explores multiple supervised models for question categorization.
- Contributions: Cosine, Jaccard, BM25, and Levenshtein similarities are combined through soft and hard voting to match user questions with knowledge-base questions.The system returns the answer associated with the most similar question subject to a threshold.
- Contributions: The paper also provides an interactive user interface for visualizing the proposed system.The paper is organized around dataset construction, method design, experimental analysis, and conclusions.
2 Literature Review
Prior medical QA research spans rule-based, information-retrieval, knowledge-graph, neural, and hybrid approaches, while Bangla systems remain constrained by data and validation gaps.
- Research landscape: Medical QA research includes rule-based, UMLS, information-retrieval, knowledge-graph, neural-network, and hybrid models.The review organizes prior work by methods, datasets, and evaluation approaches.
- Existing systems: Early UMLS-triple and semantic-web systems encountered question normalization or scalability issues.Jacquemart et al. reported that 90% of student questions followed the modeled structure, while MEANS used SPARQL and query relaxation but suffered from scalability.
- Bangla systems: Prior Bangla healthcare systems used supervised disease prediction, translated English data, or prescription and report interpretation.Examples include a TF-IDF and cosine-similarity chatbot, a translated English dataset, and a Bengali prescription-and-report QA system.
- Remaining gaps: Bangla medical QA still faces limited annotated datasets, limited sophisticated work, and a need for real-world validation.These issues are identified as persistent despite prior developments.
3 A Bangla Question Answering Dataset for Healthcare Support
The paper builds a Bangla healthcare QA dataset and supporting linguistic resources, comprising 4,493 categorized QA pairs covering 506 diseases.
- Data collection: The raw medical data were collected from diverse online sources and cleaned for quality, relevance, and meaning.Sources included health blogs, Wikipedia, newspapers, and verified social-media pages and groups.
- Dataset: 4,493 QA pairs are organized into 9 healthcare categories covering 506 diseases.Each record contains a question, its corresponding answer, and a category.
- Dataset: The category distribution is described as almost balanced, while infectious and chronic diseases constitute the largest disease types.These distributions are shown in Figure 1.
- Linguistic resources: The questions receive POS tags to support anaphora resolution in the QA-pair dataset.The paper illustrates tagging with a Bangla dengue question and its English gloss.
- Linguistic resources: A 617-word Bangla medical root-word dictionary is developed because no standard domain dictionary exists.The paper lists examples including terms for rash, hunger, dengue, disease, and ear.
- Linguistic resources: Synonym sets normalize alternative words representing the same query to support further processing.The synonym-list design is illustrated in Table 2.
4 A QA System for Medical Bangla Text
BanglaMed-QA processes Bangla medical queries through preprocessing, categorization, similarity matching, and ensemble decisions to generate answers from a knowledge base.
- System workflow: The system resolves anaphora, preprocesses and classifies each query, matches it against a database, and generates an answer in an iterative workflow.Preprocessing includes POS tagging, disease-noun tracking, punctuation removal, tokenization, stop-word removal, lemmatization, synonym substitution, and detokenization.
- Question categorization: Question categorization uses TF-IDF representations with unigram and bigram features before applying five supervised machine-learning methods.The explored classifiers are Perceptron, Multinomial Naive Bayes, Passive Aggressive Classifier, Random Forest, and Support Vector Machine.
- Base similarity methods: Four base similarity methods compare processed and stored questions: TF-IDF cosine, Jaccard lexical overlap, BM25 retrieval scoring, and normalized Levenshtein similarity.BM25 uses term frequency, inverse document frequency, and document length; Levenshtein counts minimum insertions, deletions, and substitutions.
- Ensemble similarity methods: Soft voting averages the four similarity measures and accepts a pair as similar when its score reaches the predefined threshold τ = 0.8.The combined score is S(Q, Si) = C(Q, Si) + J(Q, Si) + L(Q, Si) + Bnorm(Q, Si).
- Ensemble similarity methods: Hard voting marks a pair similar when at least three of the four binary similarity votes exceed the decision threshold.The voting logic uses normalized cosine, Jaccard, Levenshtein, and BM25 scores; the decision threshold T is set to 0.8.
5 Experimental setup, Results and Analysis
The evaluation uses held-out and manually constructed test data to assess categorization, similarity matching, and human satisfaction. The results show near-perfect categorization, faster matching, improved ensemble performance, and high user ratings.
- Experimental setup: 80% of the QA-pair dataset was used for training and 20% for testing, alongside an 880-query manually constructed test set.The manual set pairs input queries with target responses and categories and includes challenging cases.
- Evaluation metrics: Accuracy, precision, recall, F1-score, and coverage were used to evaluate categorization and overall-system predictions.Unmatched predictions were assigned to NO_PREDICTION and counted as incorrect for accuracy, precision, and recall.
- Question categorization: Near-1.0 scores across categorization metrics made SVM the best overall model, with RF and PAC following closely.The convergence pattern indicated a well-fitted SVM that was neither over-fitted nor under-fitted.
- Question matching: 1/9: Question categorization sped up the question-similarity matching approach by around one-ninth.
- Ensemble similarity methods: Combining similarity methods enhanced performance, although hard voting reduced coverage because six cases became NO_PREDICTION under tied decisions.The tie arose because an even number of base methods could split their predictions between two candidate questions.
- Human evaluation: 0.9 out of 1.0: Human evaluation averaged 0.9, with individual ratings ranging from 0.78 to 1.0.Soft voting similarity was used for the human evaluation because it outperformed the other methods.
- Qualitative analysis: The system correctly categorized questions, resolved pronoun references, and answered disease-related queries about treatment, symptoms, prevention, and cures conversationally.
6 Conclusions
BanglaMed-QA addresses medical QA challenges in a low-resource language through preprocessing, categorization, similarity voting, and automatic and manual evaluation. The authors position it as a foundation for Bangla medical NLP while identifying disease prediction, broader disease coverage, and deep learning as future directions.
- BanglaMed-QA combines POS-based pronoun disambiguation, synonym-based question rephrasing, similarity methods, voting strategies, and question categorization.The system was evaluated automatically and manually to assess answer quality.
- The system is presented as a stepping stone for further Bangla NLP development and practical applications in the medical field.
- Future work includes integrating disease prediction, expanding the range of diseases, and exploring deep learning for categorization and matching.
A Interactive User Interface (UI)
The paper includes a practical interactive user interface to visualize the usefulness of BanglaMed-QA.
- An interactive UI was developed as a practical interface for visualizing the usefulness of BanglaMed-QA.
A. INTERACTIVE USER INTERFACE (UI) 15
Figure 6 presents the interactive user interface of the proposed BanglaMed-QA system.
- Figure 6 shows the interactive UI of the proposed BanglaMed-QA system.