Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models

1 Intorduction

• Medical reports often conceal tabular data in text, so tabular ML methods cannot directly process the underlying information in that form.Text-based ML is commonly used for these problems instead.Source: §1 · p. 1
• Tabular ML can offer greater computational and statistical efficiency, interpretable features, and cross-domain applicability than text-based models.Text processing adds preprocessing steps, while textual models are often complex and may require domain-specific adjustment.Source: §1 · p. 2
• TEMED-LLM uses LLM reasoning guidelines and a validation-and-correction loop to extract tabular features, then applies interpretable models for end-to-end predictions.The evaluation includes public datasets and clinician-provided electronic health records.Source: §1 · p. 2
• The paper reports improved predictive power over state-of-the-art text classification models in medical diagnostics tasks.The contributions also include an interpretable modeling pipeline.Source: §1 · p. 2

2 Challenges

• Medical terminology, complex sentences, ambiguity, and the need for patient context make accurate manual feature extraction challenging and time-consuming.The passage notes that extraction may require experienced clinicians.Source: §2 · p. 2
• Direct textual ML can be poorly suited to complementary non-text tabular data and raises transparency concerns because advanced NLP models are mostly black boxes.Source: §2 · p. 2
• TEMED-LLM addresses extraction challenges with LLM technology while relying on tabular data rather than textual modeling.Source: §2 · p. 2

3 Related work

• Tabular medical ML analyzes records, trials, and registries, with interpretable models highlighted as important for clinical trust and application.The cited work also describes classification, regression, clustering, outcome prediction, and treatment personalization.Source: §3 · pp. 2–3
• NLP models including BERT [30], hierarchical multimodal BERT, BioBERT [24], and LLMs [12] predict from medical text, but their interpretability remains a challenge.Source: §3 · p. 3
• Unlike GReaT [11], which generates synthetic tabular data, and TabLLM [20], which serializes tabular data for LLM classification, TEMED-LLM extracts data from real-world text.Source: §3 · p. 3

4 TEMED-LLM: Medical Text to Tabular Data Conversion with Improved Interpretability and Diagnostic Performance

• TEMED-LLM has RExtract for LLM-based feature extraction, VORC for error recognition and correction, and an ML component for modeling extracted data.RExtract prompts incorporate domain knowledge, extraction instructions, reasoning, and few-shot examples.Source: §4 · p. 3, Figure 1 · p. 4

4.1 Problem formulation

• The formulation separates mapping medical texts to feature vectors from mapping feature vectors to labels with supervised ML.The text-to-feature mapping is performed by RExtract and VORC; labels may also be extracted from reports when not supplied.Source: §4.1 · p. 4

4.2 RExtract module

• RExtract combines extraction instructions, feature-specific reasoning grounded in domain knowledge, and the medical text to identify features even when names are not explicit.Reasoning improves on extraction that searches only for named features, though RExtract can still format values incorrectly.Source: §4.2 · pp. 4–5

4.3 VORC module

• VORC iteratively detects extraction discrepancies, applies rule-based corrections when possible, and otherwise prompts the LLM to correct the response.Its correction tasks include JSON and key-type errors.Source: §4.3 · pp. 5–6

4.4 ML analysis module

• The ML analysis module models extracted data with tabular techniques, emphasizing decision trees and logistic regression while allowing XGBoost when simpler models underperform.The passage notes that XGBoost is less explainable, though explanation techniques can aid interpretation.Source: §4.4 · p. 6

5 Experimental evaluation

• Evaluation used four datasets for predictive tasks; the 51-record Psychologist notes dataset was used only to assess extraction quality.Datasets were split 70% for training, 10% for validation, and 20% for testing; Patient treatment classification and Heart disease diagnosis came from a single clinic and were anonymized.Source: §5 · p. 6

5.1 Evaluation of tabular data extraction

• RExtract + VORC had the best extraction metrics in all cases, reaching 0.99 accuracy on Hepatitis and a 0.07 accuracy gain over RExtract on Heart disease diagnosis.Performance deteriorated on less structured Psychologist notes; VORC calls averaged approximately 10%, rising to 14.20% on Psychologist notes and 12.00% on Heart disease diagnosis, and falling to 2.54% on Patient treatment classification.Source: §5.1 · p. 7, Table 1 · p. 8

5.2 Evaluation of predictive performance

• XGBoost consistently outperformed the other models across datasets; logistic regression and decision trees were comparable to set-fit-mpnetv2, while BioBERT performed worse than the interpretable tabular models.In Heart disease diagnosis, logreg AUC was 0.860 and dtree AUC was 0.901, versus 0.931 for set-fit-mpnetv2.Source: §5.2 · p. 7, Table 2 · p. 8

5.3 Evaluation of interpretability

• The interpretability assessment assumes both that the tabular models are interpretable and that extraction errors do not considerably affect model behavior.It compares predictive performance and feature-importance patterns between models trained on ground-truth and extracted data.Source: §5.3 · p. 9
• Models trained on extracted tables performed least favorably on Stroke prediction, where greater extraction error coincided with greater disparity from ground-truth data.The authors conclude that interpretation quality relates to extraction quality in a predictable way.Source: §5.3 · p. 9, Table 3 · p. 9

6 Conclusions

• The authors report improved accuracy over state-of-the-art text classification models using medical text converted to tabular data and interpretable machine-learning models.The conversion uses an LLM with a data-validation and reasoning-correction feedback loop.Source: §6 · p. 9
• The authors assessed extraction faithfulness and its relationship to the interpretability of the resulting models.Source: §6 · p. 9
• The authors describe potential impact in terms of interpretable, easily trained models, lower computational and hardware costs, and medical-data management and hypothesis testing.They also identify classification accuracy and trust in automated medical decision-making as potential benefits.Source: §6 · pp. 9–10
• The authors state that TEMED-LLM requires medical-expert input to identify relevant report features, and downstream modules cannot compensate for expert errors.Source: §6 · p. 10
• The authors conclude that TEMED-LLM is a step toward intelligent and trustworthy automation in medical decision-making.Source: §6 · p. 10

A.1 Detailed Description of Datasets

• The heart-disease dataset contains 917 EHRs from a private facility, with 11 predetermined attributes and a binary heart-disease outcome.Listed features include age, sex, chest-pain type, resting blood pressure, cholesterol, fasting blood sugar, and resting ECG.Source: §A.1 · p. 13
• The listed heart-related attributes also define maximum heart rate, exercise-induced angina, Oldpeak, and ST slope categories.The passage gives a maximum-heart-rate range of 60 to 202 and describes the categorical values for angina and ST slope.Source: §A.1 · p. 13
• The treatment-classification dataset contains 718 EHRs and 10 predefined features; its target distinguishes in-care from out-care using laboratory results and physicians’ observations.Listed attributes include haematocrit, haemoglobin, erythrocyte, leucocyte, and thrombocyte results.Source: §A.1 · p. 13
• The hepatitis dataset has 589 records, 12 predetermined features, and no missing values, combining liver-function test results with physicians’ observations.Its listed attributes include age, sex, and measurements such as albumin, bilirubin, cholesterol, and protein.Source: §A.1 · p. 14
• The stroke dataset comprises 110 entries with 10 attributes for predicting stroke occurrence from factors including gender, age, prevalent diseases, and smoking status.The listed features include demographic, study, marital-status, and mental-health or treatment indicators.Source: §A.1 · pp. 14–15
• The psychologists’ notes dataset contains 51 records from private colleges in the United States and is used to assess extraction quality, not predictive tasks, because of its size.Its 10 features include student demographics, study information, mental-health indicators, and specialist-treatment status.Source: §A.1 · p. 15

A.2 Evaluating Extract-only, RExtract, and RExtract+ VORC Techniques Using ChatGPT in the TEMED-LLM Framework

• The ChatGPT evaluation compares Extract-only, RExtract, and RExtract+ VORC across all five datasets.It evaluates extraction accuracy, precision and recall with respect to missing values, and the percentage of VORC calls.Source: §A.2 · p. 15
• The authors report no significant performance difference between ChatGPT (gpt3.5) and text-davinci-003 in this evaluation, and say both can be used with RExtract+ VORC.They characterize text-davinci-003 as slightly better and attribute its edge plausibly to instruction-following training.Source: §A.2 · p. 15
• The authors say the ChatGPT evaluation parallels the earlier evaluation using text-davinci-003.Source: §A.2 · p. 15

A.3 Comparative Analysis of Feature Importance in Models Trained on Ground Truth Data Versus Tabular Data Extracted from Medical Reports

• The comparison covers logistic regression, decision tree, and XGBoost, with feature-importance scores for both extracted-data models and ground-truth models denoted with _gt.Decision-tree importance is normalized total criterion reduction, XGBoost and decision-tree scores sum to 1, and logistic-regression scores use model weights.Source: §A.3 · pp. 15–16
• Differences in feature importance between models trained on extracted and ground-truth tabular data are negligible across all datasets.Source: §A.3 · p. 16

A.4 Showcasing Examples of Prompts Using the RExtract Module

• Prompts for five datasets are organized into instructions with one-shot examples, reasoning, and the medical text to process.The examples in this part cover heart disease, patient treatment, hepatitis, and stroke; the section introduction also names psychological notes.Source: §A.4 · p. 16

A.4.1 Hearth disease diagnosis

• The medical-report prompt specifies a typed JSON schema for patient age, sex, chest-pain, blood-pressure, ECG, heart-rate and exercise-related fields, and asks for step-by-step value assignment for each key before output.Source: §A.4.1 · p. 19
• The heart-disease example demonstrates interpreting report details to assign categorical features and numerical measurements, including fasting blood sugar and exercise angina.The worked reasoning assigns fasting blood sugar 1 for a reported value above 120 mg/dL and exercise angina N when symptoms are absent during the test.Source: §A.4.1 · p. 20

A.4.2 Patient treatment classification

• The treatment-classification prompt specifies a numeric lab-and-demographic schema and asks for reasoning for each extracted key before returning JSON.Its sample note uses paraphrases such as “volume fraction of red blood cells” for haematocrit and reports the corresponding value as 35.1.Source: §A.4.2 · pp. 21–22

A.4.3 Hepatitis prediction

• The hepatitis prompt defines age, sex, and blood-marker fields, and its worked reasoning extracts values from paraphrased descriptions such as “completed 32 solar cycles.”The example maps albumin to ALB and identifies ALP, ALT, AST, BIL, CHE, CHOL, CREA, GGT, and PROT in the notes.Source: §A.4.3 · pp. 23–24

A.4.4 Stroke prediction

• The stroke-prediction excerpt specifies a student-focused schema covering demographics, study details, and mental-health or treatment indicators, then presents the required reasoning-and-JSON format.The included example begins with a 49-year-old female patient, but the supplied passage ends before showing its extraction reasoning or output.Source: §A.4.4 · p. 25
• The stroke example combines history and clinical findings with glucose, BMI, and smoking information to populate demographic and risk-factor fields in JSON.The example records elevated glucose, obesity-range BMI, active smoking, and no hypertension or heart disease; its output includes age, sex, and corresponding risk fields.Source: §A.4.4 · pp. 25–26

A.4.5 Psychologist notes

• The prompt distinguishes a valid JSON instance from an object that merely contains schema properties, illustrating the required output format.Its example treats an object with the required foo array as valid and an object containing a properties wrapper as invalid.Source: §A.4.5 · p. 27
• The psychologist-notes prompt requires a JSON instance matching a supplied schema and asks for an explanation of each key assignment before the final output.The example schema includes student demographics, course and study details, and yes/no mental-health and treatment fields.Source: §A.4.5 · p. 27
• In the example, notes describe a scholar as in his early twenties; reasoning assigns age None and extracts male gender, Islamic Studies, year 2, and CGPA 3.00 - 3.49.The reasoning also records marital_status as no.Source: §A.4.5 · pp. 27–28
• The reasoning maps reported anxiety to yes, while denied depression and panic attacks map to no.The example output also records no prior specialist treatment.Source: §A.4.5 · pp. 27–28
• The student example's final JSON includes extracted gender, course, study year, CGPA, marital status, depression, and anxiety fields.The displayed output is truncated before the panic-attack field is complete.Source: §A.4.5 · p. 28

• Figure 3 compares two decision trees on Patient treatment classification, one trained on ground-truth tabular data and the other on TEMED-LLM-extracted data.Source: §A.5 · p. 29
• The two decision trees exhibit striking similarities in their structures.Source: §A.5 · p. 29
• Only minimal differences are observed between the trees.Source: §A.5 · p. 29

A.6 Details of used ML models and their setups

• The appendix details the setups for text classifiers and tabular data models.Source: §A.6 · p. 29

A.6.1 Text classifiers

• The text-classification models are XLM-RoBERTa-Large, RoBERTa-Base, BERT-Base-Cased, BioBERT, and SetFit-MPNet-v2.The first four models were trained for three epochs with a learning rate of 2e-05; SetFit-MPNet-v2 also used three epochs and a learning rate of 2e-05.Source: §A.6.1 · p. 29
• The best text-classification model was loaded after evaluation and saving at the end of each training epoch.Source: §A.6.1 · p. 29

A.6.2 ML tabular models

• The tabular models are logistic regression, decision tree, and XGBoost, with hyperparameters tuned by grid search and no class weighting.The decision tree used Gini impurity; the compared trees in Figure 3 use ground-truth or TEMED-LLM-extracted data.Source: §A.6.2 · pp. 29, 31, Figure 3 · p. 30