PaperlayerView exact revision on arXiv

Source-linked AI summary

IndoNutriQA: An Indonesian Clinical Nutrition Question Answering Benchmark

Tirana Noor Fatyanosa, Ummu Ditya Erliana, Mahendra Data, Muhammad Nurwegiono, Eleazar Tadeo Eman, Alham Fikri Aji

arXiv:2610.31256v1cs.CL

TL;DR

Indonesian clinical QA lacks public domain-specific data, including for clinical nutrition. IndoNutriQA is an expert-curated benchmark of nutrition-report questions and answers; its strongest five-shot LLM results form a statistically tied top tier above the best fine-tuned IndoBERT baseline.

  • Problem

    No publicly available Indonesian clinical QA dataset was specifically designed for nutritional and medical decision support, despite clinical nutrition QA’s reliance on report-based information.

    Source: §1 · p. 1

  • Method

    IndoNutriQA provides 911 expert-authored question-answer pairs from real clinical-nutrition reports, using eight standardised NCP questions and evaluation with neural baselines and zero- and few-shot LLMs.

    Source: §1 · p. 2

  • Results

    The strongest five-shot LLMs form a statistically tied top tier, with Llama-3.1-8B-Instruct F1 = 55.55 and MedGemma-4B-IT 54.28, above the best fine-tuned IndoBERT baseline by +19.21 F1.

    Source: §7 · p. 9

  • Takeaways & Limitations

    IndoNutriQA is positioned as an evaluation resource for research in Indonesian clinical NLP and nutritional decision support.

    Source: §1 · p. 2

    Conclusions are limited to the eight question types, clinical reports, and tested configurations; the expert-rater audit found 6–10% of references contestable and clinical-safety scores unvalidated.

    Source: §4 · p. 5, §7 · p. 9

Abstract

from arXiv · show

Clinical Question Answering (QA) underpins Clinical Intelligent Decision Support Systems, but progress is severely limited for low-resource languages such as Indonesian by the absence of public domain-specific data. We introduce IndoNutriQA, an expert-curated Indonesian clinical QA benchmark of 911 question-answer pairs built from real clinical-nutrition reports. Rather than a large-scale training corpus, it is positioned as a high-quality evaluation resource: every pair is grounded in a clinically relevant context, enriched with structured metadata (anthropometric assessment, medical diagnosis, document provenance, chunk segmentation). Eight standardised question types following the Nutrition Care Process (NCP), including the Problem, Etiology, Sign and Symptom (PES) diagnosis statement, are answered for every report, which was written by a graduate-level dietetic intern, by one of three nutritionists. Answers are free-form clinical prose rather than extractive spans, and an expert validation study finds 6-10% of reference answers contestable. The dataset supports generative, extractive and retrieval-augmented QA as well as zero- and few-shot LLM evaluation. We report fine-tuned baselines (IndoBERT variants, mT5, RAG) with 5-fold cross-validation, learning curves and bootstrap 95% confidence intervals, and benchmark eight contemporary LLMs zero- and five-shot. We further calibrate our LLM-as-judge rubric against three expert raters (one RD, two nutritionists), finding it human-comparable on factuality and completeness but unreliable on clinical safety, and a human error analysis shows it misses unsupported content that the raters identify. IndoNutriQA is released to foster research in Indonesian clinical NLP and nutritional decision support.

1 Introduction

Indonesian clinical nutrition QA lacks a public dataset tailored to its terminology and reasoning in clinical reports. IndoNutriQA addresses this gap with expert-authored, context-grounded QA and is positioned as an evaluation benchmark rather than a training corpus.

1 Introduction

  • Indonesian clinical nutrition reports combine anthropometric assessments, diagnoses, and narrative information, but no public Indonesian clinical QA dataset was designed for nutritional and medical decision support.Existing clinical QA benchmarks also degrade in low-resource languages, nutrition subdomains, and localized clinical practice.Source: §1 · p. 1
  • IndoNutriQA provides expert-authored QA pairs from real clinical-nutrition reports, with clinical contexts and structured metadata, for extractive, generative, and retrieval-augmented QA.Metadata includes document provenance, chunk indices, anthropometric assessments, and medical diagnoses.Source: §1 · p. 1
  • Baseline experiments establish reference performance and expose challenges specific to Indonesian nutrition-focused QA, including domain terminology and numerical reasoning over anthropometric data.The introduction also identifies a five-axis evaluation protocol and an expert-rater validation study of the LLM-judge rubric, reference-answer errors, and human error analysis.Source: §1 · pp. 1–2
  • IndoNutriQA is an evaluation benchmark rather than a training corpus, with 911 pairs across 120 anonymised multi-hospital reports and eight standardised NCP questions, including PES.Each report was written by a graduate-level dietetic intern and answered by one of three nutritionists; the evaluation protocol includes cross-validation, bootstrap confidence intervals, learning curves, and zero- and few-shot LLM evaluation.Source: §1 · p. 2

2 Related Work

Prior QA benchmarks cover general, biomedical, or clinical tasks, but none combines Indonesian-language clinical nutrition with reasoning over authentic reports. IndoNutriQA also standardises the question side across reports, making it a controlled evaluation instrument.

Prior QA resources

  • General and biomedical QA resources do not target clinical reasoning over nutrition-focused real-world documentation, while Indonesian medical NLP had focused on term extraction and classification rather than QA.The cited examples include SQuAD, Natural Questions, TyDi QA, BioASQ, PubMedQA, and MedQA.Source: §2 · p. 2

Answer provenance

  • Existing resources differ in answer provenance and reasoning format; IndoNutriQA is small and fully curated, with each of its 911 answers written by a nutritionist from a real report authored by a dietitian-in-training.The comparison contrasts expert-written answers with labels derived from abstracts, exam answer keys, curated pages, and template-generated EHR pairs.Source: §2 · p. 2

Nutrition QA gap

  • Nutrition-focused QA resources remain scarce, and Indonesian open-domain QA does not cover clinical text or NCP, anthropometric, biochemical, and diet-prescription reasoning.Outside English, the cited larger resources are Chinese licensing-exam datasets; prior Indonesian medical NLP is described as term extraction and classification.Source: §2 · p. 2

Related benchmark coverage

  • IndoNutriQA occupies the intersection of Indonesian language, nutrition focus, and authentic clinical reports, with answers written by nutritionists following NCP/PES guidelines.TyDi QA includes Indonesian but is open-domain; the clinical resources discussed are English or Chinese, and none targets clinical nutrition.Source: §2 · pp. 2–3

Annotation unit

  • Eight standardised NCP/PES questions are applied uniformly to each report, fixing question phrasing and confining annotator variation to answer text.The paper presents this design as making the 911-pair corpus a controlled evaluation instrument rather than merely a small training set.Source: §2 · p. 3

3 Dataset Construction

IndoNutriQA is built from authentic Indonesian clinical-nutrition reports and expert-authored answers. Its fixed NCP/PES question set and abstractive clinical prose define both the benchmark’s controlled evaluation role and its limits for extractive QA.

3.1 Data Sources

  • The reports come from diverse Indonesian healthcare facilities and include varied hospital settings, anthropometric profiles, diagnoses, and nutritional conditions.The source documents include clinical rotation reports prepared by graduate-level dietetic interns, with case descriptions, measurements, biochemical parameters, and dietary plans; three nutritionists wrote the QA pairs.Source: §3.1 · p. 3
  • IndoNutriQA uses authentic clinical documentation and nutritionist-written answers rather than fully synthetic data.The paper presents this as reflecting the complexity, variability, and contextual reasoning of real-world clinical nutrition practice.Source: §3.1 · p. 3

3.2 Dataset Schema and Pipeline

  • Each entry records report and location information, anthropometric and diagnosis metadata, context, question, and ground-truth answer.The schema includes report title and file, page number, and chunk index.Source: §3.2 · p. 3
  • Only 8.7% of answers occur verbatim in their context, and the median answer is 30 words; the answers are authored clinical prose rather than extractive spans.The answer_start field marks the best-matching passage, not a verbatim span, which the paper identifies as the main reason extractive baselines trail prompted LLMs.Source: §3.2 · p. 3
  • The construction pipeline collects reports, anonymises and chunks them, annotates metadata, authors answers to eight fixed questions, and pairs answers with source chunks algorithmically.The same questions are applied to every report, so the annotation unit is a report paired with an answer to a fixed question; this measures eight NCP/PES steps, not open-ended clinical QA.Source: §3.2 · p. 4, Table 1 · p. 4

4 Annotation Protocol and Agreement

The annotation protocol uses one nutritionist per report to answer the same NCP/PES questions, so references are not independently annotated in parallel. Expert agreement is strong on factuality and completeness but effectively absent on clinical safety, while the audit found some reference answers contestable.

Annotation design

  • Each report’s eight answers were written by one nutritionist, and the three nutritionists covered different reports, so no report-question pair has parallel reference annotations.A conventional inter-annotator κ over the references is therefore not defined for this release.Source: §4 · p. 4
  • The fixed questions and external NCP/PES rubric constrain question-side and guideline variation, but the small author pool leaves only three writing styles.Because each nutritionist wrote answers for different reports, author effects cannot be separated from report effects.Source: §4 · p. 4

Agreement and reference audit

  • Three of 50 audited reference answers were flagged as incorrect or contestable (6%), rising to five (10%) when free-text criticism was included.Flagged disputes involved macronutrient percentages, a mid-upper-arm-circumference cut-off, and a neonatal protein prescription; flagged items are released for exclusion or re-adjudication.Source: §4 · p. 4
  • Reference-pair quadratic-weighted Cohen’s κ was 0.780 on factuality and 0.752 on completeness, but 0.160 on clinical safety.The authors describe agreement as substantial on verifiable content and effectively absent on safety judgement, which they report as unvalidated.Source: §4 · pp. 4–5
  • Two of the three audit raters also wrote references, so the 6–10% contestability estimate is not fully independent; only the RD did not write them.The authors characterise the estimate as coming from raters who are not all external to the dataset.Source: §4 · p. 5

Context matching

  • The context-matching procedure verified character-level answer_start offsets for 823 of 911 pairs (90.3%); the remaining 88 have chunk-level pointers without character offsets and are flagged in metadata.The reported six-stage matching procedure includes exact, case-insensitive, whitespace-normalised, punctuation-stripped, and fragment matching.Source: §4 · p. 5

5 Dataset Characteristics

IndoNutriQA comprises 911 pairs from 120 clinical-nutrition reports, with a naturally imbalanced disease mix and clinically realistic population characteristics. Its authentic Indonesian inpatient-report grounding distinguishes it from larger benchmarks based on exams or literature abstracts.

5.1 Dataset Statistics and Domain Coverage

  • 911 pairs from 120 reports cover 128 diagnoses; respiratory cases form the largest category at 28.33% of reports, and no category exceeds 30%.The per-pair distribution tracks the per-report distribution closely, so reports contributing unusually many pairs do not inflate a category.Source: §5.1 · p. 5

5.2 Demographics, Anthropometrics, and Syntactic Divergence

  • The population spans early childhood through adulthood, with 58.4% male participants; predicted-reference edit distance is bimodal, motivating multiple evaluation metrics.Weight and height show the expected positive correlation, while mid-upper arm circumference clusters around screening thresholds.Source: §5.2 · p. 5

5.3 Comparison with Existing Medical QA Datasets

  • Unlike larger English benchmarks based on exams or literature, IndoNutriQA uses nutritionist-authored pairs grounded in authentic Indonesian inpatient nutrition reports.The 911 pairs are fewer than the largest English resources, such as emrQA, which exceeds 400K.Source: §5.3 · p. 5

6 Experiments

Across IndoNutriQA’s test split, five-shot LLMs outperform fine-tuned baselines, with the top LLMs statistically tied. The experiments also test robustness, semantic and clinical-rubric measures, RAG components, stratified performance, and human-identified errors.

6.3 Large Language Model Evaluation

  • Five-shot prompting improves every evaluated LLM by +7.0 to +17.7 F1; smaller and language-tuned models gain more, while exact-match scores remain low despite stronger token overlap.GPT-4o’s F1 = 50.33 is matched by Llama-3.1-8B-Instruct at 55.55 under five-shot prompting, with overlapping confidence intervals.Source: §6.3 · pp. 7–8
  • Llama-3.1-8B-Instruct reaches F1 = 55.55 and MedGemma-4B-IT 54.28 in five-shot evaluation, statistically tied and +19.21 F1 above the best fine-tuned model.The benchmark’s strongest result comes from a model using five in-context exemplars without IndoNutriQA fine-tuning.Source: Table 2 · p. 7, §6.3 · p. 8

6.4 Statistical Robustness Analysis

  • Cross-validation rankings remain stable: the best extractive baseline scores 32.62 ± 1.29 F1 versus 36.34 on the held-out test set, and fold-level standard deviations stay ≤2.66 F1.The analysis also uses learning curves and bootstrap 95% confidence intervals; the top three five-shot LLMs remain well above the best fine-tuned baseline with overlapping intervals.Source: §6.4 · p. 8

6.5 Beyond Lexical Overlap: Semantic and LLM-as-Judge Evaluation

  • Top five-shot LLMs reach BERTScore F1 ≈91 and cosine ≈74, versus 46/45 for the best fine-tuned baseline; expert agreement is near inter-rater levels on factuality and completeness but weak on safety.Judge-vs-rater agreement is 0.705 vs 0.780 on factuality, 0.623 vs 0.752 on completeness, and 0.133 on safety; safety scores are treated as unvalidated.Source: §6.5 · pp. 8–9

6.6 RAG Diagnostic Evaluation (RAGAS)

  • In the tested RAG configuration, context precision is 73.43 and recall 35.98, while faithfulness is 12.86 on the 105 scorable examples, identifying generation as the binding constraint.The authors draw no broader conclusion about RAG for Indonesian clinical QA.Source: §6.6 · p. 9

6.7 Stratified Performance Analysis

  • Question-type winners diverge on longer answers—Llama-3.1-8B on recommendation at 69.36 F1, GPT-4o on inferential at 62.39, and MedGemma on factual at 33.39—while factual questions are uniformly hardest.Llama’s five-shot F1 ranges 50.08–64.17 across the seven test-set disease categories, with respiratory at 57.55.Source: §6.7 · p. 9

6.8 Human Error Analysis

  • Raters also identified NCP/PES structure violations in 6 units (12%), all on nutrition-diagnosis questions; the LLM taxonomy did not include this failure class.The errors included interpreting PES “S” as Solusi rather than Sign/Symptom or collapsing diagnoses into one statement.Source: §6.8 · p. 9
  • Raters identified unsupported additions in 5 units (10%) that the judge’s hallucination label missed, so the judge’s label is not a reliable safety indicator.The examples include an unsupported weight-loss trend despite no weight history in the patient record.Source: §6.8 · p. 9

7 Conclusion

IndoNutriQA fills a gap in Indonesian clinical QA with expert-written pairs grounded in real nutrition reports. Its strongest benchmark results come from a statistically tied tier of five-shot LLMs, while the paper limits its conclusions to the tested questions, reports, and configurations.

7 Conclusion

  • IndoNutriQA contains 911 nutritionist-written pairs grounded in graduate-level dietetic interns’ clinical-nutrition reports, and its top five-shot LLMs exceed the best fine-tuned IndoBERT baseline by +19.21 F1.The top tier is statistically tied: Llama-3.1-8B-Instruct scores F1 = 55.55 and MedGemma-4B-IT scores 54.28.Source: §7 · p. 9
  • Conclusions are limited to the eight question types, clinical reports, and configurations tested; 6–10% of reference answers were contestable, safety scores were unvalidated, and judges missed unsupported content.Future work will add cases from other facilities, extend the expert audit, and test stronger Indonesian RAG generators.Source: §7 · p. 9

Limitations

IndoNutriQA’s modest, deliberately controlled design and evaluation protocol set boundaries on the behaviors and populations its results can represent. Reference-answer uncertainty and limits in the judge, data coverage, and tested RAG configuration also constrain interpretation.

Limitations

  • 911 pairs from 120 reports make IndoNutriQA a benchmark reference, not an upper-bound estimate for data-hungry generative or retrieval-augmented systems.The authors use report-stratified five-fold cross-validation, learning curves, bootstrap confidence intervals, and zero- and five-shot evaluations, but note that modest scale limits exposure to tail behaviors.Source: Limitations · p. 10
  • Eight fixed NCP/PES questions trade coverage for control and leave open-ended dialogue and unqueried report topics outside the benchmark’s scope.Examples of unqueried topics include pharmacological interactions and longitudinal monitoring.Source: Limitations · p. 10
  • With 120 source reports, case diversity remains bounded; the held-out split over- and under-represents several disease categories and contains no cardiac or renal/urologic reports.GroupKFold cross-validation exercises every category, but transfer beyond contributing facilities, especially to primary-care puskesmas, remains untested; rare-condition estimates are unreliable.Source: Limitations · pp. 10–11
  • Single-annotator references allow clinically valid alternatives, and expert review flagged 6% as incorrect or contestable, rising to 10% including free-text criticism.The 50-unit audit was not fully independent because two of the three raters also wrote references; expansion to more units and the training split remains future work.Source: Limitations · p. 11
  • The Indonesian-only data limit direct applicability to other languages and healthcare systems; a parallel English-translated test split remains future work.The multilingual encoder and English medical LLM experiments provide partial cross-lingual signals, not a fully aligned cross-lingual evaluation.Source: Limitations · p. 11
  • Human evaluation covered only 50 stratified units, and the judge is unvalidated for clinical safety, where both judges scored above rater consensus.The authors caution that stratified-sample agreement may not transfer to the full test split and that safety scores are not clinical guarantees.Source: Limitations · p. 11
  • The hallucination label is a detection ceiling, while the human error analysis’s optional, unevenly distributed notes make its category counts lower bounds.A same-provider stylistic prior shared by both judges cannot be ruled out; the self-preference test is underpowered.Source: Limitations · pp. 11–12
  • The low RAG lexical scores localize failure to one tested configuration, with the mt5-base generator weaker than the retriever, not RAG generally.Retriever-side changes were not tested; the reported recall is 35.98.Source: Limitations · p. 12

Ethics Statement

The dataset is built from anonymized clinical-nutrition reports for research use, with standardized questions and single-nutritionist answers. Its annotation and context-matching choices support controlled evaluation while leaving specific validation and extractive-supervision limits.

Ethics Statement

  • The dataset removes patient and institution identifiers and is intended for research and education, not direct clinical deployment or automated medical decision making.The authors recommend qualified-clinician validation before derived models are used in real-world clinical settings.Source: Ethics Statement · pp. 12–13

A.1 Question Types and Complexity

  • Questions cover factual, inferential, comparative, and recommendation tasks, and the standardized NCP/PES set is applied uniformly to every patient record.The set spans assessment, clinical reasoning, and recommendation-oriented decision making.Source: §A.1 · pp. 14–15

Annotator Pool

  • Graduate-level dietetic interns wrote the 120 source reports, while three nutritionists authored the answers, each assigned different reports.Source: Annotator Pool · p. 15

Per-pair Annotation Count and Validation

Annotation Guideline

  • The eight questions map to NCP/PES sections, bounding answers by a clinical-nutrition rubric rather than free-form clinical opinion.The sections include screening, anthropometric and biochemical assessment, nutrition diagnosis, goals, principles, requirements, and guidelines.Source: Annotation Guideline · p. 15

Subjectivity and Inter-Annotator Agreement

  • Clinically valid alternative answers remain possible; the audit found 6% of references explicitly contestable, or 10% including free-text criticism.Two of the three raters also wrote references, so the audit is not fully independent.Source: Subjectivity and Inter-Annotator… · p. 15

Question-Context Pairing: Six-Stage Matching

  • A six-stage matching procedure located verified character positions for 823 of 911 answers (90.3%); the other 88 lack verified answer_start positions.Those 88 retain a chunk index and are flagged so users needing strictly extractive supervision can filter them.Source: Question-Context Pairing: Six-Stage… · p. 16

B Statistical Robustness Details

The robustness analyses support stable model rankings while distinguishing data-limited from architecture-limited systems. The learning curve shows the strongest extractive baseline still rising at full training data, whereas mT5 plateaus near zero.

Statistical Robustness: full discussion

Learning Curve

  • Across 10%, 25%, 50%, and 100% training fractions with validation and test splits fixed, the strongest extractive baseline rises overall despite a small 50% dip and still climbs at 100%, while mT5 plateaus near zero.Source: Learning Curve · p. 17

C Comparison with Existing Medical QA Datasets

The comparison positions IndoNutriQA as distinct in combining Indonesian language, nutrition, and authentic clinical documentation. A RAG diagnostic attributes the weak lexical scores in the tested configuration more to generation than retrieval.

C Comparison with Existing Medical QA Datasets

  • No other resource in the comparison combines Indonesian language, the nutrition domain, and grounding in authentic clinical documentation.TyDi QA covers Indonesian but is not clinical, while emrQA uses English and template-generated clinical records.Source: C Comparison with Existing Medical QA… · p. 17, Table 6 · p. 17
  • At 911 pairs, IndoNutriQA is the smallest listed resource, with free-text answers grounded in clinical reports.Its mean answer length is 70.66 tokens, but cross-resource length comparisons are approximate because units and tokenisation differ.Source: C Comparison with Existing Medical QA… · p. 17

RAG Diagnostic Evaluation: full discussion

  • In the tested RAG configuration, retrieval is stronger than generation, with mt5-base identified as the weaker component.Table 8 reports context precision 73.43, recall 35.98, faithfulness 12.86, and answer relevancy 0.24; faithfulness is defined for 105 examples.Source: RAG Diagnostic Evaluation: full… · pp. 17–18, Table 8 · p. 18
  • The RAG diagnosis is limited to one dense retriever with recursive chunks and an un-fine-tuned mt5-base generator.The authors call for a stronger Indonesian generator and a sweep over retrievers and chunking before establishing what RAG can achieve here.Source: RAG Diagnostic Evaluation: full… · p. 18

E Semantic and LLM-Judge Metric Details

The section tests whether semantic and LLM-judge metrics provide reliable evaluation signals, then calibrates the judge against another model and expert raters. Expert agreement is strong for factuality and completeness but weak for clinical safety.

E Semantic and LLM-Judge Metric Details

  • Removing English articles during exact-match normalization can make “vitamin A” indistinguishable from “vitamin,” affecting 67 reference answers.The same normalization is applied to predictions and references, so the passage says F1 is essentially unaffected.Source: E Semantic and LLM-Judge Metric Details · p. 18

Second-Judge Cross-Validation (Claude Haiku 4.5)

  • Across 2,879 paired judgements, GPT-4o-mini and Claude Haiku 4.5 have pooled κ = 0.652 and agree within one Likert point on 82.8%.The overall-rubric top tier is preserved across judges, although the top-ranked model changes.Source: Second-Judge Cross-Validation (Claude… · pp. 18–19

Judge Self-Preference Analysis

  • GPT-4o-mini’s additional leniency toward OpenAI systems is +0.125 rubric points and not statistically distinguishable from zero (permutation p = 0.34).The test detects provider-level favouritism, but with only four same-provider systems it cannot establish that no effect exists.Source: Judge Self-Preference Analysis · p. 19

Expert-Rater Calibration of the LLM Judge

  • GPT-4o-mini’s agreement with raters approaches inter-rater agreement on factuality and completeness, but safety scores are unreliable and substantially overstated.Factuality κ = 0.705 versus an inter-rater ceiling of 0.780; completeness κ = 0.623 versus 0.752. GPT-4o-mini’s safety scores average 4.92 against the rater consensus of 3.78.Source: Expert-Rater Calibration of the LLM… · pp. 20–21
  • Three of 50 audited units (6%) were flagged as contestable reference answers, involving nutrient distributions, a MUAC cut-off, and a neonatal protein prescription.Flagged items are released so users can exclude or re-adjudicate them.Source: Expert-Rater Calibration of the LLM… · p. 21
  • The judges’ system rankings correlate at Spearman ρ = 0.916, but the top position changes between GPT-4o-mini and GPT-4o.The paper therefore reports a top tier rather than a single rubric winner.Source: Expert-Rater Calibration of the LLM… · p. 21

F Human Error Analysis Details

The human audit identifies failure modes that the LLM-judge taxonomy misses, while stratified results show variation by question type but no systematic advantage for the largest disease category. The appendix also documents the audit’s scope and the benchmark’s standardized question and example formats.

F Human Error Analysis Details

  • The human audit’s error categories include NCP/PES violations and unsupported additions missed by the judge, while optional notes make category counts lower bounds rather than error rates.The 50 audited units span 20 systems, and raters had patient diagnosis and anthropometric information that the judge lacked.Source: F Human Error Analysis Details · pp. 21–22

G Stratified Performance Tables

  • Figure 2 summarizes lexical–semantic metric divergence, judge error labels, and five-shot prompting effects; the following tables provide stratified results.Its caption notes that LLMs cluster above the metric-agreement diagonal while mT5 falls far below it, and that prompting gains differ by model.Source: G Stratified Performance Tables · p. 22, Figure 2 · p. 23

Question-Category Breakdown

  • Recommendation and inferential questions generally receive higher F1 than shorter factual questions, so headline performance varies across reasoning types.Comparative questions are absent because each standardized question targets a single time point per patient.Source: Question-Category Breakdown · p. 22, Table 15 · p. 23

Error Taxonomy

  • Raters flagged unsupported additions in 10% of audited units, including clinically unsupported trends that the judge scored 4 on factuality; under-answering remained the dominant failure pattern.The judge’s hallucination label cannot be relied on, so the authors draw no conclusion about fabrication rates.Source: Error Taxonomy · p. 22

H Demographic, Anthropometric, and Syntactic-Divergence Figures

Standardized Clinical Nutrition Questions

Disease-Category Breakdown

J Example Question-Answer Instance

Related papers

Loading 2610.31256v1…